All Saints Day and its eve or vigil the day before weren't always celebrated in the midst of autumn. In fact, initially this feast was celebrated at the beginning of summer.
The Catholic Encyclopedia explains that May 13 was originally the date of a feast dedicated to Mary and all the Martyrs.
At the time, this anniversary feast was one of a kind and the only liturgical celebration of a large group of saints. Then "Gregory III (731-741) consecrated a chapel in the Basilica of St. Peter to all the saints and fixed the anniversary for November 1."
However, the previous feast on May 13 was still celebrated, and the Church's calendar had two feasts honoring large groups of saints.
This remained the case until "Gregory IV (827-844) extended the celebration on November 1 to the entire Church. The vigil [aka Halloween] seems to have been held as early as the feast itself."
After this point, the May 13 feast fell into disuse and the November 1 feast rose in prominence.
It is claimed by various Church historians that Gregory IV moved the feast from May 13 to November 1 to coincide more closely with the harvest (in the northern hemisphere). In a book entitled The Lives of the Saints published in 1865, the author explains that "this solemnity was afterwards removed to the other half year November 1, when the harvest had been got in."
Another book, The Sacred Vestments published in 1899, also claims that "[All Saints Day] was transferred to November 1 ... the harvest was then gathered in."
This would make sense, as the harvest would allow for a greater feast, and not interrupt the laborers out in the field.
As a result, Halloween, the eve of All Saints, has been associated with the harvest ever since it was transferred to October 31.