Throughout the liturgical year, both the Gloria and Alleluia are sung on nearly every Sunday. They are a feature of Mass that many of us are familiar with and look forward to whenever we attend Mass.
What's interesting is that the Alleluia is retained during Advent.
However, the Gloria is omitted during the penitential seasons of Advent and Lent and even the Alleluia is omitted during Lent. This omission is meant to remind us of the somber tone of these seasons and how we are called to participate in prayer, fasting and almsgiving.
Dom Prosper Guéranger first explains in his Liturgical Year that at first it seems like a contradiction.
Why is that?
[T]here is one feature which distinguishes Advent most markedly from Lent: the word of gladness, the joyful Alleluia, is not interrupted during Advent...It is sung in the Masses of the four Sundays and vividly contrasts with the sombre color of the Vestments.
Yet, there is a reason for keeping the Alleluia during Advent, as Guéranger explains.
Emmanuel is already here
[The Church] does not forget that the Emmanuel is already come to her, that he is in her and that even before she has opened her lips to ask him to save her, she has been already redeemed and predestined to an eternal union with him. This is the reason why the Alleluia accompanies even her sighs and why she seems to be at once joyous and sad waiting for the coming of that holy night, which will be brighter to her than the most sunny of days and on which her joy will expel all her sorrow.
The Advent season contains both joy and sorrow, and is different in that way from Lent.
Lent is much more a season of penance and sorrow, dwelling upon Jesus' sacrifice on the cross.
Advent, on the other hand, looks forward with joyful hearts at the coming of our Savior, who has already come, and is with us now.
It is fitting then that the Alleluia is retained and fills us with joy during a season of joyful expectation.