In the opening when the congregation is singing "In the Garden of Eden," Nelson Muntz can be seen wearing his ordinary outfit of a pink shirt with blue vest, however, in the next scene when Reverend Lovejoy is questioning the children about the hymn prank, Nelson can be seen wearing a green suit with blue bow tie.
When Rev. Lovejoy announces the name of the Hymn at the beginning of the episode, he reads the title from the handout as "In the Garden of Eden." However, when he is trying to find out who was responsible for the switch after the service, we can clearly see that the handout says "In-A-Gadda-Da-Vida."
When Rev. Lovejoy leaves Bart and Milhouse alone in the sanctuary to clean the organ, we hear a door slam. Later, Milhouse asks what all religions have to gain from "making up" a soul, we see Lovejoy sorting out the offering through an open door. We do not hear the door open again, so we shouldn't be able to see him.
Marge looks at an alligator head with sunglasses. The look of the alligator is actually much more of the look of a crocodile due to the actual fact that real alligators have no teeth on their lower jaw.
Doctor Hibbert takes his family to Moe's thinking it was a family restaurant, however he's been in Moe's in the past and should have known it was a bar.