At the start of the guitar scene where George and Seth play "Two Princes" you can hear some of the others singing "Love Take Me Down to the Streets" a song written and recorded for David Wain's previous film Role Models (2008). The song is meant to sound like the Paul McCartney band Wings, and is often believed to be a Wings song as a running gag throughout Role Models.
When George is standing in line getting coffee at the beginning of the film, the sign behind him reads "New York Office Building (actually in Atlanta)", a site gag obviously referring to the fact that the film was wholly shot in and around Atlanta.
No, George and Lindas micro-loft was created on a soundstage in Atlanta. It was Aaron Osbornes (Production Designer) favorite set. He says, It is the smallest set Ive ever designed, yet it had to hold a lot of people for filming. Everything was designed to break away and roll away. It was a little tricky, but we wanted to give the audience a sense that the characters gave everything up to be in this teeny, tiny space. It had to look good enough yet small enough so they would say, Why are we here?
What is the first song in the movie?
"Get Your Body" by Baby. The complete list of the remaining songs can be found here with scene descriptions.