Bill Murray genuinely found the Keith McCawley and Ronnie McCawley who play his sons and were very much like their screen characters, annoying, and many of the scenes where he lashes out at them and insults them were improvised.
Rosemary takes a job at a girls' private school called "The Webster Smalley School for Girls". Webster Smalley is well known for teaching playwriting at the University of Texas at Austin, which Wes Anderson and Owen Wilson both attended.
The Swiss Army Knife Dirk gives Max says "Rushmore Yankee 1985-1997". Since Max is 15, this would mean he has gone to Rushmore since age three. But we know from his first talk with Dr. Guggenheim that he has only gone to Rushmore since second grade.
When Max is riding the replacement bicycle to Blume's work to sabotage his car, we see from the side that the rack on the back of the bike has nothing in it; when he gets to the gate and we follow him in, there is now a small pack attached to the rack (from which he pulls his tools in the next scene under the car).