The painting, of two woman killing a man, that is said to be the boys' favorite is titled "Judith beheading Holofernes" and painted in 1612 by Italian artist Artemisia Gentileschi (1593-1652).
(at around 20 mins) When Miss Cathcart is investigating the classroom where many of the ghost sightings occurred, for a brief moment on the chalkboard behind her there is a quotation that says: "They shall grow not old, as we that are left grow old." This is a line from the poem "For the Fallen" by British poet Laurence Binyon. Published in 1914, it was meant as an ode to the British soldiers dying in the First World War. This hints, among other things, at connections to the boys who were shown earlier in the film in the school photos, many of whom presumably died during the War.
When Florence went into the hidden compartment and found the stuffed rabbit, the rabbit played a recorded song. At this point in history, toys only contained small music boxes, which played chiming music. The closest thing was the "Lioretgraph Jumeau" which sang a maximum of 35 words using a small phonograph. More advanced singing toys didn't make their appearance until the late 1930's and early 40's.
After finding her cigarette case in the pillow that she's ripped open, Florence goes outside with feathers stuck in her hair. In the next shot, however, the feathers are gone and her hair is clean again.