Kelsey began his scholarly career by publishing texts of ancient authors, with apparatus to aid students. His interest in contributing to the effective teaching of Latin and Greek led him to expand the conventional grammatical commentaries to include considerations of literary form and to involve the broader contexts offered by archaeology, history and topography.
The text for which he is best known, Caesar's Gallic War, was first published in 1886 and revised throughout his life. His friendship with the German archaeologist August Mau led to their collaboration on Pompeii, Its Life and Art (1899). He also began publishing the Humanistic Series under the aegis of the University of Michigan Faculty Senate.