Marking handwritten student work at scale is one of the most time-consuming tasks in undergraduate teaching, and multimodal large language models (LLMs) have quickly become the leading candidate for automating it. Recent studies show promising results for handwritten physics and mathematics problems, but performance drops when answers depend on diagrams rather than text.

Astronomy poses a particularly demanding version of this problem. Competitions such as the International Olympiad on Astronomy and Astrophysics routinely ask students to identify constellations, bright stars and planets on a projection of the night sky. Grading these answers requires spatial and geometric reasoning in a way that has not yet been tested with LLMs.

This project will build a labelled dataset of annotated sky charts and use it to measure how well current multimodal LLMs mark them against a human-defined rubric. You will investigate how accuracy depends on prompt design, rubric structure, sky projection, and how partial credit is defined. In parallel, you will implement classical computer-vision baselines to test whether purpose-built algorithms outperform general-purpose models on this narrow task.

Knowledge and interest in astronomy is essential. Basic python programming is also essential. 

Experience with computer vision is highly desireable.