Exam Feedback: Using AI Models for Medical Students

Imagine you're a medical student studying for the Progress Test Medizin. You take the exam and get your numerical score, but wouldn't it be helpful to know more? Content-based feedback can provide insight into where you went wrong and right, serving as a study aid and revision tool. But how can we make this happen? Enter Large Language Models (LLMs)! We tested two popular LLMs to see if they could generate useful feedback for medical multiple-choice questions. We compared their performance by checking how similar their outputs were. Additionally, we asked medical professionals what they thought of using LLMs for this purpose.

Both LLMs had their pros and cons. One might give slightly better feedback, but it comes at a cost—it’s a paid service. The other is free. Our survey found that the generated feedback was relevant and useful to medical practitioners and educators. They’re open to using LLMs in the future. While the AI-generated feedback isn’t perfect, it adds value to the numerical feedback students already receive. It’s like having a personal study coach always available.

Actions