Giving University Exams in the Age of Chatbots

Summary

Giving University Exams in the Age of Chatbots by Ploum on 2026-01-19 What I like most about teaching "Open Source Strategies" at École Polytechnique de Louvain is how much I learn from my students, especially during the exam. I dislike exams. I still have nightmares about exams. That’s why I try to subvert this stressful moment and make it a learning opportunity. I know that adrenaline increases memorization dramatically. I make sure to explain to each student what I was expecting and to be helpful. Here are the rules: 1. You can have all the resources you want (including a laptop connected to the Internet) 2. There’s no formal time limit (but if you stay too long, it’s a symptom of a deeper problem) 3. I allow students to discuss among themselves if it is on topic. (in reality, they never do it spontanously until I force two students with a similar problem to discuss together) 4. You can prepare and bring your own exam question if you want (something done by fewer than 10% of the students) 5. Come dressed for the exam you dream of taking! This last rule is awesome. Over the years, I have had a lot of fun with traditional folkloric clothing from different countries, students in pajamas, a banana and this year’s champion, my Studentausorus Rex! An inflatable Tyranosaurus Rex passing my exam in 2026 My all-time favourite is still a fully clothed Minnie Mouse, who did an awesome exam with full face make-up, big ears, big shoes, and huge gloves. I still regret not taking a picture, but she was the very first student to take my words for what was a joke and started a tradition over the years. Giving Chatbots Choice to the Students Rule N°1 implies having all the resources you want. But what about chatbots? I didn’t want to test how ChatGPT was answering my questions, I wanted to help my students better understand what Open Source means. Before the exam, I copy/pasted my questions into some LLMs and, yes, the results were interesting enough. So I came up with the followi...