NYU Professor Implements AI-Powered Oral Exams to Combat Academic Dishonesty

The rise of artificial intelligence (AI) has significantly impacted various sectors, including education. Educators have expressed concerns that AI tools enable students to produce polished assignments without a true understanding of the material. This phenomenon has led to debates about the integrity of traditional assessment methods, prompting educators to seek innovative solutions to ensure genuine learning.

The Cool Down

Panos Ipeirotis, a professor at New York University's Stern School of Business, observed that many of his students submitted assignments that appeared well-crafted but lacked depth in understanding. In response, he introduced AI-powered oral exams to assess students' comprehension more effectively. Ipeirotis noted that when he questioned students about their submissions, many struggled to articulate their reasoning, indicating a disconnect between their written work and actual understanding.

Business Insider

To implement this, Ipeirotis utilized ElevenLabs' conversational speech technology to create an AI examiner capable of conducting oral assessments at scale. The oral exams consisted of two parts: questioning students about their capstone projects and engaging them in discussions about class cases. Over nine days, the AI assessed 36 students, with each session lasting approximately 25 minutes. The total cost for the AI's operation was around $15, a stark contrast to the hundreds of dollars typically required for human-run exams.

Business InsiderThe Cool Down

Ipeirotis employed three AI models—Claude, Gemini, and ChatGPT—to grade the oral exams. These models independently evaluated the transcripts, collaborated on their assessments, and produced final grades. Ipeirotis reported that this "council of LLMs" provided more consistent and fair evaluations compared to human assessors. He emphasized that the AI's feedback was superior to that of human evaluators and highlighted gaps in the teaching material.

Business Insider

Despite the advantages of AI oral exams, student responses were mixed. A minority expressed a preference for this assessment format, while many found it more stressful than traditional written tests. Nonetheless, students acknowledged that the oral exams better measured their understanding of the material. Ipeirotis defended the approach, stating that the oral exams exemplified effective learning, as practice leads to improvement.

Business Insider

The introduction of AI in assessments has sparked broader discussions within academia. A study published in the journal "Assessment & Evaluation in Higher Education" described the challenges faced by educators in adapting to AI's influence on student assessment. Some faculty members view AI as a tool for enhancing learning, while others perceive it as a threat to academic integrity. LinkedIn co-founder Reid Hoffman suggested that traditional assessment formats, such as essays, may become obsolete, advocating for a reevaluation of how universities assess student learning.

Business InsiderThe Cool Down

As educational institutions continue to grapple with the implications of AI, the future of assessments remains uncertain. Ipeirotis' experiment with AI oral exams may serve as a model for other educators seeking to adapt to the evolving landscape of academic integrity and student evaluation. The ongoing dialogue about AI's role in education will likely shape the development of new assessment strategies in the coming years.

Business Insider