Episode sponsor

Maths teachers have built a free ChatGPT app that can generate worksheets directly in chatGPT, with proper mathematical notation, graphs and even diagrams!

To get started, search “Worksheet” in the ChatGPT App store. Prompt the Strive PDF Generator to create whatever you need: unit tests, homework or quizzes, then just download the fully-formatted PDF. No latex knowledge or copy-paste required.

Go to worksheet.chat to get started.

Episode details

In this in-depth interview, Daisy Christodoulou explores the evolution of AI in education assessment, focusing on the capabilities and limitations of large language models and AI auto-marking systems. She discusses the history, current state, and future potential of AI in assessment, emphasising the importance of human oversight and the challenges of AI interpretability. Daisy Christodoulou discusses the impact of AI on education, assessment, professional development, and resource creation, emphasising the importance of human oversight and strategic integration of technology.

Takeaways

1. History of AI in assessment
2. Limitations of early AI auto-markers
3. Comparison of absolute and comparative judgment
4. Second-order effects and AI biases
5. Human-in-the-loop in AI assessment AI’s role in assessment and grading
6. Importance of human oversight in AI systems
7. AI’s impact on teacher professional development
8. Resource creation and lesson planning with AI
9. Future of education technology and infrastructure

Video:

Links from Daisy

1. No More Marking Substack: substack.nomoremarking.com
2. Her books are: 7 Myths about Education, Making Good Progress, Teachers v Tech, and I Can’t Stop Thinking About VAR
3. No More Marking can be found at: nomoremarking.com

New stuff I have been working on:

1. My Tips for Teachers Guides to… series
2. Our Eedi RCT and paper with Google
3. My updated mrbartonmnaths website.

My usual plugs

• You can help support the podcast (and get an interactive transcript of this episode) via my Patreon page at patreon.com/mrbartonmaths
• If you are interested in sponsoring an episode of the show, then please visit this page
• You can sign up for my free Tips for Teachers newsletter and my free Eedi newsletter
• My online courses are here: craigbarton.podia.com
• My books are “Tips for Teachers“, “Reflect, Expect, Check, Explain” and “How I wish I’d taught maths”

Written summary:

How AI is Revolutionizing Assessment: Insights from Daisy Christodoulou

Meta Description: Discover how AI is transforming educational assessments and the potential pitfalls of relying on technology. Insights from Daisy Christodoulou, Director of Education at No More Marking.

In the ever-evolving landscape of education, the integration of artificial intelligence (AI) poses exciting opportunities and significant challenges. In a recent conversation with Daisy Christodoulou, Director of Education at No More Marking, we explored her journey with AI and its implications for assessment in schools.

About Daisy Christodoulou

Daisy Christodoulou is an influential figure in the field of education, known for her expertise in assessment and educational technology. As the author of three books, including “Seven Myths About Education,” and the Director of Education at No More Marking, Daisy has dedicated her career to improving educational outcomes through innovative approaches.

Daisy’s Journey with AI

Daisy’s journey with AI began with her work at No More Marking, which specializes in online comparative judgment writing assessments. Initially, the assessment relied entirely on human judgment, but as AI technology advanced, the conversation around its potential for automating assessments gained momentum. Daisy shares her early experiences with AI, highlighting the skepticism and caution that often accompany new technologies.

The Evolution of AI in Education

In November 2022, just before the launch of ChatGPT 3, Daisy and her team engaged in strategic discussions about the future of assessments. They considered the implications of handwriting recognition and the feasibility of AI marking written assessments. At that time, the consensus was that AI technology was still a long way from being reliable enough for educational purposes.

The Promise and Pitfalls of AI

Daisy reflects on the historical context of AI in assessment, tracing its roots back to the 1960s with Project Essay Grade. Despite the potential for AI to provide consistent and scalable grading, Daisy emphasizes that early AI models often relied on superficial features, leading to significant limitations in their effectiveness. For instance, while these models might achieve consistency in grading, they do not account for the complexities of writing quality.

Understanding AI’s Limitations

Daisy addresses the common misconception that AI can seamlessly replace human judgment in assessments. She explains that consistency, while a strength of AI, can also be its downfall. A notable example is the correlation between essay length and quality. Although longer essays tend to receive higher grades, this correlation does not imply causation. Students can exploit this weakness by artificially inflating their essay lengths without improving content quality.

The Risk of Model Collapse

The concept of model collapse emerges as a critical concern in Daisy’s analysis. As AI systems are trained on surface features, they become susceptible to manipulation, leading to discrepancies in grading standards. This phenomenon is not unique to education; it extends to other fields, such as facial recognition technology, where reliance on superficial metrics can yield flawed outcomes.

Key Takeaways from Daisy’s Insights

• AI’s Historical Context: AI in assessment is not new; its roots trace back to the 1960s with early models like Project Essay Grade.
• Consistency vs. Quality: AI models provide consistent grading but often overlook the nuanced aspects of writing quality.
• Causation vs. Correlation: The relationship between essay length and quality exemplifies the dangers of relying on superficial metrics.
• Model Collapse: The risk of model collapse underscores the need for awareness of AI limitations across various fields.

Conclusion

As we continue to explore the intersection of AI and education, it’s essential to approach these advancements with a balanced perspective. While AI has the potential to enhance assessment practices, it is crucial to remain vigilant about its limitations and the implications of relying solely on technology.

For more insights into the future of education and AI, consider listening to the full conversation with Daisy Christodoulou. Together, we can navigate this exciting landscape and harness the true potential of AI in education.