*****

Second Workshop on Automated Evaluation of Learning and Assessment Content

AIED 2025 workshop | Palermo (Italy) & Hybrid | 22-26 July 2025

https://sites.google.com/cam.ac.uk/eval-lac-2025

*****

We are happy to announce the second edition of the Workshop on Automated Evaluation of Learning and Assessment Content will be held in Palermo (Italy) & online during the AIED 2025 conference.

About the workshop

Evaluation of learning and assessment content has always been a crucial task in the educational domain, but traditional approaches based on human feedback are not always usable in modern educational settings. Indeed, the advent of machine learning models, in particular Large Language Models (LLMs), enabled to quickly and automatically generate large quantities of data, making human evaluation unfeasible. Similarly, Massive Open Online Courses (MOOCs) have numbers of attending students so large that it is unsustainable to manually provide feedback to all of them. Thus, the need for accurate and automated techniques for evaluating educational content – e.g., questions, hints, and feedback – became pressing. Building on the success of the First Workshop on the Automatic Evaluation of Learning and Assessment Content, which was held at AIED 2024, this workshop aims to attract professionals from both academia and industry, and to to offer an opportunity to discuss common challenges, share best practices, and promising new research directions.

Topics of interests include but are not limited to:

Question evaluation (e.g., in terms of the pedagogical criteria listed above: alignment to the learning objectives, factual accuracy, language level, cognitive validity, etc.).

Estimation of question statistics (e.g., difficulty, discrimination, response time, etc.).

Evaluation of distractors in Multiple Choice Questions.

Evaluation of reading passages in reading comprehension questions.

Evaluation of lectures and course material.

Evaluation of learning paths (e.g., in terms of prerequisites and topics taught before a specific exam).

Evaluation of educational recommendation systems (e.g., personalised curricula).

Evaluation of hints and scaffolding questions, as well as their adaptation to different students.

Evaluation of automatically generated feedback provided to students.

Evaluation of techniques for automated scoring.

Evaluation of pedagogical alignment of LLMs.

Evaluation of the ethical implications of using open-weight and commercial LLMs in education.

Evaluation of bias in educational content and LLM outputs.


Human-in-the-loop approaches are welcome, provided that there is also an automated component in the evaluation and there is a focus on the scalability of the proposed approach. Papers on generation are also very welcome, as long as there is an extensive focus on the evaluation step.

Important dates

Submission deadline: May 25, 2025

Notification of acceptance: June 15, 2025

Camera ready: June 22, 2025

Workshop: July 22 or July 26, 2025

Submission guidelines

There are two tracks, with different submission deadlines.

Full and short papers: We are accepting short papers (5 pages, excluding references) and long papers (10 pages, excluding references), formatted according to the workshop style (using either the LaTeX template or the DOCX template).

Extended abstracts: We also accept extended abstracts (max 2 pages), to showcase work in progress and preliminary results. Papers should be formatted according to the workshop style (using either the LaTeX template or the DOCX template).

Submissions should contain mostly novel work, but there can be some overlap between the submission and work submitted elsewhere (e.g., summaries, focus on the evaluation phase of a broader work). Each of the submissions will be reviewed by the members of the Program Committee, and the proceedings volume will be submitted for publication to CEUR Workshop Proceedings. Due to CEUR-WS.org policies, only full and short papers will be submitted for publication, not the extended abstracts.


Organisers

Luca Benedetto (1), Andrew Caines (1), George Dueñas (2), Diana Galvan-Sosa (1), Gabrielle Gaudeau (1), Anastassia Loukina (3), Shiva Taslimipoor (1), Torsten Zesch (4)

(1) ALTA Institute, Dept. of Computer Science and Technology, University of Cambridge

(2) National Pedagogical University, Colombia

(3) Grammarly, Inc.

(4) FernUniversität in Hagen