A research paper, titled “Balancing Cost and Quality: An Exploration of Human-in-the-Loop Frameworks for Automated Short Answer Scoring” was published at the 23rd International Conference on Artificial Intelligence in Education (AIED2022)* on July 27, 2022.
This study aims to guarantee high-quality predictions of exploring the use of human-in-the-loop framework for minimizing the grading cost while guaranteeing the grading quality by allowing a Short answer scoring (SAS) model to share the grading task with a human grader.
The result shows that our human-in-the-loop framework allows automatic scoring models and human graders to achieve the target scoring quality.
*The conference sets the ambitious goal to stimulate discussion on how AI shapes and can shape education for all sectors, how to advance the science and engineering of intelligent interactive learning systems, and how to promote broad adoption.
The research team includes Hiroaki Funayama, Tasuku Sato*, Yuichiroh Matsubayashi, Tomoya, Mizumoto, Jun Suzuki, and Kentaro Inui(Team Leader) of the Natural Language Understanding Team at RIKEN AIP.
*He was a former member of the team at RIKEN AIP.
Title: Balancing Cost and Quality: An Exploration of Human-in-the-Loop Frameworks for Automated Short Answer Scoring
Authors: Hiroaki Funayama, Tasuku Sato, Yuichiroh Matsubayashi, Tomoya,Mizumoto, Jun Suzuki and Kentaro Inui
This research result was delivered at the AIED 2022 at Durham University, the UK on July 27, 2022.
For more information, please see the following website.
AIED2022
