Using a Deep Recommender System for Predicting Test Item Difficulty with Content Coding

This study introduces a novel approach to item pre-calibration using item content coding. The current method associates the metadata gathered from the items —content code and examinee information — to predict item difficulty with high-performance accuracy. We demonstrated a novel application of a deep collaborative filtering system in the pre-calibration process. This study has the potential to contribute to the methodology and the current practices of test development by providing the benefits of item content coding combined with data-informed and machine-learning approaches for estimating item difficulty. The approach we illustrate helps overcome the costly process of item field testing and can be used to calibrate large numbers of items in both an efficient and cost-effective manner.

January 1, 2023

12:00 AM

Chicago, IL

AERA 2023