Using a Deep Recommender System for Predicting Test Item Difficulty with Content Coding

This study introduces a novel approach to item pre-calibration using item content coding. The current method associates the metadata gathered from the items —content code and examinee information — to predict item difficulty with high-performance accuracy. We demonstrated a novel application of a deep collaborative filtering system in the pre-calibration process. This study has the potential to contribute to the methodology and the current practices of test development by providing the benefits of item content coding combined with data-informed and machine-learning approaches for estimating item difficulty. The approach we illustrate helps overcome the costly process of item field testing and can be used to calibrate large numbers of items in both an efficient and cost-effective manner.

Creating short forms of psychological scales

In educational and psychological testing, lengthy instruments with many items may not be desirable as completing such instruments is time-consuming and cognitively demanding for test takers. Furthermore, in long self-report measures, participants may be unwilling to answer the items or show careless and insufficient effort responding. To avoid these issues, researchers have proposed various scale abbreviation methods that build short forms of long instruments (mostly psychological scales) by retaining the best items representing the target construct. Some of these methods include stepwise confirmatory factor analytical approach, Ant Colony Optimization, and Genetic Algorithm.

Considerations in Digitally-based Assessment Environment to Monitor Examinee’s Engagement and Learning Behaviours

We combined multiple advanced computational methods, including social network analysis and deep neural networks models. Our framework also models the examinee’s task-engagement status for a more accurate representation of the performance and skill demonstration in the series of interactive tasks.