HUD Colloquium Fall 2024 Presents: Dr. Chia Yi Chiu

The Human Development Colloquium Series Presents:

New Development in Cognitive Diagnosis: Nonparametric Approach for Multiple

Choice Items

Dr. Chia Yi Chiu

Associate Professor of Applied Statistics

Department of Human Development

Teachers College, Columbia University

Bio: While research on “big data” in educational measurement is thriving, Professor Chiu focused on “small data.” She is well recognized for her award-winning research on the advancement of educational assessments for the classroom that permit monitoring teaching and learning in real-time. Dr. Chiu has made groundbreaking contributions to the field of nonparametric cognitive diagnosis in developing its theory and implementing innovative algorithms for STEM assessments.

Abstract: The multiple-choice (MC) item format has been widely used in educational assessments across diverse content domains. MC items purportedly allow for collecting richer diagnostic information. The effectiveness and economy of administering MC items may have further contributed to their popularity not just in educational assessment. The MC item format has also been adapted to the cognitive diagnosis (CD) framework. Early approaches simply dichotomized the responses and analyzed them with a CD model for binary responses. Obviously, this strategy cannot exploit the additional diagnostic information provided by MC items. De la Torre’s MC Deterministic Inputs, Noisy “And” Gate (MC-DINA) model was the first for the explicit analysis of items having MC response format. However, as a drawback, the attribute vectors of the distractors are restricted to be nested within the key and each other. The method presented in this article for the CD of DINA items having MC response format does not require such constraints. Another contribution of the proposed method concerns its implementation using a nonparametric classification algorithm, which predestines it for use especially in small-sample settings like classrooms, where CD is most needed for monitoring instruction and student learning. In contrast, default parametric CD estimation routines that rely on EM- or MCMC-based algorithms cannot guarantee stable and reliable estimates—despite their effectiveness and efficiency when samples are large—due to computational feasibility issues caused by insufficient sample sizes. Results of simulation studies and a real-world application are also reported.