Methods and Statistics
Research Institute of Child Development and Education
Faculty of Social and Behavioural Sciences
University of Amsterdam

Email Letty Koopman
Academic webpage Letty Koopman

On 3 November 2022 Letty Koopman defended her thesis Nonparametric Item Response Theory for Multilevel Test Data, at the University of Amsterdam.

Summary
Psychological measurement aims at measuring a psychological construct (e.g., an attribute, skill, or ability) by means of observable variables such as items on a test or questionnaire. To relate item scores to the psychological construct, item response theory (IRT) models are used. Nonparametric IRT models fit test data relatively well compared to other IRT models. Various methods exist to evaluate the fit of nonparametric IRT models, also known as Mokken scale analysis (MSA).

In multilevel test data respondents are nested in groups, such as students in classrooms or employees in companies. A multilevel data structure can severely affect statistical analyses, possibly leading to the inclusion of items to the test that do not contribute to (or possibly negatively affect) accurate measurement, or the quality of the test may be overestimated. In addition, most methods in MSA provide results on the respondent level only, not on the group level. Hence, for multilevel test data, currently available MSA methods are only of limited value.

This thesis contributes to making MSA available for multilevel test data in the following ways. First, by discussing solutions to two computational problems in MSA. Second, by deriving, investigating, and optimizing point and interval estimates of two-level scalability coefficients and of Mokken’s scalability coefficients. Third, by introducing a test-guided automated item selection procedure. Finally, by proposing four two-level nonparametric IRT models, with their implied measurement and data properties. These developments contribute to knowledge on how tests and questionnaires can be investigated in the presence of multilevel test data.

Supervisors
Prof. L.A. van der Ark & Dr B.J.H. Zijlstra

Funder
NWO Research Talent Grant

Period
1 November 2016 – 3 November 2022