Methodology and Statistics
Tilburg School of Social and Behavioral Sciences
Tilburg University

Supervisors
Prof. K. Sijtsma & Dr L.A. Van der Ark

On July 1st 2016, Pieter Oosterwijk defended his thesis entitled

Statistical properties and practical use of classical test-score reliability methods

(Summary not available yet)

Project

Improving global and local reliability estimation in nonparametric item response theory
The goals of this project are twofold. First, investigate whether three methods from nonparametric item response theory for test-score reliability estimation are closer to the true reliability than other estimates,including Cronbach’s alpha and the greatest lower bound (GLB). Second, to propose a test information function in the context of nonparametric item response theory that expresses reliability as a function of the scale, this recognizing that measurement accuracy can vary across the scale of an attribute.
Some explanation of these goals is the following. Well-known reliability methods such as Cronbach’s alpha, the Guttman indices, and the GLB are known to be negatively biased relative to the reliability of the test score. Sijtsma and Molenaar found indications that for tests consisting of dichotonous items Mokken’s two reliability methods and their own reliability method were nearly unbiased with respect to reliability, and certainly much closer than Cronbach’s alpha and other methods. This project aims at providing more evidence for the small bias or perhaps the absence of bias for these three reliability methods and intends to generalize results to tests consisting of polytomous items.
The other aim of this project is to propose and investigate a test information function that allows for reliability assessment at different locations on the scale. The reliability coefficient is just one number, andis used for computing a standard measurement error and a confidence interval for each tested case, if is however feasible that for different location on a scale reliability of measurement also varies. A testinformation function would be a welcome addition to nonparametric item response theory, because it would further enhance the applicability of this flexible class of models for scale construction. Ramsay has provided some first attempts, which serve as point of departure in this project.

Financed by
Tilburg University