Prof. Herbert Hoijtink & Dr Rens van de Schoot
How to hedge our bets in educational testing: combining test results with teacher expertise
In the Netherlands, educational testing is heatedly debated. In specific, there is disagreement whether to use objective test results or expert knowledge (i.e., teacher expertise) for many school-track related decisions. To prevent placing one’s “bet” on either test results or teacher expertise, we propose hedging our bets using a Bayesian methodology that based judgements on the ability of children on both teacher expertise and test results. To make this possible, the following issues are investigated in my PhD:
- Experts (i.e., teachers, educational practitioners, test takers, et cetera) need to be able to express their knowledge in such a way that it can be used in statistical modelling (i.e., expert elicitation);
- The quality of test results and expert knowledge has to be evaluated to avoid inaccuracy and/or bias;
- Test results and expert knowledge need to be combined taking the relative quality of both sources into account;
- Possible conflicts between expert knowledge and test results need to be checked and the cause for these conflicts need to be evaluated.
The CITO-LVS and WISC-III IQ-test are used as case studies to provide a strong test and potential demonstration of the advantages and pitfalls of the Bayesian method in educational testing.
NWO talent grant (406-15-062)
1 September 2015 – 4 September 2020