Methods & Statistics
Faculty of Social Sciences
On March 31st 2021 Frank Bais has defended his thesis Constructing Behaviour Profiles for Answer Behaviour Across Surveys at the Utrecht University.
People may give answers to survey questions that deviate from the truth for several reasons.
We name such answers ‘undesirable answer behaviour’. The extent to which responses
deviate from the ‘true’ value is referred to as ‘measurement error’. Our purpose was to detect
respondent characteristics that are structurally related to measurement error. We selected
several answer behaviours (for instance answering ‘don’t know’) and respondent
characteristics (for instance age) to be investigated across ten population surveys from
CentERdata and Statistics Netherlands.
We introduced and explored our method of constructing behaviour profiles that summarize
and visualize answer behaviour. Such profiles can be constructed for groups of respondents
(the ‘respondent profile’) or groups of items (the ‘item profile’). We showed that a respondent
profile becomes more informative and precise as each respondent fills out more items. The
respondent profile becomes more stable and certain as the group consists of a larger number
of respondents. When a respondent profile is both precise and certain, the respondent profile
Subsequently, we examined the relation between respondent characteristics and undesirable
answer behaviour across surveys. We constructed respondent profiles and an adaptation of the
effect size Cliff’s Delta for the analyses. We did not find consistent relations between any
respondent characteristic and any answer behaviour. To examine the association between item
characteristics and measurement error in follow-up research, we investigated to what degree
item characteristics could be determined reliably. Intercoder reliability appeared to be low for
various item characteristics.
Prof. dr. ir. J.G. Schouten, dr. V. Toepoel
1 January 2014 – 31 March 2021