Methodology and Statistics
Social and Behavioural Sciences
Utrecht University

Academic webpage

Project 
Content Validity of High-Dimensional Measurement

“Incidental data” from sources like smartphones, social media and search queries are  abundant. They are collected continually and record human behaviour in natural  environments. Therefore, such data can be ideal for measuring social phenomena. In fact,  some high-impact studies have made successful predictions of variables like human values  and personality from incidental data.  

However, when the research goal is not to predict but to explain (i.e. test a theory or  estimate a causal model), which is often the case in the social sciences, at least two major  challenges arise with the use of incidental data. First, in explanatory modelling, constructed  scores (of latent constructs) must not only correlate but also be reliable and valid.  Unfortunately, incidental data tend to suffer from measurement error problems because, by  definition, they are not collected for the purpose of scientific research. Therefore,  methods like latent variable measurement models are needed to estimate and correct for  measurement error in incidental data. Second, in incidental data, indicators (e.g. social  media posts and clickstreams) are often high-dimensional and most have low relevance to  the target concepts. This raises the problem of high-dimensional measurement, which  existing latent variable measurement models cannot deal with. 

The goal of my PhD project is thus to improve measurements of theoretical (latent)  constructs (like human values and personalities) based on high-dimensional incidental data,  with a focus on content validity. Specifically, my project entails: 1) developing a variable  selection procedure for high-dimensional incidental data by leveraging knowledge from  machine learning and computational linguistics; 2) developing a method for valid cross country comparisons with incidental data; 3) developing user-friendly R packages and JASP  modules for applied social scientists to use the developed methods; 4) applying the  developed methods to study human values in Germany and the Netherlands.

Supervisors
Dr. Daniel Oberski; Dr. Dong Nguyen

Financed by
NWO Talent Programme Vidi

Period
1 June 2020 – 31 May 2024