Methodology and Statistics
Social and Behavioural Sciences
Content Validity of High-Dimensional Measurement
“Incidental data” from sources like smartphones, social media and search queries are abundant. They are collected continually and record human behaviour in natural environments. Therefore, such data can be ideal for measuring social phenomena. In fact, some high-impact studies have made successful predictions of variables like human values and personality from incidental data.
However, when the research goal is not to predict but to explain (i.e. test a theory or estimate a causal model), which is often the case in the social sciences, at least two major challenges arise with the use of incidental data. First, in explanatory modelling, constructed scores (of latent constructs) must not only correlate but also be reliable and valid. Unfortunately, incidental data tend to suffer from measurement error problems because, by definition, they are not collected for the purpose of scientific research. Therefore, methods like latent variable measurement models are needed to estimate and correct for measurement error in incidental data. Second, in incidental data, indicators (e.g. social media posts and clickstreams) are often high-dimensional and most have low relevance to the target concepts. This raises the problem of high-dimensional measurement, which existing latent variable measurement models cannot deal with.
The goal of my PhD project is thus to improve measurements of theoretical (latent) constructs (like human values and personalities) based on high-dimensional incidental data, with a focus on content validity. Specifically, my project entails: 1) developing a variable selection procedure for high-dimensional incidental data by leveraging knowledge from machine learning and computational linguistics; 2) developing a method for valid cross country comparisons with incidental data; 3) developing user-friendly R packages and JASP modules for applied social scientists to use the developed methods; 4) applying the developed methods to study human values in Germany and the Netherlands.
Dr. Daniel Oberski; Dr. Dong Nguyen
NWO Talent Programme Vidi
1 June 2020 – 31 May 2024