Project
Winner takes all: Applying MASEM with raw data”, part of the NWO VIDI research project “No data left behind. New meta-analytic structural equation models for complex data structures.

An increasingly popular meta-analytic approach is to synthesize the raw datasets instead of summary statistics. This technique goes by various names, such as meta-analysis with raw data, individual participant/patient data meta-analysis, integrative data analysis, and synthetic secondary data analysis (Curran et al., 2014; Riley et al., 2008; Shrout, 2009). The meta-analysis of raw datasets offers great promise, such as:
Evaluation of individual-level moderating effects, avoiding the ecological fallacy (Cooper & Patall, 2009)
✓ Flexibility analyzing non-normal data (such as ordinal data) ✓ Being more statistically efficient than using summary data in some cases (Chen et al., 2020)
✓ No reliance on the accuracy of reported analyses and statistics, which are often incorrect (Nuijten et al., 2015)

Simmonds et al. (2015) argue that the field of raw data meta-analysis would benefit from more random effects models, and more advanced statistical models. Project 3 combines the latest developments from the field of meta-analysis with raw data with advanced techniques from the field of (multilevel) SEM.
Currently, MASEM with raw data is mostly performed by fitting the SEM to the individual datasets, and aggregating the parameter estimates of interest (e.g., Gnambs & Staufenbiel, 2016). However, there are important limitations to this approach (Cheung & Cheung, 2016), such as the need for complete data in each study to be able to fit the same SEM in each study.
A better approach is using multilevel SEM. This is an advanced technique with which all datasets are analyzed simultaneously, and differences between studies are taken into account by allowing random effects over studies. Some advantages of using multilevel SEM are that studies with incomplete data can be included in the meta-analysis, one can use common fit indices from multilevel SEM to evaluate fit and compare models, and it makes flexible tests of measurement invariance possible (Curran et al., 2014).
A challenge of meta-analysis with raw data is the availability of raw datasets. With the open science movement and widespread adoption of data-sharing policies by journals and funding agencies, more and more raw datasets are becoming available. For example, open access to research data is actively promoted by the European Research Council, the Dutch funding institute NWO, and the National Science Foundation in the USA.
Therefore, although MASEM on summary statistics is the standard at this moment and in the coming years, in the future there will be many more raw datasets available to answer meta-analytic research questions. In this project, I will develop and evaluate new methods to apply MASEM to raw datasets, so that researchers can analyze all available data. That is, this project will make MASEM future-proof.

Supervisors
Dr. S.J. Jak
Prof. dr. F.J. Oort
Dr. K.J. Kan

Financed by
NWO (VIDI grant)

Period
January 2022 – January 2026