Project
Model Complexity in Psychology

Current psychology as a science is facing two crises at the same time. The first crisis is a crisis of theory (Eronen & Bringmann, 2021), a situation in which a lack of theory has led researchers to rely strongly on statistical models (Borsboom, 2013; Eronen & Bringmann, 2021) instead of conceptualization, improving measurement, and building new, sound theories (Bringmann et al., 2022; Fried et al., 2022). The strong reliance on statistical models directly relates to the second crisis psychology is currently facing: A crisis of confidence (Pashler & Wagenmakers, 2012). A situation in which multiple fraud cases came to light, research findings did not replicate, and the widespread use of questionable research practices affecting results of studies was uncovered (Pashler & Wagenmakers, 2012).
Both of these crises have in common that they directly relate to the modeling of psychological phenomena. The crisis of theory relates to informing researchers how we should model phenomena, whereas the crisis of confidence relates to how we should test and analyze the models we constructed based on theory. On one hand, it is reasonable to assume that theoretical relations between psychological constructs are complex, for an example see the network theory of mental disorders (Borsboom, 2017). On the other hand, often the prerequisites to test more complex models are not given and studies suffer insufficient power (Sommet et al., 2022), insufficient measurements (Bringmann et al., 2022; Flake & Fried, 2020; Fried et al., 2022), or malpractice in the application of statistical tests such as no corrections for multiple testing (Cramer et al., 2016). This went so far that Murphy and Russel (2017) even suggested to not test for moderations at all. The problem is that testing theories is an important part of forming and adjusting theories and thus these concurrent crises can lead or even may have led psychology into a vicious cycle.
Despite theoretical arguments to keep research designs and analysis strategies simple, the reality is that psychological researchers tend to favor relatively complex design choices. This proposal aims to assess the usage and usefulness of complex designs and analysis strategies over time through a literature study. What effect does the use of complex designs have on the statistical effects and interpretation of the results? Through simulations the consequences of using complex designs are investigated. By reanalyzing original study data using simpler designs the implications for scientific conclusions are explored. Finally, through a survey study, the reasons behind employing complex designs are examined.

Supervisors
Prof. dr. D. van Ravenzwaaij
Prof. dr. H.A.L. Kiers

Financed by
RUG

Period
February 2023 – February 2027