Project
Improving the Robustness of Science using Evidence-Based Journal Level Interventions

In the wake of the replication crisis, many different contributing causes to the lack of robustness in the findings of psychological science have been identified, such as a lack replication efforts (Nosek et al., 2022), publication bias (Franco, Malhotra, & Simonovits, 2014), researcher degrees of freedom (Wicherts et al., 2016), and lack of construct validity in measurement (Flake & Fried, 2020). In order to resolve these issues, numerous solutions, and initiatives have been started, such as the open science movement (Brinkman et al., 2023), new reporting guidelines (Nosek et al., 2015), and large scale replication initiatives (Ebersole et al., 2016, 2020; Klein et al., 2014, 2018, 2022).

However, uncertainty regarding the uptake and effectivity of many interventions remains. So far, evidence has suggested that still much is needed to improve in the implementation of these interventions (Hardwicke et al., 2018, 2022).

It is the goal of my PhD project to first of all, do a systematic review and meta-analysis of the implementation frequency and effectiveness of various interventions aimed to improve the robustness of science at the level of journals, since they are one of the key gatekeepers in scientific publishing (Maner, 2014). Examples of journal level interventions are Open Science Badges and new publication formats, such as Registered Reports. This overview will then be used to select one or a set of interventions and compare journals on indicators of robustness based on whether or not they implemented the intervention(s). In a final project, I intend to set up a randomized controlled trial to test an intervention at a psychological journal, and quantify its causal impact on improving scientific robustness.

This project goals are a good match with the aims and scope of IOPS. It has the potential to have a direct impact on the quality of psychological research, likely through improving aspects in methodology, statistical analyses, and/or reporting standards. I also think I would benefit greatly from the courses that IOPS offers. A thorough understanding of a variety of statistical methods used in psychology will help me, both in aiding the psychological community to improve their research, but also by ensuring my own research is of the highest methodological standards.

Supervisors
Prof. dr. J.M. Wicherts
dr. M.B. Nuijten

Period
1 September 2023 – 31 August 2027