Project
Good Research Practices in Psychology

In the past 20 years, several problems in the field of psychology have come to light. Many studies do not replicate (Open Science Collaboration, 2015), researchers admit to using Questionable Research Practices (Fiedler & Schwarz, 2015), and many studies are not analytically reproducible (Hardwicke et al., 2021; Hardwicke et al., 2018; Wicherts et al., 2011), to name a few. In response, there is more attention for Good Research Practices (GRPs) now. Some of these are newly developed, some were already established. This dissertation focuses on three GRPs in particular: data blinding, data visualization, and replication studies.

Data blinding is a method that involves temporarily hiding or changing certain parts of a dataset, so that an analysis plan can be made without making outcome-driven decisions. Condition labels can be changed or variables can be shuffled, for example. By not knowing the real outcome of the analysis, we prevent significance seeking. Once the analysis plan is finished, it can be executed on the real dataset. Although many data blinding methods already exist, they rarely get used within psychology research. Besides, an accessible yet comprehensive guide to data blinding is still lacking from the literature. Therefore, study 1 will aim to provide such a guide for researchers new to data blinding. In study 2, participating researchers will implement data blinding methods in their research projects and we will study their experience using qualitative methods. Our driving questions will be what motivates researchers to apply this new methodology, what the challenges are of applying data blinding methods, and how to resolve those challenges.

Data visualization is integral to communicating research findings and, on a more fundamental level, understanding research data (Börner et al., 2019). Visual processing is near-instantaneous whereas understanding numbers is a much slower process. Moreover, summary statistics alone do not paint a complete picture.  Two data sets with identical mean and (co)variance structures may contain distinct patterns that become apparent immediately once visualized (for examples see Franconeri et al., 2021).

It is unclear how often we use visualizations, what we visualize exactly, and how we visualize. This will be the focus of study 3. We will answer those two questions using two samples from different populations: a sample of empirical articles and a sample of scientific posters. The samples will also be compared. As a follow-up, study 4 will focus on the quality of data visualizations, and attempt to answer the question whether the data visualizations we use are potentially misleading to other researchers.

Replication studies are an essential part of the self-correcting mechanisms of science. Yet, historically the replication prevalence has been shockingly low in psychology (Makel et al., 2012). This examination took place at the start of the reproducibility revolution and is due a re-evaluation. Study 5 is a conceptual replication of Makel and colleagues (2012) and reexamines the replication prevalence in the period 2011-2020.

Supervisors
Prof. Dr. Elise M.L. Dusseldorp
Dr. Sjoerd M.H. Huisman

Financed by
The unit and Incentive Grants

Period
15 November 2025 – 14 November 2031