Psyhcometrics and Statistics
Faculty of Behavioural and Social Sciences
University of Groningen

Academic webpage Jasmine Muradchanian

Project

Improving replicability in science

This research project will be in the field of methodology and statistics. It is concerned with replication of research findings. It is motivated by the fact that large parts of the empirical literature in the social sciences seem untrustworthy (e.g., Open Science Collaboration, 2015; Camerer et al., 2018). A major culprit in this crisis of confidence is the over-reliance on inappropriate statistical inference tools (e.g., Hoekstra, Finch, Kiers & Johnson, 2006; van Ravenzwaaij & Ioannidis, 2017). Conventional statistical inference often makes use of a p-value as a vessel to quantify evidence against the null hypothesis. Unfortunately, just using a p-value to decide whether or not there is an effect will easily lead to overly strong conclusions, thus giving rise to a literature riddled with spurious research findings. However, studies differ in likelihood of having found a spurious result.

The aim of this project is to obtain more insight into the various replication success metrics. In order to achieve this, it will be studied how success of replication is defined in practice. Afterwards, traditional and state-of-the-art methods (and, possibly, extensions thereof) for predicting and quantifying replication success will be compared. The usefulness of such studies is that their results will provide more insight into the extent to which the outcomes of replication studies are in line with the underlying data structure. By using such tools to re-evaluate existing statistical evidence across scientific disciplines (including psychology and educational sciences), we can identify those (replication) studies for which reported statistical evidence is inconclusive or weak, and for which claims of the existence of an effect might have been premature. In the present project, these methods will be assessed and evaluated in the context of combining the practical and theoretical literature regarding replications.

Supervisors
Prof. dr. H. Kiers, dr. D. van Ravenzwaaij, dr. R. Hoekstra

Financed by
University of Groningen

Period

1 May 2019 – 1 May 2023