Hacking, Picking, Harking and Co.: Lessons from the Replication Crisis in Psychology

The talk focuses on statistical aspects of the so-called 'replication crisis' that has concerned psychology in recent years. Specifically, the role of questionable research practices like p-hacking, cherry picking, and HARKing (Hypothesizing After Results are Known) is illustrated. Furthermore, I demonstrate that underpowered studies not only have a low likelihood of finding an existing effect, but also contribute to a higher rate of false positives across a large number of studies. Recently, Nelson, Simmons, and Simonsohn (2018) argued that rather than focusing on the problems uncovered by the systematic (and often unsuccessful) replications of psychological experiments in recent years, the emphasis should now be on the methodological improvements that emerged as a response to this crisis, including the preregistration of studies, the sharing of data and materials as well as full disclosure with respect to statistical analyses and variables.