A Top stats errors to look out for when reading, writing and reviewing papers
Poor statistical practice can distort scientific findings and mislead readers. This presentation reviews common statistical problems that readers, authors and reviewers of journals articles should be aware of when assessing research. These include p-hacking, multiple comparisons issues, underpowered studies, correlation/causation confusion, and inappropriate statistical procedures. I discuss the methodological flaws and reasoning behind these errors, as well as ways researchers fall prey to invalid practices. Recommendations are made for identifying questionable research practices in publications. Learn to recognize what makes for credible versus misleading uses of statistics in scientific literature.
Some useful references:
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European journal of epidemiology, 31(4), 337-350.
‘Our goal is to provide a resource for instructors, researchers, and consumers of statistics whose knowledge of statistical theory and technique may be limited but who wish to avoid and spot misinterpretations.Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values. The American Statistician, 70(2), 129-133.
‘Let us be clear. Nothing in the ASA statement is new. Statisticians and others have been sounding the alarm about these matters for decades, to little avail. We hoped that a statement from the world's largest professional association of statisticians would open a fresh discussion and draw renewed and vigorous attention to changing the practice of science with regards to the use of statistical inference.’Andrade C. HARKing, cherry-picking, P-hacking, fishing expeditions, and data dredging and mining as questionable research practices. J Clin Psychiatry. 2021;82(1):20f13804. Sun, Guo-Wen, Thomas L. Shook, and Gregory L. Kay. "Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis." Journal of clinical epidemiology 49.8 (1996): 907-916.
‘The use of bivariable selection [a.k.a. univariate analysis] for selecting variables to be used in multivariable analysis is inappropriate despite its common usage in medical sciences’. – 951 citationsKent, P., Cancelliere, C., Boyle, E. et al. A conceptual framework for prognostic research. BMC Med Res Methodol 20, 172 (2020).
This paper has a great overview of variables selection methods to use instead of univariate/multivariable, including how to deal with variable selection uncertainty and a discussion on post selection inference. Gary S. Collins, Johannes B. Reitsma, Douglas G. Altman, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement. Ann Intern Med. 2015;162:55-63.
The talk will be about 40 minutes long including discussion.
Date: Thursday 18 April 2024
Time: 2.30-3.30pm
Speaker: Gordana Popovic, Statistical Consultant, UNSW Stats Central
Delivery Mode: Online
Slides