This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Finch, S. Articles by Thomason, N. Search for Related Content Social Bookmarking                   What's this?

Colloquium on Effect Sizes: the Roles of Editors, Textbook Authors, and the Publication Manual

Reporting of Statistical Inference in the Journal of Applied Psychology: Little Evidence of Reform

Geoff Cumming

La Trobe Universityg.cumming{at}latrobe.edu.au

Neil Thomason

The University of Melbourne

Reformers have long argued that misuse of Null Hypothesis Significance Testing (NHST) is widespread and damaging. The authors analyzed 150 articles from the Journal of Applied Psychology (JAP) covering 1940 to 1999. They examined statistical reporting practices related to misconceptions about NHST, American Psychological Association (APA) guidelines, and reform recommendations. The analysis reveals (a) inconsistency in reporting alpha and p values, (b) the use of ambiguous language in describing NHST, (c) frequent acceptance of null hypotheses without consideration of power, (d) that power estimates are rarely reported, and (e) that confidence intervals were virtually never used. APA guidelines have been followed only selectively. Research methodology reported in JAP has increased greatly in sophistication over 60 years, but inference practices have shown remarkable stability. There is little sign that decades of cogent critiques by reformers had by 1999 led to changes in statistical reporting practices in JAP.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				L. R. Zientek, M. M. Capraro, and R. M. Capraro Reporting Practices in Quantitative Teacher Education Research: One Look at the Evidence Cited in the AERA Panel Report Educational Researcher, May 1, 2008; 37(4): 208 - 216. [Abstract] [Full Text] [PDF]

				J. K. Byrd A Call for Statistical Reform in EAQ Educational Administration Quarterly, August 1, 2007; 43(3): 381 - 391. [Abstract] [PDF]

				K.-H. Yuan and S. Maxwell On the Post Hoc Power in Testing Mean Differences Journal of Educational and Behavioral Statistics, January 1, 2005; 30(2): 141 - 167. [Abstract] [PDF]

				R. Hubbard Alphabet Soup: Blurring the Distinctions Betweenp's anda's in Psychological Research Theory Psychology, June 1, 2004; 14(3): 295 - 327. [Abstract] [PDF]

				M. M. Capraro and R. M. Capraro Exploring the APA Fifth Edition Publication Manual's Impact on the Analytic Preferences of Journal Editorial Board Members Educational and Psychological Measurement, August 1, 2003; 63(4): 554 - 565. [Abstract] [PDF]

				D. Weisburd, C. M. Lum, and S.-M. Yang When Can We Conclude That Treatments or Programs "Don't Work"? The ANNALS of the American Academy of Political and Social Science, May 1, 2003; 587(1): 31 - 48. [Abstract] [PDF]

				F. Fidler The Fifth edition of the Apa Publication Manual: Why its Statistics Recommendations are so Controversial Educational and Psychological Measurement, October 1, 2002; 62(5): 749 - 770. [Abstract] [PDF]

				B. Thompson What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes Educational Researcher, April 1, 2002; 31(3): 25 - 32. [Abstract] [PDF]

				K. D. Bird Confidence Intervals for Effect Sizes in Analysis of Variance Educational and Psychological Measurement, April 1, 2002; 62(2): 197 - 226. [Abstract] [PDF]

				G. Cumming and S. Finch A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions Educational and Psychological Measurement, August 1, 2001; 61(4): 532 - 574. [Abstract] [PDF]