|
Sign In to gain access to subscriptions and/or personal tools.
|
Practical Significance: A Concept Whose Time Has Come
Roger E. Kirk
Baylor University
Statistical significance is concerned with whether a research result is due to chance or sampling variability; practical significance is concerned with whether the result is useful in the real world. A growing awareness of the limitations of null hypothesis significance tests has led to a search for ways to supplement these procedures. A variety of supplementary measures of effect magnitude have been proposed. The use of these procedures in four APA journals is examined, and an approach to assessing the practical significance of data is described.
Educational and Psychological Measurement, Vol. 56, No. 5,
746-759 (1996)
DOI: 10.1177/0013164496056005002
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
M. Macedoni-Luksic, L. Greiss-Hess, S. J. Rogers, D. Gosar, K. Lemons-Chitwood, and R. Hagerman
Imitation in fragile X syndrome: Implications for autism
Autism,
November 1, 2009;
13(6):
599 - 611.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. Harrison, B. Thompson, and K. J. Vannest
Interpreting the Evidence for Effective Interventions to Increase the Academic Performance of Students With ADHD: Relevance of the Statistical Significance Controversy
Review of Educational Research,
June 1, 2009;
79(2):
740 - 775.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. F. Bordley
The Hippocratic Oath, Effect Size, and Utility Theory
Med Decis Making,
May 1, 2009;
29(3):
377 - 379.
[Abstract]
[PDF]
|
|
|
|
|
|
|
F. N.-A. Alhija and A. Levy
Effect Size Reporting Practices in Published Articles
Educational and Psychological Measurement,
April 1, 2009;
69(2):
245 - 265.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Gerhart
Cross Cultural Management Research: Assumptions, Evidence, and Suggested Directions
International Journal of Cross Cultural Management,
December 1, 2008;
8(3):
259 - 274.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R. Manolov and A. Solanas
Comparing N = 1 Effect Size Indices in Presence of Autocorrelation
Behav Modif,
November 1, 2008;
32(6):
860 - 875.
[Abstract]
[PDF]
|
|
|
|
|
|
|
A. Sargeant, J. B. Ford, and J. Hudson
Charity Brand Personality: The Relationship With Giving Behavior
Nonprofit and Voluntary Sector Quarterly,
September 1, 2008;
37(3):
468 - 491.
[Abstract]
[PDF]
|
|
|
|
|
|
|
A. Somech
Managing Conflict in School Teams: The Impact of Task and Goal Interdependence on Conflict Management and Team Effectiveness
Educational Administration Quarterly,
August 1, 2008;
44(3):
359 - 390.
[Abstract]
[PDF]
|
|
|
|
|
|
|
L. R. Zientek, M. M. Capraro, and R. M. Capraro
Reporting Practices in Quantitative Teacher Education Research: One Look at the Evidence Cited in the AERA Panel Report
Educational Researcher,
May 1, 2008;
37(4):
208 - 216.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. R. Gilpin
requivalent, Meta-Analysis, and Robustness: An Empirical Examination of Rosenthal and Rubin's Effect Size Indicator
Educational and Psychological Measurement,
February 1, 2008;
68(1):
42 - 57.
[Abstract]
[PDF]
|
|
|
|
|
|
|
P. Gendreau and P. Smith
Influencing the "People Who Count": Some Perspectives on the Reporting of Meta-Analytic Results for Prediction and Treatment Outcomes With Offenders
Criminal Justice and Behavior,
December 1, 2007;
34(12):
1536 - 1559.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. K. Byrd
A Call for Statistical Reform in EAQ
Educational Administration Quarterly,
August 1, 2007;
43(3):
381 - 391.
[Abstract]
[PDF]
|
|
|
|
|
|
|
X. D. Zhang
A New Method with Flexible and Balanced Control of False Negatives and False Positives for Hit Selection in RNA Interference High-Throughput Screening Assays
J Biomol Screen,
August 1, 2007;
12(5):
645 - 655.
[Abstract]
[PDF]
|
|
|
|
|
|
|
X. D. Zhang, M. Ferrer, A. S. Espeseth, S. D. Marine, E. M. Stec, M. A. Crackower, D. J. Holder, J. F. Heyse, and B. Strulovici
The Use of Strictly Standardized Mean Difference for Hit Selection in Primary RNA Interference High-Throughput Screening Experiments
J Biomol Screen,
June 1, 2007;
12(4):
497 - 509.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Snook, J. Eastwood, P. Gendreau, C. Goggin, and R. M. Cullen
Taking Stock of Criminal Profiling: A Narrative Review and Meta-Analysis
Criminal Justice and Behavior,
April 1, 2007;
34(4):
437 - 453.
[Abstract]
[PDF]
|
|
|
|
|
|
|
W. Viechtbauer
Approximate Confidence Intervals for Standardized Effect Sizes in the Two-Independent and Two-Dependent Samples Design
Journal of Educational and Behavioral Statistics,
March 1, 2007;
32(1):
39 - 60.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. O. Monahan, C. A. McHorney, T. E. Stump, and A. J. Perkins
Odds Ratio, Delta, ETS Classification, and Standardization Measures of DIF Magnitude for Binary Logistic Regression
Journal of Educational and Behavioral Statistics,
March 1, 2007;
32(1):
92 - 109.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
Z. N. Kain and J. MacLaren
P Less Than .05: What Does It Really Mean?
Pediatrics,
March 1, 2007;
119(3):
608 - 610.
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. Natesan and B. Thompson
Extending Improvement-Over-Chance I-Index Effect Size Simulation Studies to Cover Some Small-Sample Cases
Educational and Psychological Measurement,
February 1, 2007;
67(1):
59 - 72.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R. I. Parker, S. Hagan-Burke, and K. Vannest
Percentage of All Non-Overlapping Data (PAND): An Alternative to PND
Journal of Special Education,
February 1, 2007;
40(4):
194 - 204.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. J. Lovett and L. J. Lewandowski
Gifted Students With Learning Disabilities: Who Are They?
J Learn Disabil,
December 1, 2006;
39(6):
515 - 527.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R. K. Henson
Effect-Size Measures and Meta-Analytic Thinking in Counseling Psychology Research
The Counseling Psychologist,
September 1, 2006;
34(5):
601 - 629.
[Abstract]
[PDF]
|
|
|
|
|
|
|
E. Kienzle and R. Bergler
Human-Animal Relationship of Owners of Normal and Overweight Cats
J. Nutr.,
July 1, 2006;
136(7):
1947S - 1950S.
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. Somech
The Effects of Leadership Style and Team Process on Performance and Innovation in Functionally Heterogeneous Teams
Journal of Management,
February 1, 2006;
32(1):
132 - 157.
[Abstract]
[PDF]
|
|
|
|
|
|
|
E. Kristjansson, R. Aylesworth, I. Mcdowell, and B. D. Zumbo
A Comparison of Four Methods for Detecting Differential Item Functioning in Ordered Response Items
Educational and Psychological Measurement,
December 1, 2005;
65(6):
935 - 953.
[Abstract]
[PDF]
|
|
|
|
|
|
|
Z. N. Kain
The Legend of the P Value
Anesth. Analg.,
November 1, 2005;
101(5):
1454 - 1456.
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. Wood
Bootstrapped Confidence Intervals as an Approach to Statistical Inference
Organizational Research Methods,
October 1, 2005;
8(4):
454 - 470.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. S. Lum, Y. Zhurov, E. C. Cropper, K. R. Weiss, and V. Brezina
Variability of Swallowing Performance in Intact, Freely Feeding Aplysia
J Neurophysiol,
October 1, 2005;
94(4):
2427 - 2446.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
T. C. Campbell
An Introduction to Clinical Significance: An Alternative Index of Intervention Effect for Group Experimental Designs
Journal of Early Intervention,
April 1, 2005;
27(3):
210 - 227.
[Abstract]
[PDF]
|
|
|
|
|
|
|
D. Gijbels, F. Dochy, P. Van den Bossche, and M. Segers
Effects of Problem-Based Learning: A Meta-Analysis From the Angle of Assessment
Review of Educational Research,
January 1, 2005;
75(1):
27 - 61.
[Abstract]
[PDF]
|
|
|
|
|
|
|
M. D. Hidalgo and J. A. LOPez-Pina
Differential Item Functioning Detection and Effect Size: A Comparison between Logistic Regression and Mantel-Haenszel Procedures
Educational and Psychological Measurement,
December 1, 2004;
64(6):
903 - 915.
[Abstract]
[PDF]
|
|
|
|
|
|
|
S. Nakagawa
A farewell to Bonferroni: the problems of low statistical power and publication bias
Behav. Ecol.,
November 1, 2004;
15(6):
1044 - 1045.
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. Hubbard
Alphabet Soup: Blurring the Distinctions Betweenp's anda's in Psychological Research
Theory Psychology,
June 1, 2004;
14(3):
295 - 327.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Ives
Effect Size Use in Studies of Learning Disabilities
J Learn Disabil,
December 1, 2003;
36(6):
490 - 504.
[Abstract]
[PDF]
|
|
|
|
|
|
|
M. M. Capraro and R. M. Capraro
Exploring the APA Fifth Edition Publication Manual's Impact on the Analytic Preferences of Journal Editorial Board Members
Educational and Psychological Measurement,
August 1, 2003;
63(4):
554 - 565.
[Abstract]
[PDF]
|
|
|
|
|
|
|
G. Fulcher and R. M. Reiter
Task difficulty in speaking tests
Language Testing,
July 1, 2003;
20(3):
321 - 344.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. A. Breaugh
Effect Size Estimation: Factors to Consider and Mistakes to Avoid
Journal of Management,
February 1, 2003;
29(1):
79 - 97.
[Abstract]
[PDF]
|
|
|
|
|
|
|
S. Finch, N. Thomason, and G. Cumming
Past and Future American Psychological Association Guidelines for Statistical Practice
Theory Psychology,
December 1, 2002;
12(6):
825 - 853.
[Abstract]
[PDF]
|
|
|
|
|
|
|
F. Fidler
The Fifth edition of the Apa Publication Manual: Why its Statistics Recommendations are so Controversial
Educational and Psychological Measurement,
October 1, 2002;
62(5):
749 - 770.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R. M. Capraro and M. M. Capraro
Treatments of Effect Sizes and Statistical Significance Tests in Textbooks
Educational and Psychological Measurement,
October 1, 2002;
62(5):
771 - 782.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Thompson
What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes
Educational Researcher,
April 1, 2002;
31(3):
25 - 32.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. J Huberty
A History of Effect Size Indices
Educational and Psychological Measurement,
April 1, 2002;
62(2):
227 - 240.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. K. Roberts and R. K. Henson
Correction for Bias in Estimating Effect Sizes
Educational and Psychological Measurement,
April 1, 2002;
62(2):
241 - 253.
[Abstract]
[PDF]
|
|
|
|
|
|
|
F. Baugh
Correcting Effect Sizes for Score Reliability: A Reminder that Measurement and Substantive Issues are Linked Inextricably
Educational and Psychological Measurement,
April 1, 2002;
62(2):
254 - 263.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Hess, S. Olejnik, and C. J Huberty
The Efficacy of two Improvement-Over-Chance Effect Sizes for Two-Group Univariate Comparisons under Variance Heterogeneity and Nonnormality
Educational and Psychological Measurement,
December 1, 2001;
61(6):
909 - 936.
[Abstract]
[PDF]
|
|
|
|
|
|
|
X. Fan and B. Thompson
Confidence Intervals for Effect Sizes: Confidence Intervals about Score Reliability Coefficients, Please: An EPM Guidelines Editorial
Educational and Psychological Measurement,
August 1, 2001;
61(4):
517 - 531.
[Abstract]
[PDF]
|
|
|
|
|
|
|
F. Fidler and B. Thompson
Computing Correct Confidence Intervals for Anova Fixed-and Random-Effects Effect Sizes
Educational and Psychological Measurement,
August 1, 2001;
61(4):
575 - 604.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. M. Graham
Book Review
Educational and Psychological Measurement,
August 1, 2001;
61(4):
668 - 674.
[Abstract]
[PDF]
|
|
|
|
|
|
|
S. Finch, G. Cumming, and N. Thomason
Colloquium on Effect Sizes: the Roles of Editors, Textbook Authors, and the Publication Manual: Reporting of Statistical Inference in the Journal of Applied Psychology: Little Evidence of Reform
Educational and Psychological Measurement,
April 1, 2001;
61(2):
181 - 210.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Thompson
Editor's Note on the "Colloquium on Effect Sizes: the Roles of Editors, Textbook Authors, and the Publication Manual"
Educational and Psychological Measurement,
April 1, 2001;
61(2):
211 - 212.
[PDF]
|
|
|
|
|
|
|
R. E. Kirk
Promoting Good Statistical Practices: Some Suggestions
Educational and Psychological Measurement,
April 1, 2001;
61(2):
213 - 218.
[PDF]
|
|
|
|
|
|
|
T. Vacha-Haase
Statistical Significance should not be Considered one of Life's Guarantees: Effect Sizes are Needed
Educational and Psychological Measurement,
April 1, 2001;
61(2):
219 - 224.
[PDF]
|
|
|
|
|
|
|
B. T. Gray
A Factor Analytic Study of the Substance Abuse Subtle Screening Inventory (SASSI)
Educational and Psychological Measurement,
February 1, 2001;
61(1):
102 - 118.
[Abstract]
[PDF]
|
|
|
|
|
|
|
M. Kim
Detecting DIF across the different language groups in a speaking test
Language Testing,
January 1, 2001;
18(1):
89 - 114.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R. Hubbard and P. A. Ryan
Statistical Significance with Comments by Editors of Marketing Journals: The Historical Growth of Statistical Significance Testing in Psychology--and its Future Prospects
Educational and Psychological Measurement,
October 1, 2000;
60(5):
661 - 681.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. J Huberty and L. L. Lowman
Group Overlap as a Basis for Effect Size
Educational and Psychological Measurement,
August 1, 2000;
60(4):
543 - 563.
[Abstract]
[PDF]
|
|
|
|
|
|
|
T. Vacha-Haase, J. E. Nilsson, D. R. Reetz, T. S. Lance, and B. Thompson
Reporting Practices and APA Editorial Policies Regarding Statistical Significance and Effect Size
Theory Psychology,
June 1, 2000;
10(3):
413 - 425.
[Abstract]
[PDF]
|
|
|
|
|
|
|
K. C. Mittag and B. Thompson
Research news and Comment: A National Survey of AERA Members' Perceptions of Statistical Significance Tests and Other Statistical Issues
Educational Researcher,
May 1, 2000;
29(4):
14 - 20.
[PDF]
|
|
|
|
|
|
|
P. Snyder
Guidelines for Reporting Results of Group Quantitative Investigations
Journal of Early Intervention,
January 1, 2000;
23(3):
145 - 150.
[PDF]
|
|
|
|
|
|
|
C. E. Lance and L. R. James
N2: A Proportional Variance-Accounted for Index for Some Cross-Level and Person-Situation Research Designs
Organizational Research Methods,
October 1, 1999;
2(4):
395 - 418.
[Abstract]
[PDF]
|
|
|
|
|
|
|
E. F. Krieg Jr.
Biases Induced by Coarse Measurement Scales
Educational and Psychological Measurement,
October 1, 1999;
59(5):
749 - 766.
[Abstract]
[PDF]
|
|
|
|
|
|
|
M. C. Sturman
Multiple Approaches to Analyzing Count Data in Studies of Individual Differences: The Propensity for Type I Errors, Illustrated with the Case of Absenteeism Prediction
Educational and Psychological Measurement,
June 1, 1999;
59(3):
414 - 430.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Thompson
If Statistical Significance Tests are Broken/Misused, What Practices Should Supplement or Replace Them?
Theory Psychology,
April 1, 1999;
9(2):
165 - 181.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Thompson
Statistical Significance Tests, Effect Size Reporting and the Vain Pursuit of Pseudo-Objectivity
Theory Psychology,
April 1, 1999;
9(2):
191 - 196.
[Abstract]
[PDF]
|
|
|
|
|
|
|
K. M. Kieffer
Book Review
Educational and Psychological Measurement,
August 1, 1998;
58(4):
687 - 693.
|
|
|
|
|
|
|
J. T. Austin, K. A. Boyle, and J. C. Lualhati
Statistical Conclusion Validity for Organizational Science Researchers: A Review
Organizational Research Methods,
April 1, 1998;
1(2):
164 - 208.
[Abstract]
|
|
|
|
|
|
|
J. Sanchez-Meca and F. Marin-Martinez
Weighting by Inverse Variance or by Sample Size in Meta-Analysis: A Simulation Study
Educational and Psychological Measurement,
April 1, 1998;
58(2):
211 - 220.
[Abstract]
|
|
|
|
|
|
|
B. Thompson
Book Reviews: Review Of What if there were No Significance Tests?
Educational and Psychological Measurement,
April 1, 1998;
58(2):
334 - 346.
[Abstract]
|
|
|
|
|
|
|
D. T. Morse
MINSIZE: A Computer Program for Obtaining Minimum Sample Size as an Indicator of Effect Size
Educational and Psychological Measurement,
February 1, 1998;
58(1):
142 - 153.
[Abstract]
|
|
|
|
|
|
|
H. J. Keselman, C. J. Huberty, L. M. Lix, S. Olejnik, R. A. Cribbie, B. Donahue, R. K. Kowalchuk, L. L. Lowman, M. D. Petoskey, J. C. Keselman, et al.
Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses
Review of Educational Research,
January 1, 1998;
68(3):
350 - 386.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Thompson
Rejoinder: Editorial Policies Regarding Statistical Significance Tests: Further Comments
Educational Researcher,
June 1, 1997;
26(5):
29 - 32.
[Abstract]
[PDF]
|
|
|
|
|
