This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Bray, J. H. Articles by Howard, G. S. Search for Related Content Social Bookmarking                         What's this?

Methods of Analysis with Response-Shift BIAS

Texas Woman's University

Scott E. Maxwell

University of Notre Dame

George S. Howard

University of Notre Dame

Howard and his colleagues have discovered an instrumentation related contamination which confounds the results of studies which employ self-report measures in a pre/post or posttest only design. This confounding influence is referred to as response-shift bias. Research has demonstrated that the traditional methods of analysis (i.e., analysis of posttests only, analysis of pre/post difference scores, and analysis of covariance using prescores (ANCOVA)) do not consider response-shift bias and produce biased estimates of the treatment effect. A retrospective pre/post design is recommended by Howard and his colleagues to control for response-shift bias. The only method of analysis which yields an unbiased estimate of the treatment effect is posttest minus retrospective pretest difference scores. The purpose of the present study is to determine the relative loss in statistical power of the traditional methods of analysis when response-shift bias is present. Analytic and Monte Carlo techniques were employed to compare the powers of five methods of analysis under various conditions. The results indicate that when there is a response-shift the most powerful method of analysis, overall, is the retrospective pre/post method and the loss in statistical power of the traditional methods can be substantial under many conditions. Recommendations and applications to applied research are discussed.

Educational and Psychological Measurement, Vol. 44, No. 4, 781-804 (1984)
DOI: 10.1177/0013164484444002


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				D. Moore and C. A. Tananis Measuring Change in a Short-Term Educational Program Using a Retrospective Pretest Design American Journal of Evaluation, June 1, 2009; 30(2): 189 - 202. [Abstract] [PDF]

				W. V. Pelfrey Sr and W. V. Pelfrey Jr Curriculum Evaluation and Revision in a Nascent Field: The Utility of the Retrospective Pretest--Posttest Model in a Homeland Security Program of Study Eval Rev, February 1, 2009; 33(1): 54 - 82. [Abstract] [PDF]

				A. Farel, K. Umble, and B. Polhamus Impact of an Online Analytic Skills Course Eval Health Prof, December 1, 2001; 24(4): 446 - 459. [Abstract] [PDF]

				C. C. Pratt, W. M. McGuigan, and A. R. Katzev Measuring Program Outcomes: Using Retrospective Pretest Methodology American Journal of Evaluation, September 1, 2000; 21(3): 341 - 349. [Abstract] [PDF]

				E. A. R. Robinson and H. J. Doueck Implications of the Pre/Post/Then Design for Evaluating Social Group Work Research on Social Work Practice, April 1, 1994; 4(2): 224 - 239. [Abstract] [PDF]

				K. M. Skeff, G. A. Stratos, and M. R. Bergen Evaluation of a Medical Faculty Development Program: A Comparison of Traditional Pre/Post and Retrospective Pre/Post Self-Assessment Ratings Eval Health Prof, September 1, 1992; 15(3): 350 - 366. [Abstract] [PDF]

				L. S. Aiken and S. G. West Invalidity of True Experiments: Self-Report Pretest Biases Eval Rev, August 1, 1990; 14(4): 374 - 390. [Abstract] [PDF]

				C. N. Tennis Responses to the Alpha, Beta, Gamma Change Typology: Cultural Resistance To Change Group Organization Management, June 1, 1989; 14(2): 134 - 149. [Abstract]