Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here for FREE ACCESS to this landmark database

Click here for more information on Research and Evaluation in Education and Psychology, 3e

Sign In to gain access to subscriptions and/or personal tools.
Educational and Psychological Measurement
This Article
Right arrow Full Text (PDF)
Right arrow References
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Right arrow Citing Articles via Scopus
Google Scholar
Right arrow Articles by Hurtz, G. M.
Right arrow Articles by Auerbach, M. A.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

A Meta-Analysis of the Effects of Modifications to the Angoff Method on Cutoff Scores and Judgment Consensus

Gregory M. Hurtz

Excelsior College, ghurtz{at}csus.edu

Meredith A. Auerbach

University at Albany, State University of New York

The Angoff method for setting cutoff scores on examinations has been researched extensively over the past 30 years, including investigations of several procedural modifications to the method. In the current study, the authors meta-analytically synthesized this research and evaluated the systematic effects of the most common modifications on the level of the resulting cutoff scores and the degree of consensus among judges. The authors found the following: (a) when judges use a common definition of minimally competent test-takers, this tends to increase judgment consensus; (b) when judges discuss their estimates, this tends to result in higher cutoff scores with increased consensus; and (c) when judges view normative data, this tends to systematically lower cutoff scores. In addition, the authors found an interaction effect which revealed that when judges use a common definition and later discuss their estimates, this tends to produce the highest standards on average, with the highest degree of consensus among judges. Theoretical and practical implications of these effects are discussed.

Key Words: cutoff scores • passing scores • Angoff method • standard setting • meta-analysis

Educational and Psychological Measurement, Vol. 63, No. 4, 584-601 (2003)
DOI: 10.1177/0013164403251284
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?


This article has been cited by other articles:


Home page
 Applied Psychological MeasurementHome page
G. M. Hurtz, J. P. Jones, and C. N. Jones
Conversion of Proportion-Correct Standard-Setting Judgments to Cutoff Scores on the Item Response Theory {theta} Scale
Applied Psychological Measurement, July 1, 2008; 32(5): 385 - 406.
[Abstract] [PDF]

Home page
 Eval Health ProfHome page
B. Hess, R. G. Subhiyah, and C. Giordano
Convergence Between Cluster Analysis and the Angoff Method for Setting Minimum Passing Scores on Credentialing Examinations
Eval Health Prof, December 1, 2007; 30(4): 362 - 375.
[Abstract] [PDF]