This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by von Davier, A. A. Articles by Wilson, C. Search for Related Content Social Bookmarking                         What's this?

IRT True-Score Test Equating

A Guide Through Assumptions and Applications

Educational Testing Service, avondavier{at}ets.org

Christine Wilson

Educational Testing Service

This article discusses the assumptions required by the item response theory (IRT) true-score equating method (with Stocking & Lord, 1983; scaling approach), which is commonly used in the nonequivalent groups with an anchor data-collection design. More precisely, this article investigates the assumptions made at each step by the IRT approach to calibrating items and equating tests and discusses the approaches that one might take for checking whether these assumptions are met for a particular data set. We investigated two types of tests: tests that consist of multiple-choice items only and tests that consist of both multiple-choice and free-response items. Real data from the AP^® Calculus AB exam are used to illustrate the application of the IRT true-score equating method as well as for the comparisons.

Key Words: test equating • item response theory (IRT) true-score equating method • observed-score equating methods • population sensitivity of equating function

References

• Ackerman, T.A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.[CrossRef][Web of Science]
• Browne, M.W., Cudeck, R., Tateneni, K., & Mels, G. (1999). CEFA: Comprehensive exploratory factor analysis [Computer software]. Retrieved November 10, 2004, from http://quantrm2.psy.ohio-state.edu/browne/software.htm
• Cook, L.L., Dorans, N.J., Eignor, D.R., & Petersen, N.S. (1985). An assessment of the relationship between the assumption of unidimensionality and the quality of IRT true-score equating (ETS RR-85-30). Princeton, NJ: Educational Testing Service.
• Cook, L.L., & Eignor, D.R. (1991). An NCME instructional module on IRT equating methods. Educational Measurement: Issues and Practice, 10, 37-45.
• Cook, L.L., & Petersen, N.S. (1987). Problems related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225-244.[Abstract]
• Dorans, N.J., & Feigenbaum, M.D. (1994). Equating issues engendered by changes to the SAT and PSAT/NMSQT. In I. M. Lawrence, N. J. Dorans, M. D. Feigenbaum, N. J. Feryok, A. P. Schmitt, & N. K. Wright (Eds.), Technical issues related to the introduction of the new SAT and PSAT/NMSQT (RM-94-10). Princeton, NJ: Educational Testing Service.
• Dorans, N.J., Holland, P.W., Thayer, D.T., & Tateneni, K. (2003). Invariance of score linking across gender groups for three Advanced Placement Program® exams. In N. J. Dorans (Ed.), Population invariance of score linking: Theory and applications to Advanced Placement Program examinations (ETS RR-03-27, pp. 19-36). Princeton, NJ: Educational Testing Service.
• Dorans, N.J., & Schmitt, A.P. (1991). Constructed response and di ferential item functioning: A pragmatic approach (ETS RR-91-47). Princeton, NJ: Educational Testing Service.
• Dorans, N.J., & Tateneni, K. (1993). PolyStand [Computer software.] Princeton, NJ: Educational Testing Service.
• Educational Testing Service. (2004). GENASYS [Computer software]. Princeton, NJ: Author.
• Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149.
• Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
• Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.[Abstract]
• Holland, P.W., & Thayer, D.T. (1988). Di ferential item functioning and the Mantel-Haenszel procedure (ETS RR-86-31). Princeton, NJ: Educational Testing Service.
• Jodoin, M.G., & Davey, T. (2003, April). A multidimensional simulation approach to investigate the robustness of IRT common item equating. Paper presented at the annual meeting of the American Educational Research Association (AERA), Chicago.
• Kolen, M.J., & Brennan, R.J. (2004). Test equating: Methods and practices (2nd ed.). New York: Springer.
• Lord, F.M. (1980). Application of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
• Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
• Muraki, E., & Bock, R.D. (1997). PARSCALE 3.0: IRT item analysis and test scoring for rating scale data [Computer software]. Chicago: Scientific Software International.
• Petersen, N.S., Cook, L.L., & Stocking, M.L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137-156.[CrossRef]
• Petersen, N.S., Kolen, M.J., & Hoover, H.D. (1989). Scaling, norming, and equating. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 221-262). New York: Macmillan.
• Stocking, M., & Lord, F.M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.[Abstract]
• Thissen, D., Wainer, H., & Wang, X.-B. (1994). Are tests comprising both multiple-choice and free responses items necessary less unidimensional than multiple-choice tests? An analysis of two tests. Journal of Educational Measurement, 31, 113-123.[CrossRef][Web of Science]
• von Davier, A.A. (2003). Notes on linear equating methods for the non-equivalent groups design (ETS RR-03-24). Princeton, NJ: Educational Testing Service.
• von Davier, A.A., Holland, P.W., & Thayer, D.T. (2004a). The chain and post-stratification methods for observed-score equating: Their relationship to population invariance. Journal of Educational Measurement, 41, 15-32.[CrossRef][Web of Science]
• von Davier, A.A., Holland, P.W., & Thayer, D.T. (2004b). The kernel method of test equating. New York: Springer Verlag.
• von Davier, A.A., & Wilson, C. (2006). Population invariance of IRT true-score. equating. In A. A. von Davier & M. Liu (Eds.), Population invariance of test equating and linking: Theory extension and applications across exams. (RR-06-31). Princeton, NJ: Educational Testing Service.

This version was published on December 1, 2007

Educational and Psychological Measurement, Vol. 67, No. 6, 940-957 (2007)
DOI: 10.1177/0013164407301543


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by von Davier, A. A. Articles by Wilson, C. Search for Related Content Social Bookmarking                         What's this?