|
Sign In to gain access to subscriptions and/or personal tools.
|
Estimating the Standard Error of the Maximum Likelihood Ability Estimator in Adaptive Testing Using the Posterior-Weighted Test Information Function
Randall D. Penfield
University of Miami, penfield{at}miami.edu
The standard error of the maximum likelihood ability estimator is commonly estimated by evaluating the test information function at an examinee's current maximum likelihood estimate (a point estimate) of ability. Because the test information function evaluated at the point estimate may differ from the test information function evaluated at an examinee's true ability value, the estimated standard error may be biased under certain conditions. This is of particular concern in adaptive testing because the height of the test information function is expected to be higher at the current estimate of ability than at the actual value of ability. This article proposes using the posterior-weighted test information function in computing the standard error of the maximum likelihood ability estimator for adaptive test sessions. A simulation study showed that the proposed approach provides standard error estimates that are less biased and more efficient than those provided by the traditional point estimate approach.
Key Words: computerized adaptive testing • parameter estimation • item response theory • information functions • maximum likelihood estimation
References
- Baker, F.B. (1992). Item response theory: Parameter estimation techniques. New York: Marcel Decker.
- Bock, R.D., & Mislevy, R.J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.[Abstract/Free Full Text]
- Burden, R.L., & Faires, J.D. (2001). Numerical analysis (7th ed.). Pacific Grove, CA: Brooks/Cole.
- Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 231-229.[Abstract/Free Full Text]
- Chen, S.-K., Hou, L., & Dodd, B.G. (1998). A comparison of maximum likelihood estimation and expected a posteriori estimation in CAT using the partial credit model. Educational and Psychological Measurement, 58, 569-595.[Abstract]
- Chen, S.-K., Hou, L., Fitzpatrick, S.J., & Dodd, B.G. (1997). The effect of population distribution and methods of theta estimation on CAT using the rating scale model. Educational and Psychological Measurement, 57, 422-439.[Abstract]
- Chen, S.-Y., Ankenmann, R.D., & Chang, H.-H. (2000). A comparison of item selection rules at the early stages of computerized adaptive testing. Applied Psychological Measurement, 24, 241-255.[Abstract/Free Full Text]
- Kim, J.K., & Nicewander, W.A. (1993). Ability estimation for conventional tests. Psychometrika, 58, 587-599.[CrossRef][Web of Science]
- Lee, P.M. (1997). Bayesian statistics: An introduction (2nd ed.). New York: Oxford University Press. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
- Lord, F.M. (1986). Maximum likelihood and Bayesian parameter estimation in item response theory. Journal of Educational Measurement, 23, 157-162.[CrossRef][Web of Science]
- Meijer, R.R., & Nering, M.L. (1997). Trait level estimation for nonfitting response vectors. Applied Psychological Measurement, 21, 321-336.[Abstract/Free Full Text]
- Mills, C.N., Potenza, M.T., Fremer, J.J., & Ward, W.C. (2002). Computer-based testing: Building the foundation for future assessment. Mahwah, NJ: Lawrence Erlbaum.
- Mislevy, R.J. (1993). Some formulas for use with Bayesian ability estimates. Educational and Psychological Measurement, 53, 315-328.[Abstract]
- Parshall, C.G., Spray, J.A., Kalohn, J.C., & Davey, T. (2002). Practical considerations in computer-based testing. New York: Springer.
- Penfield, R.D., & Bergeron, J.M. (2005). Applying a weighted maximum likelihood latent trait estimator to the generalized partial credit model. Applied Psychological Measurement, 29, 218-233.[Abstract]
- van der Linden, W.J. (1998). Bayesian item-selection criteria for adaptive testing. Psychometrika, 62, 201-216.[Web of Science]
- van der Linden, W. J., & Glas, C.A.W. (Eds.). (2000). Computerized adaptive testing: Theory and practice. Boston: Kluwer Academic.
- van der Linden, W.J., & Pashley, P.J. (2000). Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C.A.W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 1-25). Boston: Kluwer Academic.
- Wainer, H. (1990). Introduction and history. In H. Wainer (Ed.), Computerized adaptive testing: A primer (pp. 1-21). Hillsdale, NJ: Lawrence Erlbaum.
- Wang, T., & Hanson, B.A., & Lau, C.M. (1999). Reducing bias is CAT trait estimation: A comparison of approaches. Applied Psychological Measurement, 23, 263-278.[Abstract/Free Full Text]
- Wang, T., & Vispoel, W.P. (1998). Properties of ability estimation methods in computerized adaptive testing. Journal of Educational Measurement, 35, 109-135.[CrossRef][Web of Science]
- Warm, T.A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450.[CrossRef][Web of Science]
This version was published on December
1, 2007
Educational and Psychological Measurement, Vol. 67, No. 6,
958-975 (2007)
DOI: 10.1177/0013164407301544
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
Ying Cheng, H.-H. Chang, J. Douglas, and Fanmin Guo
Constraint-Weighted a-Stratification for Computerized Adaptive Testing With Nonstatistical Constraints: Balancing Measurement Efficiency and Exposure Control
Educational and Psychological Measurement,
February 1, 2009;
69(1):
35 - 49.
[Abstract]
[PDF]
|
|
|
|
|
