This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Stevens, J. J. Articles by Zvoch, K. Search for Related Content Social Bookmarking                         What's this?

Confirmatory Factor Analysis of the TerraNova Comprehensive Tests of Basic Skills/5

University of Oregon, stevensj{at}uoregon.edu

Keith Zvoch

University of Nevada, Las Vegas

Confirmatory factor analysis was used to explore the internal validity of scores on the TerraNova Comprehensive Tests of Basic Skills/5 using samples from a southwestern school district and standardization samples reported by the publisher. One of the strengths claimed for battery-type achievement tests is provision of reliable and valid samples of student achievement in specific content areas. Results of the present study showed that specific content areas may not be clearly represented in the test structure as there was little difference in goodness of fit between two- or three-factor structures of subtest scores. In addition, a nested series of invariance tests showed that all parameters of the three-factor model cross-validated across samples. Together with large intercorrelations between the latent achievement factors and large subtest uniquenesses, these results raise questions about the differentiation among subtest scores as well as how scores should be used and interpreted.

Key Words: validity • achievement tests • confirmatory factor analysis • cross-validation

References

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: AERA.
• Arbuckle, J.L., & Wothke, W. (1999). Amos 4.0 user's guide. Chicago, IL: Smallwaters Corporation.
• Bandalos, D.L. (1993). Factors influencing cross-validation of confirmatory factor analysis models. Multivariate Behavioral Research, 28, 351-374.[Web of Science]
• Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.[CrossRef][Web of Science]
• CTB/McGraw-Hill. (1996). TerraNova prepublication technical bulletin. Monterey, CA: Author.
• CTB/McGraw-Hill. (1997). TerraNova technical bulletin 1. Monterey, CA: Author.
• Cudeck, R., & Browne, M.W. (1983). Cross-validation of covariance structures. Multivariate Behavioral Research, 18, 147-167.[CrossRef][Web of Science]
• Graham, J.M., Guthrie, A.C., & Thompson, B. (2003). Consequences of not interpreting structure coefficients in published CFA research: A reminder. Structural Equation Modeling, 10, 142-153.[CrossRef][Web of Science]
• Henson, R.K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.[Web of Science]
• Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
• Klein, A.E. (1980). Redundancy in the Comprehensive Tests of Basic Skills. Educational and Psychological Measurement, 40, 1105-1110.[Abstract]
• Lewis, D.M., Green, D.R., Mitzel, H.C., Baum, K., & Patz, R.J. (1998). The bookmark standard setting procedure: Methodology and recent implementations. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.
• MacCallum, R.C., Roznowski, M., Mar, C., & Reith, J.V. (1994). Alternative strategies for cross-validation of covariance structure models. Multivariate Behavioral Research, 29, 1-32.[CrossRef][Web of Science]
• Marsh, H.W., Balla, J., & McDonald, R.P. (1988). Goodness-of-fit indices in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391-410.[CrossRef][Web of Science]
• Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: MacMillan.
• Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.[Abstract/Free Full Text]
• Miller, M.D. (1992). Review of the Comprehensive Tests of Basic Skills, fourth edition. In J. J. Kramer & J. C. Conoley (Eds.), The eleventh mental measurements yearbook. Lincoln, NE: Buros Institute of Mental Measurements.
• Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
• Sobel, M.E., & Bohrnstedt, G.W. (1985). Use of null models in evaluating the fit of covariance structure models. In N. B. Tuma (Ed.), Sociological methodology (pp. 152-178). San Francisco, CA: Jossey-Bass.
• Stevens, J.J. (1995). Confirmatory factor analysis of the Iowa Tests of Basic Skills. Structural Equation Modeling, 2, 214-231.
• Stevens, J.J. (2001). Confirmatory factor analysis of the CTBS5/TerraNova. Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA.
• Tucker, L.R., & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.[CrossRef]

This version was published on December 1, 2007

Educational and Psychological Measurement, Vol. 67, No. 6, 976-989 (2007)
DOI: 10.1177/0013164406299107


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This Article Abstract Free Full Text (Free PDF) Free Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Stevens, J. J. Articles by Zvoch, K. Search for Related Content Social Bookmarking                         What's this?