Project Activities
The research team is conducting a series of studies to evaluate the reliability and validity of an integrated assessment system for K-3 mathematics, mCLASS®:Math. Based on the results of these studies, the team will modify the assessment system as needed. The assessment includes screening and progress monitoring measures, and diagnostic cognitive interviews. It is intended for use with underachieving students from a variety of ethnic, linguistic, and economic backgrounds.
Structured Abstract
Setting
The schools are located in Pennsylvania, Nevada, and Missouri.
Sample
Collectively, these districts enroll 312,788 students. Of these, 39 percent are European American, 35 percent are Hispanic, and 16 percent are African American. Students are being recruited from demographically representative schools for a total of 2,400 students (800 in each district): 600 kindergarteners, 600 first-graders, 600 second-graders, and 600 third-graders.
mCLASS®:Math is a comprehensive assessment system. The system operates on a handheld computer and guides the teacher in conducting the assessment and recording the results (students do not use the device). The assessment includes screening and progress monitoring measures, and diagnostic cognitive interviews. Once the collected data are uploaded to a server, the system prepares reports for the teacher and for administrators. These reports are intended to help teachers monitor student progress and develop an understanding of those cognitive processes that impede student performance. The system also provides information about informal strategies and concepts that teachers can use to help improve student mathematics performance.
Research design and methods
In Year 1, researchers will collect reliability, validity, and item analysis data through two back-to-back administrations of the full sequence of the curriculum-based measurement and diagnostic assessments to kindergarten to third-grade students. In Year 2, the full sequence of the curriculum-based measures and diagnostic assessments will be administered to students five times throughout the school year (September, two weeks later, December, March, and May). The data from Year 2 will help establish the developmental growth models for the curriculum-based measures, the baseline predictive validity, and the baseline cut-points for screening using the curriculum-based measures. In Year 3, the students who were in grades K, 1, and 2 during Year 2 will be administered the full sequence of the curriculum-based measurement and diagnostic assessments in late May to establish predictive validity, and cross-validation of the measures and diagnostic interviews. These students will be in grades 1, 2, and 3, respectively. In Year 4, the same set of students included in Years 2 and 3 will be examined further. These students will now be in grades 2 and 3, respectively. Students will be administered the full sequence of the curriculum-based measures and diagnostic assessments in late May to complete the predictive validity studies and the development of growth curve models for the measures.
Key measures
Student mathematics achievement is measured with the Woodcock-Johnson III Broad Math Battery and state standardized achievement tests.
Data analytic strategy
Data analyses include correlational analysis and item response theory for reliability; confirmatory factor analysis, correlational analysis, and item analysis for validity; structural equation modeling using latent growth curve analysis to evaluate student growth over time; the Angoff standard setting method to determine cut-points; and correlational analysis to conduct cross validation of the assessment model.
People and institutions involved
IES program contact(s)
Products and publications
Products: Products from this project include published reports on the reliability and validity of an assessment system for K-3 mathematics.
Book
Lee, Y.S., Lembke, E., Moore, D., Ginsburg, H., and Pappas, S. (2007). mCLASS: MATH-Identifying Technically Adequate Early Mathematics Measures.Brooklyn, NY: Wireless Generation, Inc.
Lee, Y.S., Pappas, S., Chiong, C., and Ginsburg, H. (2010). mCLASS: MATH-Technical Manual.Brooklyn, NY: Wireless Generation, Inc.
Book chapter
Ginsburg, H.P., Pappas, S., Lee, Y.S., and Chiong, C. (2011). How did you get That Answer?: Computer Assessments of Young Children's Mathematical Minds in mCLASS:Math. In Noyce, P., and Hickey, D.T., (Eds.), Formative Assessment in Learning Contexts, the Next Generation (pp. 49-67). Cambridge, MA: Harvard Education Press.
Journal article, monograph, or newsletter
Hampton, D.D., Lembke, E.S., Lee, Y.-S., Pappas, S., Chiong, C., and Ginsburg, H. (2012). Technical Adequacy of Early Numeracy Curriculum-Based Progress Monitoring Measures for Kindergarten and First-Grade Students. Assessment for Effective Intervention, 37(2): 118-126.
Lee, Y.-S., Lembke, E., Moore, D., Ginsburg, H., and Pappas, S. (2012). Item-Level and Construct Evaluation of Early Numeracy Curriculum-Based Measures. Assessment for Effective Intervention, 37(2): 107-117.
Supplemental information
Purpose: The purpose of this project is to develop and evaluate a comprehensive automated assessment system that helps kindergarten through grade 3 teachers: monitor student progress in mathematics, identify students at risk of failure, and conduct clinical interviews to understand the cognitive processes underlying student performance.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
