|Title:||mCLASS:Math: Development and Analysis of an Integrated Screening, Progress Monitoring, and Cognitive Assessment System for K-3 Mathematics|
|Principal Investigator:||Ginsburg, Herbert||Awardee:||Columbia University, Teachers College|
|Program:||Science, Technology, Engineering, and Mathematics (STEM) Education [Program Details]|
|Award Period:||4 years||Award Amount:||$1,566,565|
Purpose: Teachers often do not have the means to frequently test and keep track of how well their students are performing, or have knowledge of the cognitive processes underlying their students' academic strengths or shortcomings. Frequent and efficient use of formative assessments can help teachers monitor students' performance and adjust instruction to better fit the needs of their students. The purpose of this project is to develop and evaluate a comprehensive handheld computer assessment system that helps kindergarten through grade 3 teachers: monitor student progress in mathematics, identify students at risk of failure, and conduct clinical interviews to understand the cognitive processes underlying student performance. The handheld computer assessment system provides teachers with questions to pose to students and allows the teacher to record students' responses directly to the device. Students' responses are then automatically scored and synchronized to a web-based reporting system.
Project Activities: The research team is conducting a series of studies to evaluate the reliability and validity of an integrated assessment system for K-3 mathematics, mCLASS®:Math. Based on the results of these studies, the team will modify the assessment system as needed. The assessment includes screening and progress monitoring measures, and diagnostic cognitive interviews. It is intended for use with underachieving students from a variety of ethnic, linguistic, and economic backgrounds.
Products: Products from this project include published reports on the reliability and validity of an assessment system for K-3 mathematics.
Purpose: The purpose of this project is to develop and evaluate a comprehensive automated assessment system that helps kindergarten through grade 3 teachers: monitor student progress in mathematics, identify students at risk of failure, and conduct clinical interviews to understand the cognitive processes underlying student performance.
Setting: The schools are located in Pennsylvania, Nevada, and Missouri.
Population: Collectively, these districts enroll 312,788 students. Of these, 39 percent are European American, 35 percent are Hispanic, and 16 percent are African American. Students are being recruited from demographically representative schools for a total of 2,400 students (800 in each district): 600 kindergarteners, 600 first-graders, 600 second-graders, and 600 third-graders.
Intervention: mCLASS®:Math is a comprehensive assessment system. The system operates on a handheld computer and guides the teacher in conducting the assessment and recording the results (students do not use the device). The assessment includes screening and progress monitoring measures, and diagnostic cognitive interviews. Once the collected data are uploaded to a server, the system prepares reports for the teacher and for administrators. These reports are intended to help teachers monitor student progress and develop an understanding of those cognitive processes that impede student performance. The system also provides information about informal strategies and concepts that teachers can use to help improve student mathematics performance.
Research Design and Methods: In Year 1, researchers will collect reliability, validity, and item analysis data through two back-to-back administrations of the full sequence of the curriculum-based measurement and diagnostic assessments to kindergarten to third-grade students. In Year 2, the full sequence of the curriculum-based measures and diagnostic assessments will be administered to students five times throughout the school year (September, two weeks later, December, March, and May). The data from Year 2 will help establish the developmental growth models for the curriculum-based measures, the baseline predictive validity, and the baseline cut-points for screening using the curriculum-based measures. In Year 3, the students who were in grades K, 1, and 2 during Year 2 will be administered the full sequence of the curriculum-based measurement and diagnostic assessments in late May to establish predictive validity, and cross-validation of the measures and diagnostic interviews. These students will be in grades 1, 2, and 3, respectively. In Year 4, the same set of students included in Years 2 and 3 will be examined further. These students will now be in grades 2 and 3, respectively. Students will be administered the full sequence of the curriculum-based measures and diagnostic assessments in late May to complete the predictive validity studies and the development of growth curve models for the measures.
Key Measures: Student mathematics achievement is measured with the Woodcock-Johnson III Broad Math Battery and state standardized achievement tests.
Data Analytic Strategy: Data analyses include correlational analysis and item response theory for reliability; confirmatory factor analysis, correlational analysis, and item analysis for validity; structural equation modeling using latent growth curve analysis to evaluate student growth over time; the Angoff standard setting method to determine cut-points; and correlational analysis to conduct cross validation of the assessment model.
Publications from this project:
Ginsburg, H.P., Pappas, S., Lee, Y.S., and Chiong, C. (2011). How Did You Get That Answer?: Computer Assessments of Young Children's Mathematical Minds in mCLASS:Math. In Noyce, P., and Hickey, D. T., Formative Assessment in Learning Contexts, the Next Generation. (pp. 49–67). Harvard Education Press.
Hampton, D.D., Lembke, E.S., Lee, Y.-S., Pappas, S., Chiong, C., and Ginsburg, H. (2012). Technical Adequacy of Early Numeracy Curriculum-Based Progress Monitoring Measures for Kindergarten and First-Grade Students. Assessment for Effective Intervention, 37 (2), 118–126.
Lee, Y.S., Lembke, E., Moore, D., Ginsburg, H., and Pappas, S. (2012). Item-Level andConstruct Evaluation of Early Numeracy Curriculum-Based Measures. Assessment for Effective Intervention, 37 (2): 107–117.
Lee, Y.S., Lembke, E., Moore, D., Ginsburg, H., and Pappas, S. (2007). mCLASS®:MATH –Identifying technically adequate early mathematics measures. Brooklyn, NY: Wireless Generation, Inc.
Lee, Y.S., Pappas, S., and Chiong, C., and Ginsburg, H. (2010). mCLASS®:MATH – Technical Manual. Brooklyn, NY: Wireless Generation, Inc.