Purpose: Continued poor performance in mathematics has indicated a need for more sensitive progress measures that can assist with the early identification of struggling students. Currently, there is a lack of high quality progress measures that are readily available, technically adequate, and aligned to appropriate content. More robust progress monitoring measures, independent of curriculum, could more objectively measure student progress on critical mathematics content. In addition, there is a need for such measures to be efficient to use to allow for cost-effective monitoring of student performance over time.

The purpose of this project is to develop and validate a set of online middle school mathematics progress monitoring measures aligned with critical content standards. As part of the development activities, the research team will make use of an existing web-based infrastructure to alleviate the constraints of time and logistics associated with test administration, scoring, and record keeping.

Project Activities: Over four years, the researchers will develop and validate a set of vertically scaled, comprehensive mathematics screening and progress monitoring measures for middle school special education students. During the first two years, the team will develop a benchmarking and progress monitoring assessment system and then collect evidence for reliability and validity purposes. The measures will then be vertically scaled across grades 6–8 using Item Response Theory (IRT) and follow up reliability studies include classical and G-theory techniques. The researchers will then collect evidence on the measures' internal structure, sensitivity to growth, and relation with other measures of mathematics.

Products: The products of this project include a validated set of vertically scaled, comprehensive mathematics screening and progress monitoring measures for middle school special education students. The project will also produce published reports and presentations.

Setting: The research takes place in Oregon but data is being collected from students across the country.

Population: Participants include approximately 2400 students in each grade (K–8) from four different geographic regions in the U.S. The sample is drawn from urban, rural, and suburban school districts.

Intervention: The intervention to be developed is a validated, comprehensive mathematics screening and progress monitoring measures for middle school special education students. The system will include measures of numbers and operations, geometry, algebraic relations, measurement, and analysis. In total, the team will develop 17 alternate forms of progress monitoring measures within each grade level (for every-other-week administration) for a total of 51 progress monitoring assessments in mathematics. The measures will be vertically scaled across the years to allow for interpretation of student growth between grade levels. Each measure consists of a minimum of 21 items, with at least seven items from each of the three Focal Points at that grade level, representing a range of difficulty within each Focal Point.

Research Design and Methods: The research team will conduct a two-year design and development phase followed by a two-year validation phase. The team will employ a multi-stage sampling procedure throughout the study to provide a minimum of 200 student responses per item. During instrument development in the first two years IRT scaling and alternate form and test-retest reliability analyses will be utilized. A nonequivalent group anchor test design is used for the purposes of vertical scaling. The final two years of the design focus on establishing validity of the measures through structural equation modeling and hierarchical linear modeling to document the internal structure of the measures. In addition, criterion-related evidence is gathered in the final year using an established measure of mathematics performance.

Control Condition: There is no control condition.

Key Measures: The researchers are developing progress monitoring measures for middle school as a key goal of the project but will also use the Stanford (10th edition) Math Achievement Test (intermediate level) for criterion-related validity purposes.

Data Analytic Strategy: The researchers will conduct item response theory analyses to scale the measures both within and between grades. Test-retest, internal consistency, alternate form, and G-theory analyses will be conducted to evaluate reliability. To evaluate the internal structure of the measures, invariance testing using structural equation modeling and linear regression will be utilized. Hierarchical linear modeling will be used for growth modeling, and linear regression and correlation to address criterion-related validity.

Products and Publications

Journal article, monograph, or newsletter

Anderson, D., Farley, D., and Tindal, G. (2015). Test Design Considerations for Students With Significant Cognitive Disabilities. Journal of Special Education, 49(1): 3–15. doi:10.1177/0022466913491834

Anderson, D., Irvin, P. S., Alonzo, J., & Tindal, G. A. (2015). Gauging item alignment through online systems while controlling for rater effects. Educational Measurement: Issues and Practice, 34, 22–33. doi: 10.1111/emip.12038

Anderson, D., Lai, C. F., Alonzo, J., and Tindal, G. (2011). Examining a Grade-Level Math CBM Designed for Persistently Low-Performing Students. Educational Assessment, 16(1): 15–34. doi:10.1080/10627197.2011.551084

Patarapichayatham, C., Anderson, D., and Kamata, A. (2013). Middle School Transition: An Application of Latent Transition Analysis (LTA) on EasyCBM Benchmark Mathematics Data. The International Journal of Educational Administration and Development, 4: 745–756.