Project Activities
Over four years, the researchers will develop and validate a set of vertically scaled, comprehensive mathematics screening and progress monitoring measures for middle school special education students. During the first two years, the team will develop a benchmarking and progress monitoring assessment system and then collect evidence for reliability and validity purposes. The measures will then be vertically scaled across grades 6–8 using Item Response Theory (IRT) and follow up reliability studies include classical and G-theory techniques. The researchers will then collect evidence on the measures' internal structure, sensitivity to growth, and relation with other measures of mathematics.
Structured Abstract
Setting
The research takes place in Oregon but data is being collected from students across the country.
Sample
Participants include approximately 2400 students in each grade (K-8) from four different geographic regions in the U.S. The sample is drawn from urban, rural, and suburban school districts.
The intervention to be developed is a validated, comprehensive mathematics screening and progress monitoring measures for middle school special education students. The system will include measures of numbers and operations, geometry, algebraic relations, measurement, and analysis. In total, the team will develop 17 alternate forms of progress monitoring measures within each grade level (for every-other-week administration) for a total of 51 progress monitoring assessments in mathematics. The measures will be vertically scaled across the years to allow for interpretation of student growth between grade levels. Each measure consists of a minimum of 21 items, with at least seven items from each of the three Focal Points at that grade level, representing a range of difficulty within each Focal Point.
Research design and methods
The research team will conduct a two-year design and development phase followed by a two-year validation phase. The team will employ a multi-stage sampling procedure throughout the study to provide a minimum of 200 student responses per item. During instrument development in the first two years IRT scaling and alternate form and test-retest reliability analyses will be utilized. A nonequivalent group anchor test design is used for the purposes of vertical scaling. The final two years of the design focus on establishing validity of the measures through structural equation modeling and hierarchical linear modeling to document the internal structure of the measures. In addition, criterion-related evidence is gathered in the final year using an established measure of mathematics performance.
Control condition
There is no control condition.
Key measures
The researchers are developing progress monitoring measures for middle school as a key goal of the project but will also use the Stanford (10th edition) Math Achievement Test (intermediate level) for criterion-related validity purposes.
Data analytic strategy
The researchers will conduct item response theory analyses to scale the measures both within and between grades. Test-retest, internal consistency, alternate form, and G-theory analyses will be conducted to evaluate reliability. To evaluate the internal structure of the measures, invariance testing using structural equation modeling and linear regression will be utilized. Hierarchical linear modeling will be used for growth modeling, and linear regression and correlation to address criterion-related validity.
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project include a validated set of vertically scaled, comprehensive mathematics screening and progress monitoring measures for middle school special education students. The project will also produce published reports and presentations.
Journal article, monograph, or newsletter
Anderson, D., Farley, D., and Tindal, G. (2015). Test Design Considerations for Students With Significant Cognitive Disabilities. Journal of Special Education, 49(1): 3-15. doi:10.1177/0022466913491834
Anderson, D., Irvin, P. S., Alonzo, J., & Tindal, G. A. (2015). Gauging item alignment through online systems while controlling for rater effects. Educational Measurement: Issues and Practice, 34, 22-33. doi: 10.1111/emip.12038
Anderson, D., Lai, C. F., Alonzo, J., and Tindal, G. (2011). Examining a Grade-Level Math CBM Designed for Persistently Low-Performing Students. Educational Assessment, 16(1): 15-34. doi:10.1080/10627197.2011.551084
Patarapichayatham, C., Anderson, D., and Kamata, A. (2013). Middle School Transition: An Application of Latent Transition Analysis (LTA) on EasyCBM Benchmark Mathematics Data. The International Journal of Educational Administration and Development, 4: 745-756.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
