Project Activities
Four activities are planned for the development and validation of the guidelines. First, the project will use extant datasets to review technical documentation of CBM instrumentation related to reading, analyze extant datasets, and conduct simulation studies to investigate accurate methods for estimating growth and determining decision rules. Second, progress monitoring data will be collected to evaluate the rules generated in the first activity using a subset of progress monitoring measures. Third, there will be a comparison of the results and guidelines based on simulated data from the first activity with the results from the data collection and analyses from the second activity. Fourth, the project will develop a web-based program that enables practitioners to enter and interpret data. A combination of analytic strategies, including linear mixed effects regression, ordinary least squares, and analysis of variance, will be used to determine guidelines and compare their reliability and validity across activities.
Structured Abstract
Setting
Data collection for the second activity will occur in urban, rural, and suburban schools in Minnesota.
Sample
Students included in the extant datasets are from samples participating in CBM activities across the United States who were in kindergarten through Grade 6 and defined as either having a disability or being at risk for disabilities. Students participating in primary data collection will be in Grades 1 through 5.
The project will use extant data from frequently used reading progress monitoring assessments to develop and validate guidelines. Sources of extant data include: AIMSweb, EasyCBM, Dynamic Indicators of Basic Early Literacy Skills, EdCheckup, and the Formative Assessment Instrumentation and Procedures. Based on the findings from the extant data analyses, a subset of these commonly used measures will be administered for primary data collection in the second activity.
Research design and methods
There are four activities in the development and validation of the guidelines. First, the project will review technical documentation of CBM instrumentation related to reading, analyze extant datasets, and conduct simulation studies to investigate accurate methods for estimating growth and determining decision rules. Second, the project will collect progress monitoring data to evaluate the rules generated in the first activity. Based on the findings from the extant data analyses, a subset of commonly used progress monitoring measures will be administered. Third, there will be a comparison of the results and guidelines based on simulated data from the first activity with the results from the data collection and analyses from the second activity. Fourth, the project will develop a web-based program that enables practitioners to enter and interpret data. A combination of analytic strategies, including linear mixed effects regression, ordinary least squares, and analysis of variance, will be used to determine guidelines and compare their reliability and validity across extant, simulated, and primary data collection activities.
Control condition
Due to the nature of the research design, there is no control condition.
Key measures
The measures being used are the assessment measures listed above.
Data analytic strategy
A combination of analytic strategies, including linear mixed effects regression, ordinary least squares, and analysis of variance, will be used to determine guidelines and compare their reliability and validity across extant, simulated, and primary data collection activities.
People and institutions involved
IES program contact(s)
Products and publications
Products: The primary products will consist of a fully developed and validated set of guidelines for progress monitoring data collection, interpretation, and use, as well as a web-based interface that enables practitioners to enter and interpret data.
Journal article, monograph, or newsletter
Van Norman, E.R., and Christ, T.J. (2016). . . The Folly of Data Point Decision Rules: An Exercise in Basic Probability Theory School Psychology Review Full text
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
