Purpose: There has been a national push to use growth modeling with scores from statewide standards-based summative assessments to evaluate schools, teachers, and student subgroups. Yet for students with disabilities, there has been little research exploring the use of growth modeling for this subpopulation to determine if interpretations about schools, teachers, and the academic progress of the subgroup are valid. The purpose of this study is to provide validity evidence for the use of test scores from students with disabilities on statewide standards-based summative assessments for the purposes of growth modeling and other growth-based models for accountability.

Project Activities: The grantee will obtain five states' datasets with students' results on the statewide standards-based assessment for 3 years. The number of students with disabilities in each grade between grades 3 and 8 will range from approximately 1,500 to 18,000 across states. The grantee will compare the results of different growth-based models for accountability by grade, content, and number of years of test results, and will explore the role of testing accommodations in measuring growth.

Products: The products of this project will include recommendations on growth modeling and longitudinal data collection for students with disabilities taking statewide summative assessments, peer-reviewed publications, and presentations.

Structured Abstract

Setting: The datasets to be examined are from California, Colorado, Indiana, Massachusetts, and Virginia.

Sample: Approximately 1,500 to 18,000 students with disabilities per grade in grades 3–8 will be included in the study, each with data across 3 years. The sample includes students with disabilities who took the standards-based summative assessments with or without accommodations. Students without disabilities will be included for comparison.

Assessment: The assessments are the statewide standards-based assessments in English language arts and mathematics for grades 3–8.

Research Design and Methods: The project is examining five states' extant datasets to validate inferences from growth modeling results. Databases combining 2 years or 3 years of scores for each student will be compared to evaluate whether different numbers of test scores impacts growth modeling results. In addition, the datasets will allow for comparisons across grades and content areas. There will be a total of 8 3-year datasets and 10 2-year datasets. English language learners with disabilities will be tagged in the datasets so that the analyses can include and exclude them to establish how this subpopulation compares with the overall results of students with disabilities. Descriptive analyses will document systematic factors that may impact growth modeling, including inconsistent accommodation use and subgroup performance and growth.

Control Condition: The study will include students without disabilities in the statewide datasets for comparison.

Key Measures: Due to the nature of this research, the only key measures are the statewide standards-based assessments in English language arts and mathematics for grades 3–8, which also include information on English language learner status, disability category, and accommodation use.

Data Analytic Strategy: The four-part data analysis consists of the following: (1) descriptive statistics on the characteristics of longitudinal data for students with disabilities; (2) documentation of the growth of students with disabilities as measured by statewide summative assessments; (3) a comparison of growth models on predictive accuracy (2 vs. 3 years of data), classification accuracy, and the percentage of students identified as on track to proficiency; and (4) descriptive statistics, model development, and model comparisons to explore the role of testing accommodations in growth models.

Products and Publications

Journal article, monograph, or newsletter

Buzick, H.M., and Jones, N.D. (2015). Using Test Scores From Students With Disabilities in Teacher Evaluation. Educational Measurement: Issues and Practice, 34(3): 28–38. doi:10.1111/emip.12076

Jones, N.D., Buzick, H.M., and Turkan, S. (2013). Including Students With Disabilities and English Learners in Measures of Educator Effectiveness. Educational Researcher, 42(4): 234–241. doi:10.3102/0013189X12468211?