Purpose: Proficiency in algebra is a critical building block for postsecondary education and higher wage jobs, as well as the nation's competitiveness in science, technology, engineering, and mathematics. Algebra competence is a particular concern for secondary students with disabilities, who are participating in general education mathematics courses in growing numbers and facing curriculum standards and graduation requirements that demand mastery of algebra. Empirical evidence suggests positive effects on elementary students' achievement in reading and mathematics when teachers use frequent progress monitoring to inform instruction. Unfortunately, comparable monitoring measures for advanced mathematics topics are not widely available.

The purpose of this project is to develop a series of algebra screening and progress monitoring measures intended to enable teachers of students with disabilities to better monitor students' learning in algebra. The research team will develop and pilot test an online progress monitoring assessment system that monitors progress in algebra, primarily for students with learning disabilities.

Project Activities: The screening and progress monitoring measures will be developed and pilot tested with four cohorts of general and special education teachers and more than 2,500 students learning algebra in high school settings. The research team will evaluate six types of algebra measures for screening and progress monitoring of students with or at risk for disabilities. First, the team will develop two new algebra measures targeting conceptual understanding and problem solving. Second, the researchers will evaluate these measures, along with four previously developed algebra measures, to assess their technical adequacy (reliability and validity) as screening instruments and as progress monitoring tools. The researchers will then finalize all of the measures and prepare them for dissemination.

Products: The products of this project include a fully developed algebra screening and progress monitoring assessment system, evidence of technical adequacy of the measures, and published papers and presentations.

Structured Abstract

Setting: The research will take place in high schools in Mississippi, Iowa, and Missouri.

Population: Four cohorts of general education and special education teachers and their high school students (more than 2,500 students, with and without disabilities) will participate in the development of the screening and progress monitoring measures.

Intervention: The assessment system to be created will include six algebra progress monitoring measures with three measures focusing on traditional algebra instruction (e.g., symbolic manipulation) and three reformist measures targeting conceptual understanding and problem solving. The three traditional algebra measures are Algebra Basic Skills, Algebra Foundations, and Algebra Content Analysis; and the reformist measures will be Translations along with two new measures to be developed. There will be 12 equivalent forms of each measure for a total of 72 instruments.

Research Design and Methods: The research design will include a series of studies to determine internal consistency, test-retest, and alternate form reliability, as well as validity. The researchers will conduct focus groups with students to collect feedback on comprehensibility and appropriateness of items. For each measure, the researchers will assess the validity of score interpretations by examining content validity, student response processes, internal structure, and relations to other variables. The researchers will conduct think-aloud interviews during pilot testing to examine students' response processes, analyze the dimensionality of each of the six types of measures, and use differential item functioning analyses to determine whether items function similarly for different subgroups of students. The researchers will investigate the concurrent and predictive relations between students' scores on the algebra measures and other indicators of algebra proficiency, including standardized test performance, teacher ratings, and course grades. Item response theory techniques will be used to horizontally equate parallel forms of each measure to address measurement error.

Control Condition: There is no control condition.

Key Measures: The researchers will make use of a number of instruments to help assess the reliability and validity of the screening and progress monitoring measures. These include state and district achievement tests in math, teacher ratings, and course grades. The researchers will also administer the Iowa End of Course (IEOC) Assessment–Algebra I and the Balanced Assessment in Mathematics (BAM) as pre and post measures of algebra performance.

Data Analytic Strategy: The researchers will examine item-level response data for each administered measure to determine whether item difficulty and discrimination change over time. Progress monitoring data will be analyzed at the individual level using growth curve modeling and across sites using hierarchical linear modeling. The researchers will examine the relative growth rates obtained across the different traditional algebra measures. The researchers will also use regression analyses to compare student gains on the IEOC Algebra I and the BAM-Algebra to their rates of progress and final achievement levels using the progress monitoring data. Questionnaire data will be summarized using descriptive statistics and narrative data will be transcribed and coded.