Skip Navigation
Middle School Mathematics Professional Development Impact Study Findings After the Second Year of Implementation

NCEE 2011-4024
May 2011

Data Collection and Outcome Measures

Data were collected from teachers and students in the study schools in fall and spring of the 2007–2008 and 2008–2009 school years. The two main outcome measures used in the second year of the study were constructed as follows:

  • Teacher knowledge test. Teacher knowledge was measured for all treatment and control teachers using a test constructed specifically for the study. The test consisted of multiple-choice and short-response items that were designed to measure knowledge of rational number topics. Three alternate forms of the test were administered so that individual teachers would receive different forms (i.e., different items) at each administration. In addition to a total score, the teacher knowledge test yielded two subscores for each participant, aligned with the two types of knowledge that were targeted by the PD: common knowledge of mathematics (CK) and specialized knowledge of mathematics for teaching (SK).8
  • Student achievement test. A customized, computer-adaptive rational number test was constructed for the study by the Northwest Evaluation Association (NWEA). The NWEA Rational Number Test was restricted to positive rational number content and drew on a customized item bank of nearly 1,200 rational number items abstracted from the larger NWEA item bank of scaled, operational mathematics items.9

We also surveyed treatment and control teachers to gather data on their professional backgrounds and on the amount and type of PD in mathematics they participated in during the two-year study period. Study staff obtained information on the implementation of the PD by collecting attendance records, observing the institutes and seminars, and reviewing logs maintained by coaches that recorded the nature and extent of each coach interaction with each teacher. Three Item Response Theory (IRT)-based scores were computed for each participant: a total score, a fractions and decimals score, and a ratio and proportion score.

8 CK is the knowledge of topics in rational numbers that students should ideally have after completing the seventh grade. This knowledge includes computational or procedural skills, conceptual understanding, and problem-solving skills in rational number topics. SK is the additional knowledge of rational numbers that may be useful for teaching rational number topics.
9 Each individual student was presented with 30 items from the customized item base, chosen adaptively from four topic areas: fractions (11 items), decimals (4 items), percents (4 items), and ratios/proportions (11 items). Within each topic area, items were selected for presentation in a manner that ensured distribution across the cognitive categories of concepts, operations, and applications. To aid interpretation of the total score results, NWEA also constructed customized, seventh-grade norms by reanalyzing data from its Growth Research Database—a large database compiled from NWEA testing (NWEA 2003).