Dylan Wiliam, Educational Testing Service
Caroline Wylie, Educational Testing Service
Abstract: The Diagnostic Items in Mathematics and Science (DIMS) project is focused on helping teachers make real-time adjustments to instruction to direct student learning. The project has created a bank of multiple-choice questions where both correct and incorrect responses are interpretable. A correct answer is interpretable when there is a strong warrant of student understanding of a particular concept. An incorrect answer is interpretable when it provides specific insight into students' thinking that led to the incorrect response. Teachers use individual diagnostic questions to gather immediate information from all students in a class, without having to collect and grade papers, so that the evidence of student thinking can be used to make real-time instructional adjustments to meet the specific learning needs of students.
Pilot data from 50 teachers indicate that the teachers are able to use the questions to both inform immediate instructional steps, and to provide rich discussion opportunities. From an analysis of teachers' comments a continuum of usage is emerging, from basic to sophisticated use of the diagnostic questions. The basic use was characterized by no acknowledgement that the evidence from the diagnostic question had an influence on the real-time instructional decisions and no mention even that evidence was generated. A more sophisticated use of the questions was characterized by use of the evidence generated from the diagnostic questions to influence real-time decision making such as guiding differentiated small group work, or whole class instruction, or providing a focus for an immediate whole class discussion.