Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Eliciting Mathematics Misconceptions (EM2): A Cognitive Diagnostic Assessment System
Center: NCER Year: 2011
Principal Investigator: Buffington, Pamela Awardee: Education Development Center, Inc.
Program: Cognition and Student Learning      [Program Details]
Award Period: 4 years Award Amount: $1,597,694
Goal: Measurement Award Number: R305A110306

Co-Principal Investigator: Margaret Clements

Purpose: Understanding rational numbers is an important prerequisite for success in higher level mathematics, yet many students and adults often struggle with such concepts. In particular, they have great difficulty with fractions, decimals, and percents. Despite this, there are currently few assessments that allow teachers to determine students' misconceptions. The purpose of the Eliciting Mathematics Misconceptions (EM2) Project is to address this need by applying recent advances in cognitive science and mathematics education research to develop EM2, a cognitive diagnostic assessment system appropriate for use in a variety of classroom settings and available online. This system will enable teachers to quickly and effectively diagnose commonly held student misconceptions and overgeneralizations in the areas of fractions, decimals, and operations with fractions and decimals.

Project Activities: The core focus of the project is to develop and refine valid and reliable sets of short, highly focused diagnostic measurement tools (probes). Using an iterative design process, the researchers will create a set of questions designed to assess underlying misconceptions relating to mathematical operations with fractions and decimals. Students will answer these diagnostic questions and the researchers will follow up using structured interviews to determine whether the questions captured students' true misconceptions. Throughout the test development phase, the researchers will use various statistical analyses, including latent class analysis, to test the validity of their measures. In addition to these assessments, the team will develop and test an online interface that will record student responses, allow teachers to administer the probes electronically, and produce reports on each student's misconceptions.

Products: At the conclusion of the project, the team will have a developed EM2 system and a supporting online interface. Additionally, the research team will produce a series of scholarly publications.

Structured Abstract

Setting: The research will be conducted in schools with grades 6 to 8 in Maine, New Hampshire, and New York.

Population: The participants include students in grades 6 to 8, and their mathematics teachers.

Intervention: The core focus of the project is to develop and refine three valid and reliable sets of short, highly focused diagnostic measurement tools (probes). Each set will consist of four pairs of equivalent probes, each pair suitable as "pre-test" and "post-test" forms. Each assessment probe within the pre-post set will consist of 8–12 selected response items, with each item typically consisting of one accurate response and 2 or 4 inaccurate responses. The probe selected response items will be designed to target 1–3 specific misconceptions related to the probe topic (e.g., comparing 2 fractions).

Research Design and Methods: The development of the EM2 cognitive diagnostic assessment system will be a multi-stage process in which each stage builds on the knowledge gained across four stages of work. The first stage involves developing and refining each "pre-test" probe and response patterns, conducting a small scale pilot test, and revising and re-piloting each probe as necessary. The second stage will establish the discriminant validity, reliability, and content validity of each probe. The third stage will establish the equivalence of each "pre-test" probe and "post-test" probe pair. During the final stage, the team will develop and pilot test a web-based version of the EM2 cognitive diagnostic assessment system.

Control Condition: There is no control condition.

Key Measures: Key measures include a test of the EM2 probes and assessments used to determine the content, construct, and external validity, as well as the reliability, of multiple forms of probes designed to elicit particular student misconceptions.

Data Analytic Strategy: Data analysis will be conducted using latent class analysis (LCA), a subset of structural equation modeling. The researchers will use LCA to (a) estimate latent classes of misconceptions using student responses to closed-ended questions; (b) establish the equivalence of the two forms of each probe (pre-test and post-test) by confirming that the forms elicit identical misconception classes (both in terms of the number of classes and the response patterns for each class); (c) establish the discriminant validity and content validity of the classes; and (d) establish that the results of the LCA reliably diagnose student misconceptions.