Project Activities
The core focus of the project is to develop and refine valid and reliable sets of short, highly focused diagnostic measurement tools (probes). Using an iterative design process, the researchers will create a set of questions designed to assess underlying misconceptions relating to mathematical operations with fractions and decimals. Students will answer these diagnostic questions and the researchers will follow up using structured interviews to determine whether the questions captured students' true misconceptions. Throughout the test development phase, the researchers will use various statistical analyses, including latent class analysis, to test the validity of their measures. In addition to these assessments, the team will develop and test an online interface that will record student responses, allow teachers to administer the probes electronically, and produce reports on each student's misconceptions.
Structured Abstract
Setting
The research will be conducted in schools with grades 6 to 8 in Maine, New Hampshire, and New York.
Sample
The participants include students in grades 6 to 8, and their mathematics teachers.
The core focus of the project is to develop and refine three valid and reliable sets of short, highly focused diagnostic measurement tools (probes). Each set will consist of four pairs of equivalent probes, each pair suitable as "pre-test" and "post-test" forms. Each assessment probe within the pre-post set will consist of 8–12 selected response items, with each item typically consisting of one accurate response and 2 or 4 inaccurate responses. The probe selected response items will be designed to target 1–3 specific misconceptions related to the probe topic (e.g., comparing 2 fractions).
Research design and methods
The development of the EM2 cognitive diagnostic assessment system will be a multi-stage process in which each stage builds on the knowledge gained across four stages of work. The first stage involves developing and refining each "pre-test" probe and response patterns, conducting a small scale pilot test, and revising and re-piloting each probe as necessary. The second stage will establish the discriminant validity, reliability, and content validity of each probe. The third stage will establish the equivalence of each "pre-test" probe and "post-test" probe pair. During the final stage, the team will develop and pilot test a web-based version of the EM2 cognitive diagnostic assessment system.
Control condition
There is no control condition.
Key measures
Key measures include a test of the EM2 probes and assessments used to determine the content, construct, and external validity, as well as the reliability, of multiple forms of probes designed to elicit particular student misconceptions.
Data analytic strategy
Data analysis will be conducted using latent class analysis (LCA), a subset of structural equation modeling. The researchers will use LCA to (a) estimate latent classes of misconceptions using student responses to closed-ended questions; (b) establish the equivalence of the two forms of each probe (pre-test and post-test) by confirming that the forms elicit identical misconception classes (both in terms of the number of classes and the response patterns for each class); (c) establish the discriminant validity and content validity of the classes; and (d) establish that the results of the LCA reliably diagnose student misconceptions.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
At the conclusion of the project, the team will have a developed EM2 system and a supporting online interface. Additionally, the research team will produce a series of scholarly publications.
Publications:
Journal article, monograph, or newsletter
Cramer, K., Monson, D., Ahrendt, S., Colum, K., Wiley, B., & Wyberg, T. (2015). 5 Indicators of Decimal Understandings. Teaching Children Mathematics, 22(3), 186-195.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
