Project Activities
The research team developed and validated an assessment tool to target sources of difficulties and misconceptions in middle school geometry, specifically properties of shapes, transformations, and measurement. Unlike current achievement and diagnostic tests which provide information about a students' ability within a given domain, the Diagnostic Geometry Assessment diagnosed why students struggle with a given geometric concept and provided teachers with instructional strategies and resources designed to address the targeted difficulties and misconceptions.
Structured Abstract
Setting
The research will be conducted with schools of varying socioeconomic profiles from across the United States.
Sample
The research participants will be 7th- and 8th-grade mathematics teachers and their students.
The assessment will target sources of difficulties and misconceptions in middle school geometry, specifically properties of shapes, transformations, and measurement. Unlike current achievement and diagnostic tests which provide information about a students' ability within a given domain, the Diagnostic Geometry Assessment will diagnose the reason(s) why students struggle with a given geometric concept, and it will provide teachers with instructional strategies and resources designed to address the targeted conceptions.
Research design and methods
The research team will conduct studies to examine the psychometric properties of the assessment, as well as the validity of inferences generated by the assessment of students' geometric conceptions. Research will be conducted both face-to-face and online. In the final pilot study of the assessment, 80 teachers will be randomly assigned to use the new assessment tool under one of four conditions: traditional feedback; traditional feedback and diagnostic feedback; traditional feedback, diagnostic feedback, and access to instructional resources; and traditional feedback and access to instructional resources.
Control condition
Teachers in the final pilot study who are randomly assigned to the control condition will receive only traditional feedback from the Diagnostic Geometry Assessment (i.e., the percent of correct responses and item-level information that focuses on whether the student responded correctly) and will not receive diagnostic feedback or access to the instructional resources.
Key measures
Key outcomes from the development and validation of the assessment include data indicating the reliability, unidimensionality, and the content-related, construct-related, criterion-related, and consequential-related validity of the assessment. Validity studies will examine the consistency of information provided by the Diagnostic Geometry Assessment by comparing that information to similar measures of geometric conceptions, and also through interview teaching using cognitive probes.
Data analytic strategy
Psychometric analyses will include test reliability (using Cronbach's alpha), factor analysis, and one and three parameter item response analyses. Data from the validity studies will be coded with emergent coding method using Cohen's Kappa to define acceptable agreement rates.
People and institutions involved
IES program contact(s)
Products and publications
Publications:
Masters, J. (2010). Automated Scoring of an Interactive Geometry Item: A Proof-of-Concept. The Journal of Technology, Learning and Assessment, 8(7): 1-39.
Shear, B. R., & Roussos, L. A. (2017). Validating a distractor-driven geometry test using a generalized diagnostic classification model. Understanding and Investigating Response Processes in Validation Research, 277-304.
Additional project information
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
