Project Activities
The researchers will develop a diagnostic measure for numbers (i.e., whole numbers and fractions) and operations, which encompasses a major content area of the elementary school mathematics curriculum. In phase 1, the researchers will focus on item development and think-aloud interviews. Based on existing research and teacher knowledge frameworks of content and pedagogical content knowledge needed for teaching numbers and operations, the researchers will define and operationalize the attributes, develop items, and conduct a response process validity study by conducting think-aloud interviews with 40 teachers. In phase 2, they will focus on measurement validity and reliability using a nationally representative sample of 600 elementary school teachers. Based on validity and reliability results, they will revise the assessment. In phase 3, they will examine the predictive validity, classification validity, and convergent validity of the measure based on a local sample. In phase 4, they will finalize and post the measure with training materials online.
Structured Abstract
Setting
The study will include a national sample of elementary school teachers located across the United States. The study will also take place in urban elementary schools in California.
Sample
The study will include two samples. The first sample will include 600 elementary school teachers that are nationally representative in terms of their educational and demographic background. The second sample will include 200 elementary school teachers in grades 4 through 5 that teach a diverse group of students in terms of ethnicity and free or reduced-price lunch status.
The researchers will develop a diagnostic measure for use with elementary school teachers. This measure will produce valid and reliable diagnostic feedback for teacher educators and researchers of teacher content and pedagogical content knowledge of numbers and operations. The researchers will draw content and pedagogical content knowledge attributes from literature around the knowledge and skills needed for teaching these topics. The tool, which will be administered online to teachers, will assess their understanding of these key constructs in numbers and operations so that educators and researchers can identify the specific areas where support is needed and design professional development accordingly.
Research design and methods
The researchers will develop and administer the assessment, then conduct several validity studies during the assessment development and testing process. In phase 1, the researchers will define and operationalize the attributes, develop items, and conduct a response process validity study by conducting think-aloud interviews with 40 teachers. In phase 2, the researchers will administer the diagnostic assessment to approximately 600 elementary school teachers. After administration, the researchers will analyze the assessment data and conduct several item and test validity and reliability evaluations and revise the assessment based on results. In phase 3, the researchers will use a local sample to examine the prediction validity by linking teacher scores on this assessment to their student achievement scores, as well as evaluate classification validity and convergent validity. In phase 4, the assessment will be finalized and posted with training materials online.
Control condition
There is no control condition for this study.
Key measures
For convergent validity, the researchers will administer a previously developed assessment on teacher content knowledge of numbers and operations and compare it with teacher scores on the instrument developed in this project. For predictive validity, they will measure teacher mathematics instruction by coding rubrics applied to classroom artifacts (that is, assignments, student responses to those assignments, and teacher rationale for these assignments and approach to evaluating student learning based on their responses). They will measure student math learning using student assessment data from the partner district that has agreed to share the deidentified student data of their teachers.
Data analytic strategy
The researchers will analyze the assessment data by using a diagnostic classification model. They will use several methods for validation, including multiple regression, multilevel modeling, and qualitative analysis (think-aloud protocols).
Cost analysis strategy
The researchers will follow the ingredients method and use national costs for each of the ingredients. The ingredients, or resources, required for implementation of the final version of the measure fall primarily into four categories: personnel, facilities, materials, and other inputs. The largest costs will likely be in teacher time in completing the assessment and coder time required for training and for subsequently scoring the assessments. Materials will likely make up the next largest portion of the costs, which will include training materials, costs of computers, and internet access per user. These latter costs will be adjusted to account for the proportional use of the computer and internet access.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The project will result in a diagnostic assessment tool as well as peer-reviewed publications, presentations at national conferences and to mathematics teacher educators and researchers, and workshops geared towards mathematics educators and mathematics education researchers.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Supplemental information
Co-Principal Investigators: Kho, Adam; Madison, Matthew
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
