Project Activities
Researchers will address the following aims. In Aim 1, the research team will conceptually update, align, and validate the separate teacher-, student-, and observer-report instruments. In Aim 2, the integrated CARES multi-rater assessment tool will be psychometrically validated. In Aim 3, the acceptability, feasibility, and utility of the CARES multi-rater assessment tool will be optimized for informing teacher professional development.
Structured Abstract
Setting
The multi-disciplinary multi-method research team will draw upon researchers at the University of Virginia and the University of Missouri, working in close partnership with schools in two districts in Maryland (Anne Arundel County Public Schools and Howard County Public Schools) and one district in Missouri (Jefferson City Public Schools).
Sample
Over four years, the CARES classroom assessment system will be iteratively refined and piloted with approximately 372 staff and 3,720 students sampled from 12 schools in Virginia and Missouri to be representative of diverse racial and ethnic compositions.
The CARES classroom assessment system will be a psychometrically valid and reliable, feasible and acceptable, and ultimately useful multi-informant tool that captures key domains and dimensions of culturally- and racial equity-sustaining classroom instruction and discipline practices. The final product will include three validated instruments (teacher-, student-, and observer-report), and accompanying resources and materials to support the interpretation and use of scores that result from the assessment. These accompanying materials will include a template for teacher-level score reports and interpretation guidance.
Research design and methods
Across the three project aims, researchers will complete a series of five studies using an iterative mixed methods design with rigorous qualitative and quantitative analyses to extend, refine, and streamline a multi-rater CARES classroom assessment system. In Study 1, the team will conduct cognitive interviews and focus groups with students, teachers and parents to examine content validity of newly developed CARES items focused on racial equity. In Study 2, researchers will complete a pilot of the multi-rater aligned and updated instruments to establish strong psychometric properties for each instrument, nomological validity, and convergent and divergent validity within the instrument framework and theory of change. In Study 3, the team will evaluate convergent validity when triangulating across raters in a second wave of pilot data collection. In Study 4, researchers will examine differential item functioning in various classroom racial and ethnic compositions of teacher and student (mis)match using a third wave of pilot data. In Study 5, the team will focus on feasibility and acceptability testing with end users (teachers).
Control condition
This project is not experimental; as such, there is no control condition.
Key measures
The key measures are the ones proposed for development and validation, the CARES teacher-, student- and observer-report instruments. Other measures include the student reported social-emotional, engagement, and academic outcomes (GPA, standardized test scores), and costs.
Data analytic strategy
Researchers will conduct mixed-method qualitative and quantitative data analyses for studies 1 and 5 to inform key revisions to the CARES classroom assessment system to improve its reliability, validity, and teacher receptivity to its use as a source of professional development feedback. In studies 2–4, the team will use a combination of factor analytic, item response theory (IRT), and multi-trait multi-method matrix approaches, including multilevel confirmatory factor analyses and moderated non-linear factor analyses.
Cost analysis strategy
The team will analyze costs of the CARES classroom assessment system training and administration using an "ingredients" approach and the CostOut program to track and estimate total costs for assessment, including observer training, administration, and score reporting.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The primary product is an integrated, multi-rater CARES classroom assessment system, which includes teacher-report, student-report, and observer-report instruments. The system will also include teacher-level score reports and interpretation guidance.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
