Project Activities
The research team undertook a series of studies that built upon one another using qualitative data, discovery-oriented and systematic observations, surveys, and quantitative data analysis. The studies were designed to be a fully integrated mixed-model design. The team used both existing data from prior IES projects, and new data collected under this project. The researchers carried out six studies to define and specify discrete indicators of CRPs. These included qualitative interviews with students and teachers, discovery-oriented observations in classrooms, and item-response theory analyses of CRP observations and teacher and student surveys. Next, the research team validated and refined the observation and survey tools developed in the first stage of the project. Finally, researchers used discrete observation measures of CRPs to confirm and refine their hypothesized theory of change.
Structured Abstract
Setting
This project took place in school districts across Missouri and Maryland. Within and across the two states, the participating districts were from urban, rural, and suburban settings with high concentrations of diverse racial and ethnic groups. One study also used extant data from a study taking place in Kentucky.
Sample
The project included a series of studies with multiple samples. One study involved a focus group of approximately 104 middle and high school students. A second set of studies involved interviews with 11 teachers and 15 parents from diverse backgrounds. Another set of studies included training 8 parents and 14 instructional coaches to use the observation tools and provide feedback about them. A final study included 825 middle school students across 58 classrooms.
The Double Check Student Engagement and Cultural Proficiency Program (Double-Check) identifies five core components for CRPs: Connection to Curriculum, Authentic Relationships, Reflective Thinking, Effective Communication, and Sensitivity to Students' Culture. These components were the primary malleable factors of interest in this project; other malleable factors were explored in relation to these include teacher-student relations and student autonomy, engagement, and self-efficacy.
Research design and methods
For this project, the research team took advantage of an existing network of researchers, educators, and practitioners, who were working collaboratively to improve CRPs. In the first set of studies, the team completed a series of qualitative interviews with students, teachers, parents, and administrators. Next, the team identified classrooms deemed "typical" and "exemplary" with respect to use of CRP and then conducted discovery-oriented observations. Next, the team analyzed extant observational data using their emerging observation coding system. The final study involved confirming and refining their hypothesized theory of change using path analysis. Across all studies, the team used an iterative approach to determine the malleable indicators of CRPs, refine measures of CRPs that provide objective feedback to teachers and guide professional development, and examined the hypothesized links between CRPs, student-teacher relations, student self-efficacy and engagement, and student academic and behavioral outcomes.
Control condition
Due to the exploratory nature of the research design, there was no control condition.
Key measures
Teacher self-report measures included the Double-Check Self-Reflection Tool, the Multicultural Efficacy Scale, and the Culturally-Responsive Teaching Self-Efficacy Scale. Classroom observation tools included adapted versions of the ASSIST Culture, the Brief Classroom Interaction Observation Revised (BCIO-R) and the Student Classroom Interaction Observation(STCIO). The final tool that was developed based on the overall project is the CARES 360 observation tool.
Data analytic strategy
The researchers use a grounded theory approach to code focus group data and observational data. They also used item response theory to calibrate and refine student and teacher surveys and observation protocols. They use multi-level path analyses to examine the relationship between CRPs and student achievement and behavioral outcomes.
Key outcomes
The main findings of this project are as follows:
- Improvements were made to the observational tools and surveys for schools and teachers to use to improve culturally responsive practices based on the findings from five qualitative studies with students, teachers (Debnam et al., 2023; Smith et al., 2024), administrators (Aguayo et al., 2023), and parents.
- The researchers developed the CARES 360 observation tool, a direct observation of the key dimensions of culturally-sustaining classroom practices, which now includes 12 dimensions and 36 indicators and a virtual training program to support it use (Franco et al., 2023).
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Publications:
Aguayo, D., Good, M. W., Diem, S., Herman, K. C., Burke, J., Davis, T., Hall, K., London, C., & Reinke, W. M. (2023). Promoting district-level culturally responsive practices. Educational Administration Quarterly, 59(3), 471-506.
Bottiani, J. H., McDaniel, H. L., Henderson, L., Castillo, J. E., & Bradshaw, C. P. (2020). Buffering effects of racial discrimination on school engagement: The role of culturally responsive teachers and caring school police. Journal of School Health, 90(12), 1019-1029.
Bottiani, J. H., Duran, C. A., Pas, E. T., & Bradshaw, C. P. (2019). Teacher stress and burnout in urban middle schools: Associations with job demands, resources, and effective classroom practices. Journal of School Psychology, 77, 36-51.
Debnam, K. J., Smith, L. H., Aguayo, D., Reinke, W. M., & Herman, K. C. (2023). Nominated exemplar teacher perceptions of culturally responsive practices in the classroom. Teaching and Teacher Education, 125, 104062.
Franco, M. P., Bottiani, J. H., & Bradshaw, C. P. (2024). Teachers' structuring of culturally responsive social relations and secondary students’ experience of warm demand. Contemporary Educational Psychology, 76, 102241.
Franco, M. P., Bottiani, J. H., & Bradshaw, C. P. (2023). Assessing Teachers’ Culturally Responsive Classroom Practice in PK–12 Schools: A Systematic Review of Teacher-, Student-, and Observer-Report Measures. Review of Educational Research, 00346543231208720.
Gaias, L. M., Johnson, S. L., Bottiani, J. H., Debnam, K. J., & Bradshaw, C. P. (2019). Examining teachers' classroom management profiles: Incorporating a focus on culturally responsive practice. Journal of School Psychology, 76, 124-139.
Kaihoi, C. A., Braun, S. S., Bottiani, J. H., & Bradshaw, C. P. (2023). Contextual factors contributing to variability within middle school teachers' observed classroom management and student engagement. Psychology in the Schools, 60(10), 4117-4142.
Smith, L., Debnam, K., Aguayo, D., Pandey, T., Reinke, W. M., & Herman, K.C. (in press; 2024). How can teachers improve? Using culturally responsive frameworks to examine adolescent perspectives. Journal of Adolescent Research.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
