|Title:||The Day Reconstruction Method: A New Tool for Measuring Teachers' Work and Work Contexts|
|Principal Investigator:||Camburn, Eric||Awardee:||University of Wisconsin, Madison|
|Program:||Effective Teachers and Effective Teaching [Program Details]|
|Award Period:||4 years (8/1/2016-7/31/2020)||Award Amount:||$1,376,008|
Co-Principal Investigators: Nathan Jones (Boston University), Benjamin Kelcey (University of Cincinnati), Esther Quintero-Corral (Albert Shanker Institute)
Purpose: This project is intended to develop a measure of teachers' workplaces and how teachers spend their workday. Building on a small-scale pilot study that described teacher work activities, the social context of their work, and their affect while working, researchers propose to develop and validate the Day Reconstruction Method (DRM) measure. The instrument is intended to capture the simultaneous occurrence of these constructs, thus potentially yielding insight into aspects of teachers' work that have historically been difficult to study, such as how teachers' affect varies by activity and by context, the frequency of different activities when working directly with students, and teachers' affect while working with students. When aggregated to the school level, DRM data may provide diagnostic information about schools' organizational functioning and climate.
Project Activities: The study will take place in three phases: instrument development (Year 1), pilot study (Year 2), and validation study (Year 3), with each phase building upon prior phases. Year 4 will be devoted to data analysis and dissemination of study findings. During the pilot study, researchers will attempt to identify optimal field procedures for collecting DRM data. The team will also investigate teacher compliance with DRM response protocols and whether and how teacher responses to the DRM change after completing the instrument multiple times. Via the validation study, the research team will assess construct validity by investigating the factor structure of DRM measures of teacher affect. Researchers will also assess the validity of the DRM through multiple tests of concurrent validity and predictive validity that correlate DRM data with data from an experience sampling instrument, a teacher survey, and student achievement data.
Products: Researchers will produce a fully developed and validated measure for teacher work activities, the social context of their work, and their affect while working. They will also produce peer-reviewed publications.
Setting: This study will take place in a large suburban district in Connecticut and a small urban district in New Hampshire.
Sample: Approximately 20 K–12 teachers from 3 to 4 schools will participate in the pilot study in Year 2. Three hundred K–12 teachers will participate in the validation study in Year 3.
Assessment: The Day Reconstruction Method (DRM) measure is intended to capture teacher work activities and time use devoted to those activities during their workdays, how teachers feel during those activities, and the characteristics of the contexts in which activities occur. This information is captured in two steps. First, teachers outline the episodes of activity they engaged in during a specific day. This exercise is intended to enhance memory about the specifics of each episode. In a second step, teachers answer a brief questionnaire in which they record the details about each episode.
Research Design and Methods: The proposed study will take place in three phases: instrument development (Year 1), pilot study (Year 2), and validation study (Year 3), with each phase building upon prior phases. During the instrument development phase, researchers will use cognitive interviews, observations, and follow-up interviews with teachers and expert review. During the pilot study phase, all teachers in the pilot will complete the DRM, experience sampling method (a predecessor to the DRM and a psychometrically valid tool that randomly samples how people feel during activities during the day), teacher survey instruments, and the shadowing observations. Researchers will use the pilot study to identify optimal field procedures for collecting DRM data. In the pilot phase, the team will also investigate teacher compliance with DRM response protocols and whether and how teacher responses to the DRM change after completing the instrument multiple times. During the validation study phase, researchers will conduct tests of three kinds of validity of the DRM instrument: construct validity, concurrent validity, and predictive validity. Researchers will assess construct validity by investigating the factor structure of DRM measures of teacher affect. The team will also assess the validity of the DRM through multiple tests of concurrent validity and predictive validity that correlate DRM data with data from an experience sampling instrument, shadowing observations of teachers, a teacher survey, and student achievement data.
Control Condition: There is no control condition in this study.
Key Measures: Teacher measures include the DRM, experience sampling method (ESM), teacher self-report surveys (e.g., assessing teachers' affect, divvying of their daily work into specific activities, extent to which teachers are aligned with the goals and beliefs of their colleagues, the levels of trust among teachers, and the degree that collective responsibility likely influences the degree of informal and formal collaboration among teachers throughout the school, burnout and commitment), and the shadowing observations. Researchers will measure student achievement using the Smarter Balanced assessments.
Data Analytic Strategy: Researchers will conduct multilevel confirmatory factor analyses and multitrait multi-method approach to examine the multilevel composite reliability, factorial validity, concurrent validity, predictive validity, and discriminant validity of the measure. In addition to analyzing DRM, ESM and teacher survey data from the validation study, researchers will also conduct descriptive analyses of the shadow observation data.