Project Activities
The study will take place in three phases: instrument development (Year 1), pilot study (Year 2), and validation study (Year 3), with each phase building upon prior phases. Year 4 will be devoted to data analysis and dissemination of study findings. During the pilot study, researchers will attempt to identify optimal field procedures for collecting DRM data. The team will also investigate teacher compliance with DRM response protocols and whether and how teacher responses to the DRM change after completing the instrument multiple times. Via the validation study, the research team will assess construct validity by investigating the factor structure of DRM measures of teacher affect. Researchers will also assess the validity of the DRM through multiple tests of concurrent validity and predictive validity that correlate DRM data with data from an experience sampling instrument, a teacher survey, and student achievement data.
Structured Abstract
Setting
This study will take place in a large suburban district in Connecticut and a small urban district in New Hampshire.
Sample
Approximately 20 K-12 teachers from 3 to 4 schools will participate in the pilot study in Year 2. Three hundred K-12 teachers will participate in the validation study in Year 3.
The Day Reconstruction Method (DRM) measure is intended to capture teacher work activities and time use devoted to those activities during their workdays, how teachers feel during those activities, and the characteristics of the contexts in which activities occur. This information is captured in two steps. First, teachers outline the episodes of activity they engaged in during a specific day. This exercise is intended to enhance memory about the specifics of each episode. In a second step, teachers answer a brief questionnaire in which they record the details about each episode.
Research design and methods
The proposed study will take place in three phases: instrument development (Year 1), pilot study (Year 2), and validation study (Year 3), with each phase building upon prior phases. During the instrument development phase, researchers will use cognitive interviews, observations, and follow-up interviews with teachers and expert review. During the pilot study phase, all teachers in the pilot will complete the DRM, experience sampling method (a predecessor to the DRM and a psychometrically valid tool that randomly samples how people feel during activities during the day), teacher survey instruments, and the shadowing observations. Researchers will use the pilot study to identify optimal field procedures for collecting DRM data. In the pilot phase, the team will also investigate teacher compliance with DRM response protocols and whether and how teacher responses to the DRM change after completing the instrument multiple times. During the validation study phase, researchers will conduct tests of three kinds of validity of the DRM instrument: construct validity, concurrent validity, and predictive validity. Researchers will assess construct validity by investigating the factor structure of DRM measures of teacher affect. The team will also assess the validity of the DRM through multiple tests of concurrent validity and predictive validity that correlate DRM data with data from an experience sampling instrument, shadowing observations of teachers, a teacher survey, and student achievement data.
Control condition
There is no control condition in this study.
Key measures
Teacher measures include the DRM, experience sampling method (ESM), teacher self-report surveys (e.g., assessing teachers' affect, divvying of their daily work into specific activities, extent to which teachers are aligned with the goals and beliefs of their colleagues, the levels of trust among teachers, and the degree that collective responsibility likely influences the degree of informal and formal collaboration among teachers throughout the school, burnout and commitment), and the shadowing observations. Researchers will measure student achievement using the Smarter Balanced assessments.
Data analytic strategy
Researchers will conduct multilevel confirmatory factor analyses and multitrait multi-method approach to examine the multilevel composite reliability, factorial validity, concurrent validity, predictive validity, and discriminant validity of the measure. In addition to analyzing DRM, ESM and teacher survey data from the validation study, researchers will also conduct descriptive analyses of the shadow observation data.
Key outcomes
The main outcomes of the validation study are as follows:
Stark, Jones, Camburn & Kaler (2023) examined experience sampling method data from the study. They found that teachers experienced positive affect during work much more frequently than negative affect and that variation in teachers’ affect distributed roughly equally between teachers and between daily moments within teachers. Teachers had greater levels of positive affect during instruction compared to other activities.
Jones et al. (2022) is one of the only studies to date providing direct evidence of changes in teachers’ work during the early months of the pandemic. They found that the amount of time teachers spent providing instruction to students dramatically dropped from 214 minutes per day pre-pandemic to 61 minutes per day post-pandemic and that time spent planning and grading/assessment increased substantially during this period. While positive feelings during work declined overall, positive feelings while directly teaching students increased substantially during distance learning.
Preliminary analyses reported in the study’s final report to IES indicate that on a given school day, the DRM captured activities reported by teachers on the ESM and activities observed by researchers quite accurately. Capture rates for the ESM ranged from 94% to 98% and rates for classroom observations ranged from 76% to 93%. Because it captures how teachers feel right as they are experiencing a feeling, the ESM arguably provides a strong basis of validity comparison for measures of affect. DRM were positively correlated with ESM measures of positive affect at the minute (.64) and day (.72) levels. Overall, data from the two instruments (ESM, DRM) corresponds with each other well.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Researchers will produce a fully developed and validated measure for teacher work activities, the social context of their work, and their affect while working. They will also produce peer-reviewed publications.
Additional project information
Previous award details:
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
