Project Activities
The research will be conducted in four phases. In phase 1, the researchers will focus on designing the measure using participatory involvement from diverse stakeholders (students, caregivers, secondary school staff). In phase 2, they will use field testing to establish the factor structure and internal consistency of the measure in a sample of middle and high school students from four geographic regions (the Pacific Northwest, Southern Florida, the Midwest, and the Mid-Atlantic). In phase 3, they will focus on obtaining multi-informant national norms for the SPOTS in a representative sample of adolescents as well as performing invariance testing across diverse groups of learners. Lastly, in phase 4, they will conduct a cost analysis, finalize the SPOTS technical manual, build a website, and disseminate findings from the project.
Structured Abstract
Setting
Phases 1 and 2 will include participants from four geographic regions (the Pacific Northwest, south Florida, the Midwest, and the Mid-Atlantic). Phase 3 will involve a nationally representative sample of participants from across the U.S.
Sample
Participants will include secondary school students, teachers, and caregivers. The researchers will use a purposive sampling approach to ensure a diverse sample across different categories, including by race/ethnicity, sex, school population served, stakeholder type, grade level, geographic region, and participants with and without OTMP difficulties.
The researchers will develop and validate the Student Planning, Organization, and Time Management Scale (SPOTS), a multi-informant measure of organization, time management, and planning (OTMP) behaviors for use with secondary school students. Unlike other existing measures, SPOTS will be developed, normed, and validated specifically for secondary school students, and provides behavior-based measurement (i.e., a focus on students' observable, real-world functioning) to directly inform decision making about students' intervention needs and optimal intervention/accommodation targets (e.g., teaching time management, allowing extended time to complete assignments, permitting remedial organization tools).
Research design and methods
This 3-year study will use a rigorous, iterative design to ensure a user-ready, multi-informant SPOTS measure by the end of the funding period. The researchers will conduct the study in four phases. In phase 1, they will focus on generating the measure item pool, refining the measure, conducting focus groups and an expert summit, and a pilot test of the SPOTS. In phase 2, they will recruit secondary school student-caregiver-teacher triads in four geographically diverse regions of the U.S. to field test the SPOTS to establish its factor structure and reliability. In phase 3, they will recruit secondary school student-caregiver-teacher triads to obtain multi-informant normative information for the SPOTS in a nationally representative sample. They will have a subsample of the national sample complete an enhanced battery of measures to establish the convergent, discriminant, predictive, and incremental validity of the SPOTS. In phase 4, they will conduct a cost analysis of the SPOTS assessment.
Control condition
Due to the nature of the research design, there is no control condition in this study. However, students with and without elevated OTMP problems will be compared in the full nationally representative sample, in addition to other relevant subgroups of students including middle- vs. high-school students, sex, White students vs. Black and/or Latinx students, and free/reduced lunch status.
Key measures
The researchers will use several validity measures including the Behavior Rating Inventory of Executive Function Second Edition, Expectancy Value Theory of Motivation Measure, Vanderbilt ADHD Diagnostic Rating Scale, and Impairment Rating Scale. They will also collect observed OTMP behaviors via schools' online learning support platforms and end-of-year grades.
Data analytic strategy
In phase 2, the researchers will use qualitative observational and focus group data and feedback from expert stakeholders to establish construct validity and inform SPOTS development. In phase 2, they will conduct exploratory and confirmatory factor analyses, item response theory analyses, and reliability analyses. In phase 3, they will focus on norming of the multi-informant SPOTS, in addition to invariance analyses, differential item function analyses, test-retest reliability analyses, and discriminative, convergent, concurrent, and predictive validity analyses.
Cost analysis strategy
In phase 4, the researchers will conduct a cost analysis using the ingredients approach to estimate costs for educators and/or school mental health professionals using the final SPOTS assessment. The quantity of inputs over the course of measure administration and scoring will be accounted (i.e., administration time, school staff time scoring, cost of hosting the online components of the measure/scoring system). To promote generalizability, the researchers will obtain wage estimates for school staff from the Occupational Employment Statistics data from U.S. Bureau of Labor Statistics (rather than local educational agencies). They will compare costs of the SPOTS measure to existing measures to determine any incremental costs associated with the SPOTS measures versus current practices.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
This project will yield a fully developed, validated, and normed multi-informant assessment of secondary students' organization, time management, and planning behaviors. The project will result in publications and presentations and other dissemination products (e.g., website, webinar) that will reach educators, school mental health professionals, and researchers.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
