Evaluation of Organization Skills Training Program for Upper Elementary Students
Co-Principal Investigator: Jenelle Nissley-Tsiopinis and Jennifer Mautone
Purpose: The purpose of this study is to determine the efficacy of a school-based organizational skills training program (OST-S) for students in 3rd through 5th grade with deficits in organization, time management, and planning (OTMP), key executive function skills that support success in school. The OST-S intervention was adapted from an effective clinic-based organizational skills training intervention for children diagnosed with Attention Deficit Hyperactivity Disorder (ADHD) and has been piloted in several schools as a small group intervention provided by school staff. This study addresses student OTMP challenges at a time when the demand for these skills increases at school, and fills a gap in service by considering whether schools can take a more proactive approach to supporting students with these executive function challenges before they move on to middle school where these demands become even greater.
Project Activities: The research team will randomly assign schools to implement OST-S or to continue with their typical practices. Student outcomes will be assessed immediately following intervention and 5 and 12 months after baseline assessment to determine the short- and long-term impacts of the program. The team will also explore whether improvements in OTMP skills and academic self-efficacy mediate impact of OST-S on academic outcomes. They will also examine whether variations in intervention implementation (treatment fidelity, participant engagement) or child characteristics (special education status, attention, behavioral functioning) moderate these outcomes.
Products: The products of this project include evidence of the efficacy of the school-based organizational skills training program (OST-S) to improve the organization, time management, and planning skills (OTMP) of 3rd to 5th grade students, and whether these improvements lead to improved academic self-efficacy and performance. The research team will also produce peer reviewed publications.
Setting: The participating schools are located in suburban and urban school districts in Pennsylvania and New Jersey.
Sample: In total, 280 students in 3rd, 4th, and 5th grade with OTMP deficits identified by their teachers will participate across 20 schools.
Intervention: The OST-S program is delivered by school personnel to small groups of three to four students. The program content is organized into 18 sessions that address four organizational challenges: tracking assignments, managing materials, managing time, and planning for long term assignments. Each group session includes homework review to assess completion of between-session skills implementation; modeling, shaping, guided practice, and reinforcement for organized behavior; and activities to promote generalization of skills. Session content is designed to promote student academic self-efficacy by encouraging students to believe that they can become more organized by overcoming "glitches," defined as mental lapses that cause performance errors. The research team will train school personnel to deliver the OST-S small group intervention in two full-day training sessions followed by regular consultation and coaching. Parents and teachers meet with the small group leader to learn about the purpose of the program, work collaboratively with the group leader to identify specific goals for the child's OTMP skills, and learn ways to reinforce OTMP skills at home and in the classroom.
Research Design and Methods: The research team will randomly assign schools within matched pairs to implement OST-S or continue with their typical practices. Schools will participate in two waves, with half of the schools participating in Years 1 and 3, and the other half participating in Years 2 and 4. During each year that a school is participating, the research team will recruit eligible students (determined through teacher recommendation and rating scales) to form two small groups (fall and winter) for a total of four groups of students from each of the 20 schools. Student outcomes will be assessed immediately following intervention and again at 5 and 12 months after baseline assessment.
Control Condition: This study uses a wait-list control group with schools randomly assigned to the control condition continuing their typical practices until final follow-up data is collected, at which time the OST-S intervention will be provided to eligible students in these schools by research staff.
Key Measures: Researchers will measure children's OTMP skills via parent, teacher, and child report using the Children's Organizational Skills Scale (COSS) and organization and planner checklists completed by research staff. The research team will measure academic self-efficacy using the self-efficacy subscale of the Patterns of Adaptive Learning Scales. They will assess academic functioning through a variety of measures, including teacher report (Academic Proficiency Scale, Academic Competence Evaluation Scales, Homework Performance Questionnaire-Teacher), parent report (Homework Problem Checklist), and scoring of homework and classwork samples by the researchers. Researchers will assess student characteristics using the Behavior Assessment System for Children (BASC-3), school records, and parent report. The research team will code videos of child group sessions and audiotapes of parent and teacher consultation meetings to assess implementation fidelity and participant engagement.
Data Analytic Strategy: The research team will use linear mixed effects models to determine impact of the OST-S intervention on student outcomes and possible moderation of impact by variations in intervention implementation (e.g., treatment fidelity) and child characteristics (e.g., special education status, behavioral functioning). They will also explore whether reductions in OTMP deficits and improvements in academic self-efficacy mediate the effect of OST-S on academic outcomes using marginal structural models and simulation-based methods. Finally, they will determine the incremental cost effectiveness (ICE) for each significant outcome by constructing an ICE ratio (differences in mean cost for each group divided by the mean difference in outcomes).