Can Universal SEL Programs Benefit Universally? Effects of the Positive Action Program on Multiple Trajectories of Social-Emotional and Misconduct Behaviors
Duncan, Robert; Washburn, Isaac J.; Lewis, Kendra M.; Bavarian, Niloofar; DuBois, David L.; Acock, Alan C.; Vuchinich, Samuel; Flay, Brian R. (2016). Retrieved from: https://eric.ed.gov/?id=ED590411
-
examining3,960Students, grades3-8
Single Study Review
Review Details
Reviewed: October 2022
- Single Study Review (findings for Positive Action (PA))
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a cluster randomized controlled trial with low cluster-level attrition that provides evidence of effects on clusters by demonstrating that the analytic sample of individuals is representative of the clusters.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Illinois Standard Achievement Test (ISAT) Reading |
Positive Action (PA) vs. Business as usual |
0 Days |
Full sample (all students in grades 3-8);
|
N/A |
N/A |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Illinois Standard Achievement Test (ISAT) Math |
Positive Action (PA) vs. Business as usual |
0 Days |
Full sample (all students in grades 3-8);
|
N/A |
N/A |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Absenteeism rate |
Positive Action (PA) vs. Business as usual |
0 Days |
Full sample (all students in grades 3-8);
|
N/A |
N/A |
Yes |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
School-level disciplinary referrals |
Positive Action (PA) vs. Business as usual |
0 Days |
Full sample (all students in grades 3-8) ;
|
73.22 |
137.44 |
Yes |
|
|
|
School-level suspensions |
Positive Action (PA) vs. Business as usual |
0 Days |
Full sample (all students in grades 3-8);
|
65.81 |
105.48 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Other or unknown: 100% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Illinois
-
Race Asian 4% Black 54% Other or unknown 31% White 11% -
Ethnicity Hispanic 30% Other or unknown 70% -
Eligible for Free and Reduced Price Lunch Free or reduced price lunch (FRPL) 90% No FRPL 10%
Study Details
Setting
The study took place in 14 public schools in Chicago, Illinois that served students in grades K–8.
Study sample
The researchers randomly assigned 7 schools to the intervention group and 7 schools to the comparison group. The outcomes from the study that met WWC standards were from administrative data for all students in grades 3–8 in the schools (about 3,960 students in total). Ninety percent of students in the schools were eligible for free or reduced-price lunch; 54% of students were Black, 11% were White, 4% were Asian, and 31% did not report race. Thirty percent of students were Hispanic or Latino and 70% did not report ethnicity.
Intervention Group
The intervention schools used Positive Action, a schoolwide program that combines a classroom curriculum with schoolwide activities to promote a positive school climate. Schools implemented the program over 6 years, from the 2004–05 school year to the 2009–10 school year. In addition to a schoolwide component intended to improve schools' climate, Positive Action includes a curriculum component that was delivered to all K–8 classrooms which consisted of six units: (1) self-concept, (2) positive actions for mind and body, (3) getting along with others, and (4) managing, (5) being honest with, and (6) continually improving oneself. The program aims to improve students’ self-concept; healthy behaviors; positive feelings, thoughts, and actions; and motivation to learn. Students in grades K–6 received four 15–20-minute lessons per week (for a total of 140 lessons). Students in grades 7–8 receive two 20-minute lessons per week (for a total of 70 lessons).
Comparison Group
The comparison schools did not offer Positive Action.
Support for implementation
The program developer provided about 4 hours of teacher and staff training to each school during the first year of implementation and 2 hours in subsequent years. Implementation coordinators visited the schools and assessed the extent to which the program components were in place, identified challenges, and provided technical support and training.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Washburn, I. J., Acock, A., Vuchinich, S., Snyder, F., Li, K. K., Ji, P., Day, J., DuBois, D., & Flay, B. R. (2011). Effects of a social-emotional and character development program on the trajectory of behaviors associated with social-emotional and character development: Findings from three randomized trials. [Study 2: Chicago Schools]. Prevention Science, 12(3), 314–323. https://doi.org/10.1007/s11121-011-0230-9.
-
Flay, Brian R. (2014). Replication of Effects of the "Positive Action" Program in Randomized Trials in Hawai'i and Chicago Schools. [Study 2: Chicago Schools]. Society for Research on Educational Effectiveness.
-
Duncan, R. J., Rolan, E., Marceau, K., Lewis, K. M., Bavarian, N., DuBois, D. L., & Flay, B. (2019). Childhood protective factors and a prevention program reduce later problem behaviors. Journal of Applied Developmental Psychology, 65. https://doi.org/10.1016/j.appdev.2019.101063.
-
Bickman, Leonard; Riemer, Manuel; Brown, Joshua L.; Jones, Stephanie M.; Flay, Brian R.; Li, Kin-Kit; DuBois, David; Pelham, William, Jr.; Massetti, Greta. (2009). Approaches to Measuring Implementation Fidelity in School-Based Program Evaluations. Journal of Research in Character Education, v7 n2 p75-101.
-
Lewis, K. M., Bavarian, N., Snyder, F. J., Acock, A., Day, J., Dubois, D. L., Ji, P., Schure, M. B., Silverthorn, N., Vuchinich, S., & Flay, B. R. (2012). Direct and mediated effects of a social-emotional and character development program on adolescent substance use. The International Journal of Emotional Education, 4(1), 56–78. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3845413/.
-
Silverthorn, Naida; DuBois, David L.; Lewis, Kendra M.; Reed, Amanda; Bavarian, Niloofar; Day, Joseph; Ji, Peter; Acock, Alan C.; Vuchinich, Samuel; Flay, Brian R. (2017). Effects of a School-Based Social-Emotional and Character Development Program on Self-Esteem Levels and Processes: a Cluster-Randomized Controlled Trial. SAGE Open, v7 n3.
-
Lewis, Kendra M.; Vuchinich, Samuel; Ji, Peter; DuBois, David L.; Acock, Alan; Bavarian, Niloofar; Day, Joseph; Silverthorn, Naida; Flay, Brian R. (2016). Effects of the "Positive Action" Program on Indicators of Positive Youth Development among Urban Youth. Applied Developmental Science, v20 n1 p16-28.
-
Lewis, K., M., DuBois, D., L., Bavarian, N., Acock, A., Silverthorn, N., Day, J., Ji, P., Vuchinich, S., & Flay, B., R. (2013). Effects of Positive Action on the emotional health of urban youth: A cluster-randomized trial. Journal of Adolescent Health, 53(6), 706–711. https://pubmed.ncbi.nlm.nih.gov/23890774/.
-
Lewis, K. M., Schure, M. B., Bavarian, N., DuBois, D. L., Day, J., Ji, P., Silverthorn, N., Acock, A., Vuchinich, S., & Flay, B. R. (2013). Problem behavior and urban, low-income youth: A randomized controlled trial of positive action in Chicago. American Journal of Preventive Medicine, 44(6), 622–30. https://pubmed.ncbi.nlm.nih.gov/23683980/.
-
Haegerich, T. M. & Metz, E. (2009). The social and character development research program: Development, goals, and opportunities. Journal of Research in Character Education, 7(2), 1–20. https://psycnet.apa.org/record/2011-19986-001.
-
Li, K. K., Washburn, I., DuBois, D. L., Vuchinich, S., Ji, P., Brechling, V., Day, J., Beets, M. W., Acock, A. C., Berbaum, M., Snyder, F., & Flay, B. R. (2011). Effects of the Positive Action program on problem behaviors in elementary school students: A matched-pair randomized control trial in Chicago. Psychology and Health, 26(2), 187–204. https://pubmed.ncbi.nlm.nih.gov/21318929/.
-
Day, J., Ji, P., DuBois, D. L., Silverthorn, N., & Flay, B. (2016). Cumulative social-environmental adversity exposure as predictor of psychological distress and risk behavior in urban youth. Child and Adolescent Social Work Journal, 33(3), 219–235. https://link.springer.com/article/10.1007/s10560-015-0421-5.
-
Bavarian, N., Lewis, K. M., Acock, A., DuBois, D. L., Yan, Z., Vuchinich, S., Silverthorn, N., Day, J., & Flay, B. R. (2016). Effects of a school-based social-emotional and character development program on health behaviors: A matched-pair, cluster-randomized controlled trial. The Journal of Primary Prevention, 37, 87–105. https://doi.org/10.1007/s10935-016-0417-8.
-
Bavarian, Niloofar; Lewis, Kendra M.; DuBois, David L.; Acock, Alan; Vuchinich, Samuel; Silverthorn, Naida; Snyder, Frank J.; Day, Joseph; Ji, Peter; Flay, Brian R. (2013). Using Social-Emotional and Character Development to Improve Academic Outcomes: A Matched-Pair, Cluster-Randomized Controlled Trial in Low-Income, Urban Schools. Journal of School Health, v83 n11 p771-779.
-
Lewis, K. M. (2012). Evaluation of a social-emotional and character development program: methods and outcomes [Doctoral dissertation, Oregon State University]. Oregon State University. https://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/7d278w417.
-
Washburn, I. J. (2010). Evaluation of a social-emotional and character development program [Doctoral dissertation, Oregon State University]. [Study 2: Chicago Schools]. Oregon State University. https://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/7h149s57c.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).