Effects of the Lessons in Character English Language Arts Character Education Program on Behavior and Academic Outcomes
Regional need and study purpose
Partially in response to unacceptable levels of student misbehavior and inadequate endorsement of good character values, character education has become one of the fastest growing reform movements in K–12 education today (Williams 2000). Between 1993 and 2004, 23 states passed laws mandating or recommending some aspect of character education. Such programs also have strong support from parents, teachers, and school administrators (Glanzer and Milzen 2006). But despite such widespread popularity, relatively few randomized controlled trials have examined the impact of character education programs on the character traits, behavior, and academic outcomes of students (What Works Clearinghouse 2009).
This study examines the impact of Lessons in Character—an English language arts-based character education program—on student academic performance, social skills, problem behaviors, and school climate. Since 1995, Lessons in Character has been implemented in more than 15,000 schools in every state except Alaska. The program consists of literature-based supplementary material aligned with state English language arts standards and designed to integrate easily into the current English language arts curriculum. The result is an ease of implementation that distinguishes the program from other character education programs.
The following research questions guide this study:
- Does the Lessons in Character program increase student achievement and teacher perceptions of academic competence?
- Do students who participate demonstrate greater social skills and fewer problem behaviors compared with control group students?
- Do students at schools implementing Lessons in Character report a greater sense of student belonging and more school expectations consistent with character development than control group students do?
In addition to these primary research questions, the study also examines whether participation in Lessons in Character is associated with student academic engagement, self-control, and other measures of student well-being and school climate.
One limitation of the study is that relatively few data are collected on classroom implementation and implementation fidelity. Due to cost considerations, a detailed process study was not conducted. The sample size—inadequate for examining differential impacts on student subgroups—was another limitation. Differences in program impacts for student subgroups can be examined only in an exploratory manner.
Intervention description
Developed by Dr. B. David Brooks, with support from Young People's Press, the Lessons in Character curriculum is delivered by classroom teachers, with implementation support from Dr. Brooks. Through multicultural literature (lap books) and audiocassettes, it teaches character and integrates the language of character into the English language arts curriculum. Designed for grades K–9, the curriculum is aligned with California education standards. Teachers in schools randomly assigned to the treatment condition participate in a one-day training session and in the fall semester receive a day of coaching support to deliver the curricular material.
The curricular material comprises two components, both designed to reinforce good character and support language arts learning standards: the core curriculum, Lessons in Character, and the supplemental materials, Daily Oral Language with Character and Writing with Character. The core curriculum, with a focus on teaching for understanding, explicitly integrates the language of character into the curriculum—emphasizing civility, respect, responsibility, trustworthiness, fairness, caring, loyalty, and self-control. A literature-based language arts program, it relies on multicultural literature, enrichment activities, cross-curricular activities, read-aloud books, and questioning that help instill habits of good character. The core curriculum consists of 24 twenty-minute lessons; study participants are asked to implement at least 19 lessons during the academic year. The program begins with a decisionmaking model—STAR, or stop, think, act, review—that is used throughout the year as a classroom management tool and reinforcement of the program lessons.
Daily Oral Language with Character and Writing with Character are optional for teachers. Daily Oral Language with Character materials consist of sentence correction activities followed by short writing assignments that emphasize decisionmaking, goal setting, civic responsibility, and other character education components. Designed for daily use, Daily Oral Language with Character augments the regular language arts program with lessons that take no longer than five minutes. Writing with Character materials (for grades 3–8) consist of 36 weekly twenty-minute writing assignments that focus on the mechanics of writing as well as character education.
Although Lessons in Character is a comprehensive schoolwide character education program, it is the program's integration into the curriculum that secures teacher support. Teachers become program experts through daily teaching, and this acquired expertise helps build support for comprehensive schoolwide character education policies and practices.
To date, there has been one randomized controlled trial to investigate the short-term effectiveness of Lessons in Character. Dietsch, Bayha, and Zheng (2005)—in a sample of 372 grade 4 students in Louisiana and Florida—compare outcomes for 11 classrooms that used the program for one semester and 10 control classrooms. Statistically significant results favoring the treatment group were found for attendance and for reading and mathematics grades. And while item-level analyses of student surveys favored the treatment group on character-related knowledge, attitudes, and values, only a few differences were statistically significant.
Study design
Designed as an experimental trial, the study runs from spring 2007 to spring 2010 in 50 volunteer California elementary schools with teachers of grades 2–5. Recruited using mass mailings and established WestEd marketing channels, schools are randomly assigned to treatment and control groups. The intervention involves a one-day training course for treatment teachers in year 1—who in turn are expected to incorporate between 19 and 24 supplementary lessons into their classroom instruction for two academic years. No Lessons in Character professional development activities or coaching occurs in year 2 of implementation. Two cohorts of elementary schools are participating. Cohort 1 implementation is taking place in 2007/08 and 2008/09, with teacher professional development and coaching in late summer and early fall of year 1 (2007). Cohort 2 schools were recruited in spring 2008, with implementation scheduled for 2008/09 and 2009/10. Both cohorts are pooled in the data analyses. Teachers in control schools, meanwhile, continue with their regular professional development activities and instructional practices.
The study sample consists of around 15,000 students in grades 2–5 in 50 public elementary schools in California—34 in the Los Angeles and San Diego Metropolitan Statistical Areas, 13 in northern California, and 3 in central California. Thirty schools are in a large city or urban fringe of a large city, six in a mid-size city (population of 25,000–250,000), and fourteen in a small town or rural area. Each school serves around 350 students, though nine have fewer than 150 students, and two have more than 600. Approximately 58 percent of the students are eligible for free or reduced-price lunch, 49 percent are Hispanic, 30 percent are English language learner students, and 33 percent are White (California Department of Education 2009).
With 25 schools per condition (treatment or control) and an expected average of at least 224 students in each school, the projected sample size is sufficient for detecting program impacts on student outcomes (0.17–0.23 standard deviation). Specifically, that sample size can detect an effect size equal to about two-fifths of a year of growth experienced by grade 2 students on norm-referenced standardized tests in reading (Hill et al. 2008).
Key outcomes and measures
Table 1 lists the study's key outcome variables—student academic achievement, social skills, behavior, and school climate—and their measures. These outcomes are assessed with standardized achievement tests, teacher reports on the Social Skills Rating System, student surveys of fourth and fifth graders, and teacher surveys
- State assessments. Student achievement data from state-mandated standardized assessments of English language arts and mathematics (the California Standards Tests) are collected for the years before and during program implementation. Criterion-referenced to state standards, the California Standards Tests in English language arts and mathematics are administered to students in grades 2–11.
- Social Skills Rating System teacher report. Gresham and Elliott's (1990) Social Skills Rating System is used to assess student social skills, problem behaviors, and academic competence. Teacher reports assess the subdomains of cooperation, self-control, and externalization/internalization of problems.
- Student survey. A 35-minute survey assessing behaviors, attitudes, and values consistent with the goals of character education is administered to all fourth and fifth graders in the fall and spring of both implementation years. Using items and subscales from validated instruments, the survey assesses student altruism (Characterplus 2002), aggression (Opinas and Frankowski 2001), delinquent behavior (Kisker et al. 2004), and empathy (Funk et al. 2003) as well as school belonging and expectations (Characterplus 2002).
- Teacher surveys. All teachers are surveyed before random assignment to treatment and control groups in the spring of year 0 (pretest) and are surveyed again in the spring of year 1 and year 2. The pretest survey assesses teacher background and school climate—teacher education, professional development experiences in language arts and character education, activities related to social and character development that they have implemented in their classroom, and school and classroom climate (students' feelings of belonging and school expectations). The posttest survey also contains questions that ask about other professional development teachers participated in during program implementation and questions that assess teachers' communication and collaboration.
Table 1. Impact analysis outcomes
| Outcomes | Measure |
| Academic achievement | |
| English language arts standardized tests | State assessment |
| Mathematics standardized tests | State assessment |
| Academic competence | Teacher survey |
| Social skills | |
| Social Skills Rating Scale total score | Teacher survey |
| Altruism | Student survey |
| Empathy | Student survey |
| Problem behaviors | |
| Externalization of problems | Teacher survey |
| Aggression | Student survey |
| Delinquent behavior | Student survey |
| School and classroom climate | |
| Students' feeling of belonging | Teacher and student survey |
| School expectations | Teacher and student survey |
Data are collected on three broad areas: student outcome, teacher background and school climate, and implementation. Except for teacher interviews, program teacher implementation logs, and survey items on program implementation—which are collected only for treatment teachers—all measures are collected in both treatment and control sites. Table 2 summarizes the data collection schedule.
Table 2. Data collection schedule
| Data | Year 0 | Year 1 | Year 2 |
| Student outcome measures | |||
| State assessments | Spring | Spring | Spring |
| Student surveys | Fall and spring | Fall and spring | |
| Teacher Social Skills Rating System report | Spring | Spring | Spring |
| Teacher background and school climate measures | |||
| Teacher surveys | Spring | Spring | Spring |
| Implementation measures | |||
| Teacher implementation logs (treatment group only) | Fall and spring | Fall and spring | |
Analysis plan
To estimate program impacts, outcomes for students and classrooms in treatment schools are compared with those for students and classrooms in control schools. Multilevel regression models are used to analyze the effects of Lessons in Character, to account for data clustering by school (Goldstein 1987; Raudenbush and Bryk 2002; Murray 1998). The impact analyses control for all baseline (pretest) measures of outcome variables and other student- and school-level covariates.
The procedures described by Schochet (2008) are used to account for testing multiple hypotheses involving the study's numerous outcome variables. Specifically, four outcome domains are delineated—student academic achievement, social skills, and problem behaviors and school and classroom climate—with primary and secondary outcomes within each domain. Multiple comparison procedures are used for each outcome domain to reduce the probability of finding statistically significant program impacts when impacts are due to chance alone. Exploratory analyses also examine potential impacts on other areas, such as staff culture of belonging and student academic engagement, cooperation, and disciplinary referrals. In addition to estimating program impacts, the study also conducts exploratory analyses to investigate differences in program impacts by gender, race/ethnicity, and limited English proficiency. Multiple comparison procedures are not used for the exploratory analyses of outcomes.
Thomas Hanson, PhD, Regional Educational Laboratory West and Barbara Dietsch, PhD, Regional Educational Laboratory West
Contact information
Dr. Thomas Hanson
Regional Educational Laboratory West
4665 Lampson Avenue
Los Alamitos, CA 90720-5139
Voice: (562) 799-5170
Fax: (562) 799-5151
Email: thanson@wested.org
Region: West
References
California Department of Education. (2009). Academic performance index data file, 2007 Base. Retrieved March 13, 2009, from www.cde.ca.gov/ta/ac/ap/apidatafiles.asp.
Characterplus. (2002). Evaluation resource guide: Tools and strategies for evaluating a character education program. St. Louis, MO: Characterplus.
Dietsch, B, Bayha J., and Zheng, H. (2005, April). Short-term effects of a literature-based character education program among fourth grade students. Paper presented at the 2005 Meeting of the American Educational Research Association, Montreal, Quebec, Canada.
Funk, J., Elliott, R., Bechtoldt, H., and Pasold, T. (2003) The attitudes toward violence scale: child version. Journal of Interpersonal Violence, 18 (2), 186–96.
Glanzer, P.L., and Milson, A.J. (2006). Legislating the good: A survey and evaluation of character education laws in the United States. Educational Policy, 20 (3), 525–50.
Goldstein, H. (1987). Multilevel models in educational and social research. London: Oxford University Press.
Gresham, F.M., and Elliott, S.N. (1990). Social Skills Rating System (SSRS). Bloomington, MN: Pearson Assessments.
Hill, C.J., Bloom, H.S., Black, A.R., and Lipsey, M.W. (2008) Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2(3), 172–77.
Kisker, E, Kalb, L, Miller, M., Sprachman, S., Carey, N, Schochet, P, and James-Burdumy, S. (2004). Social and character development research program evaluation: Supporting statement for request for OMB approval of SACD evaluation. Princeton, NJ: Mathematica Policy Research.
Murray, D.M. (1998). Design and analysis of group randomized trials. New York: Oxford University Press.
Raudenbush, S.W., and Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications.
Schochet, P.Z. (2008). Guidelines for multiple testing in experimental evaluations of educational interventions. Mathematical Policy Research, Inc.
What Works Clearinghouse. (2007). Character education topic report. Washington, DC: U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. Retrieved February 2, 2009, from http://ies.ed.gov/ncee/wwc/pdf/CE_TR_06_04_07.pdf.
Williams, M. (2000). Models of character education: Perspectives and developmental issues. Journal of Humanistic Counseling, Education & Development, 39 (1), 32–41.