I. General Inclusion Criteria

• Populations to be Included
• Types of Interventions to be Included
• Types of Research Studies to be Included

Populations to be Included

The population of interest includes K-12 general and/or special education students attending elementary, middle, or high school at any type of school (public, private, parochial, etc.). The students must attend a school in the U.S. or its territories or tribal entities, or in an English-speaking country with a sufficiently similar cultural context that the study can be realistically replicated in the U.S.

Types of Interventions to be Included

The interventions considered for inclusion are identified through an exhaustive search of the published and unpublished literature as well as a review of nominations submitted to the WWC. For an intervention to be included in the review, the main goal and focus in its lessons, activities, and/or organization must be the direct or indirect teaching of a range of core values. The intervention does not necessarily have to be referred to as a character education program if it otherwise meets this criterion. Only interventions that are replicable (i.e., documented well enough that they can be distinctly identified and reproduced) will be included. Finally, all interventions should be delivered during the regular school day and school year; after-school and summer programs will not be included.

Within the above constraints, three broad types of interventions are eligible for inclusion:

1. Programs. Examples of school-based character education programs include:
   • A curriculum to promote character values among high school students that integrates character education activities into several academic subject areas including social studies, English, and math.
   • An elementary school program in which the entire school becomes a community in which all participants—children and adults alike—work together to foster values of caring, helping, respect, and tolerance. The approach involves training the adults in a child’s life to model, teach, and embody core principles.
   • A literature-based program to promote core values. The program consists of in-class lessons that include stories, puppets, finger plays, rhymes, songs, and activity books.
2. Practices or strategies. Examples of character education practices include:
   • Processes in which school staff and community members reach consensus about what a character education program should include and/or participate in the ongoing implementation of the program.
   • Teaching strategies (e.g., cooperative learning focused on promoting helping/caring behaviors or acceptance of diversity, classroom disciplinary system to enable student responsibility).
   • Professional development for school staff on character education topics or methods.
   • Procedures for assessing and recognizing relevant values and attributes (e.g., awards, recognition ceremonies).
3. Policies. Examples of character education policies include:
   • Systemwide or schoolwide behavior codes and discipline policies that focus on promoting core values in character education.
   • School policies that require moral leadership from school administrators, staff and students.

For this WWC review, character education products (e.g., workbooks, videos, and interactive CD-ROMs) are not included, except as they are incorporated into a program, practice, or policy as described above. A large variety of products are offered to elementary, middle, and high schools for individual or group use. Scrutiny of the full list is beyond the scope of this review. In addition, it is believed that little research exists on the outcomes produced by the use of such products.

Types of Research Studies to be Included

The review focuses on well-conducted randomized and nonrandomized controlled trials of character education programs, practices, and policies. To be eligible, studies must have been published in 1983 or later and report outcome measures taken on students, not solely teachers, parents, or other participants in the program or school. At least one of the outcome measures must focus on character education outcomes or academic performance and demonstrate adequate reliability or face validity.

II. Specific Topic Parameters

The following parameters specify which studies will be coded for the review and the aspects of those studies that are critical to how they are coded.

1. Commonly shared and/or theoretically derived characteristics of the intervention that should be reflected in its definition and implementation
2. Important characteristics of the intervention that must be known in order to reliably replicate it
3. Important classes of outcomes
4. Evidence sufficient for an outcome measure to demonstrate reliability
5. Interval of time within which studies should have been conducted
6. Necessary characteristics that define the target population
7. Important characteristics of participants that might be related to the intervention's effect that must be equated if a study does not employ random assignment.
8. Relevant subgroups
9. Relevant settings
10. Appropriate interval for measuring the intervention's effect
11. Differential attrition assumed to be problematic
12. Overall attrition assumed to be severe

1. Commonly shared and/or theoretically derived characteristics of the intervention that should be reflected in its definition and implementation.

• The primary emphasis of the intervention is on instilling, teaching, or promoting core values that can be generalized across contexts (that is, are not focused on a single domain such as conflict resolution, drug use, or unsafe sexual behavior). Values of this sort include, for example, honesty, courage, perseverance, self-discipline, responsibility, integrity, caring, respect, trustworthiness, fairness, citizenship, patriotism, justice, and the like (described also as moral values, virtues, character traits, or principles). Qualifying interventions will have most or all of their lesson units, prescribed activities or events, and/or training and program materials dedicated to this objective in readily apparent ways.
• Attention is given to a range of values of the sort identified above; that is, multiple values can be identified in lesson plans, prescribed activities, etc. in contrast to a focus on a single value or a set of very closely related values (such as those having to do only with peer relationships or only with community service activities).
• The intervention is a program, practice, or policy but not exclusively a product (e.g., workbook, video, interactive CD-ROM), and it is replicable. Replicability will be assumed for branded programs that have a name and are marketed to schools, and for distinctly identifiable programs, practices, or policies (i.e., those that appear under the same name in different places or are associated with a particular developer or disseminator) that are documented well enough to be implemented by a provider other than the developer.
• The intervention is implemented in an elementary, middle, or high school during the regular school day and academic year; after-school and summer programs will not be included.
• In addition, interventions will be characterized for descriptive purposes as comprehensive, modular, or (if necessary) something in between, with all these variants eligible for inclusion.

2. Important characteristics of the intervention that must be known in order to reliably replicate it with different participants, in other settings, at other times. An intervention is considered to be replicable if:

• The intervention is “branded.”
• The intervention is not “branded” but meets the following conditions:
  • The intervention is described in general terms.
  • The duration of the intervention is described.
  • The characteristics of the individuals who are expected to deliver the intervention are described.

3. Important classes of outcomes.

• Character development as reflected in understanding, caring about, and/or enacting core ethical values (knowledge, attitudes/values, or behavior)
• Prosocial behavior or reduction of problem behaviors that represent a behavioral expression of core values
• Academic performance

4. Evidence sufficient for an outcome measure to demonstrate each type of reliability (internal consistency, temporal stability/test-retest, and inter-rater).

Standard WWC values will be applied to these reliability considerations as follows:

• Internal consistency: .60
• Temporal stability/test-retest: .40
• Inter-rater reliability: .50

5. Interval of time within which studies should have been conducted to be appropriate for the Evidence Report.

Studies must have been conducted within the past 20 years (i.e., with a publication date of 1983 or later). This is the default time interval for all WWC reviews. Timing will be judged by the date of earliest publication of any report on the study outcomes. The publication date for unpublished studies will be the date the document reached its final form.

6. Necessary characteristics that define the target population.

Students are between the ages of 5 and 21 and are enrolled in grades K through 12. Students attend an institution within the United States (including U.S. territories and tribal entities) or within an English-speaking cultural setting for which the study could realistically be replicated in the U.S. In addition, interventions will be characterized for descriptive purposes with regard to the population with which they are implemented, including special subpopulations of children (e.g., children with learning disabilities or language impairments, English language learners, etc.).

7. Important characteristics of participants that might be related to the intervention's effect that must be equated if a study does not employ random assignment.

Studies that do not employ random assignment must demonstrate the initial equivalence of the comparison groups, or ensure it through statistical control, with regard to:

• Pretest scores for at least one outcome measure
• Grade level or age
• Gender
• Any special status such as special education, ELL, etc.
• Location of the schools involved (urban, suburban, or rural; geographical region)

8. Relevant subgroups of interest for this review, in addition to age or grade, gender, disability status, and race/ethnicity.

• Second-language learners or children with cultural backgrounds different from the local dominant culture
• Children with a history of school disciplinary infractions or who are otherwise identified as at risk for anti-social behavior

9. Relevant settings of interest for this review.

• Schools in urban, suburban, or rural locations
• Public, private, and special-purpose schools

10. Appropriate interval for measuring the intervention's effect relative to the end of the intervention.

The benefits of a character education intervention are expected to appear by the end of the intervention and to be retained past that point. Thus measures at the end of an intervention and any time thereafter are appropriate.

11. Amount of differential attrition from the intervention and control groups assumed to be problematic.

The WWC default value will be used: Differential attrition is defined as a differential loss of greater than 7%. If differential attrition is less than or equal to 7%, the associated bias will be assumed minimal. If differential attrition is greater than 7%, the study must show that it did not bias the effect size estimate by demonstrating post-attrition group equivalence on the pretest for at least one major outcome variable. This may be accomplished by showing nonsignificant differences with an adequately powered (.80) statistical significance test or a standardized mean difference between groups of d = .10 or less.

12. Amount of overall attrition from the study sample assumed to be severe.

The WWC default value will be used: Severe overall attrition is defined as a loss of more than 20% of the sample assigned to the relevant study conditions. If overall attrition is less than or equal to 20%, the associated bias is assumed to be minimal. If overall attrition is greater than 20%, the study must show that it did not bias the effect size estimate by demonstrating post-attrition group equivalence on the pretest for at least one major outcome variable. This may be accomplished by showing nonsignificant differences with an adequately powered (.80) statistical significance test or a standardized mean difference between groups of d = .10 or less.

III. Methodology

• Literature Search Strategies
• Key Word List
• Supplementary List of Journals to be Hand Searched
• Supplementary List of Organizations
• Personal Contacts

Literature Search Strategies

The WWC Evidence Report Team employs comprehensive and systematic literature search strategies to identify the population of published and unpublished relevant studies. This section contains topic specific elements of the literature search (e.g., search terms, additional journals, and associations).

Key Word List

A combination of Boolean terms such as AND and OR will be used with this keyword list. The librarian at AIR will be consulted as to the appropriate combination to use and then tailor according to each specific electronic database.

Supplementary List of Journals to be Hand Searched

1. Journal of Research in Character Education
2. Journal of Adolescent Research
3. Journal of Consulting and Clinical Psychology
4. Journal of Research on Crime and Delinquency
5. Journal of Quantitative Criminology
6. Criminology
7. Journal of Community Psychology
8. Psychology in the Schools
9. Criminal Justice and Behavior
10. Justice Quarterly
11. Crime and Delinquency
12. Evaluation Review

Additional journals will be added to the supplementary list when a few articles appearing in them are identified through other search procedures.

Supplementary List of Organizations

1. Association for Supervision and Curriculum Development
2. Tanglewood Research (http://www.tanglewood.net)
3. Character Education Partnership (http://www.character.org)
4. Collaborative for Academic, Social, and Emotional Learning (http://www.casel.org/sel_resources/linksindex.php)
5. SAMHSA's National Registry of Effective Programs (NREP)
6. Office of Safe and Drug Free School Programs Exemplary and Model Programs (http://www.ed.gov/admins/lead/safety/exemplary01/index.html)
7. The Center for Information and Research on Civic Learning and Engagement (http://www.civicyouth.org/)
8. Michigan Model Partnership for Character Education (http://www.emc.cmich.edu/charactered)
9. National Service Learning Partnership (http://www.service-learningpartnership.org)
10. National Center for Learning and Citizenship, Education Commission of the States (http://www.ecs.org/html/projectsPartners/clc/clc_main.htm)
11. Service Learning Research and Development Center (http://gse.berkeley.edu/research/slc/)
12. Center for the 4th and 5th R’s (http://www.cortland.edu/c4n5rs/)
13. indexes of similar organizations embedded in the above websites

Personal Contacts

The WWC Character Education Evidence Report Team will solicit studies directly from knowledgeable informants in the field of character education. This will include members of organizations associated with character education, researchers who publish on character education, grantees, evaluators funded for character education programs, and other such persons.

Another source of contacts will be the developers of programs identified as relevant to the topic. The WWC Character Education Evidence Report Team will solicit studies and any additional information about the program from the developers.

A third source of personal contacts will occur after the identification of studies to be reviewed. We will contact the authors of these studies to answer specific queries and request additional materials.