Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Initial Efficacy Study of Classcraft, A Gamified Approach to Classroom Management
Center: NCER Year: 2023
Principal Investigator: Sumi, W. Carl Awardee: SRI International
Program: Social, Emotional, and Behavioral Context for Teaching and Learning      [Program Details]
Award Period: 4 years (09/01/2023 – 08/31/2028) Award Amount: $3,746,025
Type: Initial Efficacy Award Number: R305A230392
Description:

Co-Principal Investigator: Davies-Mercier, Betsy

Purpose: In this project, researchers will evaluate the efficacy of Classcraft, a gamified technology-based approach designed to help teachers establish a positive, cooperative classroom atmosphere and improve classroom management and student engagement. Although Classcraft is widely used by over 30,000 teachers and 1 million active student participants worldwide, it has not been rigorously studied in a randomized controlled trial (RCT). Furthermore, little is known about the effect of Classcraft on young adolescents. Digital gamified approaches to learning may be particularly appealing for young adolescents, especially when there are opportunities to collaborate and build positive relationships with teachers and peers. Using an RCT research design, researchers will examine the impact of Classcraft on middle school student social-emotional, behavioral, and academic skills, teacher classroom management practices and perceived self-efficacy, and student perceptions of classroom climate.

Project Activities: The research team will randomly assign middle school classrooms to implement Classcraft across 2 academic years and compare outcomes to business-as-usual control classrooms. They will also test the effect of Classcraft on teacher use of effective classroom management practices and self-efficacy as well as student perceptions of classroom climate to determine whether Classcraft improves student academic, social-emotional, and behavioral skills and reduces behavior problems and discipline referrals.

Products: Researchers will produce evidence of the efficacy of Classcraft for middle school students and their teachers. They will disseminate findings through peer-reviewed publications, conference presentations, reports, blogs, and an active social media presence targeting educators, administrators, parents, students, and researchers.

Structured Abstract

Setting: The study will be set in middle schools in urban, suburban, and rural communities in multiple states.

Sample: The sample will include 84 English language arts and mathematics teachers and approximately 840 students in grades 6 to 8 receiving instruction from one of the participating teachers.

Intervention: Classcraft (https://www.classcraft.com) is a gamified, technology-based approach to improving student engagement and education outcomes. During academic instruction, Classcraft is intended to run in the background, with the teacher and students having access to the platform to view progress and interact with behavior-related content. Students can choose and customize their own avatar (Warrior, Mage, or Healer), with each role having different powers, health and energy stats, and responsibilities. Teachers are expected to use Classcraft to teach, prompt, and reinforce student behaviors by awarding and deducting points within the game and administering real-life rewards and consequences. Students participate in teams of 4 to 6 members. This team approach encourages collaboration, cooperation, and team building because student choices have direct impacts on their teammates. As students move through the game, they build a sense of competence and mastery and identify them as positive behaviors. These experiences within the game enable them to "level up" and unlock privileges that also correspond to real-life rewards.

Research Design and Methods: Researchers will randomize teachers to the intervention or control condition and utilize an independent cohort longitudinal design. To limit selection-related threats to internal validity, researchers will block by schools, grade levels, and academic subjects. For each of the 2 independent cohorts, half of participating teachers (n = 21) will begin Classcraft planning and training processes, while half (n = 21) will be assigned to the services-as-usual control condition. For each cohort, researchers will collect baseline data in the fall, and in the spring, they will collect posttest measures, school records, and back-end usage. Researchers will also collect implementation measures to test the hypothesis that, when implemented with fidelity, Classcraft significantly improves middle school students' academic, social, and behavioral skills as well as teachers' disciplinary practices, perceived self-efficacy, and student perception of classroom climate.

Control Condition: The control condition consists of business-as-usual classrooms that will not receive the Classcraft program.

Key Measures: From students, the researchers will collect student self-report of self-efficacy and cognitive/behavioral engagement with school and student-report of teacher-student relationship. From teachers, they will collect teacher-report of student social-emotional, behavioral, and academic skills. and teacher self-report of self-efficacy and classroom management skills. They will collect administrative data on student behavior problems and discipline referrals; observation of student collaboration, cooperation, and team building; and computer-adaptive Renaissance STAR 360 Mathematics and Reading Assessment to assess student academic achievement. They will also collect observations of teacher classroom management skills and classroom climate and measures of student characteristics (such as gender, race) and teacher and classroom characteristics (such as class size, course, teacher years of experience). Additionally, they will monitor intervention dosage and fidelity through training attendance logs, teacher surveys, and back-end usage data.

Data Analytic Strategy: Researchers will assess the fidelity of implementation in intervention classrooms and the degree to which Classcraft affects student behavior and academic outcomes using linear regression or hierarchical linear modeling. In addition, researchers will examine the outcomes of subgroups as defined by potential individual-level moderators (for example, grade level) and classroom-level mediators (for example, teacher self-efficacy) that may uncover underlying mechanisms that lead to observed outcomes.

Cost Analysis: Researchers will use the ingredients method to account for the costs incurred during Classcraft program implementation and opportunity costs. Researchers will follow the steps in the ingredients method for both the intervention and control group. In measuring both key outcomes and program costs, researchers will calculate a cost-effectiveness ratio. This ratio allows stakeholders to understand the resources needed to produce the observed effects for Classcraft.


Back