Description:

Co-Principal Investigator: Tally, William

Purpose: Mission US is an award-winning series of interactive games that is available at no cost for students in grades 5 to 9 for increase knowledge of U.S. history. In the most recent National Assessment of Educational Progress, only 15% of students in grade 8 performed at or above the basic level of proficiency in U.S. history. One way to engage students in learning history is to create history learning resources that are designed to be relevant and appealing to young people's interests and regular activities, while simultaneously immersing students in historical events.

Project Activities: During Year 1, researchers will refine the games to provide teachers and researchers with back-end data about student game-play and pilot test the procedures and outcome measures with 10 teachers. During Year 2, 35 history or social studies teachers in New York City and New Jersey will be randomly selected to use Mission US materials with their students for 3 months in addition to their regular curriculum, and 35 continue with their regular curriculum. Students will be assessed before and after the intervention to measure history content knowledge, ability to analyze and interpret documents, motivation to study history, and historical perspective-taking. During the final year of the project, the project team will analyze data and present findings to researchers, educators, administrators, and policymakers at conferences, and through publications, webinars, blogs, and other relevant venues.

Products: The team will share results with educators regarding the impact of Mission US on student learning and the real costs of implementation.

Structured Abstract

Setting: Research will be conducted in Grade 8 classrooms in 35 public schools in New York City and New Jersey, with one treatment and one control teacher in each school.

Population/Sample: Participants will include 70 Grade 8 social studies or history teachers and approximately 1,750 students (25 per class) evenly divided between treatment and control conditions. Students will represent a range of ability levels and racial, ethnic, and socioeconomic backgrounds. The sample is intended to represent the typical Grade 8 population of U.S. urban public schools.

Intervention: Teachers in the intervention classrooms will receive guidance about implementing the Mission US digital suite and will use the intervention over three months. The intervention consists of three Mission US narrative games and their curricular supports, covering key topics from 1900–1945, all tied together via a teacher dashboard that enables progress monitoring and in-depth classroom discussion. Curricular supports consist of teacher guides, web-based materials for classroom implementation, including student writing prompts, discussion questions, vocabulary lists, document-based exercises, and quizzes.

Research Design and Methods: The study will employ a three-level block-randomized design in which teachers and their classrooms are assigned to condition within schools. During Year 1, the researchers will finalize and pilot the procedures and research instruments. During the Year 2, the researchers will conduct the randomized trial and begin analyses, assessing student performance at baseline and after completion of the intervention. The researchers will collect data on teacher instructional practices and costs during the intervention. Year 3 will be dedicated to finalizing analyses and dissemination. Cost analyses will be planned for and conducted throughout the project.

Control Condition: Teachers and students in 35 Grade 8 classrooms will be randomly assigned to the control condition and will experience business-as-usual teaching with the standard history curriculum. Control teachers and students will have access to the games after the study period.

Key Measures: Researchers will use student grades on standardized measures, including the NY State Regents exams, to measure historical content knowledge and students' ability to analyze and interpret documents; the School Motivation Questionnaire (SMQ) and Individual Interest Questionnaire (IIQ) to assess motivation to study history; and both the Basic Empathy Scale (BES) and the Historical Perspective Taking assessment to approximate a measure of historical empathy.

Data Analytic Strategy: Researchers will carry out a three-level model, with students nested within classrooms nested within schools, with the treatment indicator at the teacher level. The researchers will examine the mediating impact of historical empathy and the influence of student- and teacher-level moderating variables, as well as the influence of background curriculum.

Cost Analysis: Researchers will evaluate the cost of implementing the Mission US intervention as compared to business-as-usual in control classrooms using the ingredients method. Initially, total cost of implementation will be used to provide cost per teacher and cost per student of implementing the program. How costs vary across sites and numbers of teachers or students served will be explored, as will the cost per Mission US game. To calculate a cost-effectiveness ratio, the incremental costs of implementing the Mission US intervention above and beyond the costs of the business-as-usual curriculum will be determined.

Related IES Projects: Mission US: An Interactive Solution for Middle School History Learning (EDIES13C0027)