Description:

Purpose: In this study, researchers will create and test Interactive Virtual Training (IVT), a video game technology in which early career teachers can hone their classroom management skills in a virtual classroom. Difficulty managing classrooms and engaging learners are two of the strongest predictors of attrition from the profession of teaching for those early in their careers. This project builds on a recently completed IES Development and Innovation study, Enhancing Effectiveness and Connectedness among Early Career Teachers in Urban Schools, in which the researchers created a model designed to support early career teachers around those challenges. The model includes school-based professional learning communities, in addition to group seminars and coaching for early career teachers, all designed to build their feelings of effectiveness in managing classroom behavior and engaging learners and to promote feelings of connectedness with colleagues. Findings were promising but insufficient in the original study; improvements in classroom management were slowly realized and teachers reported a preference for online delivery of content.

Project Activities: In Year 1, researchers will develop the IVT prototype, including disruptive student avatars and virtual classrooms. In Year 2, preservice teachers will try IVT and provide feedback on usability. In Year 3, researchers will conduct an initial field test with early career teachers in elementary school classrooms. In Year 4, the team will carry out a randomized pilot study to determine the promise of the model for improving early career teachers' feelings of effectiveness and connectedness, and for improving student engagement, behavior, and academic achievement.

Products: The research team will produce a fully developed virtual learning environment to help new teachers learn how to effectively manage classrooms and engage learners in elementary school. Researchers will also produce peer-reviewed publications.

Structured Abstract

Setting: This study will take place in high poverty urban schools in New Jersey.

Sample: In this study, 16 preservice teachers will provide feedback on usability of the prototype. Sixty-four early career teachers (24 in Year 3 field test and 40 in Year 4 pilot test) will participate across about 8 schools (6 in the pilot study). About 200 school personnel are expected to participate in the professional learning communities (PLCs). The pilot study will include 136 elementary school students (K to 5) considered at-risk for behavior problems.

Intervention: Interactive Virtual Training (IVT) uses video game technology to help early career teachers learn effective classroom management skills and strategies to engage students in learning. IVT includes "practice" students (disruptive avatars) in a virtual classroom. IVT is designed to augment the professional development model created in the prior development study, which consists of professional learning communities (PLCs) for all school personnel and group seminars and coaching for early career teachers. IVT utilizes the content of the group seminar curriculum and is hypothesized to provide early career teachers with more opportunities to reflect on, problem solve, and practice seminar content in a realistic yet low-stakes virtual training environment.

Research Design and Methods: In Year 1, researchers will develop the IVT prototype, including disruptive student avatars and virtual classrooms. Experienced teachers provide substantive feedback on the early prototypes. In Year 2, the research team will evaluate IVT for usability of the interface and its instructional design goals. In addition, preservice teachers will participate in formative testing sessions to inform any usability problems that are not identified through the expert evaluation. Researchers will devote Year 3 to field testing with two consecutive cohorts of early career teachers in elementary school classrooms. Revisions to IVT are incorporated in an iterative fashion throughout the first three years of the project. In Year 4, the team will conduct a randomized pilot study to test the promise of the model for improving early career teachers' feelings of effectiveness and connectedness, and for improving student engagement, behavior, and academic achievement. Schools are randomly assigned to the full model (IVT plus PLCs and coaching) or to serve as controls, and outcomes are assessed at baseline, mid-way through the school year, and in the spring to determine impacts on early career teachers and their disruptive students. To determine if IVT leads to more rapid acquisition of behavior management skills, a quasi-experimental design is used to compare rate in change of slope from pre- to post-test between early career teachers randomly assigned to the full model in this study and early career teachers who participated in the recently completed development study (n = 15).

Control Condition: Schools randomly assigned to the control group continue to provide standard professional development opportunities available in the district.

Key Measures: The research team will measure usability using the Heuristic Checklist-Adapted, Concurrent Think Aloud Protocols, semi-structured interviews, and questionnaires. Researchers will measure fidelity of implementation using IVT tracking logs. Researchers will use the Classroom Assessment Scoring System (CLASS), the Teaching Strategies Questionnaire, and focus groups to assess feasibility of the IVT system. Researchers will assess student outcomes using the Behavioral Observation of Students in Schools (BOSS), the Social Skills Improvement System – Problem Behavior Subscale, and school standardized achievement test scores in reading and math. Teacher outcomes are measured using the CLASS Behavior Management Dimension Score, sociometric interviews, the Professional Community Index, and the Teacher Trust Scale.

Data Analytic Strategy: Researchers will analyze usability and feasibility data from the questionnaires and heuristic checklists using descriptive statistics. They will also analyze the qualitative data (concurrent think aloud protocols, semi-structured interviews, focus groups) to identify emerging themes. Researchers will use three-level mixed effects models to analyze the pilot study data, with relevant student covariates (gender and grade) and teacher covariates (prior gaming and computer experience) included in the models.

Related IES Projects: Enhancing Effectiveness and Connectedness among Early Career Teachers in Urban Schools (R305A090085)