Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: An Evaluation of the Efficacy of Virtual Enterprises
Center: NCER Year: 2021
Principal Investigator: Daugherty, Lindsay Awardee: RAND Corporation
Program: Career and Technical Education      [Program Details]
Award Period: 5 years (07/01/2021 - 06/30/2026) Award Amount: $3,799,857
Type: Initial Efficacy Award Number: R305A210198
Description:

Co-Principal Investigators: Opper, Isaac; Hughes, Katherine; Ozek, Umut

Purpose: For this project, the research team will provide evidence on the efficacy and cost-effectiveness of Virtual Enterprises (VE), a virtual work-based learning (WBL) program. Many high schools are struggling to ensure that all graduates are prepared for college and the workforce, and employers consistently report difficulty in finding workers with sufficient employability skills, such as teamwork and communication. A growing number of education practitioners and researchers have argued that incorporating career and technical education (CTE), including WBL, into high school curricula can improve students' engagement and their readiness for postsecondary education and the workplace. School-based enterprises (SBE) are one form of WBL, in which students operate a business that produces and sells goods or services, and may offer unique benefits relative to other types of WBL by providing opportunities for more students to participate, reducing the need for transportation, and allowing students more room to make, and learn from, their mistakes. However, there is very little causal evidence showing the impacts of CTE or WBL on student academic or employment outcomes, and none showing the impacts of SBE. VE has been adopted by over 430 schools across 18 states, and prior implementation studies indicate that it has the potential to increase student motivation and employability skills.

Project Activities: The research team will rely on propensity score matching and difference-in-differences frameworks in five districts across two states and approximately 35 high schools that offer the VE program. The primary samples for the analysis will be two cohorts of seniors who will be surveyed between 2023-24 and 2025-26. The researchers will use a combination of administrative data and surveys during and after the intervention to identify comparable students to those enrolled in the VE course and compare academic and career readiness outcomes for eligible students who were assigned to the VE program and those who were not.

Products: The research team will produce a report documenting findings and two published journal articles.  The research team will disseminate shorter products for education practitioners, policymakers, and CTE and workforce development organizations. These products will be shared via conferences and social media. The research team will also present results to participating schools.

Structured Abstract

Setting: The study will take place in high schools in New York and California. The participating schools come from four urban and suburban counties.

Sample: Approximately 2,288 12th grade students enrolled in the Virtual Enterprises course and 6,864 comparison students assuming three nearest neighbors are selected. The student demographics in the targeted schools range from 1 to 28 percent African American, from 22 to 53 percent Hispanic, and from 23 to 30 percent white. Between 29 and 50 percent of students are eligible for free or reduced-price lunch.
Intervention:  VE is a year-long high school class in which students run a virtual firm. Individual students assume specific roles, such as CEO, marketing manager, or human resources employee. Together, the students develop and implement a strategy for the firm and participate in a virtual economy with other VE firms. The curriculum is provided by Virtual Enterprises International (VEI), a non-profit organization that also provides training for the VE teachers (known as facilitators), provides virtual start-up capital to the firms, and runs the online marketplace in which firms sell their goods or services. Treatment students will be assigned to the VE class, which has already been adopted in all participating schools. Treatment students will also have the option to take advantage of several other opportunities provided by VEI, including participating in business plan competitions at which they pitch their ideas to business professionals, and attending workshops on leadership and business skills.
Research Design and Methods: In this project, the research team will use a quasi-experimental design to identify comparison students who are similar to VE participants based on observable student characteristics, prior academic progress, and baseline outcomes (e.g., career readiness, work-based learning participation, interest in business). All participating schools currently have VE programs. Researchers will measure the main outcomes using administrative data held by the district (including National Student Clearinghouse data). The research team will also conduct student baseline (at the beginning of the intervention school year) and endline (at the end of the intervention school year) surveys that include assessments of students' leadership and professionalism skills. They will rely on site visits, teacher surveys, and student focus groups to assess fidelity of implementation.
Control Condition: Control students will not take the VE class, but the exact nature of the business-as-usual counterfactual may vary depending on the alternative courses available at the school. The researchers will conduct analysis of administrative data, hold discussions with school administrators, and use student surveys to identify whether control students engage in any activities (e.g. business plan competitions) similar to those offered to VE students.
Key Measures: Confirmatory outcome measures will be senior year GPA, attendance, college credits earned in high school, and postsecondary enrollment. The researchers will also explore several exploratory outcomes. Mediators include college and career readiness, academic motivation, financial literary, leadership, and professionalism. Moderators will include pre-program college and career readiness and academic motivation, position held in the VE firm, fidelity of implementation, and student demographics.
Data Analytic Strategy: The research team will combine propensity score matching with a difference-in-difference approach (when baseline outcomes are observed) that accounts for time-invariant and time-varying student characteristics. The team will assess mediators through structural equation modeling and will examine moderation by interacting treatment status with each moderator. To identify implementation fidelity and challenges to implementation, they will systematically code and analyze interview and focus group data using a coding structure based on the adherence constructs and variables.
Cost Analysis: The research team will use the "ingredients method," summing up the costs of all "ingredients" required for implementation of the VE program. They will group costs into four categories: program payments to VEI, personnel costs, materials, and facilities. They will calculate the cost-effectiveness ratio (CER) by dividing the per-student cost by the estimated effect sizes.


Back