Co-Principal Investigators: Philip Poekert (University of Florida), George Michailidis (University of Florida), and Sidney D'Mello (University of Notre Dame)

Project Website: http://www.virtuallearninglab.org/

Purpose: The Virtual Learning Lab will bring together expertise in informatics (the study of information processing in the context of computer systems), mathematics education, and teacher professional development to launch the emerging field of precision education, in which prior students' data are used to support decisions about the learning opportunities provided to future students. Precision education is now becoming possible due to the availability of large data sets accumulated as students work with technology-based systems, combined with advances in learning analytics, an approach to data collection and analysis that seeks to identify optimal learning sequences for individual students. The Lab's program of research is organized around the goal of improving the impact of Algebra Nation, a free online learning platform for students enrolled in algebra and their teachers, which is designed to promote mastery of basic algebra. Algebra Nation is currently in use throughout the state of Florida, with plans for expansion to other states.

Research Projects: The Virtual Learning Lab will conduct a focused program of research to learn whether precision education can improve students' ability to pass an end-of course exam. The researchers will address the challenge of improving low-achieving students' achievement in algebra, using Algebra Nation, a free online learning platform for students and teachers designed to promote mastery of basic algebra, as a virtual laboratory. The team will personalize instruction using learning analytics; develop sensor-free estimates of engagement during learning (i.e., using students' interactions with the technology and contextual cues to infer engagement); and design professional development to help teachers use learning analytics to differentiate their instruction. This research program will serve as a national model for developing online systems that can use prior student data to optimize, update, and refine the instruction provided to current users.

Population/Sample: Participating students will typically be in ninth grade and will participate through their algebra courses. Three large school districts and the Florida Department of Education have formally agreed to be partners in the research. These districts reflect the broad cultural and economic diversity of Florida.

Measurement Study 1: In Years 1 and 2, the research team will use data from prior students of Algebra Nation to develop recommendations for new users of the platform. They will use student records from across the state of Florida for three school years (a separate group of students each year) in which Algebra Nation was implemented, including students' prior year's test scores, demographic information, and end-of-year test scores. The research team will mine the data to identify paths taken through Algebra Nation by students who performed well and those who did not. These analyses will inform the development of a personalized recommendation system, which will be implemented in Algebra Nation. The research team will pilot test a personalized version of Algebra Nation in six to eight classrooms to evaluate feasibility and to solicit feedback from students about whether the personalized recommendations for resources and activities are helpful to them and are at the appropriate difficulty level. Analyses will also examine optimal classroom implementations of Algebra Nation, which will provide the foundation for the development of an effective professional development program to help teachers implement Algebra Nation in their classrooms and use the rich student data available to personalize instruction.

Measurement Study 2: In Years 1 and 2, the research team will develop an automated measure of engagement, which will be used to enhance the personalized learning models that will be implemented in Algebra Nation. The engagement measure will be developed by applying machine learning methods (i.e., methods that use data to develop models and algorithms that can make predictions) to data about students' actions in Algebra Nation and relevant contextual cues. In small-scale studies in school computer labs, students will interact with Algebra Nation, and self-reports of engagement will be administered. Researchers will use these data to validate the accuracy of the engagement measure. In a subset of the computer lab sessions, the researchers will conduct live field observations and record videos of student engagement, affect, and behavior. These data will be used to obtain evidence for the convergent validity of the automated engagement measure.

Experimental Study 1: In Year 3, the research team will conduct a pilot study using a quasi-experimental design. They will implement the personalized version of Algebra Nation in 30 classrooms (approximately 750 students) and will use 120 classrooms that used the original Algebra Nation in the previous year as the control condition. The Florida end-of-year course exam for algebra will serve as the primary outcome measure, along with estimates of engagement derived from log file data using the automated techniques described above, with the prediction that estimated engagement levels should be higher for students who work with the personalized version of Algebra Nation.

Experimental Study 2: In Year 4, the research team will conduct a randomized controlled trial that will compare the personalized version of Algebra Nation to the original, non-personalized version. A total of 111 schools, with two classes taught by different teachers from each school, will participate in this study. Researchers will recruit classrooms in Florida and other states, depending on the progress of Algebra Nation's national expansion plan. Within each school, the research team will randomly assign one classroom to the personalized version of Algebra Nation and the other to the original version of Algebra Nation. Each classroom is expected to include 25 students on average, for a student sample of 2,825. Key measures will include the researcher-developed measure of engagement, the Florida Comprehensive Assessment Test, and the Florida Standards Assessment. In addition, the research team will collect student data (demographic data, archival data, process data from interactions with the Algebra Nation system, and student performance on the end-of-course algebra exam, class grades, and pass/fail outcomes), data about teachers' experience and credentials, and log file data to assess fidelity of implementation of Algebra Nation. The research team will use multi-level modeling to analyze the data. Data analysis and dissemination will continue through Year 5.

Leadership and Dissemination Activities: The Virtual Learning Lab will serve as a national hub for the emerging field of precision education and the use of learning analytics to improve online learning platforms for teachers and students. Outreach activities will include summer workshops for students, an annual analytics competition, collaborative sessions at conferences, and supplemental studies in which researchers can collaborate with the team on projects involving shared data and analytic approaches. The Lab will communicate its findings through scholarly outlets, traditional news media, social media, and their website in order to reach researchers, practitioners, policymakers, and the public.