Inside IES Research

Notes from NCER & NCSER

A2i: From Research to Practice at Scale in Education

This blog post is part of an interview series with education researchers who have successfully scaled their interventions.

Assessment-to-Instruction (A2i) is an online Teacher Professional Support System that guides teachers in providing Kindergarten to Grade 3 students individualized literacy instruction and assessments. Students complete the assessments independently online without the teacher taking time away from instruction. A2i generates instantaneous teacher reports with precise recommendations for each student and group recommendations. See a video demo here. Between 2003 and 2017, researchers at Florida State University (FSU) and Arizona State University (ASU), led by Carol Connor, developed and evaluated A2i with the support of a series of awards from IES and the National Institutes of Health. Findings from all publications on the A2i are posted here.

While results across seven controlled studies demonstrated the effectiveness of A2i, feedback from practitioners in the field demonstrated that implementation often required substantial amounts of researcher support and local district adaptation, and that the cost was not sustainable for most school district budgets. In 2014, the development firm Learning Ovations, led by Jay Connor, received an award from the Department of Education (ED) and IES’s Small Business Innovation Research program (ED/IES SBIR) to develop an technologically upgraded and commercially viable version of A2i to be ready to be used at scale in classrooms around the country. In 2018, with the support of a five-year Education Innovation and Research (EIR) expansion grant from ED totaling $14.65 million, A2i is now used in more than 110 schools across the country, with plans for further expansion. 

 

Interview with Carol Connor (CC) and Jay Connor (JC)

From the start of the research in the early 2000s, was it always the goal to develop a reading intervention that would one day be used on a wide scale?
CC: Yes and no. First, we had to answer the question as to whether individualization was effective in achieving student literacy outcomes. Once the research established that, we knew that this work would have wide-scale application.

When did you start thinking about a plan for distribution
CC: Before embarking on the cumulative results studies, in 2008, Jay said that we needed to know who the “customer” was… i.e., how purchasing decisions were made at scale.  His 2008 Phase I ED/IES SBIR was critical in shifting our research focus from individual classrooms to school districts as the key scaling point. 

Did you work with a technology transfer office at the university?
CC: Only to the extent of contractually clarifying intellectual property (IP) ownership and licensing. 

Who provided the support on the business side?
CC: Jay, who has an MBA/JD and has been a senior officer in two Fortune 100 companies was very instrumental in guiding our thinking of this evolution from important research to practical application. 


 Do you have any agreement about the IP with the university? What were the biggest challenges in this area?

JC: Yes, Learning Ovations has a 60-year renewable exclusive licensing agreement with FSU Foundation. FSU couldn’t have been better to work with.  Though there were expected back-and-forth elements of the original negotiations, it was clear that we shared the central vision of transforming literacy outcomes.  They continue to be a meaningful partner.

When and why was Learning Ovations first launched?
JC: In order to pursue SBIR funding we needed to be a for-profit company.  At first, I used my consulting business – Rubicon Partners LLP – as the legal entity for a 2008 Phase I award from ED/IES SBIR.  When we considered applying (and eventually won) a Fast Track Phase I & II award from SBIR in 2014, it was clear that we needed to create a full C – Corp that could expand with the scaling of the business, thus Learning Ovations was formed.

Who has provided you great guidance on the business side over the year? What did they say and do? 
JC: Having run large corporate entities and worked with small business start-ups in conjunction with Arizona State University (Skysong) and the University of California, Irvine (Applied Innovation at The Cove) and having taught entrepreneurship at The Paul Merage School of Business at UC Irvine, I had the experience or network to connect to whatever business guidance we needed.  Further, having attended a number of reading research conferences with Carol, I was quite conversant in the literacy language both from the research side and from the district decision maker’s side.

How do you describe the experience of commercializing the A2i? What were the biggest achievements and challenges in terms of preparing for commercialization?

JC: Having coached scores of entrepreneurs at various stages, I can safely say that there is no harder commercialization than one that must stay faithful to the underlying research.  A key strategy for most new businesses: being able to pivot as you find a better (easier) solution.  It is often circumscribed by the “active ingredients” of the underlying research.  Knowing this, we imbued Learning Ovations with a very strong outcomes mission – all children reading at, or above, grade level by 3rd grade.  This commitment to outcomes certainty is only assured by staying faithful to the research.  Thus, a possible constraint, became our uncontroverted strength.

Do you have advice for university researchers seeking to move their laboratory research in education into wide-spread practice? 
JC:  Start with the end in mind.  As soon as you envision wide-scale usage, learn as much as you can about the present pain and needs of your future users and frame your research questions to speak to this.  Implementation should not be an after-the-fact consideration; build it into how you frame your research questions. On one level you are asking simultaneously “will this work with my treatment group” AND “will this help me understand/deliver to my end-user group.”  I can’t imagine effective research being graphed onto a business after the fact.  One key risk that we see a number of researchers make is thinking in very small fragments whereas application (i.e., the ability to go to scale) is usually much more systemic and holistic.

In one sentence, what would say is most needed for gaining traction in the marketplace?
JC: If not you, as a researcher, someone on your team of advisors needs to know the target marketplace as well as you know the treatment protocols in your RCT.

____________

Carol Connor is a Chancellor’s Professor in the UC Irvine School of Education. Prior she was a professor of Psychology and a Senior Learning Scientist at the Learning Sciences Institute at ASU. Carol’s research focuses on teaching and learning in preschool through fifth grade classrooms – with a particular emphasis on reading comprehension, executive functioning, and behavioral regulation development, especially for low-income children.

Joseph “Jay” Connor, JD/MBA, is the Founder/CEO of Learning Ovations, Inc, the developer of the platform that has enabled the A2i intervention to scale.  Jay has 20+ years of experience in senior business management at the multi-billion dollar corporate level, and has experience in the nonprofit and public policy arenas.

This interview was produced by Edward Metz of the Institute of Education Sciences.

Inside IES Special Interview Series: From University Research to Practice at Scale in Education

Over two decades, the National Center for Education Research and the National Center for Special Education Research at IES have built a knowledge base to inform and improve education practice. This work has also spurred the development of evidence-based tools, technological products, training guides, instructional approaches, and assessments. 

While some IES-supported interventions are used on a wide scale (hundreds of schools or more), we acknowledge that a “research to practice gap” hinders the uptake of more evidence-based interventions in education.  The gap refers to the space between the initial research and development in university laboratories and pilot evaluations in schools, and everything else that is needed for the interventions to be adopted as a regular practice outside of a research evaluation.

For many academic researchers, advancing beyond the initial stage of R&D and pilot evaluations is complex and often requires additional time, financing, and specialized expertise and support. For example, interventions often need more R&D to ready interventions for scale—whether to ensure that implementation is turnkey and feasible without any researcher assistance, that interventions work the same across divergent settings and across different populations, or to bolster technology systems to be able to process huge amounts of data across numerous sites at the same time. Advancing from research to practice may also entail commercialization planning to address issues such as intellectual property, licensing, sales, and marketing, to facilitate dissemination of interventions from a university to the education marketplace, and to sustain it over time by generating revenue or securing other means of support.

Special Inside IES Research Interview Series

This winter and spring, Inside IES Research is publishing a series of interviews with the teams of researchers, developers, and partners who successfully advanced IES-funded education research from the university laboratory to practice in schools at scale.  Collectively, the interviews illustrate a variety of models and approaches for scaling evidenced-based interventions and for disseminating and sustaining the interventions over time.

Each interview will address a similar set of questions:

  • Was it part of the original plan to develop an intervention that could one day be used at scale in schools?
  • Describe the initial research and development that occurred. 
  • What role did the university play in facilitating the research to practice process? 
  • What other individuals or organizations provided support during the process?
  • Beyond the original R&D process through IES or ED grants, what additional R&D was needed to ready the intervention for larger scale use?
  • What model was used for dissemination and sustainability?
  • What advice would you provide to researchers who are looking to move their research from the lab to market? What steps should they take? What resources should they look for?

Check this page regularly to read new interviews.

We hope you enjoy the series.

This series is produced by Edward Metz of the Institute of Education Sciences

Calling All Students to the Mars 2020 “Name the Rover” Contest

On August 27, 2019, NASA launched a national contest for Kindergarten to Grade 12 students to name the Mars 2020 rover, the newest robotic scientist to be sent to Mars.  Scheduled to launch aboard a rocket in July 2020 from Cape Canaveral Air Force Station in Florida and touch down on Mars in February 2021, the to-be-named rover weighs more than 2,300 pounds (1,000 kilograms) and will search for astrobiological signs of past microbial life, characterize the planet’s climate and geology, collect samples for future return to Earth, and pave the way for human exploration of the Red Planet.

By focusing the Mars 2020 “Name the Rover” contest on K to 12 students, NASA seeks to engage U.S. students in the engineering and scientific work that makes Mars exploration possible. The contest also supports national goals to stimulate interest in science, technology, engineering, and mathematics (STEM) and help create the next generation of STEM leaders.

Students can sign up and submit their entries for the competition at https://www.futureengineers.org/nametherover. Entries must include a proposed name for the rover and a short essay of 150 words or less explaining the reasons for the name. NASA will select 156 state winners (one from each state and age group), before narrowing down to the top 9 entries that will be part of a public poll. The grand prize winner who will name the rover will be selected and announced in spring of 2020.

Even if you are not a student you can still participate. US residents over the age of 18 can apply to be judges for the contest to help NASA make their selection.

The Mars 2020 Project at NASA’s Jet Propulsion Laboratory manages rover development for NASA’s Science Mission Directorate. NASA’s Launch Services Program, based at the agency’s Kennedy Space Center in Florida, is responsible for launch management.

NASA Partners with an ED/IES SBIR Awardee to Run the Contest

The education technology firm that NASA selected to help run the competition is Burbank, California-based, Future Engineers.  The “Name the Rover” contest leverages Future Engineers’ online challenge platform, which was developed with the support of a 2017 award from the US Department of Education and Institute of Education Sciences’ Small Business Innovation Research program (ED/IES SBIR).  The platform will receive, manage, display, and judge what is anticipated to be tens of thousands or more student submissions from around the country.

Future Engineers has a history of collaborating on space-themed student challenges. The company previously ran a national competition series in 2018 for the ASME Foundation with technical assistance from NASA, where K-12 students submitted digital designs of useful objects that could be 3D printed on the International Space Station, resulting in the first student-designed 3D print in space.

Future Engineers developed its platform to be an online hub for classrooms and educators to access free, project-based STEM activities, and to provide a portal where students submit and compete in different kinds of maker and innovation challenges across the country. The Mars 2020 “Name the Rover” contest will be the first naming challenge issued on its platform.

We look forward to the results of the competition!

Originally posted on the U.S. Department of Education’s Homeroom blog.


Edward Metz is a research scientist at the Institute of Education Sciences in the US Department of Education.

Bob Collom is an integration lead in the Mars Exploration Program at NASA Headquarters.


About ED/IES SBIR

The U.S. Department of Education’s Small Business Innovation Research program, administered by the Institute of Education Sciences (IES), funds projects to develop education technology products designed to support students, teachers, or administrators in general or special education. The program emphasizes rigorous and relevant research to inform iterative development and to evaluate whether fully-developed products show promise for leading to the intended outcomes. The program also focuses on commercialization once the award period ends so that products can reach students and teachers and be sustained over time. ED/IES SBIR-supported products are currently used by millions of students in thousands of schools around the country.

About NASA’s Mars Exploration Program (MEP)

NASA’s Mars Exploration Program (MEP) in the Planetary Science Division is a science-driven program that seeks to understand whether Mars was, is, or can be, a habitable world. To find out, we need to understand how geologic, climatic, and other processes have worked to shape Mars and its environment over time, as well as how they interact today. To that end, all of our future missions will be driven by rigorous scientific questions that will continuously evolve as we make new discoveries. MEP continues to explore Mars and to provide a continuous flow of scientific information and discovery through a carefully selected series of robotic orbiters, landers and mobile laboratories interconnected by a high-bandwidth Mars/Earth communications network.

Computational Thinking: The New Code for Success

Computational thinking is a critical set of skills that provides learners with the ability to solve complex problems with data. The importance of computational thinking has led to numerous initiatives to infuse computer science into all levels of schooling. High-quality research, however, has not been able to keep up with the demand to integrate these skills into K–12 curricula. IES recently funded projects under the Education Research Grants, the Small Business Innovation Research, and the Low-Cost, Short-Duration Evaluation of Education Interventions programs that will explore computational thinking and improve the teaching and learning of computer science.

 

  • Greg Chung and his team at the University of California, Los Angeles will explore young children’s computational thinking processes in grades 1 and 3. The team will examine students’ thought processes as they engage in visual programming activities using The Foos by codeSpark.
  • The team from codeSpark will develop and test a mobile game app for grade schoolers to learn coding skills through creative expression. The game supports teachers to integrate computational thinking and coding concepts across different lesson plans in English Language Arts and Social Studies.
  • VidCode will develop and test a Teacher Dashboard to complement their website where students learn to code. The dashboard will guide teachers in using data to improve their instruction.
  • Lane Educational Service District will work with researchers from the University of Oregon to evaluate the impact of the district’s Coder-in-Residence program on student learning and engagement.

IES is eager to support more research focused on exploring, developing, evaluating, and assessing computational thinking and computer science interventions inclusive of all learners. IES program officer, Christina Chhin, will speak at the Illinois Statewide K-12 Computer Science Education Summit on September 20, 2019 to provide information about IES research funding opportunities and resources focusing on computer science education.

IES Makes Two New Awards for the Development of Web-based Tools to Inform Decision Making by Postsecondary Students

In June, the Institute of Education Sciences (IES) announced two new awards to technology firms to develop web-based tools that inform student decision making in postsecondary education. The projects will focus on generating a measure of the return of investment (ROI) for different educational training programs and careers so that high school and college students have access to data-driven information to guide their decisions.

The awards were made through a special topic offered by the IES Small Business Innovation Research (known as ED/IES SBIR) program, which funds the research and development of commercially viable education technology. (For information on the 21 awards made through the IES 2019 standard solicitation, read here.)

Background and Awards

While websites like College Scorecard and CareerOneStop provide information to explore training programs in colleges and occupations of interest, there is no tool that helps students understand the costs and benefits of individual postsecondary programs in an integrated, customizable, and user-friendly manner.  

The special topic SBIR solicitation requested proposals from small businesses to develop new ROI tools that would combine information on fees, time to complete, and projected earnings so that students can easily compare college and career pathways. The IES-funded ROI tools aim to improve student program completion rates, with higher employment and earnings, less education-related debt, and more satisfaction with their selected paths. The special topic SBIR solicitation offered up to $200,000 for firms to develop and evaluate a prototype of their ROI tool. 

Two awards were made through this special topic:

  • Illinois-based BrightHive, Inc. is developing a prototype of the Training, Education, and Apprenticeship Program Outcomes Toolkit (TEAPOT). Designed to inform student training and educational decision making over a variety of potential pathways, TEAPOT will improve the flow and accuracy of data resulting in improved estimates of the ROI for different postsecondary education pathways.  The team will develop a data interoperability system and simplified toolkit for states and local postsecondary and workforce development organizations. The toolkit will provide more high quality, consistent, and granular information on postsecondary outcomes. The prototype will calculate ROI using student information, programmatic information (with an emphasis on net program costs to allow for variations by program type at the same institution), and access to wage and employment data sets.
  • Virginia-based Vantage Point Consultants is developing a prototype of a user-contextualized ROI tool that prospective students will use to make meaning of lifetime costs and opportunity tradeoffs associated with different degree programs offered by postsecondary institutions. The ROI tool will incorporate information on student goals and academic, professional, and personal characteristics.  The prototype will include an interface to present information to guide decision making based on an ROI calculation that discounts earning cash-flows under current and future state career and education assumptions, while subtracting college cost. In the first phase of work, the project will use information from data partners including Burning Glass Technologies and from public sources at the Department of Labor and Department of Education.

After developing prototypes, researchers will analyze whether the tools function as intended and are feasible for students to use. Research will also test if the tool shows promise for producing a meaningful and accurate measure of ROI.  Both firms are eligible to apply for additional funding to complete the full-scale development of the ROI tool, including developing an interface to improve user experience and conducting additional validation research.

Stay tuned for updates on Twitter (@IESResearch) as IES projects drive innovative forms of technology.

Written by Edward Metz, program manager, ED/IES SBIR