Inside IES Research

Notes from NCER & NCSER

NCER’s Investments in Education Research Networks to Accelerate Pandemic Recovery Network Lead Spotlight: Dr. Rebecca Griffiths, LEARN Network

Welcome to the first installment of the NCER research network leads spotlight series! With funding from the American Rescue Plan (ARP), NCER has invested in research grants that will generate information about accelerating learning that is useful, usable, and used. The awardees, who are members of these new research networks, are addressing the urgent challenges faced by schools as they support students’, teachers’, and school districts’ recovery in the aftermath of the pandemic.

Today, we’ll take you through our conversation with Dr. Rebecca Griffiths, senior principal education reporter at SRI International, and hear about the Leveraging Evidence to Accelerate Recovery Nationwide Network (LEARN Network).

 

NCER: What are some of the biggest challenges facing education systems, teachers, and learners post-COVID, and what are some ways that education researchers can help to target solutions to those challenges?

Dr. Rebecca Griffiths: The biggest challenges facing education systems, teachers, and learners post-COVID are not new, for the most part; rather, they are long-standing problems and inequities that have worsened. To put a finer point on it: while all students lost ground academically, students from underserved and underresourced communities were disproportionately impacted by the pandemic, further exacerbating existing academic disadvantages. 

For education researchers hoping to support COVID recovery by introducing evidence-based programs and practices, timing is an issue. Designing a new curriculum or intervention typically takes years of development and testing, and we (as a country) don’t have that luxury. Fortunately, we don’t have to reinvent the wheel. What we can do is focus attention on solutions that already have proven effectiveness, which means making sure that educators know they exist and have support for implementing them. Some adaptation may be needed given the urgency of the current circumstances. For example, if the number of students reading two years below grade level jumped from a handful to a large share, teachers will need a different way to meet that need. So an intervention or online tutoring system may need to be adapted to serve many more kids. 

There are a few implications here for researchers who develop products such as curriculums and interventions that are intended to impact student learning:  

  1. We should be thinking about how our products and interventions can be adapted to meet needs with greater urgency at a larger scale. This may mean that the implementation process needs to be simplified, streamlined, or reconfigured to support new participants (such as parent tutors) in the educational process.  
  2. We should ensure that the products and interventions we provide fit with the needs, environments, and decision-making processes of educators. Gold-standard efficacy studies will not make a solution attractive to users if the solution doesn’t address a high-priority need, is overly difficult and expensive to implement, or doesn’t fit the criteria of various stakeholders who have a say in selecting products and interventions for their schools. We need to attend to the user environment, which we can do by ensuring the communities we aim to serve have a voice in designing solutions. 
  3. We can do a much better job with how we typically disseminate information about evidence-based products. “Dissemination” sounds a bit like dropping a bunch of leaflets out of an airplane, but actually requires a much more energetic stance than this word implies. Effective dissemination integrates at least four activities that commercial providers typically undertake to get their solutions out into the world: building interest in and awareness of a solution (marketing); persuading people that a solution is the best choice for their needs and that they should dedicate resources to it (sales); making sure that people have access to a solution (distribution); and making sure that people have the support they need to implement a solution with integrity (customer support). Those of us who develop educational products and interventions need to think beyond journal publications and academic conferences if we want to reach a meaningful share of our target users. Of course, not all researchers have the capacity or desire to undertake these activities, and in these cases, we might consider alternate pathways to scale, such as licensing our intellectual creations to others (e.g., curriculum publishers or entrepreneurs) who are equipped and appropriately motivated to take these steps.

NCER: What is your view of a research network, and how does it differ from a traditional education research project?

Dr. Griffiths: A research network seeks to amplify the impact of its members so that the whole is greater than the sum of its parts, and to generate lessons learned for a broader community. We can do this by identifying promising strategies and shared challenges within the network, facilitating a learning community to share expertise and experience among network members in service of overcoming those challenges, and then documenting these processes and successes to share with the broader field. 

NCER: What are the specific goals for this network, and how does it support the goals of the ARP?

Dr. Griffiths: The LEARN Network (which stands for Leveraging Evidence to Accelerate Recovery Nationwide) is led by SRI and includes four teams of researchers focused on scaling existing EBPs in K–8 literacy or mathematics: Targeted Reading Instruction  (TRI) for students in K–3, integrating  for students in grades 2–6, Peer-Assisted Learning Strategies (PALS)  Reading to improve learning for underrepresented student groups, and an adaptation of the Strategic Adolescent Reading Intervention (STARI) for underserved middle-grade students.

The network has two related goals. One is to adapt and scale adoption of evidence-based practices and products that can help educators address the negative impacts of the COVID-19 pandemic on learning, particularly for students who were disproportionately harmed. Of our four product teams, three are focused on K–8 literacy, especially supporting students who are reading below grade level, and the fourth is focused on an intervention for fifth-grade math. As the 2022 NAEP scores showed, early math and literacy are critical areas in which we need to help students make up lost ground. As network lead, SRI will help these projects prepare for scaling.

Longer term, the LEARN Network will provide a model, tools, and resources to inform the development and adoption of educational innovations. These resources will help education researchers ensure that their innovations are designed from the ground up with the potential to achieve impact at scale for students. They will also help practitioners and policymakers know what to consider when investing in educational innovations.

NCER: How do you envision this network working to reach those goals? What’s the value added for building this network relative to a set of independent research teams?

Dr. Griffiths: In the LEARN Network, SRI is assembling and deploying a pool of scaling, equity, and research experts to accelerate progress. The product teams are experts in their respective fields of literacy and math education, which we can augment with a more demand-driven perspective of their research products. We can take them through a structured process of investigating stakeholders’ needs, aligning products and practices to educators’ environments and selection criteria, and developing effective dissemination strategies. SRI brings unique expertise to this task, given our history of transitioning inventions from laboratories to market.

We are also assembling a group of external advisors with networks and expertise in rural and small-town schools, state educational systems, ed-tech investing and entrepreneurship, and implementation science.

Through the LEARN Network, product teams have access to a rich set of perspectives and expertise that would be impractical to build for individual projects.        

In addition, SRI will conduct some original research to advance our understanding of how educators and educational agencies select and adopt products and interventions. We know these processes are confounding to many researchers, in part because they vary so much by school, district, and state. At the same time, we believe we can help our network and the broader research community by shedding some light on a few key questions, such as how these processes may differ by product type (e.g., complete curriculum vs. supplementary resources), district characteristics (e.g., size, locale), and other key factors. Our research will also explore current barriers or challenges to identifying EBPs aligned with their contexts and students’ needs and explore what resources or tools would make it easier to do so.

Last, the product teams include seasoned researchers with decades of experience developing and disseminating evidence-based practices and products. They bring valuable perspectives from these experiences, and they are also investigating some similar questions about how educators discover and decide what tools to use. As network lead, we aim to create spaces and facilitate conversations so that all the teams can learn from each other.

NCER: What approaches do you propose to use to cultivate a meaningful connection among the research teams in the network? What are some challenges in bringing independent research teams together like this?

Dr. Griffiths: Our aim is to be very responsive to what the product teams tell us they want help with, while encouraging them to aim high with their scaling goals. In education research, we often think of scaling in terms of growing implementation from a few schools to a few dozen schools. What if we reframed our perspective to consider “reach”? As in, What share of the nation’s 100,000 public schools, or a particular population of students, are we reaching? That really shifts how we think about what kind of organizational infrastructure or strategic choices are needed to have a meaningful impact. 

Our purpose as network lead is to help network members be successful, and to do that, we know that we need to demonstrate our ability to add value. The product teams all have ambitious goals and tight timelines, and we are mindful of that. Fortunately, the product teams were already aware of the Invent-Apply-Transition framework that SRI pioneered and saw how it could be helpful to them. In order to support meaningful connections among the teams, we are facilitating regular cross-team meetings, each focused on a particular challenge (for example, stakeholder mapping, product-user fit, dissemination strategies). In these working sessions we will draw upon expertise residing in the product teams, in SRI’s education division and our unit that is focused on transitioning inventions to market, and among our external advisors. We anticipate that these will be rich, generative sessions that will provide the product teams (and SRI) with new insights about pathways to scale.

NCER: Are there some generalizable tools or lessons learned that are likely to come out of this network project that you think will benefit the education research community as a whole?

Dr. Griffiths: As I mentioned, we are drawing heavily from SRI’s Invent-Apply-Transition framework to guide product teams through the process of preparing to scale. As we do this, we are developing tools and resources specifically for scaling education products that will be accessible to a broader community of researchers who aspire to have a wide-reaching impact. We also expect to learn some things through our work with the product teams that we can share through briefs and presentations. In addition, we are considering how we might design engagements for a broader community of researchers that allow for more-interactive sharing of tools, resources, and lessons. Stay tuned!


Thank you for reading our conversation with Dr. Rebecca Griffiths! Come back tomorrow for our next network lead spotlight!  

 

Save the Date: Leveraging Evidence to Accelerate Recovery Nationwide (LEARN) Network Launch Event

Join us on January 19, 2023, from 3pm EST-4:30pm EST, as members of the IES-funded LEARN (Leveraging Evidence to Accelerate Recovery Nationwide) Network convene publicly for the first time to share their network's goals and vision. Learn more from the network teams during this virtual event

and hear from IES Director Mark Schneider about his hopes for the LEARN Network in the coming years as IES looks to the future with a focus on progress, purpose, and performance.

The LEARN Network was established to focus on adapting and preparing to scale existing, evidence-based products to address learning acceleration and recovery for students in K-12, particularly for students from underrepresented groups disproportionately affected by the COVID-19 pandemic. In addition to generating solutions to the nation’s most pressing challenges to COVID-19 recovery within the education sector, IES expects that the combined efforts of this network will lead to the establishment of best practices for how to prepare to effectively scale evidence-based products.

The LEARN Network includes a scaling lead and four product teams. The scaling lead, led by a team at SRI International, is facilitating training, coaching, and collaboration activities with product teams; ensuring educator needs and perspectives are addressed; and providing a model for the field that ensures evidence-based products are developed with the potential to achieve impact at scale for students—particularly those in most need—from the start. Product teams are focused on preparing to scale literacy products for students in K-3 (Targeted Reading Instruction; Grantee: University of Florida), 4th-5th grade (Peer-Assisted Learning Strategies; Grantee: AIR), and middle school (Strategic Adolescent Reading Intervention; Grantee: SERP) as well as a math product for students in 5th grade (Classwide Fraction Intervention combined with Math Peer-Assisted Learning Strategies; Grantee: AIR).

Registration is now open, and we hope to see you there! For more information on the event and to register, visit https://learntoscale.org/

New Grant to Develop a Learning Game About the Supreme Court Features ED/IES SBIR Education Technology Platform

In April 2022, iCivics was awarded a $400,000 grant from the National Endowment for the Humanities (NEH) to develop Supreme Justice, a live-action multiplayer experience that simulates the deliberation process used by Supreme Court justices.  The game is being developed in partnership with small business Gigantic Mechanic and deployed using their VOXPOP platform, which was developed through the ED/IES SBIR program with awards in 2018 and 2019.

VOXPOP is a technology-enabled, class-wide, role-playing game for high school students. Using any web browser, teachers access a library of simulations on a range of topics drawn from the AP U.S. history curriculum and Common Core History Standards. VOXPOP’s platform provides resources to guide implementation, including videos, individualized student profiles, and real-time voting, and facilitates each student playing a unique historical role. Throughout the experience, the software guides participants with facts and primary sources, with students engaging in face-to-face discussions, and debating issues central to the simulation. All VOXPOP content can be found at www.voxpop.io

On July 15, VOXPOP was selected as the winner for the “Best Civics Game” through the Games For Change 2022 Awards. This annual award competition recognizes the year's best games for social impact and learning. All awards are competitive with submissions are evaluated by expert jurors.

In the new Supreme Justice game, students in grades 6 to 12 participate in a live-action multiplayer simulation focused on freedom of speech, freedom of expression, and due process rights. Classrooms are divided into different groups to play petitioners, respondents, and justices as they argue and consider cases of constitutional law. Supreme Justice will model deliberation and critical thinking in a civic setting, grounded in historical cases and relying heavily on the U.S. Constitution as evidence. The experience will engage students in face-to-face discussions and debates, while collaborating to craft arguments central to civic and government life. Once the game is developed, it will be freely available on the iCivics website.

 

 

Along with the iCivics NEH grant, Gigantic Mechanic is partnering with other organizations and museums, such as Revolutionary Spaces (see video below), to develop custom role-plays and simulations for their platforms and spaces.

 

 


Stay tuned to @IESResearch for news and updates on research, initiatives, and project updates in the area of tutoring to accelerate learning.

For more information, please contact Edward Metz (Edward.Metz@ed.gov), research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.

 

Technology Facilitated Tutoring Programs to Accelerate Learning

It goes without saying, challenges caused by COVID-19 in the field of education remain widespread and have the potential to be long lasting. A recent report confirmed that during the pandemic the move to full-time remote schooling was related to a decrease in student achievement, especially in high-poverty districts. Now over two years later, school leaders continue to employ strategies to address learning loss, such as intensive tutoring programs. This is because decades of research support the effectiveness of in-person tutoring to accelerate learning, and recent research also shows positive effects for high dosage virtual tutoring for struggling learners during the pandemic.

While the human-to-human interaction will always be central to a quality tutoring experience, technology offers unique functionalities to enrich and extend tutoring. For example, new models of technology facilitated tutoring programs—

  • Engage students with game-based and multi-media content that adjusts to the level of the individual and generate real-time tips to scaffold learning
  • Employ tools such as virtual whiteboards and data visualizations to enrich the virtual workspace for the tutor and student
  • Use dashboards to present real-time data driven insights for tutors to track student progress and individualize instruction
  • Provide automated professional development and training opportunities to prepare tutors

Technology also enables schools and community organizations to offer remote tutoring programs at scale—to reach students anywhere and anytime, including after school and during the summer. This of course depends on student access to technology and the availability of qualified tutors.  

Four IES Supported Technology-Based Tutoring Programs

In spring 2020, to help address the crisis in education caused by the pandemic, four teams of IES-funded developers adapted and extended their learning technologies for remote tutoring to be ready to be used at scale. The technologies are all research based and offer unique capabilities to strengthen the tutoring experience and to allow programs to reach more students. Each of the following programs described below can now deliver tutoring in schools or remote settings.

A2i by Learning Ovations. A2i (watch video) is a web-based product for students in kindergarten to grade 3 that continually assesses reading and generates data-driven recommendations to inform instruction. A2i is used in hundreds of schools by tens of thousands of students each year. Originally developed for in-school use through multiple IES and other government awards, research demonstrates the efficacy of A2i to improve student reading.

At the beginning of COVID-19, the Community Literacy Support System was designed to extend A2i for tutoring at student homes and other non-school locations. The program provides customized lessons and data visualization tools for tutors and parents. In the past two years, the tutoring program has been used by 9,200 students across 5 different states, at 23 different community sites (such as this one), and within almost 8,000 homes.

ASSISTments by ASSISTments. ASSISTments (watch video) is used by teachers to assign problems to students from curricula, such as EngageNY, Illustrative Mathematics, and Open Up Resources. Students receive real-time instructional feedback while doing problems online, while teachers receive reports with actionable insights to inform instruction. Initially developed through multiple awards from IES and other sources, in 2021-22, ASSISTments was used by over 5,000 educators and 200,000 students at schools around the country. Research by SRI International demonstrated that classrooms that used ASSISTments increased in learning course content compared to a control group.

At the onset of COVID-19, ASSISTments designed TutorASSIST, a tool to present data visualizations for tutors to target specific student needs through remote (or in-person) sessions. More than 750 students across Louisiana, Georgia, and Maryland used the tool to support remote tutoring during the pandemic. With a 2021 ED/EIR grant, ASSISTments is further developing its core product and tutoring tool to serve historically underserved students, including starting a school tutoring pilot program in eight schools in partnership with EnCorps Tutors, with a focus on optimizing the technology for tutoring.

SAGA Coach by SAGA Education and Simbulus. SAGA Education and Woot Math (watch video) employ interactive and game-based activities to support student math learning and a dashboard that generates data-driven insights to promote dialogue and discussion of complex topics between teachers and students. The school-based intervention reaches approximately 75,000 students per year. Woot Math was developed through an ED/IES SBIR award and through other sources. In 2021, Woot Math was acquired by and integrated within SAGA Education’s online math program.  

During the pandemic, SAGA Coach was designed to extend Saga Education and Woot Math for remote tutoring through the addition of an interactive whiteboard shared by tutors and students, and online training materials for tutors. During the 2020-21 school year, 5,500 students used SAGA Coach for high-dosage remote tutoring, and in 2022-23, the Boys and Girls Clubs of America will use SAGA Coach to deliver a remote tutoring program.

Lightning Squad by Sirius Thinking and Success For All. Lightning Squad (watch video) is a multimedia platform where pairs of students in grades 1 to 3 who are struggling readers collaborate to read stories and play games that are presented by the computer, while a tutor provides targeted support. Developed through an ED/IES SBIR award, the product is currently being evaluated through a multiyear efficacy trial and will be used in 50 Baltimore City elementary schools in fall 2022.  

At the onset of COVID-19 in 2020, Lightning Squad was adapted for remote delivery in eight Baltimore City schools serving principally low-income students. In the remote version, tutors use a video platform (for example, Zoom) with pairs of students in their respective homes. Students proceed with the activities of the software and respond verbally while a tutor types responses on the screen in real-time for each team member to see. In the 2020-21 school year, 16 Baltimore City schools used Lightning Squad with over 800 students, most for remote tutoring. Research conducted during the project by Success For All (not yet reviewed by the Department) found that students who were able to maintain consistent participation with remote tutoring gained 1.5 years of progress as measured by their initial placement and end of year placement, double the expected gains in reading during the period of school closures.  An additional 1,200 students were served remotely using Lightning Squad in other states during this same period.

Additional Related Resources on Tutoring to Accelerate Learning

  • Lightning Squad and SAGA Coach are part of Proven Tutoring, a coalition of technology-delivered tutoring programs with a mission to help educators learn about and access tutoring programs. These evidence-based programs have the potential to increase the achievement of students performing far below grade level due to COVID school closures or other factors.
  • In June 2021 during the ED Games Expo, IES partnered with AmeriCorps to host a webinar focusing on government and community partner initiatives to support remote tutoring to accelerate student learning during COVID-19.

In April 2022, AmeriCorps partnered with ED to produce a webinar on lessons from the field on the topic of high-dosage tutoring. AmeriCorps and the Department, along with Johns Hopkins University’s Everyone Graduates Center, are partners in the National Partnership for Student Success (NPSS). Launched in July 2022, the NPSS is committed to engaging 250,000 adults as tutors, mentors, and coaches in evidence-based programs designed to accelerate students’ recovery from the pandemic. 


Stay tuned to @IESResearch for news and updates on research, initiatives, and project updates in the area of tutoring to accelerate learning.

Edward Metz is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.

Melissa Moritz was the Afterschool and Summer Learning Fellow in the National Center for Education Evaluation and Regional Assistance (NCEE) at the US Department of Education’s Institute of Education Sciences. She currently serves as the Director of Policy for the STEM Next Opportunity Fund.

Please contact Edward.Metz@ed.gov with questions or for more information.

 

Google Acquires Intellectual Property for IES-Supported Education Technology Products Moby.Read and SkillCheck

On April 1, 2022, Google acquired the intellectual property (IP) rights for Moby.Read and SkillCheck, education technology products developed through IES programs by California-based Analytics Measures, Inc. (AMI). AMI will continue as a small business and is honoring school contracts that use Moby.Read and SkillCheck until 2024.

Moby.Read is a technology-delivered, fully automated, oral reading fluency (ORF) assessment that is self-administered by grade school students. As students read a passage aloud into a tablet, the speech-recognition software generates an assessment of ORF in real time through natural language processing software that analyzes text passages of the read-aloud performances. SkillCheck is a component of Moby.Read that employs natural language processing software to analyze recordings and produce interactive report pages that rate and illustrate the student's basic reading skills.

 

 

The technologies were developed over two decades with IES funding. Beginning in 2002, AMI designed several early prototypes to be used for ORFs as a part of the National Assessment of Educational Progress and other national assessments administered by IES’s National Center for Education Statistics. In 2016 and 2017, the IES Small Business Innovation Research program (ED/IES SBIR) funded AMI to develop and test Moby.Read to be used in schools at scale. With 2020 and 2021 ED/IES SBIR awards, AMI developed the SkillCheck as an additional component of Moby.Read to provide educators activities to inform instruction. AMI conducted research at key points over 20 years to validate the results of the assessment.

Since commercial launch in 2019, the Moby.Read and SkillCheck have been used for more than 30,000 student assessments in 30 states. Google acquired the Moby.Read and SkillCheck IP with plans to incorporate these tools into Google suite of products for education.

For additional information on the research, development, and commercialization of these technologies, see this Success Story on the ED/IES SBIR website.


Edward Metz is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences. Please contact Edward.Metz@ed.gov with questions or for more information.