Inside IES Research

Notes from NCER & NCSER

Exploring the Intersection of Special Education, Learning Analytics, and Psychometrics: A Journey in Education Research

This year, Inside IES Research is publishing a series of blogs showcasing a diverse group of IES-funded education researchers and fellows that are making significant contributions to education research, policy, and practice. In recognition of Asian American and Pacific Islander Heritage Month, in this interview blog we asked Dr. Xin Wei, a senior quantitative researcher at Digital Promise to discuss her career journey. Dr. Wei’s current IES-funded study uses statistical and machine-learning techniques to understand the test-taking behavior of National Assessment of Educational Progress (NAEP) grade 8 learners with and without disabilities.

How did you become interested in a career in education research?

As a child, I aspired to become a teacher, and in college I decided to pursue a degree in child development. During my senior year of college, I worked as a research assistant on a project studying statistical and psychometric methods used to analyze learning differences among children. This experience sparked my interest in education research and revealed the potential for statistical analysis to inform and enhance teaching practices.

Graduate studies at the University of Wisconsin-Madison and Stanford University helped me gain a deeper understanding of quantitative methods in education research. Through applying and improving quantitative methods, I discovered how national and state longitudinal datasets can help us understand the learning, social, and emotional needs of students with disabilities and which policy interventions can help us achieve better outcomes. This opportunity helped me understand the challenges students with disabilities face in the education system and deepened my appreciation for secondary data analysis and its power to inform intervention research.

Currently, my research focuses on analyzing log/process data to understand how digital learning and assessments can facilitate student learning, accurately measure progress, and improve outcomes for students with disabilities. Through this work, I am committed to advancing the education research field at the intersection of special education, learning analytics, and psychometrics.

What has been the biggest challenge you have encountered, and how did you overcome the challenge?

When I came to the United States to pursue a graduate degree at the age of 23, I faced a host of challenges that forced me out of my comfort zone. Navigating a new culture and adapting to academic expectations and research demands was overwhelming. Additionally, understanding U.S. K-12 education policies and practices was no easy feat. However, I was fortunate enough to have incredible mentors, professors, peers, and colleagues who provided me with guidance, support, and patience when I needed it most. These individuals played a crucial role in helping me grow as a researcher.

The most important lesson I learned from the challenges I faced was the value of continuous learning and growth in my career. These experiences have strengthened my commitment to making a positive impact in education and helping others who may be facing similar obstacles.

How can the broader education research community better support the careers and scholarship of researchers from underrepresented groups?

The student population in the United States is diverse, and it is essential that the education research community reflects that diversity by including scholars who bring unique perspectives and experiences.

One way to do this is by actively seeking out and valuing diverse voices in research, teaching, and leadership positions. This includes promoting diversity in conference panels, as well as actively recruiting and hiring researchers from underrepresented groups. By creating a culture of inclusivity, the education research community can better support the careers and scholarship of researchers from underrepresented groups.

Another way to better support the careers and scholarship of researchers from underrepresented groups is through mentoring programs, summer internships, and postdoc positions. These opportunities can provide valuable professional development and collaboration opportunities. In addition, research grants specifically targeted toward underrepresented groups can also help support their work and advance their careers. It is essential to widely advertise these opportunities and make them accessible to ensure that all researchers have an equal chance to participate.

In your area of research, what do you see as the greatest research needs or recommendations to address diversity, equity, and inclusion and to improve the relevance of education research for diverse communities of students and families?

To address diversity, equity, and inclusion in education research, it is crucial to adopt an asset-based approach when working with neurodiverse students. By shifting the focus from deficits to strengths, we can recognize and leverage their unique abilities, promoting more equitable educational practices. Additionally, targeted support should be provided to address the specific challenges underserved students face, ensuring inclusive learning environments. For instance, my research findings indicate that students with autism exhibit strengths in visuospatial reasoning and are drawn to STEM fields. However, autistic students may benefit from extra support to develop perseverance and improve their weaker areas (such as word problems) in math.

Furthermore, there is a need for more research focusing on understanding how students with disabilities or other underserved groups engage with and benefit from digital learning and assessment systems. This entails investigating their cognitive processes, level of engagement, needs, and barriers within these contexts.

To address this gap, I am currently analyzing the NAEP process/log, performance, and survey data to study the impact of digital tools (such as text-to-speech) on student performance. This line of research is crucial and should be expanded to gather new insights on inclusive and accessible learning possibilities as technologies continue to develop.

In addition, research efforts should extend beyond traditional methods and incorporate the analysis of multimodal data. By considering a range of data sources, including behavior log/process data, speech, facial expressions, and eye-tracking data, we can gain deeper insights into how students interact with digital learning and assessments. This comprehensive approach enables us to capture nuanced aspects of their experiences and informs the design and implementation of effective educational interventions and digital learning platforms.

What advice would you give to emerging scholars from underrepresented, minoritized groups that are pursuing a career in education research?

First and foremost, seek out a great mentor and research team. Having someone to guide and support you in the field can be tremendously beneficial to your career. Look for someone who shares your research interests, is supportive of your goals, and is committed to helping you succeed. Learning from others in your team is a great way to improve your skills and knowledge.

Second, don’t be afraid of change. The greatest opportunities often require stepping out of your comfort zone and exploring new research areas or methodologies. Be open to feedback and new perspectives that can help you grow as a researcher.

Third, be brave! It is important to recognize that your unique experiences and perspectives are valuable assets to the research community. Do not be afraid to share your ideas and contributions with others. Being proactive about your work can be a great way to build your network and collaborate with other researchers in the field.

Lastly, know that you have the potential to lead a research team yourself. Keep working hard, stay focused on your goals, and do not be afraid to take on leadership roles when the opportunities arise. Pursuing this career as an emerging scholar from an underrepresented or minoritized group can be challenging but also incredibly rewarding, and you can make a meaningful impact in the field and inspire others to follow in your footsteps.


Dr. Xin Wei is currently a senior quantitative researcher at Digital Promise. Prior to joining Digital Promise, she held the position of principal research scientist at SRI International for a duration of 15 years. She specializes in using applied experimental design, statistical and machine-learning techniques to evaluate and improve instruction, interventions, assessments, and policies. In addition to her current IES study, Dr. Wei has designed and directed statistical analysis of more than 26 grants funded by federal agencies.

Produced by NCER program officer Wai Chow (Wai-Ying.Chow@ed.gov) and Virtual Student Federal Service intern Audrey Im.

ED/IES SBIR: Highlights from 2022 & Announcing the New 2023 Program

The Department of Education’s Small Business Innovation Research Program (SBIR), which IES administers, funds the research, development, and evaluation of new, commercially viable education technology products. Known as ED/IES SBIR, the program’s goal is to grow a portfolio of products that are research-based and ready to be widely deployed to address pressing educational needs.

Over the past decade, the program has become known for investing in new entrepreneurial innovations, such as products by Future Engineers and Schell Games, as well as for supporting the transfer of evidence-based research into products that can be used in practice at scale, such as projects by Learning Ovations and Analytic Measures.

ED/IES SBIR: A Look Back at 2022

ED/IES SBIR products were used by millions of students, educators, and administrators to support remote and in-person learning.  Many companies, including MidSchoolMath, Education Modified, Sirius Thinking with partner Success For All, and PocketLab, earned new district contracts and licensing agreements to adopt their technologies at scale. Many companies also won industry awards for innovations on the basis of their ED/IES SBIR products.

With its 2022 awards, ED/IES SBIR continued to invest in emerging areas of education technology, funding projects that use artificial intelligence to personalize learning and generate real-time insights for educators to inform instruction, facilitate real-world learning, and support integrating arts in education and learning. Also in 2022, ED/IES SBIR launched a new “Direct to Phase II” program to support the scale up of existing evidence-based researcher developed innovations through the development of new education technology products. One award was made through this program.

Checkout the IES/ED SBIR News Archive for more information about our 2022 highlights.

ED/IES SBIR Releases Three 2023 Program Solicitations

On January 12, 2023, ED/IES SBIR released three solicitations, requesting proposals for Phase IA, Phase IB, or Direct to Phase II projects. The submission deadline for all three solicitations is March 13, 2023. The URL links to each solicitation on SAM.gov can be found on this page.

This year’s Phase I program introduces a new, two-track approach to stimulating innovation and research.

  • A “Phase IA” solicitation requests proposals for projects to develop a prototype of an entirely new education technology product, where no previous technological development has occurred. The goal of the Phase IA track is to stimulate novel approaches to solve pressing problems in education.
  • A “Phase IB” solicitation requests proposals for projects to develop a prototype of a new component to be added to an existing education technology prototype or product. The goal of the Phase IB track is to strengthen existing research-based prototypes or products in addressing pressing problems in education. Offerors interested in submitting a proposal for Phase IB must demonstrate that the existing prototype or product is research-based and that an additional investment in a new component to be integrated with what already exists is warranted. All Phase IA and IB proposals are for projects lasting 8 months for $250,000. All successful 2023 Phase I awardees will be eligible to submit a Phase II proposal in 2024 for $1M for full-scale development and evaluation.

A “Direct to Phase II” solicitation requests proposals for 2-year projects for $1,000,000 for the full-scale R&D and evaluation of new education technology products to ready existing evidence-based innovations (products, programs, or practices) for use at scale in education settings, and to plan for commercialization.  The existing education innovation is required to have originally been created by researchers at either universities (or other academic institutions) or non-profit education research organizations. Proposals must be submitted by a for-profit small business per the eligibility requirements of the SBIR program.


Stay tuned for updates on Twitter, Facebook, and LinkedIn as IES continues to support innovative technology and research.

Edward Metz (Edward.Metz@ed.gov) is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.

New Grant to Develop a Learning Game About the Supreme Court Features ED/IES SBIR Education Technology Platform

In April 2022, iCivics was awarded a $400,000 grant from the National Endowment for the Humanities (NEH) to develop Supreme Justice, a live-action multiplayer experience that simulates the deliberation process used by Supreme Court justices.  The game is being developed in partnership with small business Gigantic Mechanic and deployed using their VOXPOP platform, which was developed through the ED/IES SBIR program with awards in 2018 and 2019.

VOXPOP is a technology-enabled, class-wide, role-playing game for high school students. Using any web browser, teachers access a library of simulations on a range of topics drawn from the AP U.S. history curriculum and Common Core History Standards. VOXPOP’s platform provides resources to guide implementation, including videos, individualized student profiles, and real-time voting, and facilitates each student playing a unique historical role. Throughout the experience, the software guides participants with facts and primary sources, with students engaging in face-to-face discussions, and debating issues central to the simulation. All VOXPOP content can be found at www.voxpop.io

On July 15, VOXPOP was selected as the winner for the “Best Civics Game” through the Games For Change 2022 Awards. This annual award competition recognizes the year's best games for social impact and learning. All awards are competitive with submissions are evaluated by expert jurors.

In the new Supreme Justice game, students in grades 6 to 12 participate in a live-action multiplayer simulation focused on freedom of speech, freedom of expression, and due process rights. Classrooms are divided into different groups to play petitioners, respondents, and justices as they argue and consider cases of constitutional law. Supreme Justice will model deliberation and critical thinking in a civic setting, grounded in historical cases and relying heavily on the U.S. Constitution as evidence. The experience will engage students in face-to-face discussions and debates, while collaborating to craft arguments central to civic and government life. Once the game is developed, it will be freely available on the iCivics website.

 

 

Along with the iCivics NEH grant, Gigantic Mechanic is partnering with other organizations and museums, such as Revolutionary Spaces (see video below), to develop custom role-plays and simulations for their platforms and spaces.

 

 


Stay tuned to @IESResearch for news and updates on research, initiatives, and project updates in the area of tutoring to accelerate learning.

For more information, please contact Edward Metz (Edward.Metz@ed.gov), research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.

 

Technology Facilitated Tutoring Programs to Accelerate Learning

It goes without saying, challenges caused by COVID-19 in the field of education remain widespread and have the potential to be long lasting. A recent report confirmed that during the pandemic the move to full-time remote schooling was related to a decrease in student achievement, especially in high-poverty districts. Now over two years later, school leaders continue to employ strategies to address learning loss, such as intensive tutoring programs. This is because decades of research support the effectiveness of in-person tutoring to accelerate learning, and recent research also shows positive effects for high dosage virtual tutoring for struggling learners during the pandemic.

While the human-to-human interaction will always be central to a quality tutoring experience, technology offers unique functionalities to enrich and extend tutoring. For example, new models of technology facilitated tutoring programs—

  • Engage students with game-based and multi-media content that adjusts to the level of the individual and generate real-time tips to scaffold learning
  • Employ tools such as virtual whiteboards and data visualizations to enrich the virtual workspace for the tutor and student
  • Use dashboards to present real-time data driven insights for tutors to track student progress and individualize instruction
  • Provide automated professional development and training opportunities to prepare tutors

Technology also enables schools and community organizations to offer remote tutoring programs at scale—to reach students anywhere and anytime, including after school and during the summer. This of course depends on student access to technology and the availability of qualified tutors.  

Four IES Supported Technology-Based Tutoring Programs

In spring 2020, to help address the crisis in education caused by the pandemic, four teams of IES-funded developers adapted and extended their learning technologies for remote tutoring to be ready to be used at scale. The technologies are all research based and offer unique capabilities to strengthen the tutoring experience and to allow programs to reach more students. Each of the following programs described below can now deliver tutoring in schools or remote settings.

A2i by Learning Ovations. A2i (watch video) is a web-based product for students in kindergarten to grade 3 that continually assesses reading and generates data-driven recommendations to inform instruction. A2i is used in hundreds of schools by tens of thousands of students each year. Originally developed for in-school use through multiple IES and other government awards, research demonstrates the efficacy of A2i to improve student reading.

At the beginning of COVID-19, the Community Literacy Support System was designed to extend A2i for tutoring at student homes and other non-school locations. The program provides customized lessons and data visualization tools for tutors and parents. In the past two years, the tutoring program has been used by 9,200 students across 5 different states, at 23 different community sites (such as this one), and within almost 8,000 homes.

ASSISTments by ASSISTments. ASSISTments (watch video) is used by teachers to assign problems to students from curricula, such as EngageNY, Illustrative Mathematics, and Open Up Resources. Students receive real-time instructional feedback while doing problems online, while teachers receive reports with actionable insights to inform instruction. Initially developed through multiple awards from IES and other sources, in 2021-22, ASSISTments was used by over 5,000 educators and 200,000 students at schools around the country. Research by SRI International demonstrated that classrooms that used ASSISTments increased in learning course content compared to a control group.

At the onset of COVID-19, ASSISTments designed TutorASSIST, a tool to present data visualizations for tutors to target specific student needs through remote (or in-person) sessions. More than 750 students across Louisiana, Georgia, and Maryland used the tool to support remote tutoring during the pandemic. With a 2021 ED/EIR grant, ASSISTments is further developing its core product and tutoring tool to serve historically underserved students, including starting a school tutoring pilot program in eight schools in partnership with EnCorps Tutors, with a focus on optimizing the technology for tutoring.

SAGA Coach by SAGA Education and Simbulus. SAGA Education and Woot Math (watch video) employ interactive and game-based activities to support student math learning and a dashboard that generates data-driven insights to promote dialogue and discussion of complex topics between teachers and students. The school-based intervention reaches approximately 75,000 students per year. Woot Math was developed through an ED/IES SBIR award and through other sources. In 2021, Woot Math was acquired by and integrated within SAGA Education’s online math program.  

During the pandemic, SAGA Coach was designed to extend Saga Education and Woot Math for remote tutoring through the addition of an interactive whiteboard shared by tutors and students, and online training materials for tutors. During the 2020-21 school year, 5,500 students used SAGA Coach for high-dosage remote tutoring, and in 2022-23, the Boys and Girls Clubs of America will use SAGA Coach to deliver a remote tutoring program.

Lightning Squad by Sirius Thinking and Success For All. Lightning Squad (watch video) is a multimedia platform where pairs of students in grades 1 to 3 who are struggling readers collaborate to read stories and play games that are presented by the computer, while a tutor provides targeted support. Developed through an ED/IES SBIR award, the product is currently being evaluated through a multiyear efficacy trial and will be used in 50 Baltimore City elementary schools in fall 2022.  

At the onset of COVID-19 in 2020, Lightning Squad was adapted for remote delivery in eight Baltimore City schools serving principally low-income students. In the remote version, tutors use a video platform (for example, Zoom) with pairs of students in their respective homes. Students proceed with the activities of the software and respond verbally while a tutor types responses on the screen in real-time for each team member to see. In the 2020-21 school year, 16 Baltimore City schools used Lightning Squad with over 800 students, most for remote tutoring. Research conducted during the project by Success For All (not yet reviewed by the Department) found that students who were able to maintain consistent participation with remote tutoring gained 1.5 years of progress as measured by their initial placement and end of year placement, double the expected gains in reading during the period of school closures.  An additional 1,200 students were served remotely using Lightning Squad in other states during this same period.

Additional Related Resources on Tutoring to Accelerate Learning

  • Lightning Squad and SAGA Coach are part of Proven Tutoring, a coalition of technology-delivered tutoring programs with a mission to help educators learn about and access tutoring programs. These evidence-based programs have the potential to increase the achievement of students performing far below grade level due to COVID school closures or other factors.
  • In June 2021 during the ED Games Expo, IES partnered with AmeriCorps to host a webinar focusing on government and community partner initiatives to support remote tutoring to accelerate student learning during COVID-19.

In April 2022, AmeriCorps partnered with ED to produce a webinar on lessons from the field on the topic of high-dosage tutoring. AmeriCorps and the Department, along with Johns Hopkins University’s Everyone Graduates Center, are partners in the National Partnership for Student Success (NPSS). Launched in July 2022, the NPSS is committed to engaging 250,000 adults as tutors, mentors, and coaches in evidence-based programs designed to accelerate students’ recovery from the pandemic. 


Stay tuned to @IESResearch for news and updates on research, initiatives, and project updates in the area of tutoring to accelerate learning.

Edward Metz is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.

Melissa Moritz was the Afterschool and Summer Learning Fellow in the National Center for Education Evaluation and Regional Assistance (NCEE) at the US Department of Education’s Institute of Education Sciences. She currently serves as the Director of Policy for the STEM Next Opportunity Fund.

Please contact Edward.Metz@ed.gov with questions or for more information.

 

Google Acquires Intellectual Property for IES-Supported Education Technology Products Moby.Read and SkillCheck

On April 1, 2022, Google acquired the intellectual property (IP) rights for Moby.Read and SkillCheck, education technology products developed through IES programs by California-based Analytics Measures, Inc. (AMI). AMI will continue as a small business and is honoring school contracts that use Moby.Read and SkillCheck until 2024.

Moby.Read is a technology-delivered, fully automated, oral reading fluency (ORF) assessment that is self-administered by grade school students. As students read a passage aloud into a tablet, the speech-recognition software generates an assessment of ORF in real time through natural language processing software that analyzes text passages of the read-aloud performances. SkillCheck is a component of Moby.Read that employs natural language processing software to analyze recordings and produce interactive report pages that rate and illustrate the student's basic reading skills.

 

 

The technologies were developed over two decades with IES funding. Beginning in 2002, AMI designed several early prototypes to be used for ORFs as a part of the National Assessment of Educational Progress and other national assessments administered by IES’s National Center for Education Statistics. In 2016 and 2017, the IES Small Business Innovation Research program (ED/IES SBIR) funded AMI to develop and test Moby.Read to be used in schools at scale. With 2020 and 2021 ED/IES SBIR awards, AMI developed the SkillCheck as an additional component of Moby.Read to provide educators activities to inform instruction. AMI conducted research at key points over 20 years to validate the results of the assessment.

Since commercial launch in 2019, the Moby.Read and SkillCheck have been used for more than 30,000 student assessments in 30 states. Google acquired the Moby.Read and SkillCheck IP with plans to incorporate these tools into Google suite of products for education.

For additional information on the research, development, and commercialization of these technologies, see this Success Story on the ED/IES SBIR website.


Edward Metz is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences. Please contact Edward.Metz@ed.gov with questions or for more information.