Inside IES Research

Notes from NCER & NCSER

Conducting Education Research During Covid-19

Since the start of the pandemic, we have all heard about the unprecedented changes to schooling in the U.S. and the ways that educators, students, and families have been adapting to the new reality.

Education researchers have also been adapting their work due to school closings, canceled testing, and different school reopening plans in the 2020-21 school year.

How have education researchers handled the new reality?

Some researchers have been busy compiling and disseminating research findings to support districts and schools to continue instruction during the pandemic. For example, evidence-based recommendations were made available to help parents and schools pivot to a virtual environment (from very young children up to the postsecondary level), maintain engagement, address mental health (including in rural areas), protect against learning loss, and decide how to prioritize needs when considering re-opening. And many education technology researchers and developers have provided online resources to schools.

Other researchers have been working hard to understand the overall disruption to schooling due to COVID-19 and the ramifications on student learning around the world.  For example, there have been efforts to keep track of school closures, document what is happening in schools across the country (including in rural districts), study the switch to online learning and attend to unequal access to technology for remote learning, forecast funding scenarios, and examine changes in teacher recruitment.

In addition, education researchers are thinking about new ways to conduct research in light of the changes to schooling. They are looking at alternatives to standardized testing, new approaches to teaching and learning to strengthen schools moving forward, and ways to rebuild our education systems after the pandemic. Indeed, there are myriad ways that education researchers can and are using their skills to continue to support education during this unprecedented time.

How has COVID-19 impacted IES-funded education research studies?

IES realizes that the pandemic has changed things in ways that may complicate education research – both how it is conducted and how it is interpreted. So, we are actively working with grantees to help ensure the integrity of their work and to respond to the needs, interests, and concerns of the schools and colleges they are working with and the communities they are trying to help. In a follow-up to an IES-funded study on students in foster care, a researcher-practitioner partnership in Colorado is examining the implications of challenging circumstances such as COVID-19 on the postsecondary education of vulnerable youth.

Many IES-funded researchers have had to alter their research plans to accommodate the needs of their partner schools and overcome the challenges posed by the abrupt transition to virtual learning. Because of continued uncertainty, they may need to change plans again. Program officers at IES have been working with grantees on a case-by-case basis to adapt their timelines and, in some cases, their research designs.

IES’s priority is to help researchers maintain scientific rigor while holding a realistic view of what can and cannot be done this year. As we work with our grantees, we take into consideration where the project is in its overall timeline. For example, if the project has collected all of its data and is in the final analysis stage, the remaining work may not be affected. Or, if a project has not yet started to begin an intervention in schools, it can pause during the 2020-2021 academic school year and resume in 2021-2022. Still, other projects may find themselves unable to either continue or pause. These projects may not be able to achieve their initial purpose and may need to end.

Despite some of the challenges, the pandemic offers a unique natural experiment for learning and instruction, as well as opportunities for innovation that can ultimately benefit education. IES, our funded researchers, and the communities that rely on research evidence continue to pull in the same direction: building strong evidence to inform policy and practice. Through collaboration and dialog, we will work together to ensure that data and results are meaningful, valid, and as timely as possible. IES will continue to focus on high-quality education research to improve student learning and achievement both now and in the future.

Stay tuned for future blog posts on what our researchers are doing to address some of the challenges that face educators, families, and policymakers during this unprecedented time!


Written by Corinne Alfeld (Corinne.Alfeld@ed.gov), National Center for Education Research.  

Facilitating Causal Research in CTE: Notes from the Network

The Career and Technical Education (CTE) Research Network just entered its third year, and it is time to share recent accomplishments with the IES community! As a reminder, the CTE Research Network (led by the American Institutes for Research, or AIR) was created to expand the evidence base on the impact of CTE programs on student outcomes using causal research methods.

 

Research

In June 2020, AIR released a preliminary report of CTE sites that are ready for causal evaluation. Designed to support researchers interested in studying the impact of CTE on student outcomes, the report details the history, theory of change, student enrollments, and other information for 4 selected CTE sites around the country. The Network hopes that researchers will use the information in the report to design evaluation studies of these programs.

In July 2020, a fifth research project joined the network. A team from MDRC, led by Rachel Rosen, was recently funded by IES to study the Impact of Technology-Based Career Advising Tools on High School Students' CTE Choices and Academic Performance. In partnership with Communities in Schools (CIS), the study will use a three-arm, school-level random assignment research design (RCT) to assess the effects of Navience and YouScience on students' self-expressed attitude and interest in career pathways, CTE course taking patterns, and engagement with and progress towards graduation. We welcome the team to the Network and look forward to learning whether and how these advising tools influence student thinking about career options, choice of relevant CTE coursework and work-based learning options, and decisions about CTE concentration in available pathways and programs of study.

A small group of researchers from different Network teams collaborated on and recently released a technical working paper on counterfactuals in CTE. It can be challenging to identify comparison groups for CTE students because it is an elective into which they self-select. The paper describes a variety of rigorous methods of comparing CTE students to valid counterparts and provides case studies that illustrate how to use these methods.

 

Training

The CTE Research Network is committed to increasing the number of researchers trained to study CTE using causal methods. It is notoriously challenging to isolate the effects of CTE from other influences on student outcomes. In August 2020, the Network hosted 18 researchers for a week-long virtual summer training institute on how to design a causal study to examine the impact of CTE. During the week, participants learned how to implement randomized-control trials (RCTs), regression discontinuity designs (RDDs), and comparative interrupted time series (CITS) in a CTE context. After learning about each method, participants worked in small groups to apply the method to real data and had access to the instructors to ask questions. The feedback about the training was overwhelmingly positive. The lecture portions of the training will be posted soon to the training page of the Network’s website. Another week-long training institute will be held in summer 2021 (hopefully, in person!)

The Network is currently developing a series of online modules for CTE practitioners and state agency staff to strengthen capacity to access, conduct, understand, and use CTE research. There will be a presentation to preview the modules at the Association of Career and Technical Education (ACTE) Best Practices and Innovations Conference on October 9, 2020 and a longer and more in-depth session at ACTE’s Career Tech VISION conference the first week of December. These practitioner training modules will be available for free on the website in late Fall 2020.

 

Leadership and Dissemination

The CTE Research Network is regularly updating its resources page with publications of interest to the CTE research field. The most recent is a report of findings from MDRC’s study of P-Tech high schools. The Network’s equity workgroup (a group of researchers from across the Network’s member projects) also published a popular blog this summer on applying an equity lens to CTE research. The Network also posts outside resources such as a REL self-study tool on career readiness and evaluation reports from other researchers.

 


For more about the CTE Research Network, you can sign up to receive the Network’s quarterly newsletter at the bottom of their website’s home page and follow them on Twitter (@CTEResNetwork) and LinkedIn.

If you are interested in learning more about the CTE Research Network, contact the Director, Kathy Hughes (khughes@air.org).

If you are interested in discussing CTE research opportunities at IES, contact Corinne Alfeld (Corinne.Alfeld@ed.gov).

Research on Adult Literacy: A History of Investment in American Adults

Reading is fundamental, but it is also difficult to master, taking thousands of hours of instruction and practice. Roughly 52 percent of U.S. adults over the age of 16 may struggle with everyday literacy tasks. Of these adults, approximately 20 percent may perform at very low levels of literacy. For adults who are still mastering of this skill, the task can seem overwhelming.

Luckily, IES-funded researchers have been working towards solutions for adults with low basic reading skills and are creating and refining assessments, curricula, and software. These innovations aim to help adult learners, the instructors and tutors who work with them, and the programs that support them.

As part of our commemoration of National Adult Education and Family Literacy Week (September 20-26, 2020), we would like to recognize the history of adult literacy research at IES and its National Center for Education Research.  

Since 2004, IES-funded researchers have been developing assessments to help identify the needs of adults struggling with literacy and working on solutions to build adult literacy skills. This work fed into the measurement component of IES’s Reading for Understanding Initiative in 2010 and later returned back to addressing adult basic literacy measurement in 2016.

In 2012, IES funded the Center for the Study of Adult Literacy (CSAL), which developed a curriculum and technology for adults reading between the 3rd- and 8th-grade levels. CSAL demonstrates how adult literacy research benefits by integrating research conducted with students with disabilities and those in K-12 and postsecondary settings. In fact, the researchers pulled upon findings from eight prior IES grants funded by NCER and NCSER.

Our researchers are also developing a clearer picture of the adults who fall into the broad category of those with low literacy. They are leveraging the PIAAC data set to conduct exploratory work that informs both our understanding of those at the very low ends of literacy and also of whether basic skills may predict success in postsecondary career and technical education programs.

In 2020, IES funded additional development research to help refine an interactive, online reading comprehension program, AutoTutor for Adult Reading Comprehension (AT-ARC). Another project will recruit and train postdoctoral fellows to cultivate the next generation of researchers who can continue to build a research base for improving adult literacy outcomes.

Although IES researchers are making great strides to build knowledge, the field needs more information, and adult learners deserve tools and innovations developed for their specific needs and goals. IES hopes to continue to support such work.

 


To learn more about IES-wide efforts to understand and improve adult learners’ outcomes, visit the Adult Basic Skills topic page. Contact Dr. Meredith Larson for more information about the research supported by NCER.

 

Building Knowledge About Adult Education

Adult education programs aim to support the millions of American adults who wish to strengthen their basic skills, earn a high school degree or equivalent, or become U.S. citizens. During the National Adult Education Family and Literacy Week (September 20-26, 2020), IES is highlighting the ongoing research it supports to help expand our knowledge of and innovations for the adult education system and the communities they support.

Roughly 30 million U.S. adults may lack a high school degree or equivalent. And according to an assessment conducted in 2017, approximately 30 percent of adults may have very low numeracy skills, and approximately 20 percent may have very low literacy skills. Adults with low levels of academic attainment or low basic skills may face barriers to full economic or civic engagement.

Adult education programs aim to help such adults. These programs often run on lean budgets and need to meet the needs of a highly diverse population with a multitude of learning goals.

In order to help understand and improve the ability of the adult education system to provide services, IES researchers have been conducting various research projects in partnership with adult education providers.

For example, Career Pathways Programming for Lower-Skilled Adults and Immigrants conducted mixed-methods research in partnership with Chicago, Houston, and Miami. This work helped each city understand what types of career pathways adult education programs were offering and who was participating in such programing. This type of information helps programs, cities, and the system more broadly understand and adjust to the needs of learners and communities.

The New York State Literacy Zone Researcher-Practitioner Partnership focused on improving the ability of case managers to help adult learners leverage wrap-around services and access and succeed in adult education and training programs. This project helped develop tools and training for case managers and conduct an exploratory pilot study of these tools to see if they predicted learner outcomes, such as persistence in a program.

The Georgia Partnership for Adult Education and Research (GPAER) is a collaboration among researchers at Georgia State University and leadership at the Georgia Office of Adult Education: Technical College System of Georgia to help understand adult literacy programs across the state. This ongoing work is conducting mixed-methods studies to understand program features, learner characteristics, and indicators of beneficial learner outcomes.

There is still much to learn to help improve the ability of programs to find and support adult learners. IES encourages additional research to further help us understand the landscape of adult education, the needs and interests of adult learners and their instructors, and the outcomes and impacts of improving adult basic skills.


For more information about adult education research at the National Center of Education Research, contact Dr. Meredith Larson.

 

 

Back to School During COVID19: Developers and Researchers Continue to Respond to Support In-Class and Remote Teaching and Learning

Many programs across the Federal government, such as the ED/IES Small Business Innovation Research (SBIR) and the IES Research Grants programs, fund projects to develop and evaluate new forms of education technology and interventions that can be implemented to support instruction and learning at schools and for remote learning. More than 150 of these technologies were demoed in January 2020 at the ED Games Expo, a showcase for learning games and technologies developed with support from IES and more than 30 other Federal programs.

Since the global outbreak of COVID19 and the closure of schools across the United States and the world, a group of government-supported developers and researchers responded to provide resources to educators, students, and families to facilitate remote learning. More than 50 developers and researchers offered 88 learning games and technologies at no cost through the end of the school year for use in distance learning settings with internet access (see this blog for the list). In addition, many of the developers and researchers provided technical assistance directly to individual teachers to support implementation at a distance, and many created new materials and worked to refine and adapt their products to optimize usability and feasibility for fully remote use. More than a million students and thousands of educators used these learning technologies during the spring.

In April and May 2020, more than 70 developers and researchers partnered to produce and participate in a series of free day-long virtual events, which were called “unconferences.” The events featured presentations on innovative models and approaches to teaching and learning remotely and provided an in-depth look at the learning games and technologies created by the presenters. More than 25,000 educators attended these virtual events in real-time, hundreds asked questions and made comments through chats during the events, and many thousands more have accessed these videos after the events. See this blog for the list of archived videos.

A New Resource: Guides to Education Technologies that are Ready Now

As schools begin re-opening for the new school year, a group of 70 developers and researchers have collaborated to produce a new series of Guides to Education Technologies. The guides present information on government-supported education technology products that are ready now for in-class and remote learning. All the resources are web-based and can be used on either computers, tablets, or personal devices. The resources in the guides include a mix of no-cost products as well as ones that are fee-based.  

With awards from government programs, all of the resources were developed through an iterative process with feedback from teachers and students, and most were evaluated through small pilot studies to measure the promise of the technologies to support improvements in student learning and relevant educational outcomes. All the products were used and demonstrated to be feasible for use in remote settings in the spring after the onset of the pandemic.

The guides present resources appropriate for young children through postsecondary students in education and special education, for English learners, and for teachers in education and special education across a wide range of educational topics. Many of the technologies personalize learning by adjusting content to students as they go and present information to educators to inform instruction.

The Guides focus on the following areas and can be accessed below:

 

Stay tuned to the Inside IES Blog for more information and resources about the response to the COVID-19 in education.


Edward Metz (Edward.Metz@ed.gov) is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.