Inside IES Research

Notes from NCER & NCSER

Investing in Next Generation of Education Technologies to Personalize Learning and Inform Instructional Practice

The Institute of Education Sciences Small Business Innovation Research program (ED/IES SBIR) funds entrepreneurial developers to create the next generation of education technology for students, teachers, and administrators in general and special education. The program emphasizes an iterative research and development process and pilot studies to evaluate the feasibility, usability, and promise of new products to improve educational outcomes. The program also focuses on commercialization after development is complete, so that the products can reach schools and be sustained over time.

In recent years, millions of students in tens of thousands of schools around the country have used technologies developed through ED/IES SBIR. And in the past four months, about one million students and teachers used the technologies for remote teaching and learning, as many ED/IES SBIR-supported developers made their products available at no cost in response to the COVID-19 pandemic and the closure of schools.

 

ED/IES SBIR Announces its 2020 Awards

This week, ED/IES SBIR announced the results of its 2020 award competition. Of the 22 new awards, 16 are for prototype development and 6 are for full-scale development. IES also announced two additional awards through a special topic solicitation in postsecondary education. Read about these awards here.

 

 

Each of the new awards supports a project to develop a product to personalize the student learning experience or generate information that educators can use to guide practice.

Most of the projects are developing a software component (for example, algorithms, artificial intelligence, machine Learning, or natural language processing) that continually adjusts the difficulty of content, provides prompts to support individual students if support is needed, or generates real-time actionable information for educators to track student progress and adjust instruction accordingly. Other projects are developing technologies to promote student learning through self-directed, hands-on, simulated, and immersive experiences. If the future of education includes a blend of in-class and remote learning due to public health crises, or for whatever reasons, technologies such as these will be ready to keep the learning going.

The projects address different ages of students and content areas.

In science, LightHaus is fully developing a virtual reality (VR) intervention for students to explore plant heredity; LightUp is fully developing an augmented reality (AR) app for students to perform hands-on physical science investigations with their own on-device camera; and Myriad Sensors is developing a prototype artificial intelligence formative assessment system that generates feedback in real time as students do hands-on laboratory experiments.

In math, Muzology is creating a prototype for students to create music videos to learn algebra, and Teachley is creating a prototype transmedia kit with videos, comics, and pictures to enhance teaching and learning of hard to learn concepts.

In engineering and computer science, Parametric Studios is fully developing an augmented reality puzzle game for early learners, and Liminal eSports, Makefully, and Beach Day Studios are creating prototype components that each provide feedback to students as they engage in activities to learn to code.

In English Language Arts, Analytic Measures and Hoogalit are each employing natural language processing to develop new prototypes to facilitate speech acquisition, and Learning Ovations is developing a prototype data engine to make recommendations for what individual children should read.

For English learners, KooApps is developing an artificial intelligence prototype to support vocabulary acquisition, and Kings Peak Technologies is employing machine learning to generate passages that blend English and Spanish words together to improve reading comprehension.

For early learners, Cognitive Toybox is fully developing an observation and game-based school readiness assessment.

For postsecondary students, Hats & Ladders is fully developing a social skills game to foster career readiness skills.

In special education, Attainment Company is developing a prototype to support student’s self-management, and Alchemie is developing a prototype of an augmented reality science experience for visually impaired students.

To support school administrators and teachers, LearnPlatform is fully developing a dashboard that generates reports with insights for teachers to implement education technology interventions, and Zuni Learning Tree, Teachley and LiveSchool are developing prototype dashboards to organize and present results on student progress and performance in real time.

 

Stay tuned for updates on Twitter and Facebook as IES continues to support innovative forms of technology.


Written by Edward Metz (Edward.Metz@ed.gov), Program Manager, ED/IES SBIR

Developing and Piloting the Special Education Research Accelerator

The traditional approach to research involves individual researchers or small teams independently conducting a large number of relatively small studies. Crowdsourcing research provides an alternative approach that combines resources across researchers to conduct studies that could not be done individually. As such, it has the power to address some challenges with the traditional research approach, including limited diversity of research participants as well as researchers, small sample sizes, and lack of resources. In 2019, the National Center for Special Education Research funded a grant to the University of Virginia to develop a platform for conducting crowdsourced research with students with or at risk for disabilities—the Special Education Research Accelerator (SERA).

Below, the Principal Investigators of this grant – Bryan Cook, Bill Therrien, and Vivian Wong – tell us more about the problems they intend to address through SERA, its potential, and the activities involved in its development and testing.

What’s the purpose of SERA?

SERA is a platform for conducting research in special education with large and representative study samples across multiple research sites and researchers. We are developing SERA to address some common concerns in education research, such as (a) studies with small, underpowered, and non-representative samples; (b) lack of resources for individual investigators to engage in the high-quality research that they have the skills to conduct; and (c) scarce replication studies. The issue of small, underpowered, and non-representative samples is especially acute in research with students with low-incidence disabilities, with whom few randomized controlled trials have been conducted. SERA seeks to leverage crowdsourcing to flip “research planning from ‘what is the best we can do with the resources we have to investigate our question,’ to ‘what is the best way to investigate our question, so that we can decide what resources to recruit’” (Uhlmann et al., 2019, p. 713). Conducting multiple, concurrent replication studies will allow us to not only examine average effects across research sites, but also to examine variability between sites.

How do you plan to develop and test SERA?

To pilot SERA, we are currently developing the infrastructure (project website, training materials, etc.) and procedures—including for data management—to be applied in a study that will be conducted in the 2020/21 academic year. In that study, we will conceptually replicate Scruggs, Mastropieri, and Sullivan (1994) by examining the effects of direct and indirect teaching methods on the acquisition and retention of science facts among elementary-age students with high-functioning autism. Students will be randomly assigned to one of three conditions: (a) control, in which students are told 14 science facts (as an example, frog eggs sink to the bottom of the water); (b) interventionist-provided explanations, in which students are told 14 science facts with explanations from the interventionist (frog eggs sink to the bottom to avoid predators at the top of the water); and (c) student-generated explanations, in which the interventionist provides scaffolds to the student to generate their own explanation of each science fact (frog eggs sink to the bottom – why do you think they do?; what is at the top of the water that could harm the eggs?). Acquisition of facts and explanations will be assessed immediately after the intervention, and retention will be assessed after approximately 10 days. Twenty-three research partners, representing each of the nine U.S. Census districts, have agreed to conduct the intervention with a minimum of five students.

One challenge with building an infrastructure platform for conducting replication studies is that the “science” of replication as a method has yet to be fully established. That is, there is not consensus on what replication means, how high-quality replication studies should be conducted in field settings, and appropriate statistical criteria for evaluating replication success. To address these concerns, the research team is collaborating with The University of Virginia’s School of Data Science to create the pilot SERA platform to facilitate distributed data collection across independent research sites. The platform is based on the Causal Replication Framework (Steiner, Wong, & Anglin, 2019; Wong & Steiner, 2018) for designing, conducting, and analyzing high-quality replication studies and utilizes data-science methods for efficiently collecting and processing information. Subsequent phases of SERA will focus on expanding the platform so that it is available for systematic replication research for the broader education research community.

How does SERA align with the IES Standards for Excellence in Education Research (SEER)?

With its focus on systematically conducting multiple replication studies across research sites, SERA aligns closely with and will address the following SEER principles.

  • Pre-register studies: To be implemented with fidelity across multiple research partners and sites, crowdsourced study procedures have to be carefully planned and documented, which will facilitate pre-registration. We will pre-register the SERA pilot study in the Registry of Efficacy and Effectiveness.
  • Make findings, methods, and data open: Because of the data platform being developed to merge study results across more than 20 research sites, data will be in a clean and sharable format upon completion of the study. We are committed to the principles of open science and plan to share our data, as well as freely accessible study materials and research reports, on the Open Science Framework.
  • Document treatment implementation and contrast: Using audio transcripts of sessions and fidelity rubrics, SERA will introduce novel ways for utilizing natural language processing methods to evaluate the fidelity and replicability of treatment conditions across sites. These measures will allow the research team to assess and improve intervention delivery while researchers are in the field, as well as to characterize and evaluate treatment contrast in the analysis phase.
  • Analyze interventions' costs: It will not only be important to examine the costs for implementing SERA as a whole, but also the costs of the intervention implemented by the individual research teams. To this end, we are adapting and distributing easy-to-use tools and resources that will allow our research partners to collect data on ingredients and costs related to implementing a pilot intervention and replicating study results.
  • Facilitate generalization of study findings: Because SERA studies involve large, diverse, and representative samples of research participants; multiple and diverse research locations; and multiple and diverse researchers, results are likely to generalize.
  • Support scaling of promising results: Crowdsourced studies, by their nature, examine scaling by investigating whether and how findings replicate across multiple samples, locations, and researchers.

Conducting research across multiple sites and researchers raises important questions: What types of interventions can be implemented? What is the most efficient and reliable approach to collecting, transferring, and merging data across sites? It will also lead to challenges (such as IRB issues, promoting and assessing fidelity) that we are working to address in our planning and pilot study. Despite these challenges, we believe that crowdsourcing research in education may provide important benefits.

This blog was co-authored by Bryan Cook (bc3qu@virginia.edu), Bill Therrien (wjt2c@virginia.edu), and Vivian Wong (vcw2n@virginia.edu) at the University of Virginia and Katie Taylor (Katherine.Taylor@ed.gov) at IES

 

Meeting the Literacy Needs of Students with Autism: What Do We Know and Where Do We Need to Go?

April is Autism Awareness Month, which celebrates the importance of people with autism, the contributions they make every day to our world, and what we are learning about improving outcomes for the growing number of people with autism. IES supports research in this area, primarily through grants funded by the National Center for Special Education Research. Dr. Emily Solari at the University of Virginia (UVA) was awarded an IES grant in 2018 to lead an autism-focused postdoctoral training program. This program provides postdoctoral fellows with extensive research training in the academic, behavioral, and social development of students with autism spectrum disorders (ASD) from pre-K through their transition out of secondary school. Currently, their research focuses on literacy development, including reading comprehension and writing, in children with ASD.

Below we share a recent conversation with Dr. Solari about the importance of this work and what she and researchers at UVA are learning about literacy development.

What do we know about the needs of students with ASD in the area of literacy?

Children with ASD have a unique constellation of strengths and weaknesses that impacts their academic development. Several studies by our research group and others have shown that children and adolescents with ASD are at risk for difficulties in the area of literacy. Some individuals with ASD show a particular strength in alphabet knowledge, including letter names and sounds, as well as reading words. A strength in word reading ability does not always translate into adequate reading comprehension. Many adolescents with ASD who can successfully read words still demonstrate difficulties with reading comprehension, especially comprehension that requires inferencing. Difficulties may be due, in part, to the highly social content that is embedded in stories. Children with ASD often struggle in the area of social communication and theory of mind (understanding others’ mental states), which may inhibit their ability to comprehend narrative texts. Additionally, we know that vocabulary and oral language are both important for reading comprehension; therefore, difficulties in these areas – often seen in individuals with ASD – may impact reading comprehension as well.

Similarly, the existing data show that children with ASD have a more difficult time with writing-related tasks, such as composition. Our work in this area suggests that these writing difficulties may be due to broader difficulties related to language development and social communication skills. 

What research is being done to address the needs of students at different ages?

While we are beginning to understand developmental trajectories of reading for this population, very little research has been conducted on specific interventions for reading and writing. Our research group has begun to look at early elementary (K-3rd grade) language and reading comprehension interventions for students with ASD. Our initial studies have shown that when we implement highly interactive language and listening comprehension instruction, these students show gains in oral language and listening comprehension. We have found that instructional strategies that use shared book reading, where the teacher reads aloud from a book and asks children targeted questions about the characters in the story, are effective. Our instruction also provides students practice with vocabulary words and opportunities to respond to texts both orally and through writing.

There are also other research groups investigating emergent literacy (prekindergarten years) with this population of students. For example, Jaclyn Dynia at The Ohio State University has engaged in work investigating strengths and weaknesses in emergent literacy skills such as phonological awareness, alphabet knowledge, and print awareness.  Also, in the early childhood years, Dr. Veronica Fleury at Florida State University is engaged in some work in this area, including an IES-funded study aimed at developing and testing the feasibility of an adaptive shared book reading intervention for preschoolers with ASD. 

To address difficulties in reading comprehension with older students, Michael Solis and his team at the University of California, Riverside are using IES funding to develop and test explicit instructional routines and curricular materials for a reading comprehension and behavior intervention for students with ASD in upper elementary and middle school.

In collaboration with our colleagues at the University of California, Davis, we continue to analyze and publish developmental studies examining literacy skills. Additionally, we have become increasingly interested in understanding the transition from prekindergarten to kindergarten and early elementary school and how literacy is developing during this time. At UVA, we have started a longitudinal data collection project to investigate the relations among early reading, oral language, social attention, and cognition variables in young children with higher functioning ASD.

Our group is also starting to think about how we can design interventions that specifically target early reading skills and language development as well as social communication skills. Children’s books often provide very rich opportunities to engage around events and feelings that could be used to teach children with ASD social communication skills. Our next school-based intervention study will combine our previously successful language and listening instruction with targeted social communication instruction.

What recommendations or resources do you have for parents who are supporting children with ASD as they learn from home during the pandemic?

Here are some tips for reading at home with children. Additional resources are below.

  • Friends on the Block was developed through an IES grant as an early literacy curriculum for children with disabilities.  They have provided some free content online for use at home by caregivers.
  • Self-regulated strategy development (SRSD) is an approach that emphasizes direct instruction of writing strategies, knowledge, and self-regulation skills via flexible, recursive instructional stages. SRSD approaches have been shown to be effective for some elementary and middle school children with autism.
  • Book Share Time provides read aloud texts and allows caregivers to filter the books based on specific speech or language goals.

This blog was co-authored by Sarah Brasiel (Sarah.Brasiel@ed.gov), Amy Sussman (Amy.Sussman@ed.gov), Katie Taylor (Katherine.Taylor@ed.gov) at IES and Emily Solari (ejs9ea@virginia.edu), and her IES funded postdoctoral fellows (Alyssa Henry & Matthew Zajic) at UVA.  IES hopes to encourage more research on students with ASD in the coming years in order to increase the evidence base and guide program and policy decisions.

What Are we Learning about Applied STEM CTE Course-taking by Students with Disabilities?

February was National Career and Technical Education (CTE) Month, which celebrates the importance of CTE and the achievements and accomplishments of CTE programs across the country. IES supports research in this area, including grants funded through the National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER).

Dr. Michael Gottfried at the University of California Santa Barbara was awarded a 2-year grant in 2019 from NCSER to investigate whether participating in applied STEM career and technical education (AS-CTE) courses in high school is related to pursuing and persisting in STEM majors and/or careers for students with learning disabilities (SWLDs). Although a significant number of SWLDs participate in CTE courses, little is known about the types of AS-CTE courses they take and the extent to which taking these courses is related to postsecondary and employment outcomes. This project uses data from two nationally representative, longitudinal studies, the Education Longitudinal Study of 2002 (ELS:2002) and the High School Longitudinal Study of 2009 (HSLS:2009).

The descriptive evidence resulting from this project will have important implications for policymakers and educators about promoting SWLDs’ interests in CTE and STEM and facilitating their access to these courses. We take a moment to share our recent conversation with Dr. Gottfried to understand the importance of this project and what he is learning.

Tell us about your project and what you hope to accomplish (or why a focus on STEM and SWDs?)

Our project is investigating the pathway students take in the pursuit of and persistence in STEM majors and careers for SWLDs. SWLDs are currently underrepresented in STEM fields throughout the STEM pipeline from high school to college to career. This SWLD-STEM college and career gap will continue to persist and potentially worsen unless there are efforts to lessen this underrepresentation of SWLDs in STEM fields. To address this, our research team is exploring AS-CTE courses that SWLDs take in high school and the extent to which taking these courses promotes advancement towards postsecondary success and careers in STEM. Unlike traditional STEM courses, AS-CTE courses emphasize the application of math and science concepts directly to practical job experiences by offering “hands-on” logic and problem-solving skills. They are designed to reinforce traditional academic STEM learning and motivate students’ interests and long-term pursuits in STEM areas. Using two nationally representative samples of high school students, we are examining whether high school AS-CTE can help prepare SWLDs for college, STEM fields of study in college, and careers in STEM or with STEM applications. We hope that this project will provide new evidence for policymakers and educators that will help facilitate access to AS-CTE courses in schools in order to promote short- and long-term interest in STEM for SWLDs.

What are applied STEM career and technical education courses students with disabilities can take in high school?

AS-CTE courses encourage the alignment of applicable job-related skills with academically challenging coursework targeted to students at all ability levels. These courses fall into two of the sixteen broad CTE categories: engineering technology and information technology. Some examples of engineering courses offered in high schools include Biotechnical Engineering, Wind Energy, Laser/Fiber Optics, Aerospace Engineering, and Computer-Aided Design Software. Some examples of information technology courses are Database Management and Data Warehousing, Business Computer Applications, Web Page Design, Geospatial Technology, and Networking Systems. 

What have you learned so far about enrollment in CTE and applied CTE courses by students with disabilities and related outcomes for those students?

We are currently in the beginning stages of our project, but through our analyses thus far we have found that SWLDs are more likely to take CTE courses than the general student population but less likely than other students to enroll in AS-CTE courses. In other words, SWLDs are taking CTE courses, just not in STEM areas. We see this pattern becoming even more prominent in the recent years.

What are some of the challenges with your research?

Although using large national datasets such as ELS and HSLS provides extremely rich information and data about high school students across the nation, there are some limitations to the conclusions that we can draw when using extant longitudinal data. First, although we are able to examine AS-CTE course taking patterns for high school students, no data exist in either dataset on why students chose to take AS-CTE courses. Second, there is no detailed information available in these datasets about course content, including design, curriculum, rigor, and quality of an academic course, all of which affect student achievement. Third, the datasets identify SWLDs based on parent survey responses about whether a doctor, healthcare provider, teacher, or school official had ever told them that their student had a learning disability. There is no verification that the student has an official special education label of LD, so there may be some variability in the population of SWLDs in the datasets, which could impact what we find.

What other research is needed to improve CTE course-taking and outcomes for students with disabilities?

Our project is quantitative, which has many strengths such as identifying patterns and trends. Yet, we believe that a future qualitative project would be useful to complement the work we are doing. For instance, there are many lingering questions that we cannot address. For example, why are SWLDs taking fewer AS-CTE courses? What does SWLDs’ sense of STEM self-efficacy look like in these AS-CTE courses? What types of supports are teachers providing in these courses? These types of questions could be addressed with future qualitative research in which teachers and students can be interviewed and followed.

What other recommendations do you have to support research in this area?

For our work, we plan to address diversity within the SWLD group. For instance, we are going to explore differences by gender for SWLDs taking AS-CTE courses. We propose that future research could consider this type of heterogeneity.

Dr. Gottfried also has funding from NCER in a related project exploring whether and how AS-CTE course-taking can help prepare low-income students for college and for careers in STEM or with STEM applications.

If your state, district, or school is looking for resources for developing and improving the quality of your CTE program, the Association for Career and Technical Education has many high-quality CTE tools, including a Quality CTE Program of Study Framework. The National Technical Assistance Center on Transition also has many resources to increase engagement in CTE by students with disabilities, including the on-demand webinar, Toward Best Practices: Programs that Work, Models Toward Success. This webinar was recorded on December 19, 2019 with a panel of experts and practitioners in workforce education and CTE led by Dr. Michael Harvey, Professor of Education in the Workforce Education and Development academic program at the Pennsylvania State University. Advance CTE connects CTE leaders across states and has resources to support CTE at the state level.

This blog series was co-authored by Sarah Brasiel (Sarah.Brasiel@ed.gov) and Jacquelyn Buckley (Jacquelyn.Buckley@ed.gov) at IES and Michael Gottfried (mgottfried@education.ucsb.edu). IES began funding research grants in CTE in 2017 and established a CTE Research Network in 2018 through NCER. NCSER started funding research grants in special education in CTE in 2019. IES hopes to encourage more research on CTE in the coming years in order to increase the evidence base and guide program and policy decisions.

IES-Funded Researchers Receive Awards from the Council for Exceptional Children

In February, the Council for Exceptional Children (CEC) held its annual Convention and Expo, during which scholars were recognized for their research contributions to the field. Several investigators funded through IES were among those honored by the CEC.

 

Nancy Jordan (University of Delaware) received the 2020 Kauffman-Hallahan Distinguished Researcher Award. This honor, awarded by the CEC Division for Research, recognizes individuals or research teams who have made outstanding scientific contributions in special education over the course of their careers, leading to better education or services for exceptional individuals. Dr. Jordan has been the Principal Investigator (PI) on a number of IES awards. With support from the National Center for Special Education Research (NCSER), she is currently developing and testing a fraction sense intervention for middle school students with or at risk for mathematics difficulties. She also served as PI for the large-scale National Research and Development Center on Improving Mathematics Instruction for Students with Mathematics Difficulties, which conducted exploratory research on fractions and related cognitive process as well as developed interventions for fraction understanding among students with mathematics difficulties. In addition, Dr. Jordan has received funding from the National Center for Education Research (NCER), including a grant to refine and validate a number sense screener for students from prekindergarten through Grade 1 and to train postdoctoral fellows to apply cognitive science principals to crucial issues in education such as mathematics, language development, and early learning. She also co-authored a synthesis of IES-funded research focusing on mathematics learning and teaching from kindergarten through secondary school and has served as an expert panelist for the What Works Clearinghouse practice guides in mathematics.

 

Tim Lewis (University of Missouri) received the CEC J.E. Wallace Wallin Lifetime Achievement Award, which recognizes an individual who has made continued and sustained contributions to the education of children and youth with exceptionalities. Dr. Lewis is currently the PI on a NCSER-funded grant to evaluate the efficacy of Check-in/Check-out for improving social, emotional, and academic behavior of elementary school students at risk for emotional and behavioral disorders, as well as co-PI on a project to further develop and refine the Resilience Education Program, a tier 2 intervention for elementary students at risk for internalizing problems. He also served as co-PI on the large-scale National Research and Development Center on Serious Behavior Disorders at the Secondary Level, focused on developing and evaluating the efficacy of a package of intervention strategies designed to reduce the significant behavioral and academic challenges experienced by high school students with behavior disorders.

 

Sara McDaniel (University of Alabama) received the 2020 Distinguished Early Career Research Award from CEC’s Division for Research. This award recognizes individuals who have made outstanding scientific contributions in basic and/or applied special education research within the first 10 years after receiving a doctoral degree. Dr. McDaniel is a co-Investigator on a NCSER supported grant to develop and test Racial equity Assessment of data, Cultural adaptation, and Training (ReACT), a professional development intervention aimed at reducing racial/ethnic disproportionality in school discipline and special education referrals.

 

Congratulations to the winners!