Inside IES Research

Notes from NCER & NCSER

Spotlighting Doug and Lynn Fuchs: Two Decades of Innovation in Special Education Research

Doug and Lynn Fuchs

During our 20-year anniversary, IES would like to reflect upon the important work of Drs. Doug and Lynn Fuchs, who have received multiple IES grants over the years to explore important factors associated with learning and develop interventions aimed at improving outcomes for low-achieving learners and learners with disabilities in math and literacy. Their work as “trailblazers in the field of special education” was recognized in 2021 when they received the “Nobel Prize of education,” the Harold W. McGraw, Jr. Prize in Education.

Doug and Lynn Fuchs are internationally recognized for their intervention work in Response to Intervention, or Multi-Tiered Systems of Support (MTSS), tiered models that include teacher collection of progress monitoring data and offer progressively intensive support for students who are not performing at grade level. Their research and development work has provided training for educators and research project staff and intervention materials to use in tiered interventions for students who are struggling in the areas of reading and mathematics. Their research has also included exploratory work and measurement development to better understand and measure factors associated with risk of disability in reading and math in elementary school children. Their innovative intervention designs take into consideration different cognitive factors such as working memory and executive functioning.

Although Doug Fuchs is well known for his work within MTSS frameworks, one of his early IES grants in 2004 focused on teachers tailoring instruction to meet individual student needs in elementary schools with a diverse range of students. The goal of the project was to scale up Peer-Assisted Learning Strategies (PALS), an instructional approach developed by the Fuchses in 1997 with increasing instructional differentiation and evidence of reading achievement. For this scale-up study, his research team collected and analyzed data across 2 years from three sites. They demonstrated that implementation of PALS with onsite support for teachers led to significant reading achievement gains, an effect that was strongly influenced by whether teachers were encouraged to modify the PALS program to suit the needs of their particular students. With a NCSER grant in 2009, Doug Fuchs and his research team (including Lynn Fuchs) developed and tested interventions in reading and math to prevent or mitigate disability among first grade students with or at risk for disabilities in these outcome areas. One of the interesting findings from this research related to students with comorbid math and reading disabilities (LD). They found that students with comorbid LD respond differently than those with only math disabilities, depending on the nature of mathematics intervention. However, students with reading disabilities responded similarly whether they had a disability only in reading or in both reading and math. Recently, Doug Fuchs has become passionate about assessment, critiquing how reading comprehension is often assessed in an article he co-authored with Nathan Clemens, “Commercially Developed Tests of Reading Comprehension: Gold Standard or Fool’s Gold?

Lynn Fuchs is a leader in improving outcomes for students with or at risk of math disabilities. Through a 2009 grant, she and her research team (including Doug Fuchs) developed a measure to predict first graders’ calculation skills and word problem development using dynamic assessment. The measure was found to be more predictive than traditional assessments for early identification of students at risk for a math disability. The team concluded that language, reasoning, and mathematical cognition were important in predicting calculation and word problem solving for these early learners. Lynn Fuchs continued this work in math and cognition with students in second grade, exploring connections between cognition and student calculation, word problem solving, and pre-algebraic knowledge with funding from the National Institute of Child Health and Human Development. With a 2015 IES grant, she applied what she learned about the importance of cognition to test the efficacy of a math intervention that embedded working memory training into a previously validated math problem-solving intervention (Pirate Math) for students with poor problem-solving skills. The results of this study showed that general working memory training with ongoing math practice improves working memory and word problem skills; however, working memory training alone is not sufficient to improve word problem solving.

More recently, Lynn Fuchs received a 2020 grant to further research a fraction intervention for fourth grade students with disabilities developed through her work as co-principal investigator on the 2010 National Research and Development Center on Improving Mathematics Instruction for Students with Mathematics Difficulties. In the current replication study, she and her research team are testing the Inclusive Fraction Intervention as a class-wide intervention taught by general education teachers to understand the effect on students with and without math learning disabilities. Lynn Fuchs also chaired the panel for the most recent WWC Practice Guide, Assisting Students Struggling with Mathematics: Intervention in the Elementary Grades. This guide provides six evidence-based practices that can help teachers tailor their instructional approaches and/or their mathematics intervention programs to meet the needs of their students. All six practices in this guide are supported with strong evidence due, in part, to the research conducted by Lynn Fuchs. By the start of 2023, this practice guide has had 62,346 views and 10,468 downloads.

Together, Doug and Lynn Fuchs have pushed the field forward with their leadership. Through their 2013 A3 Initiative project, they developed and tested the efficacy of intensive reading and math interventions for learners in upper elementary grades. The research team demonstrated that both the math and reading interventions were effective in improving  outcomes for students with disabilities. As part of this work, the Fuchses led a meeting with a group of experts to discuss evidence to support the importance of moderator analysis in intervention research. This effort resulted in a special journal issue with several articles on this topic. Their leadership role extends beyond IES-funded work to their involvement in several other national projects, such as the National Center on Intensive Intervention, funded by the Office of Special Education Programs and other national thought leadership activities, such as their webinar on  intensive intervention. The Fuchses have also published articles in practitioner journals outlining how their research-based practices can be implemented by teachers in the classroom, such as “What is intensive intervention and why is it important?

The impact of Doug and Lynn Fuchs research is far reaching. In addition to leading research projects and publishing articles, Doug and Lynn Fuchs have truly developed capacity in the field of special education research through mentoring and collaborating with junior researchers. The following are examples of researchers who worked with Doug and Lynn Fuchs in the past as graduate research assistants, post-doctoral researchers, or research associates who now lead their own IES-funded research:

Doug and Lynn Fuchs have pushed the fields of assessment and intervention development forward, providing new opportunities to understand and support math and literacy outcomes for students with or at risk for disabilities. We are proud to have funded their work over the years, and we are excited to see how they continue to advance the field.

This blog was authored by Sarah Brasiel, program officer at NCSER.

Assessing Math Understanding of Students with Disabilities During a Pandemic

For almost two decades, IES/NCSER has funded Brian Bottge and his teams at the University of Kentucky and University of Wisconsin-Madison to develop and test the efficacy of a teaching method called Enhanced Anchored Instruction (EAI), which helps low-achieving middle school students with math disabilities develop their problem-solving skills by solving meaningful problems related to a real-world problem. The research findings support the efficacy of EAI, especially for students with math disabilities. Most recently, Bottge and his team have been researching innovative forms of assessment that more adequately capture what students with disabilities know both conceptually and procedurally in solving math problems. With supplemental funding, IES/NCSER extended Dr. Bottge’s latest grant to test the use of oral assessment to measure student knowledge and compare that with the knowledge demonstrated on a pencil and paper test. The COVID-19 pandemic introduced added challenges to this work when schools closed and students shifted to online education.

Below we share a recent conversation with Dr. Bottge about the experience of conducting research during a pandemic and what he and his team were still able to learn about the value of oral assessment in mathematics for students with disabilities.

What changes did you observe in the intervention implementation by teachers due to the COVID-related shift to online learning?

Photo of Dr. Brian Bottge

The shift to online learning created changes in class size and structure. For 38 days (22 days in classroom, 16 days online through a virtual meeting platform), the middle school special education teacher first taught concepts through a widely used video-based anchored problem, the Kim’s Komet episode of the Jasper Project, in which characters compete in a “Grand Pentathlon.” The teacher then engaged the students in a hands-on application of the concepts by running a live Grand Pentathlon. In the Grand Pentathlon, students make their own cars, race them on a full-size ramp, time them at various release points on the ramp, and graph the information to estimate the speed of the cars. The purpose of both units was to help students develop their informal understanding of pre-algebraic concepts such as linear function, line of best fit, variables, rate of change (slope), reliability, and measurement error. Midway through the study, in-person instruction was suspended and moved online. Instead of working with groups of three to four students in the resource room throughout the day, the teacher provided online instruction to 14 students at one time and scheduled one-on-one sessions with students who needed extra help.

What challenges did you observe in the students interacting with the activities and their learning once they shifted to online learning?

All students had access to a computer at home and they were able to use the online platform without much confusion because they had used it in other classes. The screen share feature enabled students to interact with much of the curriculum by viewing the activities, listening to the teacher, and responding to questions, although they could not fully participate in the hands-on part of the lessons. Class attendance and student behavior were unexpectedly positive during the days when students were online. For example, one student had displayed frequent behavioral outbursts in school but became a positive and contributing member of the online class. The ability to mute mics in the platform gave the teacher the option of allowing only one student to talk at a time.

Were students still able to participate in the hands-on activities that are part of the intervention?

For the hands-on activities related to the Grand Pentathlon competition, the teacher taught online and a research staff member manipulated the cars, track, and electronic timers from campus. Students watched their computer screens waiting for their turn to time their cars over the length of the straightaway. The staff member handled each student’s cars and one by one released them from the height on the ramp as indicated by each student. After students had recorded the times, the teacher asked students to calculate and share the speeds of their cars for each time trial height.

Do you have any other observations about the impact of COVID-19 on your intervention implementation?

One of the most interesting observations was parent participation in the lessons. Several parents went beyond simply monitoring how their child was doing during the units to actively working out the problems. Some were surprised by the difficulty level of the math problems. One mother jokingly remarked: I thought the math they were going to do was as easy as 5 + 5 = 10. The next time my son might have to be the parent and I might have to be the student. You all make the kids think and I like that.

When COVID-19 shut down your participating schools, how were you able to adjust your data collection to continue with your research?

We used the same problem-solving test that we have administered in several previous studies (Figure 1 shows two of the items). On Day 1 of the study (pre-COVID), students took the math pretest in their resource rooms with pencil and paper. Due to COVID-19 school closures, we mailed the posttest and test administration instructions to student homes. On the scheduled testing day during an online class session, students removed the test from the envelope and followed directions for answering the test questions while we observed remotely. On Days 2 and 3 of the study (pre-COVID), an oral examiner (OE) pretested individual students in person. The OE asked the student questions, prompting the student to describe the overall problem, identify the information needed for solving the problem, indicate how the information related to their problem-solving plan, and provide an answer. Due to COVID-19, students took the oral posttests online. The teacher set up a breakout room in the platform where the OE conducted the oral assessments and a second member of the research team took notes.

A picture depicting two sample questions. The first shows a graph of two running paths along with the text, "3. The total distance covered by two runners is shown in the graph below. a. How much time did it take runner 1 to go 1 mile? b. About how much time after the start of the race did one runner pass the other?" The second image features a marble on top of a ramp accompanied with the question "What is the speed of a marble (feet per second) let go from the top of the ramp? (Round your answer to the nearest tenth.)"Figure 1. Sample Items from the Problem-Solving Test

During the testing sessions, the OE projected each item on the students’ computer screens. Then she asked the student to read the problem aloud and describe how to solve it. The OE used the same problem-solving prompts as was used on the pretests. For problems that involved graphs or charts, the OE used the editing tools to make notations on the screen as the students directed. One challenge is that oral testing online made it more difficult to monitor behavior and keep students on task. For example, sometimes students became distracted and talked to other people in their house.

What were the results of this study of oral assessment in mathematics for students with disabilities?

Our results suggest that allowing students to describe their understanding of problems in multiple ways yielded depth and detail to their answers. We learned from the oral assessment that most students knew how to transfer the data from the table to an approximate location on the graph; however, there was a lack of precision due to a weak understanding of decimals. For item 4 in Figure 1, the use of decimals confused students who did not have much exposure to decimals prior to or during the study. We also found that graphics that were meant to help students understand the text-based items were in some cases misleading. The representation in item 4 was different than the actual ramp and model car activity students experienced virtually. We have used this math test several times in our research and regrettably had no idea that elements of the graphics contributed to misunderstanding.

Unfortunately, our findings suggest that the changes made in response to COVID-19 may have depressed student understanding. Performances on two items (including item 4 in Figure 1) that assessed the main points of the intervention were disappointing compared to results from prior studies. The increase in class size from 3–4 to 14 after COVID and switching to online learning may have reduced the opportunity for repetition and practice. There were reduced opportunities for students to participate in the hands-on activities and participate in conversations about their thinking with other students.

We acknowledge the limitations of this small pilot study to compare knowledge of students when assessed in a pencil and paper format to an oral assessment. We are optimistic about the potential of oral assessments to reveal problem-solving insights of students with math disabilities. The information gained from oral assessment is of value if teachers use it to individualize their instruction. As we learned, oral assessment can also point to areas where graphics or other information are misleading. More research is needed to understand the value of oral assessment despite the increase in time it might add to data collection efforts for students with math disabilities. This experience highlights some of the positive experiences of students learning during COVID-19 virtually at home as well as some of the challenges and risks of reduced outcomes from these virtual learning experiences, especially for students with disabilities.

This blog was written by Sarah Brasiel, program officer for NCSER’s Science, Technology, Engineering, and Math program.

Responding to COVID19 in Education: ED/IES and Government Supported Developers Offer Virtual Resources and Activities for Distance Learning

We recently posted this blog listing more than 80 learning games and technologies that are available at no cost until the end of the school year in response to the closure of schools due to the COVID crisis. The resources were created by education technology developers with support from the Small Business Innovation Research (SBIR) Programs at ED/IES and other agencies, as well as through programs at IES and across government. In recent weeks, more than 100,000 teachers and students around the country have accessed these learning technologies at a distance.

Today, we are sharing more resources and activities that this group of developers is making available to the education community in response to COVID19.

A Series of Day-Long Virtual Unconferences

Over the coming weeks, developers are hosting a series of free virtual “Unconferences” on different topics for educators, parents, and students. The events will feature innovative models and approaches to teaching and learning during this time of distance learning and in-depth looks at the learning games and technologies created by the presenters, available at no cost until the end of the school year. While presenters will describe the delivery of online interventions via computers and devices, sessions will also focus on innovative approaches to implementing the interventions in low-resource settings.

The events are called “Unconferences” because the sessions are informal in nature and attendees can select sessions to join across the day. Attendees can participate by asking the presenters questions through the chat box and by responding to polls that capture reactions and views on topics.

Schedule and Information about the Virtual Unconferences in Education:

National K12 Student Challenge

ED/IES SBIR awardee Future Engineers (@K12FutureE) launched a nation-wide challenge for K12 students to submit entries to “invent a way to make someone smile or feel appreciated during COVID19.” Teachers can sign up a class to participate or students can participate on their own. See this page for more information and to submit an entry.

Stay tuned to the Inside IES Blog for more information and resources about the response to the COVID19 in education.

 


Edward Metz is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences. Please contact Edward.Metz@ed.gov with questions or for more information.

 

ASSISTments: From Research to Practice at Scale in Education

ASSISTments is a free, web-based formative assessment platform for teachers and students in Grades 3 through 12. The tool is designed for teachers to easily assign students math problems from OER textbooks such as Illustrative Math and EngageNY, existing item banks, or items they have developed on their own. ASSISTments will continually assess students as they solve math problems and provide immediate feedback and the chance to try again. The computer-generated reports provide teachers with information to make real-time adjustments to their instruction. Teachers can use it with their school’s existing learner management systems, such as Google Classroom and Canvas. Watch a video here.

 

 

Over the past 13 years, ASSISTments was developed and evaluated with the support of a series of IES and National Science Foundation awards. With a 2003 IES award to Carnegie Mellon University and Worcester Polytechnic Institute (WPI), researchers created the first version of ASSISTments. The system was populated with Massachusetts high-stakes mathematics test questions and the tutoring for the questions was authored by WPI staff with assistance from local teachers. After students completed problems assigned by the teacher, reports provided teachers with information about question difficulty and the most commonly submitted wrong answers, initiating class discussions around the completed assignments. In 2007, researchers at WPI received an award to build additional functionalities in the ASSISTments program so that teachers could assign supports (called “skill builders”) to students to help them master content.  An additional eight grants allowed the researchers to create other features. 

With a 2012 IES research grant award, SRI evaluated the efficacy of the ASSISTments program as a homework tool for academic learning.  In the study, the researchers took all 7th grade textbooks in the State of Maine and added answers to homework problems into ASSISTments.  The results of the efficacy trial demonstrated that teachers changed their homework reviewing behavior, mathematical learning improved an extra three quarters of a year of schooling, and using ASSISTments reliably closed achievement gaps for students with different achievement levels. ASSISTments is currently being evaluated again through two IES studies, with over 120 schools, to attempt to replicate this result. To view all publications related to ASSISTments, see here.

As of 2020, ASSISTments has been used by approximately 60,000 students with over 12 million problems solved.

 

Interview with Neil Heffernan and Cristina Heffernan

From the start of the project, was it always a goal that ASSISTments would one day be used on a wide scale?

We created ASSISTments to help as many teachers and students as possible. After we learned that the ASSISTments intervention was effective, we set the goal to have every middle school student in the country get immediate feedback on their homework. We created ASSISTments to be used by real teachers and have been improving it with each grant. Because of the effectiveness of ASSISTments, we kept getting funded to make improvements allowing our user base to grow.

At what point was ASSISTments ready to be used at a large scale in schools?

We were ready in year one because of the simplicity of our software. Now that we integrated seamlessly with Google Classroom, most teachers can use the system without training!

ASSISTments is backed by a lot of research, which would make some think that it would be easy for many schools to adopt. What were (or are) the biggest obstacles to ASSISTments being used in more schools?

A big obstacle has been access to technology for all students. The current environment in schools is making that less and less of a barrier. Now, teachers are looking for effective ways to use the computers they have.      

What options did you consider to begin distributing ASSISTments?

We had major companies try to buy us out, but we turned them all down. We knew the value was being in control so we could run research studies, let others run research studies and AB test new ideas. It was important to us to keep ASSISTments free to teachers. It is also a necessity since we crowdsource from teachers.

How do you do marketing?

Our biggest obstacle is marketing. But we are lucky to have just received $1 million in funding from a philanthropy to create a nonprofit to support the work of making our product accessible. Foundation funding has allowed us to hire staff members to write marketing materials including a new website, op-eds, blog posts and press releases. In addition to our internal marketing staff member, we work extensively with The Learning Agency to get press and foundation support for ASSISTments.

What costs are associated with the launch and distribution of ASSISTments, including marketing? Will a revenue model needed sustain ASSISTments over time?

When creating ASSISTments, we didn’t want a traditional business model based on schools paying. Our vision for future growth, instead, focused on crowdsourcing ideas from teachers and testing them. We are trying to replicate the Wikimedia platform idea created by Jimmy Wales. He crowdsources the content that makes up the encyclopedia, so it must be free. We envision using ASSISTments to help us crowdsource hints and explanations for all the commonly used questions in middle school mathematics.  

Do you have any agreement about the IP with the universities where ASSISTments was developed?

The ASSISTments Foundation was founded in 2019 and supports our project work in tandem with Worcester Polytechnic Institute due to our integration with research. The close relationship takes care of any issues that would arise with intellectual property. Additionally, the fact that we are a nonprofit helps address these issues.

How do you describe the experience of commercializing ASSISTments? What would you say is most needed for gaining traction in the marketplace?

Even though we are free, we do have several competitors. To gain traction, we have found that word of mouth is an effective disseminator and our positive efficacy trial result. Currently, there are many teachers on Facebook sharing how much they like ASSISTments. We also attend conferences and are working on an email campaign to get new users onboard.

Do you have advice for university researchers seeking to move their laboratory research into widespread practice?

Make sure your work is accessible and meaningful! We are solving a super-pervasive problem of homework in schools. Everyone finds meaning in making homework better.


Neil Heffernan (@NeilTHeffernan) is a professor of computer science and Director of Learning Sciences and Technologies at Worcester Polytechnic Institute.  He developed ASSISTments not only to help teachers be more effective in the classroom but also so that he could use the platform to conduct studies to improve the quality of education.  

Cristina Heffernan (@CristinaHeff) is the Lead Strategist for the ASSISTments Project at WPI. She began her career in education as a math teacher in the Peace Corps and after went on to teach middle school math in the US.  She began working with teachers while a graduate student at the University of Pittsburgh. As one of the co-founders of ASSISTments, Cristina has nurtured the system to be a tool for teachers to improve what they already do well in the classroom. 


This interview was produced by Edward Metz of the Institute of Education Sciences. This is the fourth in an ongoing series of blog posts examining moving from university research to practice at scale in education.​

Activities for Students and Families Stuck at Home due to COVID-19 (Coronavirus)

As I write this blog post, my 4-year-old is spraying me with a water sprayer while I am desperately protecting my computer from a direct hit. Earlier, while I was listening in on a meeting, she yelled out “hi!” anytime I took myself off mute. Balancing work and raising kids in this bizarre situation we find ourselves in is an overwhelming experience. When schools started closing, some parents resorted to posting suggested schedules for kids to keep up a routine and deliver academic content during the day. These were wonderful suggestions. As someone whose dissertation focused on how people learn, I should be applauding such posts, but instead, they filled me with a sense of anxiety and guilt. How am I supposed to balance getting my work done while also designing a rigorous curriculum of reading, writing, and math instruction for a kid whose attention span lasts about 10-20 minutes and who needs guidance and adult interaction to learn effectively? Let’s take a step back and recognize that this situation is not normal. We adults are filled with anxiety for the future. We are trying to manage an ever-growing list of things—do we have enough food? Do we need to restock medications? What deadlines do we need to hit at work?

So here is my message to you, parents, who are managing so much and trying desperately to keep your kids happy, healthy, and engaged: recognize that learning experiences exist in even the simplest of interactions between you and your kids. For example—

  • When doing laundry, have your child help! Have them sort the laundry into categories, find the matching socks, name colors. Create patterns with colors or clothing types (for example, red sock, then blue, then red, which comes next?).
  • Find patterns in your environment, in language (for example, nursery rhymes), and when playing with blocks or Legos. Researchers have shown that patterning is strongly related to early math skills.
  • Talk about numbers when baking. I did this with my daughter yesterday morning. We made muffins and had a blast talking about measuring cups, the number of eggs in the recipe, and even turning the dial on the oven to the correct numbers. Older kids might be interested in learning the science behind baking.
  • Take a walk down your street (practicing good social distancing of course!) and look for different things in your environment to count or talk about.
  • Bring out the scissors and paper and learn to make origami along with your kids, both for its benefits for spatial thinking and as a fun, relaxing activity! In this project, researchers developed and pilot tested Think 3d!, an origami and pop-up paper engineering curriculum designed to teach spatial skills to students. The program showed promise in improving spatial thinking skills.
  • If you choose to use screen time, choose apps that promote active, engaged, meaningful, socially interactive learning.
  • If you choose to use television programs, there is evidence showing that high quality educational programs can improve students’ vocabulary knowledge.

Hopefully these examples show that you can turn even the most mundane tasks into fun learning experiences and interactions with your kids. They may not become experts in calculus at the end of all of this, but maybe they will look back fondly on this period of their life as a time when they were able to spend more time with their parents. At the end of the day, having positive experiences with our kids is going to be valuable for us and for them. If you have time to infuse some formal learning into this time, great, but if that feels like an overwhelmingly hard thing to do, be kind to yourself and recognize the value of even the most simple, positive interaction with your kids.

Written by Erin Higgins, PhD, who oversees the National Center for Education Research (NCER)'s Cognition and Student Learning portfolio.