Inside IES Research

Notes from NCER & NCSER

Spotlight on FY 2023 Early Career Development Grant Awardees: Supporting HBCU Students with Trauma Informed Online Teaching

This summer, IES awarded the first grant within the Early Career Development and Mentoring Program for Faculty at Minority Serving Institutions to Dr. Virginia Byrne, an assistant professor at Morgan State University, a historically black university (HBCU) located in Maryland. In recognition of HBCU week, we asked Dr. Byrne to share with us her career journey and her new IES-funded early career project that explores how evidence-based models of trauma informed online teaching (TIOT) may benefit students taking online classes at HBCUs.

Tell us about your current IES early career project.

I began my career as a student affairs practitioner focused on how the Internet was expanding access for activism, community engagement, and higher education, particularly among students of minoritized and marginalized groups. Now I study online teaching and learning in higher education with a focus on which students tend to thrive in online classes versus those who struggle.

My IES early career project is driven by the urgent need to understand how to best support the academic success of HBCU students in online learning environments. Today’s college students are likely to have survived some form of traumatic or adverse event either prior to or during their undergraduate enrollment, including the complexities of grief, illness, financial difficulties, and social disruption such as the COVID-19 pandemic. As a result, many trauma-affected students struggle to engage in traditional school activities, regulate their effort and motivation, authentically participate in classroom discussions, and successfully graduate. To ensure that trauma-affected students are persisting in their college courses, educators are encouraged to adopt trauma-informed teaching practices.

While research exists on the value of trauma-informed teaching in face-to-face K-12 classrooms and, to a lesser extent, higher education contexts, it has yet to rigorously explore the impact of these trauma-informed practices on online college students. It is still unclear how trauma-informed teaching principles align with the leading online teaching literature, how students feel about these practices, or how they relate to achievement and persistence outcomes, particularly among trauma-affected students who might struggle with effort regulation. My three-year project, The Learning and Engaging at a Distance (LEAD) Initiative, seeks to fill this gap by posing an evidence-based model of TIOT practices in higher education and rigorously exploring how these practices might benefit HBCU students in online classes. I hypothesize that the adoption of TIOT practices will be helpful to all students, but especially those who are trauma-affected with reduced effort regulation. 

What are trauma-informed online teaching practices?

A trauma-informed approach to teaching consists of shifts in education practices, pedagogies, and policies as faculty learn about the role of trauma in students’ lives and how classrooms can perpetuate it. Trauma-informed teaching emphasizes an anti-racist, asset-based approach to provide all students with trauma-informed care, not just those who report their traumatic experiences (a universal approach). In my current IES project, I weave together the online learning, college teaching, and trauma-informed teaching literatures to pose the TIOT model, an evidence-based model consisting of seven related principles:

  1. Collaboration and Mutuality: Faculty welcome student input in collaboration and decision-making to share power and co-construct knowledge.
  2. Emotional, Social, and Academic Safety: Faculty foster an online learning environment that respects the need for safety, respect, and authenticity.
  3. Empowerment, Voice, and Choice: Faculty make space for and empower students to make choices so that the course is relevant to their professional interests and aligned with their personal development goals.
  4. Resilience, Growth, and Change: Faculty take an anti-deficit, growth-oriented approach by forwarding the idea that all students are emerging scholars capable of learning and succeeding. They use formative feedback to cultivate a community of learning and to reinforce self-efficacy.
  5. Social Justice: Faculty cultivate an equitable and anti-oppressive learning environment that prioritizes social justice in the course design and curriculum.
  6. Support and Connection: Faculty provide students with structures and resources to support their community building, academic achievement, and professional development.
  7. Trustworthiness and Transparency: Faculty foster a sense of trust and transparency among students by providing clear expectations, being reliable and consistent, and establishing healthy boundaries.

I theorize that by adopting a universal, trauma-informed approach to online teaching, instructors can create a more supportive and equity-centered learning environment for all students to thrive.

What advice would you give to emerging scholars from HBCUs who are pursuing a career in education research?

First, recognize the strengths of your institution as an HBCU. For example, my project leverages Morgan State University’s focus on high-touch teaching both in-person and online. By emphasizing what our faculty are already doing well and building on our existing expertise, I was able to gain campus buy-in on my project. Remember that being at an HBCU is an asset.

Second, even if your campus is small and research is not on the top of everyone’s to-do list, there are probably faculty who are doing rigorous research projects. Find them, even if they are outside of your field or discipline. Ask them how they found support on campus to do this work. Who is their go-to person for budget questions? For grant submission issues? Is there any unadvertised money for professional development? There are likely those on campus who want to mentor emerging scholars on research and grant-writing, even if it is not their full-time job. Reach out to them. 

How can the broader education research community better support the careers and scholarship of researchers from HBCUs?

The broader community often misses out on the tremendous excellence that HBCU faculty and graduate student researchers have to offer. There is a LOT of excellence here!

That said, we are limited by a lack of administrative research support and training, such as grant-writing support staff, in-house budget teams, graduate assistantship funding, high-quality financial software, and conference travel funding. This is caused, in part, by historic and systemic underfunding. Members of the broader community could better support HBCU researchers by educating themselves on the funding histories of the public HBCUs in their states and talking about it with colleagues, reaching out to HBCU faculty to build collaborative research partnerships, recruiting HBCU students to join their research teams as interns, research assistants, and post-docs, and sharing access to existing resources on the hidden curriculum around research (for example, how to write a grant proposal, how to craft a budget, how to effectively manage a large grant).

Incredible research is already happening at HBCUs. These are just a few ways that the education research community could get more involved.


Virginia L. Byrne, PhD, is an assistant professor of higher education and student affairs at Morgan State University in the School of Education and Urban Studies. Her research falls at the intersection of higher education, online learning, and the learning sciences to investigate issues of climate and equity in online and technology-enhanced learning environments. Dr. Byrne earned her PhD from the University of Maryland, College Park in technology, learning and leadership and her master’s degree in student affairs from Florida State University.

This blog was produced by Katina Stapleton (Katina.Stapleton@ed.gov), program officer for the Early Career Mentoring Program for Faculty at Minority Serving Institutions (Early Career MSI). This program supports grants that prepare faculty at minority-serving institutions to conduct high-quality education research that advances knowledge within the field of education sciences and addresses issues important to education policymakers and practitioners. In FY 2024, IES is accepting applications for the Early Career MSI program as well as the new Early Career Development and Mentoring Program for Education Research.

 

Congratulations Dr. Roddy Theobald on Winning the 2022 AEFP Early Career Award!

Headshot of Roddy TheobaldEach year, the Association for Education Finance and Policy (AEFP) recognizes one outstanding early career scholar whose research makes a significant contribution to the field of education finance and policy. In 2022, Dr. Roddy Theobald was the recipient of the Early Career award from AEFP. Congratulations to Dr. Theobald!

Dr. Theobald is a principal researcher in the Center for Analysis of Longitudinal Data in Education Research (CALDER) at the American Institutes for Research (AIR). CALDER, a collaboration among researchers at AIR and several universities around the United States, uses longitudinal data to explore a wide range of policy-relevant topics in education. Dr. Theobald’s research focuses on the teacher pipeline and its implications for student outcomes. Over the years, he has been involved in multiple IES-funded projects. These projects reflect a clear commitment to improving the teacher workforce and promoting positive outcomes for students. Dr. Theobald became interested in education policy research and studying the teacher workforce as a result of his experience as a 7th grade math teacher in the Oakland Unified School District. He is particularly interested in better understanding teacher shortage areas and what schools and districts can do to address them. 

As principal investigator (PI) on a recently completed researcher-practitioner partnership project, Dr. Theobald and his team worked in partnership with the Massachusetts Department of Elementary and Secondary Education to investigate the predictive validity of the state’s pre-service teacher evaluation systems and later in-service teaching outcomes and student outcomes. Key findings showed that teacher candidate performance on the Massachusetts Candidate Assessment of Performance, a practice-based assessment of student teaching, was predictive of their in-service summative performance ratings a year later. In examining the predictive validity of the Massachusetts Tests for Educator Licensure, results indicated that pre-service teacher scores were positively and significantly related to in-service performance ratings and value-added modeling of student test scores.

Dr. Theobald is currently the PI of a research grant that examines associations between pre-service teacher experiences (coursework, student teaching placements, and the match between student teaching experiences and early career experiences), special education teacher workforce entry and retention, and student academic outcomes. Using data on graduates of special education teacher education programs in Washington state, he found that the rate of special educator attrition is between 20-30%, which includes teachers that left public schools as well as those who moved to general education classrooms. Interestingly, the research team found that while dual endorsement in special and general education is positively associated with retention in the teaching workforce, it is negatively associated with retention in special education classrooms specifically. In terms of factors that promote retention, the research team found that better coherence between teacher preparation and early career experiences is associated with greater retention and that being supervised by a cooperating teacher endorsed in special education as part of student teaching is associated with a higher likelihood of becoming a special education teacher. The research team also found a link between preservice teacher experiences and student outcomes: students demonstrate larger reading gains when their district and the program from which their teacher graduated emphasized evidence-based literacy decoding practices and when a more experienced cooperating teacher supervised their teacher’s student teaching placement.

When we asked Dr. Theobald about the direction in which this line of research is heading, he explained, “immediate next steps in this line of work include looking at the employment outcomes of individuals trained to be special education teachers who never enter public school teaching or leave the teacher workforce, as well as better understanding the paraeducator workforce in public schools. It is also essential to understand how the special educator workforce has changed in response to the COVID pandemic, and we hope to study these changes in the years to come!”

This blog was authored by Kaitlynn Fraze, doctoral student at George Mason University and IES intern, and Katie Taylor (Katherine.Taylor@ed.gov), program officer at the National Center for Special Education Research.

A Lightbulb Moment: How IES Sparks Research, Teaching, and Practice

We know a lot about how people learn and the strategies and principles that promote learning and retention, but much of it gets stuck in translation between research and practice. Through the Cognition and Student Learning program, the Institute of Education Sciences (IES) supports projects that try to bridge that gap. Pooja Agarwal has been a researcher for multiple Cognition and Student Learning projects. She was also a Harry S. Truman Foundation Scholarship recipient, which gave her the opportunity to work as an intern at IES for a summer. She received her PhD from Washington University of St. Louis and is currently an Assistant Professor at Berklee College of Music and founder of RetrievalPractice.org. Here she shares some reflections on how IES and its grants programs have influenced her career and the field, culminating in a book she recently published with collaborator Patrice Bain, Powerful Teaching: Unleash the Science of Learning.

 

Fifteen years ago, I was double majoring in cognitive neuroscience and elementary education at Washington University in St. Louis. One semester, I was taking a psychology class on human memory on one side of campus and a class on K–12 social studies teaching methods on the other side of campus. I felt frustrated that the psychology class was too esoteric, thinking that’s not how memory works in the “real world.” Meanwhile, I felt the social studies methods class was too anecdotal; we were being told to teach the way the professor taught without any evidence to support their methods.

Around the same time, the Institute of Education Sciences (IES) was born, bridging the gap between research and practice. I was enthralled by the legislative authorization bill (yes, I was that college student). The more I read about IES, the more I was convinced that transformation of the education system through research was not only possible–it was starting to happen. Simply put, this was my coveted “lightbulb moment.”

Subsequently, I spent a summer as an intern at IES, and the following fall, my mentor Henry L. Roediger, III (“Roddy”) and colleagues received a Cognition and Student Learning (CASL) grant from IES (2006-2010). This opportunity was a perfect fit with my growing passion: I would be embedded in K–12 classrooms leading rigorous research on learning and memory.

Our research centered around extending a laboratory-based principle, retrieval practice, into the classroom. Retrieval practice is the process by which learners recall or retrieve information they have previously learned, which subsequently improves their long-term retention for that information. For example, do you know the fourth president of the United States? Your mental struggle is referred to as a “desirable difficulty,” which will help you remember the name of the president (it’s listed at the end of this blog).

Our first experiments on retrieval practice were conducted with Patrice Bain, a 6th grade world history teacher at Columbia Middle School in Columbia, Illinois. Initially, we compared student performance after lessons with brief quizzes vs. lessons without quizzes. Importantly, Patrice’s curriculum stayed the same; we simply included frequent retrieval practice. In one set of experiments, for example, retrieval practice boosted grades from a C to an A level, with benefits lasting nine months later, until the end of the school year. By year two, we were collecting data on a scale we never had in the lab, in various grade levels and content areas. We were fortunate to continue our research with a second CASL grant (2011-2014), publishing numerous peer-review publications, presenting at academic conferences, and creating a practice guide for educators on the research and implementation of retrieval practice in the classroom. I recently completed a review of the literature on retrieval practice, screening more than 2,000 abstracts and narrowing them down to 50 selected experiments. The majority of experiments demonstrated that retrieval practice consistently boosted student learning, regardless of type (for example, multiple-choice or short answer), spacing over time, or education level.

To get this information into the hands of educators around the world, my fellow cognitive scientists and I have written and disseminated 6 practice guides available in 6 languages, which have been downloaded more than 100,000 times. I continue to develop a community of more than 15,000 educators around the world via social media (Twitter and Facebook), a weekly newsletter, and articles and podcasts. Of course, our own research on retrieval practice informs my own teaching as a college professor on a daily basis.

Most recently, I co-authored a book, Powerful Teaching: Unleash the Science of Learning, with aforementioned collaborator Patrice Bain. Educators are given the impossible challenge of seeking out good research, making sense of it, and applying it in the classroom. It is impossible because this research isn’t accessible–literally and figuratively. In line with IES’s mission, we aimed to increase access to cognitive science research and make it applicable for today’s classrooms.

In Powerful Teaching, we focused on four teaching strategies we call Power Tools: retrieval practice, spacing, interleaving, and feedback-driven metacognition, all of which are supported by research from the CASL grants program.

  1. Retrieval practice boosts learning by pulling information out of students’ heads, rather than cramming information into students’ heads.
  2. Spacing boosts learning by spreading lessons and retrieval opportunities out over time, so learning is not crammed all at once. In this way, forgetting is a good thing for learning.
  3. Interleaving boosts learning by mixing up closely related topics, encouraging discrimination between similarities and differences.
  4. Feedback-driven metacognition boosts learning by providing the opportunity for students to know what they know and know what they don’t know.

The four Power Tools are flexible, practical, and quick to implement. By focusing on just a few carefully selected strategies, educators are empowered to harness cognitive science, without being stretched too thin. We have found these elements–accessibility and feasibility–to be critical if educators are to implement research-based strategies in their classrooms.

I say all this because I want to emphasize that IES–and the people behind the scenes–is much more than a granting agency. IES provides opportunities and support for applied research in education that can inform practice and make a difference in the classroom. Sometimes, all it takes is a lightbulb moment to spark a transformation.

 

Endnote: The fourth president of the United States was James Madison.


Pooja K. Agarwal, PhD (@RetrieveLearn) is a cognitive scientist, conducting research on how students learn since 2005. She is the author of the book Powerful Teaching: Unleash the Science of Learning and an Assistant Professor of Psychology at the Berklee College of Music in Boston. Pooja is also the Founder of RetrievalPractice.org, a source of research-based teaching strategies for more than 15,000 educators around the world.

Using Mistakes as a Vehicle for Learning in Mathematics: From Research to Practice at Scale in Education

Every student makes mistakes. But not every student is given the opportunity to learn from mistakes. Left unaddressed, the mathematical misconceptions that underlie many mistakes can keep students from progressing in mathematics.

 

At the request of districts in the Minority Student Achievement Network (MSAN), a Strategic Education Research Partnership (SERP) team was convened in 2007 to address a widening achievement gap in Algebra I. The team was charged with identifying an intervention strategy, subject to several district constraints:

  1. The solution would need to be applied to all students in the regular classroom to avoid the stereotype threat associated with separating students based on performance and to protect the intervention from budget cuts that target supplemental, after-school, and summer programs first.
  2. A new curriculum was off the table because it would create upheaval for a time and would be followed by a decline in student performance during the period of adjustment.
  3. Extensive teacher training was considered undesirable because it would be costly and because algebra teachers consider themselves more expert in mathematics teaching than central office staff who would be requiring the training.

 

Julie Booth joined the partnership, and with funding from IES, led the iterative development and testing of worked example assignments that, with the input of teachers and administrators, fit within the routines of the classroom. The result—AlgebraByExample—consists of 42 uniquely designed assignments that address misconceptions, harness the power of explanation, and use mistakes as a vehicle for learning.

Typical math assignments require students to solve problems on their own. If a student’s work is incorrect, the student may never focus on what went wrong. ByExample assignments also give students problems to solve, but they first provide a solution to a similar problem that is marked right or wrong. Students are prompted with questions that target common misconceptions and errors before solving a similar problem on their own. Each assignment contains several strategically designed item pairs:

 

 

Designed in collaboration with teachers from districts in several states, the assignments can be easily incorporated into any Algebra I curriculum and teachers can choose in what way and in what order to use them. The assignments were tested in randomized trials in classrooms in eight districts with more than 6,000 students. Not only did students using AlgebraByExample improve an average of 7 percentage points on an assessment of standardized test items, students at the lower end of the distribution improved the most. The PDF downloads of the assignments are freely available for anyone to use.

The success of AlgebraByExample  led to  further IES funding of MathByExample for Grades 4 and 5 and GeometryByExample for high school geometry .

 

Resources:

AlgebraByExample website

MathByExample website

Booth et al, 2015

NSF Stem for All Video Submission 2019

 

Interview with Dr. Suzanne Donovan (SERP), Dr. Julie Booth (Temple University), and Allie Huyghe (SERP), the developers of the ByExample interventions.

 

 

Was it part of the original plan to develop an intervention that could one day be used at scale in schools?

Yes. SERP partnerships begin with problems of practice nominated by district partners, but the partnership agreement distinguishes SERP from a consultant. The intention from the start is to frame the problem and design a solution that can be used at scale. SERP has developed in-house, user-centered design expertise so that resources (such as the ByExample products) developed through partnerships meet the needs of teachers and students. Products scale when they improve the experience of teachers and students. Both the model and the internal design capacity allow SERP to move from problem framing through research, development, and dissemination of a product with IES grant funding.

 

Describe the initial research and development that occurred.

Dr. Julie Booth drafted initial assignments drawing on the mathematics misconceptions literature. SERP held regular partnership meetings with teachers and administrators at which assignments were reviewed and additional misconceptions were nominated for attention in the assignments. Administrators agreed to randomization of the assignments across classrooms and within-teacher. Assignments were first tested in individual topic blocks and revised in accordance with student performance data, observations, and teacher feedback. A year-long pilot study was then conducted using the full set of assignments.

 

Beyond IES or ED grants, what additional funding was needed to develop the intervention?

For the ByExample work, additional funding was provided by the Goldman Sachs Foundation in the initial phase to support partnership formation, problem framing, and the solution generation. IES grants funded the research and development, along with initial dissemination activities to make the materials available to the public. We were also able to develop an online platform to allow for digital use with the IES grant funds.

 

What model was used for dissemination and sustainability?

The assignments are available as free downloads on SERP’s website, and as printed workbooks through SERP’s partner print-on-demand company. They have been publicized through online communications, journal articles, presentations at conferences of various types, social media, and word of mouth. There will be a small fee for use of the digital platform to support its maintenance, but the PDFs will remain as free downloads. We have been able to sustain the collaboration of the partnership team by responding to requests from educators to expand the approach to other grade levels and submitting additional proposals to IES that have been awarded.

 

What advice would you provide to researchers who are looking to move their research from the lab to market? What steps should they take? What resources should they look for?

First, I would note that it is difficult to persuade educators to use a product that solves a problem they don’t believe they have. Listen to educators and apply research expertise to address the challenges that they experience on a day-to-day basis. Design for ease of use by teachers. No matter how good your strategy or marketing is, if it’s too much work for an already busy teacher to use, you may get uptake by a few committed teachers, but not at scale. Finally, pay attention to where teachers get their information. For AlgebraByExample, we got a big boost from the Marshall Report, produced by a teacher for other teachers to call attention to usable research.  

 

In one sentence, what would you say is most needed for gaining traction and wide scale use by educators?

Design for the routines of the classroom.

 


Suzanne Donovan, PhD, is the founding Executive Director of the SERP Institute, an education research, development, and implementation organization incubated at the National Academies. SERP leads collaborations of educators, researchers, and designers to generate research-based, scalable, and sustainable solutions to critical problems of practice. 

Julie Booth, PhD, is a Professor of STEM Education and Psychology and the Deputy Dean of Academic and Faculty Affairs at Temple University’s College of Education and Human Development. Her work focuses on translating between cognitive science and education to better understand students’ learning and improve instruction, primarily in mathematics education. She is currently an Executive Editor for the Journal of Experimental Education.

Allie Huyghe is the Assistant Director of the SERP Institute, where she manages several projects, including the IES-funded MathbyExample and GeometryByExample projects. She is also intricately involved with other SERP areas of work, participating in the design of materials from early development through release to the public.

 

This interview was produced by Christina Chhin (Christina.Chhin@ed.gov) and Edward Metz (Edward.Metz@ed.gov) of the Institute of Education Sciences. This is the fifth in an ongoing series of blog posts examining moving from university research to practice at scale in education.​

 

 

Investing in Next Generation of Education Technologies to Personalize Learning and Inform Instructional Practice

The Institute of Education Sciences Small Business Innovation Research program (ED/IES SBIR) funds entrepreneurial developers to create the next generation of education technology for students, teachers, and administrators in general and special education. The program emphasizes an iterative research and development process and pilot studies to evaluate the feasibility, usability, and promise of new products to improve educational outcomes. The program also focuses on commercialization after development is complete, so that the products can reach schools and be sustained over time.

In recent years, millions of students in tens of thousands of schools around the country have used technologies developed through ED/IES SBIR. And in the past four months, about one million students and teachers used the technologies for remote teaching and learning, as many ED/IES SBIR-supported developers made their products available at no cost in response to the COVID-19 pandemic and the closure of schools.

 

ED/IES SBIR Announces its 2020 Awards

This week, ED/IES SBIR announced the results of its 2020 award competition. Of the 22 new awards, 16 are for prototype development and 6 are for full-scale development. IES also announced two additional awards through a special topic solicitation in postsecondary education. Read about these awards here.

 

 

Each of the new awards supports a project to develop a product to personalize the student learning experience or generate information that educators can use to guide practice.

Most of the projects are developing a software component (for example, algorithms, artificial intelligence, machine Learning, or natural language processing) that continually adjusts the difficulty of content, provides prompts to support individual students if support is needed, or generates real-time actionable information for educators to track student progress and adjust instruction accordingly. Other projects are developing technologies to promote student learning through self-directed, hands-on, simulated, and immersive experiences. If the future of education includes a blend of in-class and remote learning due to public health crises, or for whatever reasons, technologies such as these will be ready to keep the learning going.

The projects address different ages of students and content areas.

In science, LightHaus is fully developing a virtual reality (VR) intervention for students to explore plant heredity; LightUp is fully developing an augmented reality (AR) app for students to perform hands-on physical science investigations with their own on-device camera; and Myriad Sensors is developing a prototype artificial intelligence formative assessment system that generates feedback in real time as students do hands-on laboratory experiments.

In math, Muzology is creating a prototype for students to create music videos to learn algebra, and Teachley is creating a prototype transmedia kit with videos, comics, and pictures to enhance teaching and learning of hard to learn concepts.

In engineering and computer science, Parametric Studios is fully developing an augmented reality puzzle game for early learners, and Liminal eSports, Makefully, and Beach Day Studios are creating prototype components that each provide feedback to students as they engage in activities to learn to code.

In English Language Arts, Analytic Measures and Hoogalit are each employing natural language processing to develop new prototypes to facilitate speech acquisition, and Learning Ovations is developing a prototype data engine to make recommendations for what individual children should read.

For English learners, KooApps is developing an artificial intelligence prototype to support vocabulary acquisition, and Kings Peak Technologies is employing machine learning to generate passages that blend English and Spanish words together to improve reading comprehension.

For early learners, Cognitive Toybox is fully developing an observation and game-based school readiness assessment.

For postsecondary students, Hats & Ladders is fully developing a social skills game to foster career readiness skills.

In special education, Attainment Company is developing a prototype to support student’s self-management, and Alchemie is developing a prototype of an augmented reality science experience for visually impaired students.

To support school administrators and teachers, LearnPlatform is fully developing a dashboard that generates reports with insights for teachers to implement education technology interventions, and Zuni Learning Tree, Teachley and LiveSchool are developing prototype dashboards to organize and present results on student progress and performance in real time.

 

Stay tuned for updates on Twitter and Facebook as IES continues to support innovative forms of technology.


Written by Edward Metz (Edward.Metz@ed.gov), Program Manager, ED/IES SBIR