IES Blog

Adult education programs aim to support the millions of American adults who wish to strengthen their basic skills, earn a high school degree or equivalent, or become U.S. citizens. During the National Adult Education Family and Literacy Week (September 20-26, 2020), IES is highlighting the ongoing research it supports to help expand our knowledge of and innovations for the adult education system and the communities they support.

Roughly 30 million U.S. adults may lack a high school degree or equivalent. And according to an assessment conducted in 2017, approximately 30 percent of adults may have very low numeracy skills, and approximately 20 percent may have very low literacy skills. Adults with low levels of academic attainment or low basic skills may face barriers to full economic or civic engagement.

Adult education programs aim to help such adults. These programs often run on lean budgets and need to meet the needs of a highly diverse population with a multitude of learning goals.

In order to help understand and improve the ability of the adult education system to provide services, IES researchers have been conducting various research projects in partnership with adult education providers.

For example, Career Pathways Programming for Lower-Skilled Adults and Immigrants conducted mixed-methods research in partnership with Chicago, Houston, and Miami. This work helped each city understand what types of career pathways adult education programs were offering and who was participating in such programing. This type of information helps programs, cities, and the system more broadly understand and adjust to the needs of learners and communities.

The New York State Literacy Zone Researcher-Practitioner Partnership focused on improving the ability of case managers to help adult learners leverage wrap-around services and access and succeed in adult education and training programs. This project helped develop tools and training for case managers and conduct an exploratory pilot study of these tools to see if they predicted learner outcomes, such as persistence in a program.

The Georgia Partnership for Adult Education and Research (GPAER) is a collaboration among researchers at Georgia State University and leadership at the Georgia Office of Adult Education: Technical College System of Georgia to help understand adult literacy programs across the state. This ongoing work is conducting mixed-methods studies to understand program features, learner characteristics, and indicators of beneficial learner outcomes.

There is still much to learn to help improve the ability of programs to find and support adult learners. IES encourages additional research to further help us understand the landscape of adult education, the needs and interests of adult learners and their instructors, and the outcomes and impacts of improving adult basic skills.

For more information about adult education research at the National Center of Education Research, contact Dr. Meredith Larson.

Many programs across the Federal government, such as the ED/IES Small Business Innovation Research (SBIR) and the IES Research Grants programs, fund projects to develop and evaluate new forms of education technology and interventions that can be implemented to support instruction and learning at schools and for remote learning. More than 150 of these technologies were demoed in January 2020 at the ED Games Expo, a showcase for learning games and technologies developed with support from IES and more than 30 other Federal programs.

Since the global outbreak of COVID19 and the closure of schools across the United States and the world, a group of government-supported developers and researchers responded to provide resources to educators, students, and families to facilitate remote learning. More than 50 developers and researchers offered 88 learning games and technologies at no cost through the end of the school year for use in distance learning settings with internet access (see this blog for the list). In addition, many of the developers and researchers provided technical assistance directly to individual teachers to support implementation at a distance, and many created new materials and worked to refine and adapt their products to optimize usability and feasibility for fully remote use. More than a million students and thousands of educators used these learning technologies during the spring.

In April and May 2020, more than 70 developers and researchers partnered to produce and participate in a series of free day-long virtual events, which were called “unconferences.” The events featured presentations on innovative models and approaches to teaching and learning remotely and provided an in-depth look at the learning games and technologies created by the presenters. More than 25,000 educators attended these virtual events in real-time, hundreds asked questions and made comments through chats during the events, and many thousands more have accessed these videos after the events. See this blog for the list of archived videos.

A New Resource: Guides to Education Technologies that are Ready Now

As schools begin re-opening for the new school year, a group of 70 developers and researchers have collaborated to produce a new series of Guides to Education Technologies. The guides present information on government-supported education technology products that are ready now for in-class and remote learning. All the resources are web-based and can be used on either computers, tablets, or personal devices. The resources in the guides include a mix of no-cost products as well as ones that are fee-based.  

With awards from government programs, all of the resources were developed through an iterative process with feedback from teachers and students, and most were evaluated through small pilot studies to measure the promise of the technologies to support improvements in student learning and relevant educational outcomes. All the products were used and demonstrated to be feasible for use in remote settings in the spring after the onset of the pandemic.

The guides present resources appropriate for young children through postsecondary students in education and special education, for English learners, and for teachers in education and special education across a wide range of educational topics. Many of the technologies personalize learning by adjusting content to students as they go and present information to educators to inform instruction.

The Guides focus on the following areas and can be accessed below:

• Early Learning
• Mathematics
• Science and Engineering
• Social Studies
• Special Education

Stay tuned to the Inside IES Blog for more information and resources about the response to the COVID-19 in education.

Edward Metz (Edward.Metz@ed.gov) is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.

We know a lot about how people learn and the strategies and principles that promote learning and retention, but much of it gets stuck in translation between research and practice. Through the Cognition and Student Learning program, the Institute of Education Sciences (IES) supports projects that try to bridge that gap. Pooja Agarwal has been a researcher for multiple Cognition and Student Learning projects. She was also a Harry S. Truman Foundation Scholarship recipient, which gave her the opportunity to work as an intern at IES for a summer. She received her PhD from Washington University of St. Louis and is currently an Assistant Professor at Berklee College of Music and founder of RetrievalPractice.org. Here she shares some reflections on how IES and its grants programs have influenced her career and the field, culminating in a book she recently published with collaborator Patrice Bain, Powerful Teaching: Unleash the Science of Learning.

Fifteen years ago, I was double majoring in cognitive neuroscience and elementary education at Washington University in St. Louis. One semester, I was taking a psychology class on human memory on one side of campus and a class on K–12 social studies teaching methods on the other side of campus. I felt frustrated that the psychology class was too esoteric, thinking that’s not how memory works in the “real world.” Meanwhile, I felt the social studies methods class was too anecdotal; we were being told to teach the way the professor taught without any evidence to support their methods.

Around the same time, the Institute of Education Sciences (IES) was born, bridging the gap between research and practice. I was enthralled by the legislative authorization bill (yes, I was that college student). The more I read about IES, the more I was convinced that transformation of the education system through research was not only possible–it was starting to happen. Simply put, this was my coveted “lightbulb moment.”

Subsequently, I spent a summer as an intern at IES, and the following fall, my mentor Henry L. Roediger, III (“Roddy”) and colleagues received a Cognition and Student Learning (CASL) grant from IES (2006-2010). This opportunity was a perfect fit with my growing passion: I would be embedded in K–12 classrooms leading rigorous research on learning and memory.

Our research centered around extending a laboratory-based principle, retrieval practice, into the classroom. Retrieval practice is the process by which learners recall or retrieve information they have previously learned, which subsequently improves their long-term retention for that information. For example, do you know the fourth president of the United States? Your mental struggle is referred to as a “desirable difficulty,” which will help you remember the name of the president (it’s listed at the end of this blog).

Our first experiments on retrieval practice were conducted with Patrice Bain, a 6th grade world history teacher at Columbia Middle School in Columbia, Illinois. Initially, we compared student performance after lessons with brief quizzes vs. lessons without quizzes. Importantly, Patrice’s curriculum stayed the same; we simply included frequent retrieval practice. In one set of experiments, for example, retrieval practice boosted grades from a C to an A level, with benefits lasting nine months later, until the end of the school year. By year two, we were collecting data on a scale we never had in the lab, in various grade levels and content areas. We were fortunate to continue our research with a second CASL grant (2011-2014), publishing numerous peer-review publications, presenting at academic conferences, and creating a practice guide for educators on the research and implementation of retrieval practice in the classroom. I recently completed a review of the literature on retrieval practice, screening more than 2,000 abstracts and narrowing them down to 50 selected experiments. The majority of experiments demonstrated that retrieval practice consistently boosted student learning, regardless of type (for example, multiple-choice or short answer), spacing over time, or education level.

To get this information into the hands of educators around the world, my fellow cognitive scientists and I have written and disseminated 6 practice guides available in 6 languages, which have been downloaded more than 100,000 times. I continue to develop a community of more than 15,000 educators around the world via social media (Twitter and Facebook), a weekly newsletter, and articles and podcasts. Of course, our own research on retrieval practice informs my own teaching as a college professor on a daily basis.

Most recently, I co-authored a book, Powerful Teaching: Unleash the Science of Learning, with aforementioned collaborator Patrice Bain. Educators are given the impossible challenge of seeking out good research, making sense of it, and applying it in the classroom. It is impossible because this research isn’t accessible–literally and figuratively. In line with IES’s mission, we aimed to increase access to cognitive science research and make it applicable for today’s classrooms.

In Powerful Teaching, we focused on four teaching strategies we call Power Tools: retrieval practice, spacing, interleaving, and feedback-driven metacognition, all of which are supported by research from the CASL grants program.

1. Retrieval practice boosts learning by pulling information out of students’ heads, rather than cramming information into students’ heads.
2. Spacing boosts learning by spreading lessons and retrieval opportunities out over time, so learning is not crammed all at once. In this way, forgetting is a good thing for learning.
3. Interleaving boosts learning by mixing up closely related topics, encouraging discrimination between similarities and differences.
4. Feedback-driven metacognition boosts learning by providing the opportunity for students to know what they know and know what they don’t know.

The four Power Tools are flexible, practical, and quick to implement. By focusing on just a few carefully selected strategies, educators are empowered to harness cognitive science, without being stretched too thin. We have found these elements–accessibility and feasibility–to be critical if educators are to implement research-based strategies in their classrooms.

I say all this because I want to emphasize that IES–and the people behind the scenes–is much more than a granting agency. IES provides opportunities and support for applied research in education that can inform practice and make a difference in the classroom. Sometimes, all it takes is a lightbulb moment to spark a transformation.

Endnote: The fourth president of the United States was James Madison.

Pooja K. Agarwal, PhD (@RetrieveLearn) is a cognitive scientist, conducting research on how students learn since 2005. She is the author of the book Powerful Teaching: Unleash the Science of Learning and an Assistant Professor of Psychology at the Berklee College of Music in Boston. Pooja is also the Founder of RetrievalPractice.org, a source of research-based teaching strategies for more than 15,000 educators around the world.

With a 2010 IES research grant, researchers at Teachers College, Columbia University conducted basic research and created prototype software programs for children in mathematics. In 2011, three members of the research team launched a startup and submitted a successful proposal to IES’s Small Business Innovation Research programs. With awards in 2012 and 2013, the developers created a suite of math game apps that support fact fluency and promote math strategy development. The apps all connect with a teacher dashboard that provides in-depth reports in real time and supports differentiation in math instruction. In 2013, Teachley’s Addimal Adventure won an Apple Design Award as one of the 12 best apps of 2014. Since their commercial launch in 2014, Teachley Apps have been downloaded 1.5 million times, and the Teachley suite of products are currently used in all 50 states and 2,000 schools.

Interview with Kara Carpenter, co-founder of Teachley

The three co-founders of Teachley were all classroom teachers before you met at Teachers College as graduate students in 2010. What led to your decision to go to graduate school to earn PhDs as researchers?

While teaching 2nd grade, I had the opportunity to receive professional development focused on elementary math content, and I became fascinated with how children develop their mathematical thinking. Years later, when I was getting a master’s in curriculum & teaching at Teachers College, I pursued a work study opportunity with Professor Herb Ginsburg, who focuses on early childhood math thinking. At the time in 2009, my cofounder Rachael Labrecque was already working with Professor Ginsburg, and the three of us submitted an application to IES to develop math software for young learners. That fall, I went back to classroom teaching, but when the application was funded in 2010, I decided to take the leap and accept a research fellowship to pursue a PhD. My other co-founder, Dana Pagar, joined our research team that fall, and the three of us decided to start Teachley in 2012 to bring all the great research on how kids learn math into marketable products.

Tell us about the research projects that you were involved with in graduate school.

We worked on a project developing math software for grades pre-K to 3, called MathemAntics. We developed dozens of activities and conducted small learning studies along the way. In the third year, we conducted an RCT with approximately 400 students in grades PreK - 2. Each of our dissertations involved different elements of the project. Mine focused on teaching and detecting kids’ single-digit addition strategies. Dana’s focused on continuous versus discrete blocks, while Rachael studied teachers’ preparedness to integrate technology into their classrooms.

How did you come up with the idea to develop apps that would be used in schools on a wide scale basis?
Originally, we were looking for a company who might want to take these research findings and turn them into commercial products. We were meeting with various business leaders, and one of them turned to us and said, “You should do this. You should start a company to bring your ideas to market.” That’s the push we needed to think of ourselves as potential startup founders.

How did you find out about the SBIR program at the US Department of Education’s Institute of Education Sciences? How important was the first SBIR award for launching Teachley?

Once we decided to start Teachley, we knew that SBIR would be a great resource for us. The MathemAntics project had actually started out as an NIH SBIR Phase I with a different company. That first ED/IES SBIR award is the reason that Teachley became a company. Without that funding, we would not have been able to prove ourselves capable of bringing a product to market. Institutional investors aren’t taking those kinds of risks, and angel investment is too tied into social networks and who you know.

Was Teachers College supportive of its graduate students starting a small business and getting an award to develop apps? Did anyone at the university offer advice or guidance on how to operate a small business?

Leaving the university was tricky because we had research fellowships when we started the company. However, the Teachers College president at the time, Susan Fuhrman, and the provost, Tom James, were supportive of our startup. We speak and participate in various discussions and events at Teachers College, which keeps us connected to the university and the research.

How does Teachley ensure that research is integrated into your development and validation process?

Before developing any new product idea, we look to the research to see what’s already been learned about the topic, especially as it relates to struggling learners. During the early stages of development, we rely on close observations of students as they use pencil/paper mockups and early software builds. As a team, we closely review videos of students working through problems, looking to find better, more intuitive ways to support students’ thinking. Once we have a functional prototype, we use more formal evaluative techniques to determine our impact on student learning.

What models have you used to commercialize Teachley on a widespread basis?

We have tried out many different revenue models. Initially, we tried publishing the games for free and charging schools for the formative assessment data. However, we soon found that bundling the games and data together into a single subscription worked better for schools. With our latest game, Market Bay, we are trying a new model where educators create a free account, and parents subscribe to have access at home. Schools who subscribe to Teachley get home access to Market Bay and our other games for all of their families.

Have you raised funds from venture capitalists? Why or why not?

Not yet. Raising money from venture capitalists can put you on a succeed-or-fail-fast treadmill that isn’t always a great fit for the education market. Many investors are looking for a 70x return within just a few years or they abandon ship. Developing great educational software takes time for both the iterative design process and the research to prove your effectiveness. We are just now at the stage where raising venture capital may soon make sense because we have enough content to scale our school/district sales.

When COVID-19 emerged and schools closed, you made your apps freely available to teachers and students in their classes, and 15,000 teachers and students were able to access your products. What was that experience like?

Teachers are looking for digital products that will deeply engage students and support true learning. We’re a great fit. However, schools across the country are suffering budget shortfalls at the same time as they need to spend more to ensure they meet safety standards. We’re working with schools and teachers to find alternative ways to fund our program, from parent organizations to Donors Choose to corporate partnerships.

None of you had had formal business training prior to founding Teachley. Do you have advice for those who are interested in starting an entrepreneurial small business to develop education technology that can be used in schools?

My advice would be to know your users and implementation deeply. If you don’t have a background in teaching, spend time volunteering in schools. Become a close observer of children and their thinking, so you can create products that support and bring out children’s genius.

 ____________________________________________________________________________

Kara Carpenter is cofounder of Teachley (@teachley), an edtech startup focused on promoting deep math thinking and learning. Kara has over 10 years of teaching experience and was a National Board Certified Teacher with a PhD in Cognition and Learning from Teachers College, Columbia University. Her dissertation went on to become an Apple Design Award winning app, Addimal Adventure.

This interview was produced by Ed Metz (Edward.Metz@ed.gov) of the Institute of Education Sciences. This post is the sixth in an ongoing series of blog posts examining moving from university research to practice at scale in education.

In 2020, as part of a wider IES investment in resources around cost, IES funded the Cost Analysis in Practice (CAP) Project, a 3-year initiative to support researchers and practitioners who are planning or conducting a cost analysis of educational programs and practices. The CAP Project Help Desk provides free on-demand tools, guidance, and technical assistance, such as support with a cost analysis plan for a grant proposal. After inquiries are submitted to the Help Desk, a member of the CAP Project Team reaches out within two business days. Below is a list of resources that you can access to get more information about cost analysis.

STAGES FOR CONDUCTING A COST ANALYSIS

CAP Project Resources

Cost Analysis Standards and Guidelines 1.0: Practical guidelines for designing and executing cost analyses of educational programs.

IES 2021 RFAs Cost Analysis Requirements: Chart summarizing the CAP Project’s interpretation of the IES 2021 RFAs cost analysis requirements.

Cost Analysis Plan Checklist: Checklist for comprehensive cost analysis plans of educational programs and interventions.

Introduction to Cost Analysis: Video (17 mins).

General Cost Analysis Resources

The Critical Importance of Costs for Education Decisions: Background on cost analysis methods and applications.

Cost Analysis: A Starter Kit: An introduction to cost analysis concepts and steps.

CostOut®: Free IES-funded software to facilitate calculation of costs once you have your ingredients list, includes database of prices.

DecisionMaker®: Free software to facilitate evidence-based decision- making using a cost-utility framework.

Cost-Effectiveness Analysis of Early Reading Programs: A Demonstration With Recommendations for Future Research: Open access journal article.

*More resources available here.

The content for this blog has been adapted from the Cost Analysis in Practice Project informational flyer (CAP Project, 2020) provided by the CAP Project Team. To contact the CAP Help Desk for assistance, please go to https://capproject.org/. You can also find them on Twitter @The_CAP_Project.