Help Us Grow Seedlings

Mark Schneider, Director of IES | January 23, 2024

Many of you, I'm sure, have been following the ongoing efforts of IES and its supporters to establish NCADE (aka ARPA-ED). Congress needs to create NCADE, and we continue to see slow but steady progress there. In the meantime, IES is carving out a set of innovative, risk-informed, high-reward activities that fit within our existing legislative authority (and budget), pending the more extensive changes that NCADE will enable.

Among the ARPA-like activities moving forward is our From Seedlings to Scale (S2S) program that will support new work that integrates high quality product development with high quality research to transform breakthrough ideas into facts on the ground.

A while ago, we put out a Request for Information (RFI) asking the field to share their thoughts about S2S as a part of our larger ARPA-ED strategy. We received over 60 comments in response to the RFI (thanks to all of you who responded). Building on these comments, IES is working on a Request for Applications (RFA) to launch the program. Here, I want to highlight some of the themes that were raised in response to the RFI. I also want to invite any further thoughts you might have.

Creating a three-stage funding strategy

IES's regular grant competitions usually make 5-year awards. While grantees are required to file annual program reviews, the assumption (and practice) is that funding will continue throughout the entire five-year period.

The S2S program will use a different approach. True, we envision that a full life cycle for an S2S project could last 6 years, but here's the critical difference—multiple year funding is not guaranteed. Instead, we envision three rounds of competitive and progressively larger grants, and winning in an early round does not guarantee funding in subsequent rounds.

The RFI describes the three rounds in some detail, but briefly: Round 1 will focus on the ideas, team, and market possibilities of a project. Demonstrating a successful prototype of the product and articulating a clear theory of change will be key performance indicators in Round 1. Round 2 will focus on developing a minimum viable product (MVP), user testing, and evidence of impact on education outcomes, as well as a continuing focus on marketability. Round 3 will focus on scaling. We envision a cohort model, where say a dozen grants are issued in year 1. For rounds 2 and 3, IES will select among grantees from previous rounds, based on how well the project is progressing. From our experiences with the SBIR program, we expect an attrition rate of around 40–50 percent between rounds.

Here are some of the issues raised in responses to the RFI:

How much time do we need for Stage 1's initial prototype development and testing?

We envision Stage 1 to be around one year. The question that follows is obvious: what can reasonably be accomplished in this time? IES is not expecting a fully developed model with empirical evidence of effectiveness or marketability in that window. Rather, we are working on a more reasonable (but demanding) set of accomplishments that focus on idea validation (including updating the theory of change proposed at the beginning of the project), evidence of a functioning team, and initial feasibility testing.

What will successful partnerships look like?

In our transformative research program, we require broader partnerships than what we have historically asked for in research to practice partnerships. Rather than just researchers and SEAs or LEAs, we require these partnerships to include a product development partner. Because of the type of work the S2S program is requiring, interdisciplinary teams and complex partnerships will be essential. These partnerships will need time to build and gel—and we will monitor their progress and accomplishments in S2S as well as in the transformative research program. Other funding agencies are working on how best to support expanded partnerships. For example, NSF's Convergence Accelerator program calls for funded teams to go through a design thinking and team building curriculum before they can move to Phase 2. We will continue to look for the best strategies to support more complex partnerships.

Does the timeline allow for meaningful change?

A few responses to the RFI asked whether the S2S timelines allow for development of solutions that aim for systems change. IES will continue to think about S2S program design and how that intersects with the kinds of change we can support through this program, but systems change, while important, is not central to the mission of S2S. Several other comments noted that the outcomes S2S hopes to improve may take a long time to develop. We agree that IES will need to identify both short-term and intermediate outcomes against which project progress can be evaluated.

How will marketability and effectiveness be weighed in assessing a project?

There are many examples of widely used products that are not effective and many examples of effective products that fail in the market. The challenge for S2S is to incentivize and support both product marketability and effectiveness throughout all phases of the work. This will be challenging at first as product developers and researchers learn to combine their expertise and approaches in ways that reflect a balanced focus on research and development. We expect that the lessons we are learning from working with SRI and the LEARN network will help us identify skills needed for bringing effective products to market and scaling them.

What about implementation?

IES has long recognized that implementation challenges may derail even the most promising programs. The S2S program must lean even more heavily on the study of implementation.

How can the field benefit from projects that do not make it through subsequent rounds?

Many responses to the RFI noted the need for a cultural shift in IES that would embrace failure—and, even more importantly, a culture in which learning from failure is central. Indeed, we need to learn and communicate what did not work; knowing what has already been tried and discarded is critical to breakthroughs. If teams keep details of their approach and research confidential, it will limit the ability of S2S projects to learn from each other.

Will there be a role for other funders?

Several responses to the RFI noted the importance of partnering with philanthropies to help bring products to market and provide needed financial and technical support. Indeed, IES has recently built strong relationships with several foundations to bring innovative ideas to fruition. We will continue down that road.

What is required to ensure the success of this approach?

For S2S to successfully support more risk-informed, high-reward programs, IES will need to build its own capacity by developing more flexible funding mechanisms and by hiring entrepreneurial program managers. But IES will also need to help build the field's capacity to engage in more entrepreneurial activities. I am optimistic—in part because I am reminded of just how far the field has developed its capacity to undertake rigorous cost analysis. When IES first introduced the requirement that certain types of projects include cost analysis, the results were not pretty. In the first year, IES did not review somewhere around two-thirds of grant applications that required a cost analysis because they were non-responsive to the requirement. At the same time, IES began to provide technical assistance to help researchers master those skills. Now, plans for rigorous cost analysis are getting better all the time—and only a small number of proposals are deemed non-responsive. We will extend this approach to help build capacity for risk-informed, high-reward education research.

As we continue to move down the path for NCADE and S2S, IES will continue to support the field to develop better and more effective projects that help us identify what works for whom under what circumstances.

What an exciting time!

As always, please feel free to contact me: mark.schneider@ed.gov