Inside IES Research

Career and Technical Education (CTE) has been evolving and expanding at a rapid pace in recent years as industry and education leaders focus on students’ readiness for college and careers. While some studies have shown positive effects of CTE on students, the evidence base is thin. To learn more about the research needs of the CTE field, the National Center for Special Education Research (NCSER) and the National Center for Education Research (NCER) at the Institute of Education Sciences (IES) convened a group of experts in policy, practice and research related to CTE.  The discussion held by the Technical Working Group (TWG) led both NCER and NCSER to increase their investments in CTE for fiscal year 2019: NCSER included a CTE special topic, and NCER changed its CTE topic from a special topic to a standing topic. Applications to both are due August 23, 2018.  Both research centers hope to fund more studies that will help us better understand this growing aspect of education.

The TWG focused on the following four questions:

1. Who is served by CTE and who is left behind? From national CTE statistics, we know that 82% of all public high schools offer CTE. And, 85% of students earn at least one credit in CTE with the average high school student earning 2.5 CTE credits. However, TWG members noted that research is lacking on specific subpopulations in CTE, such as students from various demographic backgrounds and students with disabilities. Disaggregated data on these dimensions are needed to better understand the CTE experiences of the range of students being served. Such data may help educators improve equity of access to high quality programs for all students.
2. What do we know―and need to know―about CTE policies, programs, and practices at the secondary and postsecondary levels?  TWG experts discussed the need to know more about industry-recognized credentials and about business and industry engagement in CTE at the secondary level. They argued that we do not know if credentials align with industry requirements, nor do we understand the impact of different types of credentials on student outcomes and wage trajectories. TWG members also noted that the higher the perceived quality or prestige of the CTE program, the more exclusive it becomes, and the more difficult it is for disadvantaged students to obtain access. TWG members also expressed concerns about CTE teacher training, particularly for experts who are recruited from industry without prior teacher preparation. As the experts discussed postsecondary CTE, they suggested that the field would be best served by framing the conversation about secondary to postsecondary pathways as a continuum that enables transparent and sequential transitions from secondary to 2-year and then to 4-year programs or to training or employment, with guidance for students to understand possible sequences.
3. What are the critical methodological issues in CTE?  TWG members noted that, with a few notable exceptions (e.g.,  a 2008  MDRC study on career academies in New York and a recent study of CTE high schools in Massachusetts), few causal studies on CTE have been conducted. There is an urgent need for more high quality, causal research on CTE policies and programs. In addition, the experts noted that there is almost no research on students with disabilities in CTE. TWG members concluded that the field needs to re-conceptualize CTE research – including better defining CTE students, instructors, programs, and measures – and identify the critical research questions in order to encourage more research in this field.
4. What is needed to advance CTE research?  State CTE administrators want to know how to identify quality CTE programs so they know how to spend their dollars most effectively on programs that best meet the needs of students. Policymakers also want to know what “works” and what the benefits are of such investments. The TWG members encouraged studies that examine the educational benefits of particular instructional approaches. They also highlighted the importance of collaborative cross-institutional and cross-agency efforts to advance CTE research.

Readers are invited to read the summary of the TWG discussion.

By Corinne Alfeld (NCER program officer) and Kimberley Sprague (former NCSER program officer)

On Wednesday June 27, 2018, the Smithsonian’s National Air and Space Museum held Space Innovation Day, an event to celebrate space exploration, STEM education, and students as makers. The event was co-developed by the museum and Future Engineers, a technology firm that is a current awardee of the U.S. Department of Education and Institute of Education Sciences’ Small Business Innovation Research Program (ED/IES SBIR). 

In the morning, the event featured a live conversation (called a “downlink”) between NASA astronaut Serena Auñón-Chancellor on the International Space Station and Washington, D.C.-area students at the museum. After a brief introduction of Auñón-Chancellor as she floated around in the space station, students asked her a series of questions such as “What it is like to experience space?” and “What does it take to be an astronaut?”

The morning also included on-stage interviews with three students who won the Future Engineers Two For the Crew Challenge. Through this national competition, sponsored by the ASME Foundation with technical assistance from NASA, K-12 students submitted a digital design of an astronaut tool intended to be manufactured on the International Space Station using a 3-D Printer. This tool allows innovative solutions to be provided to the astronauts immediately and means that NASA does not need to ship tools into space. One of the student winners designed “2 Pliers + 1 Handle,” a set of tool parts including needle-nose and lineman’s pliers with attachable handles. The 3-D printed multi-purpose tool can be customized into many different configurations when in space.

The challenge competition was run through a web-based platform that Future Engineers is developing with the support of a 2017 award from ED/IES SBIR.  The platform provides an online hub for students to create and submit solutions to innovation design challenges. Future Engineers is planning to launch the school version of their platform in the 2018-19 school year, with the goal of bringing many different kinds of maker design challenges to classrooms around the country across many areas of STEM for grades K to 12.

The afternoon of the event featured hands-on exhibits with educational opportunities for hundreds of students and museum attendees, including a 3-D design makerspace by Future Engineers, an augmented reality solar system experience by the Space Foundation, and a virtual reality space station experience by NASA.

We look forward to more maker design challenge events in the future!

Edward Metz is a program officer at the Institute of Education Sciences.

About ED/IES SBIR

The U.S. Department of Education’s Small Business Innovation Research program, administered by the Institute of Education Sciences (IES), funds projects to develop education technology products designed to support students, teachers, or administrators in general or special education. The program emphasizes rigorous and relevant research to inform iterative development and to evaluate whether fully-developed products show promise for leading to the intended outcomes. The program also focuses on commercialization once the award period ends so that products can reach students and teachers and be sustained over time. ED/IES SBIR-supported products are currently used in thousands of schools around the country.

Funding research that makes it into classrooms and impacts the lives of students is an important goal for IES. One of the ways in which the Institute strives to make sure the work it funds is useful to education practitioners is through the funding of Researcher-Practitioner Partnerships (RPPs), which are grants to support researchers and education agencies working together to answer questions of high priority for the education agency. IES began funding these RPP projects in 2013 with the goal of supporting partnership development, initial research on problems of high priority for the education agency, and increasing the education agency’s capacity to use and understand research.

Because the RPP program is relatively new and little is known about the best way to frame and support these partnerships, IES funded the National Center for Research in Policy and Practice (NCRPP; an IES-funded national research and development center on knowledge utilization) to conduct a supplemental study to look at how researcher and practitioner partners form and maintain their partnerships, and strategies they use to increase research use by practitioners at the education agency. NCRPP also wanted to understand more about the IES RPP grant program in terms of funding amount, time frame, and Institute leadership and monitoring.

NCRPP just released its final report on the findings from the RPP study. In two waves of data collection, NCRPP distributed surveys and conducted interviews with key researchers and practitioners involved in 27 of the 28 RPPs funded by IES between 2013 and 2015. Findings from the report show that partnerships reported progress on goals related to developing findings that apply to other organizations and improving students’ socio-emotional outcomes. Additionally, researchers and practitioners both reported valuing their participation in the partnership work. Finally, when compared to a national sample of school and district leaders, practitioners who participate in an IES-funded RPP are much more likely to name journal articles as useful pieces of research and to indicate they integrate research processes into their own work.

You can read a summary about the findings on the NCRPP website here, or download the full interim report here.

Becky McGill-Wilkinson, NCER Program Officer for the Knowledge Utilization Research and Development Centers

The IES Goal Structure was created to support a continuum of education research that divides the research process into stages for both theoretical and practical purposes. Individually, the five goals – Exploration (Goal 1), Development and Innovation (Goal 2), Efficacy and Replication (Goal 3), Effectiveness (Goal 4), and Measurement (Goal 5) – were intended to help focus the work of researchers, while collectively they were intended to cover the range of activities needed to build evidence-based solutions to the most pressing education problems in our nation. Implicit in the goal structure is the idea that over time, researchers will identify possible strategies to improve student outcomes (Goal 1), develop and pilot-test interventions (Goal 2), and evaluate the effects of interventions with increasing rigor (Goals 3 and 4).

Over the years, IES has received many applications and funded a large number of projects under Goals 1-3.  In contrast, IES has received relatively few applications and awarded only a small number of grants under Goal 4. To find out why – and to see if there were steps IES could take to move more intervention studies through the evaluation pipeline – IES hosted a Technical Working Group (TWG) meeting in 2016 to hear views from experts on what should come after an efficacy study (see the relevant summary and blog post). IES also issued a request for public comment on this question in July 2017 (see summary).

The feedback we received was wide-ranging, but there was general agreement that IES could do more to encourage high-quality replications of interventions that show prior evidence of efficacy. One recommendation was to place more emphasis on understanding “what works for whom” under various conditions.  Another comment was that IES could provide support for a continuum of replication studies.  In particular, some commenters felt that the requirements in Goal 4 to use an independent evaluator and to carry out an evaluation under routine conditions may not be practical or feasible in all cases, and may discourage some researchers from going beyond Goal 3.   

In response to this feedback, IES revised its FY 2019 RFAs for Education Research Grants (84.305A) and Special Education Research Grants (84.324A) to make clear its interest in building more and better evidence on the efficacy and effectiveness of interventions. Among the major changes are the following:

• Starting in FY 2019, Goal 3 will continue to support initial efficacy evaluations of interventions that have not been rigorously tested before, in addition to follow-up and retrospective studies.
• Goal 4 will now support all replication studies of interventions that show prior evidence of efficacy, including but not limited to effectiveness studies.
• The maximum amount of funding that may be requested under Goal 4 is higher to support more in-depth work on implementation and analysis of factors that moderate or mediate program effects.

The table below summarizes the major changes. We strongly encourage potential applicants to carefully read the RFAs (Education Research, 84.305A and Special Education Research, 84.324A) for more details and guidance, and to contact the relevant program officers with questions (contact information is in the RFA).

Applications are due August 23, 2018 by 4:30:00 pm Washington DC time.

By Thomas Brock (NCER Commissioner) and Joan McLaughlin (NCSER Commissioner)

In recent years, thousands of schools around the country have used technologies developed through the Small Business Innovation Research program at the U.S. Department of Education’s Institute of Education Sciences (ED/IES SBIR). The program emphasizes a rapid research and development (R&D) process, with rigorous research informing iterative development and evaluating the promise of products for improving the intended outcomes. ED/IES SBIR also focuses on the commercialization after development is complete so that products can reach schools and be sustained over time.

This month, IES announced 21 new awards for 2018. Of these, 15 are Phase I projects to develop and test a prototype, and six are Phase II projects to fully develop and evaluate an education technology product for students, teachers, or administrators to use in classrooms and schools. A playlist of videos from the Phase II projects is available below. 

Many of the new projects continue trends that have emerged across the portfolio in recent years, including creating learning games and dashboards that present data to inform learning and instruction in core subjects like reading and math. Several other awards are for projects that promote learning in new areas, such as computer science and Career and Technical Education.

Trend #1: Learning Games

Games are increasingly being used to engage students in learning by presenting content in new ways. For the eighth straight year, many new ED/IES SBIR awardees will be developing game-based learning products.

• Phase II awardee Electric Funstuff and Phase I awardee Schell Games are developing virtual reality (VR) games to immerse students in history in 360-degree environments, and Phase I awardee Gigantic Mechanic is developing a role-playing game on civic discourse facilitated by tablet-based computers.

• With Phase I funding, Fablevision and Sirius Thinking are creating interventions that employ game mechanics to improve reading.
• Several project teams are embedding storylines within learning games, including Phase II awardee MidSchoolMath for algebra, and Phase I awardees Codespark for computer science and Immersed Games for ecosystems science.
• The 3C Institute is creating a game-based assessment of early grade science learning.

Trend #2: Dashboards for Students, Teachers and Administrators

Modern technologies provide the opportunity to organize and present data in real-time to students, teachers, and administrators to inform learning and decision-making. Several new awards are developing data dashboards.

• Phase II awardees are fully developing dashboards across several areas. LiveSchool will generate reports on students’ behavior across classes with a recommendation engine for administrators and teachers to address challenges, StoryWorld will provide teachers of English Learners insights into students’ language acquisition, and Simbulus and Myriad Sensors will present real-time information to enrich classroom discussions on math and science topics.
• Phase I projects by Appendis and Graspable are creating adaptive learning technologies which include teacher dashboards to present results on student performance to guide instruction. VidCode is creating a dashboard for teachers to monitor student progress in learning coding. Education Modified is developing a dashboard to provide special education teachers guidance on student IEPs, or individual educational plans.

New Areas of Focus

Along with continuing to support projects in the areas above, a series of Phase I 2018 awards are focusing on areas new to ED/IES SBIR.  VidCode, CodeSpark, and Zyante are focusing on computer science learning and Core Learning is seeking to build capacity in Career and Technical Education (CTE). Language Learning Partners is developing an automated avatar tutor to support English Learners through conversation. And in special education, Attainment Company is developing an app for supporting student writing.

Stay tuned for updates on Twitter and Facebook as IES continues to support innovative forms of technology.

Written by Edward Metz, Program Manager, ED/IES SBIR