Inside IES Research

Notes from NCER & NCSER

An Example of the Unquantifiable Effect of Research on Practice

At IES, we continue to think about ways to positively impact education practice through research. It is relatively straightforward to count and share the publications and research outputs produced by our grants. A bigger challenge is measuring the impact IES-funded research has on implementing evidence-based practice after the research project is complete. So we were thrilled when we received the following letter from Patrice Bain—a middle school teacher, author, education specialist, speaker, and consultant—who has worked closely with IES for many years.

I used to think of government agencies as impersonal bureaucracies often hidden from the public eye. One agency, IES, not only proved me wrong, it positively changed my life.

In 2006, Drs. Henry Roediger III and Mark McDaniel from Washington University in St. Louis obtained a grant from IES to research how students learn in an authentic classroom. The classroom where this research began was mine. And this is where the life-changing impact began.

The IES grant paid for technology to be used in my school’s classrooms and research assistants to aid our teachers. Heading up the research at my school was Pooja Agarwal, and this began a collaboration lasting over a decade.

In 2007, IES invited me to be the sole K-12 educator to co-author a practice guide. The large organization, to me, now had a face: Elizabeth Albro, who warmly welcomed me. I clearly recall sitting at a large table in Washington, DC, surrounded by my cognitive science superheroes: Drs. Hal Pashler, Mark McDaniel, Brian Bottge, Art Graesser, Janet Metcalfe, and Ken Koedinger. Each talked about important research that would impact learning in classrooms, and I knew my newly-expanded teaching repertoire now would be based in the science of learning. The final result of our meetings became the highly cited practice guide Organizing Instruction and Study to Improve Student Learning. In addition, information from this guide was featured on the website Doing What Works.

As Pooja and I delved into how retrieval, spacing, and metacognition played a role in student learning at my school, I was contacted by REL Mid-Atlantic, a part of IES that offers research-based professional development in Delaware, Washington, DC, New Jersey, Pennsylvania, and Maryland. Touting the benefits of teaching using the seven recommendations in Organizing Instruction and Study, I gave professional development presentations in the Mid-Atlantic regions with Drs. Hal Pashler, Ken Koedinger, and Nate Kornell.

Pooja and I also gave several presentations that included the research happening in my classroom. With IES funding, that research became a multi-year project involving over 1500 middle and high school students. With the passing of each year and research on learning becoming more defined, I was able to develop strategies utilizing retrieval, spacing, and metacognition. Pooja and I continued our collaboration. I was seeing success in the eyes of my students: I wasn’t just teaching content, I was teaching them how to learn.

A wealth of information on the science of learning seemed to be making a mark. Yet learning myths—those based on anecdotes and fads—were still circulating. To combat this, IES and NCER invited me to be on a working task group to tackle Neuromyths vs. Neurotruths. Once again, as I sat around a table in Washington, DC with learning superheroes, we explored how to begin to dispel prevalent myths of learning.

Because of IES and the opportunities I was given, I wanted to shout from a mountaintop that we can transform teaching. I’ve seen it. I’ve done it in my classroom. I realized a book started to brew within me. I’m not sure how the decision occurred, but I knew my collaboration with Pooja Agarwal was worthy of documenting. And so it began. We wanted to write a practical, evidence- and research-based book. Books had been written by cognitive scientists; books had been written by teachers. However, our book would be the first written by a cognitive scientist and an educator.

Powerful Teaching: Unleash the Science of Learning was released in June 2019. The ideas have resonated with educators across the globe. We are transforming education.

And it all started with IES approving a grant.

 

IES Celebrates Computer Science Education Week and Prepares for the 2020 ED Games Expo at the Kennedy Center

This week is Computer Science Education Week! The annual event encourages students from Kindergarten to Grade 12 to explore coding, with a focus on increasing representation among girls, women, and minorities. The event honors the life of computer scientist Grace Hopper, who broke the mold in the 1940s as a programming pioneer. Coding and computer science events are occurring in schools and communities around the country to celebrate the week.

This week is also a great time to highlight the computer science and engineering projects that are coming to Washington, DC for the 2020 ED Games Expo at the Kennedy Center on the evening of January 9, 2020. Developed with the support of the Institute of Education Sciences and other federal government offices, the projects provide different types of opportunities for students to learn and practice computer science and engineering skills with an eye toward examining complex real-world problems.

At the Expo, expect to explore the projects listed below.

  1. In CodeSpark Academy’s Story Mode, children learn the ABCs of computer science with a word-free approach by programming characters called The Foos to create their own interactive stories. In development with a 2019 ED/IES SBIR award.
  2. In VidCode, students manipulate digital media assets such as photos, audio, and graphics to create special effects in videos to learn about the coding. A teacher dashboard is being developed through a 2019 ED/IES SBIR award.
  3. Future Engineers uses its platform to conduct STEM challenges for Kindergarten to Grade 12 students. Developed with a 2017 ED/IES SBIR award.
  4. Fab@School Maker Studio allows students to design and build geometric constructions, pop-ups, and working machines using low-cost materials and tools from scissors to inexpensive 3-D printers and laser cutters. Developed with initial funding in 2010 by ED/IES SBIR.
  5. In DESCARTES, students use engineering design and then create 3-D print prototypes of boats, gliders, and other machines. Developed through a 2017 ED/IES SBIR award.
  6. In Ghost School, students learn programming and software development skills by creating games. In development with a 2018 Education Innovation and Research grant at ED.
  7. In Tami’s Tower, children practice basic engineering to help Tami, a golden lion tamarin, reach fruit on an overhanging branch by building a tower with blocks of geometric shapes. Developed by the Smithsonian Institution.
  8. In the Wright’s First Flight, students learn the basics of engineering a plane through hands-on and online activities, then get a firsthand look at what it looked (and felt) like to fly it through a virtual reality simulation. Developed by the Smithsonian Institution.
  9. In EDISON, students solve  engineering problems with gamified design software and simulate designs in virtual and augmented reality. In development with support from the National Science Foundation. 
  10. May’s Journey is a narrative puzzle game world where players use beginning programming skills to solve puzzles and help May find her friend and discover what is happening to her world. Developed with support from the National Science Foundation. 
  11. In FLEET, students engineer ships for a variety of naval missions, test their designs, gather data, and compete in nationwide naval engineering challenges. Developed with support from the U.S. Navy’s Office of Naval Research.
  12. Muzzy Lane Author is a platform for authoring learning games and simulations without requiring any programming skills. Developed in part with a Department of Defense award.

About the ED Games Expo: The ED Games Expo is the Institute’s and the Department of Education's annual public showcase and celebration of educational learning games as well as innovative forms of learning technologies for children and students in education and special education. At the Expo, attendees walk around the Terrace Level Galleries at the Kennedy Center to discover and demo more than 150 learning games and technologies, while meeting face-to-face with the developers. The Expo is free and open to the public. Attendees must RSVP online to gain entry. For more information, please email Edward.Metz@ed.gov.

Edward Metz is the program manager for the ED/IES Small Business Innovation Research program.

Christina Chhin is the program officer for the Science, Technology, Engineering, and Mathematics (STEM) Education research program.

Measuring Social and Emotional Learning in Schools

Social and emotional learning (SEL) has been embraced by many schools and districts around the country. Yet in the rush to adopt SEL practices and support student SEL competencies, educators often lack assessment tools that are valid, reliable, and easy to use.

 

Washoe County School District in Nevada has moved the needle on SEL assessment with support from an IES Researcher-Practitioner Partnership grant. The district partnered with the Collaborative for Academic, Social, and Emotional Learning (CASEL) to develop the Social and Emotional Competency Assessments (WCSD-SECAs)—free, open-source instruments that schools can use to measure SEL competencies of students in 5th through 12th grade.

Long and short versions of the SECA are available to download from the school district’s website, along with a bank of 138 items across 8 SEL domains that schools around the country can use to modify SECA assessments for their local context. The long-form version has been validated and aligned to the CASEL 5 SEL competency clusters and WCSD SEL standards (self-awareness, self-management, social awareness, relationship skills, and responsible decision making). The assessment is also available in Spanish, and the Metro Nashville Public schools offer the assessment in 8 additional languages.  

Students complete the long-form SECA as part of Washoe’s Annual Student Climate Survey by rating how easy or difficult SEL skills are for them. Under the Social Awareness domain, students respond to items such as “Knowing what people may be feeling by the look on their face” or “Learning from people with different opinions than me.” Under the Responsible Decision Making domain, students rate themselves on skills such as “Saying ‘no’ to a friend who wants to break the rules” and “Thinking of different ways to solve a problem.”

The SECA is one component of Washoe County’s larger School Climate Survey Project that is marking its 10th anniversary this year. Washoe provides district-level and school-level reports on school climate to support the district’s commitment to providing safe, caring, and engaging school environments for all of Washoe’s students and families.  

Written by Emily Doolittle, NCER’s Team Lead for Social Behavioral Research

IES Honors Dominic Gibson as Outstanding Predoctoral Fellow

Each year, IES recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. The 2018 winner, Dr. Dominic Gibson completed his Ph.D. in Developmental Psychology at the University of Chicago. He is currently a Postdoctoral Researcher at the University of Washington where he specializes in understanding how children learn words and mathematical concepts. In this blog, Dominic discusses his research and his experience as an IES fellow.  

What inspired you to focus your research on early mathematics?

So many everyday activities as well as many of humanity’s greatest achievements rely on math. Simple math becomes so second nature to us that it is often difficult for older students to conceptualize what it would be like to not have a basic understanding of numbers. But children take months and often years to learn the meanings of just the first few number words (one, two, three) and to learn how the counting procedure really works. Children’s acquisition of other math terms (angle, proportion, unit of measurement) is similarly marked by misconceptions and slow, difficult learning.  

Overcoming these learning challenges relies on an interesting mixture of uniquely human abilities (like language) and skills we share with other animals. Moreover, children’s ability to master early math concepts predicts their future academic success. Therefore, by studying how children learn about math, we can better understand the sources of humanity’s unique achievements and apply this knowledge to reducing early achievement gaps and maximizing our potential.

Based on your research, what advice would you give parents of pre-kindergartners on how to help their children develop math skills?

My biggest piece of advice is to talk to children about numbers and other basic math concepts. Children benefit from abundant language input in general, and “math talk” is no different. Even simply talking about different numbers of things seems to be particularly important for acquiring early math concepts. Numbers can be easily incorporated into a variety of activities, like taking a walk (“let’s count the birds we see”) or going to the grocery store (“how many oranges should we buy?”). Likewise, good jumping off points for using other types of early math talk such as relational language are activities like puzzles (“this one is too curvy to fit here—we need to find a piece with a flat edge”) and block building (“can you put this small block on top of the bigger one?”).

It also may be useful to note that even when a child can say a word, they may not fully understand what it means. For instance, two- to four-year-old children can often recite a portion of the count list (for example, the numbers one through ten) but if you ask them to find a certain number of items (“can you give me three blocks?”) they may struggle when asked for sets greater than two or three. Therefore, in addition to counting, it is important to connect number words to specific quantities (“look there are three ducks”). It may be especially helpful to connect counting to the value of a set (“let’s count the ducks—one, two, three—there are three!”).

My last piece of advice is to be careful about the types of messages we send our children about math. Many people experience “math anxiety,” and if we are not careful, children can pick up on these signals and become anxious about math themselves or internalize negative stereotypes about the types of people who are and are not good at math. Ensuring that children feel empowered to excel in math is an important ingredient for their success.

How has being an IES predoctoral fellow helped your development as a researcher?

The diverse group of people and perspectives I encountered as an IES predoctoral fellow made a huge impact on my development as a researcher. As an IES predoctoral fellow pursuing a degree in psychology, I met many students and faculty members who were interested in the same questions that interest me but who approached these questions from a variety of other disciplines, such as economics, public policy, and sociology. I also connected with networks of educators and policymakers outside of academia who alerted me to important issues that I may have missed if I had only worked within my own discipline. Through these experiences, I gained new tools for conducting my research and learned to avoid the types of blind spots that often develop when approaching a problem from a single perspective. In particular, I gained an appreciation for the challenges of translating basic science to educational practice and the number of interesting research questions that emerge when attempting to do this work.

Compiled by Katina Rae Stapleton, Education Research Analyst and Program Officer for the Predoctoral Interdisciplinary Research Training Programs in the Education Sciences, National Center for Education Research

What Do State CTE Directors Want to Learn from the Research Community?

Career Technical Education (CTE) is gaining widespread interest and support from state policymakers, who see it as a strategy to expand access to opportunity and meet employer needs. Between 2014 and 2018, states enacted roughly 800 policies related to CTE, and in 2019, workforce development was one of the top education-related priorities mentioned by governors in their state-of-the-state addresses.

What’s more, in 2018 Congress passed the Strengthening Career and Technical Education for the 21st Century Act (Perkins V), which reauthorized the federal law for CTE and invests around $1.2 billion a year to strengthen and expand CTE programs. The law was enacted in July 2019 and will be in full effect in July 2020 after states submit their four-year plans for CTE to the U.S. Department of Education (see more about the Perkins V planning process here).

With CTE in the spotlight, State CTE Directors are working hard to improve quality and equity in CTE. But state CTE offices often do not have the staffing or resources to conduct rigorous program evaluations to learn what’s working and what needs improvement. By partnering with CTE researchers, State Directors can gain critical insights into the impact of CTE programs, policies, and practices.

While the design, governance and delivery of CTE varies from state to state, there are several common questions and challenges across the country that CTE researchers can help address, particularly in light of Perkins V implementation:

Improving program quality: State leaders are working to improve CTE program quality by connecting secondary and postsecondary coursework, integrating academic and technical learning, aligning programs with labor market needs and expectations, and preparing learners to earn industry-recognized credentials of value. Tennessee, for example, recently revised its secondary CTE program standards and developed model CTE programs of study that meet statewide workforce needs. Answers to the following research questions would help fuel these efforts:

  • What set of experiences at the secondary and postsecondary levels (CTE coursework, work-based learning, dual enrollment, etc.) best prepares learners for postsecondary enrollment and completion, certificate and degree attainment, and high-wage employment?
  • Do these vary by region of the country, Career Cluster® or program of study?
  • Does the delivery mechanism (comprehensive high schools, career academies, area technical centers, technical colleges) matter?

Ensuring equitable access and success in CTE: To reverse historical inequities in CTE, state leaders are using data to identify disparities and ensure each learner can access, fully participate in, and successfully complete a high-quality CTE program of study. In Rhode Island, the Department of Education repurposed $1.2 million in state funds to launch an Innovation & Equity grant initiative, which provided resources to local recipients to recruit and support underrepresented student populations in high-quality programs. CTE researchers can help these efforts by addressing the following questions:

  • What are the classroom and workplace conditions in which CTE students of color are most likely to develop the interests, knowledge, and skills that prepare them to earn postsecondary credentials of value and obtain high-wage employment in their careers of choice?
  • What interventions, accommodations, and instructional strategies best prepare learners with disabilities to transition successfully into the workforce?
  • How does gender inform the development of occupational identity, and what can educators do to limit the effects of stereotyping on the career aspirations of learners?

Improving the quality and use of CTE data: Most State Directors believe improving and enhancing their CTE data systems is a priority, but only 45 percent say they have the information they need at both the secondary and postsecondary levels to improve program quality. States like Minnesota (through the State Colleges and University System) are working to improve the validity and reliability of their data by collaborating with industry-recognized credential providers to obtain data for their students. CTE researchers can help state leaders improve data quality in two ways:

  • Identifying relevant data sources and matching student records to allow for a comprehensive examination of student pathways and outcomes
  • Developing and sharing guidance for collecting, validating, and matching student data relevant to CTE

Fostering collaboration and alignment across state agencies: Supporting learner success requires cross-agency collaboration and coordination. State leaders are working to create seamless pathways by sharing data, coordinating program design, and braiding resources to achieve economies of scale. One example is Massachusetts, where Governor Charlie Baker established a cross-agency workforce skills cabinet to coordinate education, workforce, housing, and economic development. The following research questions would help accelerate the work in Massachusetts and other states:

  • Do states with policies that foster cross-agency coordination see better education and employment outcomes for students? Can merging datasets across agencies help states better understand and respond to student needs?
  • Does credit for prior learning and/or credit transfer between institutions decrease time to credential attainment and entry into employment?
  • How does the integration of support services—such as financial aid, Medicaid, Temporary Assistance for Needy Families, and other state and federal programs—impact the likelihood of student success?

Expanding career advisement opportunities: School counselors are the most trusted source of information on CTE and career options, and states are working to bolster their career advisement systems by reducing the counselor-to-student ratio, requiring each student to complete an individualized graduation plan, and developing user-friendly platforms for career exploration. In Oklahoma, for example, it is now policy for all students to identify their career and academic goals through the state’s new Individual Career and Academic Planning program. CTE researchers can help address the following questions:  

  • Do career and academic planning programs increase the likelihood that learners will complete CTE programs of study, graduate from high school, and earn postsecondary credentials?
  • How does early career exposure through job shadowing, career fairs and career counseling inform student course taking, academic achievement, and future employment and earnings?

As states chart a vision and path for the future of CTE, they can and should use their data to inform decisions. Researchers can help them collect and analyze high quality data to understand the relationships between CTE program elements and various learner outcomes. This can help them understand what is and isn’t working with current policy and practice and identify how to focus their efforts to improve quality and equity in CTE. In addition, researchers can help state directors plan and conduct rigorous evaluations as they roll out new CTE policies and programs. Over the next few months, Advance CTE and the Institute of Education Sciences (IES) will feature a series of successful partnerships between states and CTE researchers and explore how those projects provided critical data and insights to inform state policy.

This blog series was co-authored by Corinne Alfeld at IES (corinne.alfeld@ed.gov) and Austin Estes from Advance CTE (aestes@careertech.org). IES began funding research grants in CTE in 2017 and established a CTE Research Network in 2018. IES hopes to encourage more research on CTE in the coming years in order to increase the evidence base and guide program and policy decisions. At the same time, Advance CTE has been providing resources to help states improve their CTE data quality and use data more effectively to improve CTE program quality and equity.