IES Blog

By Corinne Alfeld and Meredith Larson, NCER Program Officers

Since IES was founded more than a dozen years ago, it has built a reputation for funding rigorous research to measure the causal effects of education policies and programs.  While this commitment remains solid, we also recognize the value of well-designed qualitative research that deepens understanding of program implementation and other educational processes and that generates new questions or hypotheses for study. In this blog post, we highlight the outcomes from a recent meeting we hosted focused on the use of mixed methods – that is, studies that combine qualitative and quantitative methods – and share some of the ways in which our grantees and other researchers incorporate mixed methods into their research.

On May 29, 2015, 10 researchers with experience designing and conducting mixed methods research met with staff from the two IES research centers in a technical working group (TWG) meeting. The TWG members shared their experiences carrying out mixed methods projects and discussed what types of technical assistance and resources we could provide to support the integration of high-quality mixed methods into education research. There was consensus among the TWG members that qualitative data is valuable, enriches quantitative data, and provides insight that cannot be gained from quantitative research alone.  Participants described how mixed methods in currently used in education research, proposed potential NCER and NCSER guidance and training activities to support the use of high-quality mixed methods, and offered suggestions for researchers and the field. Below are just a few examples that were shared during the meeting:

• Dr. Carolyn Heinrich and colleagues used a longitudinal mixed method study design to evaluate the efficacy of supplemental education services provided to low-income students under No Child Left Behind. One of the critical findings of the study was that there was substantial variation across school districts in what activities were included in an hour of supplemental instruction, including (in some cases) many non-instructional activities.  This was revealed as the team examined the interview data describing what activities lay behind the shared metric of an hour of instructional time.  Having that level of information provided the team with critical insights as they examined the site-by-site variation in efficacy of supplemental education services.  Dr. Heinrich emphasized the need for flexibility in research design because the factors affecting the impact of an intervention are not always apparent in the design phase. In addition, she reminded the group that while statistical models provide an average impact score, there is valuable information included in the range of observed impacts, and that that variability is often best understood with information collected using in-depth field research approaches.

• Dr. Mario Small used mixed methods research to examine social networks in childcare centers in New York City. Using observational methods, he discovered that variations in the level of networking among mothers depended on the individual child care center, not the neighborhood. He hypothesized that child care centers that had the strictest rules around pick-up and drop-off, as well as more opportunities for parent involvement (such as field trips), would have the strongest social networks. In such settings, parents tend to be at the child care center at the same time and, thus, have more interaction with each other. Dr. Small tested the hypotheses using analysis of survey and social network data and found that those who developed a social network through their child care center had higher well-being than those who did not. He concluded from this experience that without the initial observations, he would not have known that something small, like pick-up and drop-off policies, could have a big effect on behavior.

• Dr. Jill Hamm described a difficult lesson learned about mixed methods “after the fact” in her study, which was funded through our National Research Center on Rural Education Support. In planning to launch an intervention to be delivered to sixth-grade teachers to help adolescents adjust to middle school, she and her colleagues worked with their school partners to plan for possible challenges in implementation. However, because some of the qualitative data collected in these conversations were not part of the original research study – and, thus, not approved by her Institutional Review Board – the important information they gathered could not be officially reported in publications of the study’s findings. Dr. Hamm encouraged researchers to plan to use qualitative methods to complement quantitative findings at the proposal stage to maximize the information that can be collected and integrated during the course of the project.

• In a study conducted by Dr. Tom Weisner and his colleagues, researchers conducted interviews with families of children with disabilities to determine the level of “hassle” they faced on a daily basis and their perceptions of sustainability of their family’s routines. Findings from these interviews were just as good at predicting family well-being as parental reports of coping or stress on questionnaires. The findings from the analysis of both the qualitative and quantitative data collected for this study enhanced researchers’ understanding of the impact of a child’s disability on family life more than either method could have alone. Dr. Weisner observed that the ultimate rationale of mixed methods research should be to gather information that could not have been revealed without such an approach. Because “the world is not linear, additive, or decontextualized,” he suggested that the default option should always be to use mixed methods and that researchers should be required to provide a rationale for why they had not done so, where feasible.

Curious to learn more about what was discussed? Additional information is available in the meeting summary.

Comments? Questions? Please email us at IESResearch@ed.gov.

Featuring Jennifer Ledford, Vanderbilt University

By Liz Berke, NCSER Intern and Kristen Rhoads, NCSER Program Officer

Welcome to our second blog post featuring the Principal Investigators of the inaugural NCSER Early Career Development and Mentoring program grants.  This week we are excited to feature the work of Dr. Jennifer Ledford from Vanderbilt University and former Special Education teacher from Georgia.

Dr. Ledford is being mentored by Dr. Joseph Wehby (Vanderbilt University), Dr. David Gast (University of Georgia) and Dr. Kevin Ayres (University of Georgia).  In her IES-funded project, she is further developing and testing a small-group intervention designed to improve the academic and social skills of children with autism.  Dr. Ledford is using single case designs to study whether the intervention improves child outcomes and teachers can effectively implement it. 

We had the chance to sit down with Dr. Ledford and ask her about the challenges she faces as well as get advice from her for others like her who are early in their research careers.

What are some of the biggest challenges that you face as a young researcher? How do you hope this award will help you overcome those challenges?

Early career research in education is hard for numerous reasons—not yet having established relationships with teachers and schools, relative inexperience with balancing research with other tasks (i.e., teaching, advising, service), and, of course, lack of funding. The early career award actually helps in all of these areas. It is much easier to establish relationships with schools when you are an early career researcher if you have a well-considered and funded series of studies and if you’ve aligned yourself well with more advanced researchers. In addition, the funding potentially allows you to reduce time spent on teaching and other activities, so that you have additional time to contribute to research efforts. Funding student support has been especially crucial in running my complex single case studies that require considerable personnel resources. Finally, the mentorship and training associated with grant have provided a flexible but structured framework for improving my ability to conduct high-quality research.

What advice would you give to young researchers?

I’m not sure I feel ready to give advice to fellow early career investigators! I think taking advantage of the knowledge of senior researchers has been key for me—both in my official mentee role and just in the day-to-day conduct of research outside of this grant.  

What is your favorite aspect of working with your mentors?

It is great to have a structured and focused mentoring program—it makes it easy to forge a relationship and to continue working with your mentor over time. Without this structure, I think it may have been easy to let the mentoring take a back seat to other responsibilities. It’s great to have an excuse to meet with and learn from experienced and invested leaders in the field.

What made you decide to apply to for the early career development and mentoring award? Is there anything you wish you had known before you applied?

When I read the RFA for the new competition, I think my first thought was probably something like “I might actually be competitive for this grant!” The training and mentoring components and competition with other early career investigators makes it a less daunting prospect.  While I was applying, I wish I had realized and taken advantage of the potential value of the Program Officer during the application process and the tremendous benefit of asking for input from colleagues. 

Comments? Questions? Please write to us at IESresearch@ed.gov.

By Liz Albro, NCER Associate Commissioner of Teaching and Learning

Summer Conference Season

Many IES-funded researchers have been sharing the findings of their studies at academic conferences this past month.  Want to learn more? Lists of presentations describing IES-funded research at the Society for Text & Discourse and Society for the Scientific Study of Reading annual meetings are available on our conferences page.

A Busy Month for IES Research in the News

Have you visited our IES Research in the News page lately? It’s a great way to learn more about IES-funded research.  Not only can you read more about the new awards that have been recently made, you can learn about findings from recent studies. We do our best to keep up, but if we’re missing something, send us a note at IESResearch@ed.gov.

More Recognition for ED/IES SBIR Products

ED/IES SBIR supported games by Triad Interactive Media (PlatinuMath) and Electric Funstuff won Gold at the Serious Play Conference.  And ED/IES SBIR awardee Fluidity Software won 1st Place in the “Best Performing Office Add-On” category, for their FluidMath app, which teachers and students use to create dynamic math and physics formulas.

Summer Research Training Institute on Cluster-Randomized Trials in Education Sciences

Congratulations to the 29 participants who completed the ninth Summer Research Training Institute on cluster-randomized trials (CRTs) in education sciences!

The purpose of this training is to prepare current education researchers to plan, design, conduct, and interpret cluster-randomized trials. A tenth Institute will be held in summer 2016, so be sure to follow us on Twitter or subscribe to the IES Newsflash to get application information as soon as it is available. 

Please send any questions or comments to IESResearch@ed.gov.

By Sammi Plourde, NCSER Intern; Kristen Rhoads, NCSER Program Officer; and Becky McGill-Wilkinson, NCER Program Officer

IES-funded research by Dr. Compton and his colleagues was recently awarded the International Literacy Association’s (ILA) Albert J. Harris Award!  ILA is an advocacy organization that publishes current research on literacy and provides resources for practitioners, students, and leaders involved in facilitating literacy development across the world.  The Albert J. Harris Award is given annually to a recently published journal article or monograph that contributes to better understanding of prevention or measurement of learning disabilities or reading disabilities.

The winning article by Jennifer K. Gilbert, Donald L. Compton, Douglas Fuchs, Lynn S. Fuchs, Bobette Bouton, Laura A. Barquero, and Eunsoo Cho entitled “Efficacy of a First-Grade Responsiveness-to-Intervention Prevention Model for Struggling Readers,” features findings from a NCSER-funded measurement study focused on identifying and intervening with struggling readers as early as first grade.  The article describes effects of intensive intervention within a multi-tiered prevention model. Struggling readers who were randomly assigned to receive an intensive, small-group intervention had better reading gains compared to students who received classroom instruction as usual. However, some students continued to struggle despite receiving the intensive intervention.  Those students were then randomly assigned to receive the intensive intervention in a one-on-one format or to continue in a small-group format. Results indicated that no differences in performance existed between the two formats.  They also found that more than half of the students who participated in the intervention failed to achieve average reading scores by the end of third grade.  These findings suggest that students with persistent reading problems need intervention as early as possible that spans multiple years.  They also suggest that instruction for the students should be tailored to meet individual needs.  

Dr. Compton and his colleagues are continuing this research with IES.  They were funded by NCER to conduct a follow-up research study to identify characteristics of children who begin elementary school with typical reading development but are then later identified as having a reading disability. This work will provide information on how to guide instruction for students who have these characteristics.

Congratulations to Dr. Compton and his colleagues for making such an important contribution to identifying, preventing, and treating reading disabilities!

Questions? Comments? Please send them to IESResearch@ed.gov. 

By Liz Albro, NCER Associate Commissioner of Teaching and Learning

Welcome to our second “Month in Review” post! In addition to writing blogs, both NCER and NCSER have been busy making new awards this month, and preparing abstracts describing our newly funded projects published on our website. 

New Research Awards

Across the two research centers, IES awarded 148 new discretionary grants to support research and research training activities. I hope that you will take the time to dip into our abstracts describing the individual projects. From early childhood to postsecondary, from basic cognitive science to system-level analysis, from exploration to impact, the projects reflect the wide scope of education research questions that the IES research centers support. To learn more about the awards, click here to read about the new NCSER awards, and here for information about the new NCER awards. Be sure to check back on Monday, July 6^th, to read a new blog from Commissioner Brock discussing the 2015 awards and the forecast for 2016.

IES Funded Research in the News

Research findings from the Cognition and Student Learning portfolio were featured in two EdWeek articles in June. These articles describe some of the exciting work being done to address long-standing questions of transferring knowledge learned in one class or context to support new learning in mathematics  and science.

The ED/IES Small Business Innovation Research (SBIR) portfolio was featured in three different articles in June! Read more about how games developed with SBIR funding are being used to teach students about a wide variety of topics, like algebra, environmental science, and social skills.

IES Staff Presentations

On June 16-17, NCSER co-sponsored a Technical Working Group Meeting with the Department of Education’s Office of Special Education Programs (OSEP) and Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) on Evidence-Based and Emerging Practices: State of Science and Practice for Children with Disabilities.  The meeting was an important opportunity for leaders in the field of special education to share what has been learned across a number of pivotal areas in research and practice and also to identify some promising next steps. A synthesis of the meeting is underway and will be available later this year.

ED/IES SBIR program officer Ed Metz participated in the National SBIR Conference, and led a panel on games for learning.

Applying for IES Research Funding This Summer? Missed Our Webinars?

No problem. PowerPoint presentations and transcripts from the webinars led by our program officers are available on our website. Click here to access information about preparing grant applications for IES.