Inside IES Research

This blog post featuring advice from IES-funded Hispanic Serving Institutions on developing research training programs, is part of an ongoing series featuring IES training programs as well as our blog series on diversity, equity, inclusion, and accessibility (DEIA) within IES grant programs

In 2015, IES launched the Pathways to the Education Sciences Research Training Program to encourage undergraduate, postbaccalaureate, and master’s students from diverse backgrounds to pursue careers in education research. The Pathways program grants were made to minority-serving institutions (MSIs) and their partners to provide one year of mentored research training. We asked the leadership teams from our six initial Pathways Programs to share their lessons learned on establishing research training programs. In part one of this blog, we share the lessons learned from the Pathways programs based at Hispanic Serving Institutions (HSIs). In part two, we share lessons learned from the Pathways programs based at historically Black colleges and universities (HBCUs) and their partner institutions.

Pathways: Successful Transitions to and Through Higher Education

California State University, Sacramento (HSI, AANAPISI)

Leadership: Jana Noel, Timothy Fong, Jacqueline Brooks, Erica Zamora

We have five areas for universities to consider that wish to develop undergraduate training programs:

Create an interdisciplinary training team. Draw on the strengths of the wide range of researchers on your campus and beyond. An interdisciplinary team provides an expanded range of perspectives on both the research methods and the questions that are important to pursue within education. All are important and valuable to the expansion of research that will make a difference to the lives of underrepresented students.

Develop partnerships. Develop partnerships across departments and colleges as well as in the community. Our apprenticeship sites span the university and into the community and include university research centers and institutes, K-12 offices of education, and non-profit public policy centers. Apprenticeship partnerships provide fellows with the opportunity to be part of a team that actively conducts research into pressing educational issues and contributes to the research needed to make practice and policy decisions within your region and state.

Intentionally match mentors to fellows. As much as possible, match the fellows’ diversity when selecting research mentors. Mentors provide support on learning new research methodology, asking new questions, working as a team, preparing to present research at conferences, and preparing for graduate school.

Choose a broad research theme. Choose a broad research theme that will appeal to a wide range of students. In our case, we study the barriers and supports for underrepresented students in K-12, community college, and higher education. This allows underrepresented students at MSIs to know that they are welcome in the program and that their experiences and voices will be valued.

Provide continuity across cohorts. Fellows in our program speak at recruiting events for future cohorts, participate in panel discussions for future cohorts, and truly serve as our best source of encouragement for future fellows. The continuity persists during and beyond the program as fellows engage in their academic journeys together.

AWARDSS Training Program

University of Arizona (HSI)/College of Applied Science and Technology at the University of Arizona

Leadership: Michelle Perfect, Brandy Perkl, Sara Chavarria, Andrew Huerta

Our number one piece of advice for establishing undergraduate research training programs is to add in bridges over the biggest barriers to URM participation.

Our Pathways program (AWARDSS) was built on the idea that (1) support from campus programs and (2) intentional mentoring are vital aspects of promoting participation in research from traditionally underrepresented students. For that reason, we have learned that undergraduate research training programs within MSIs need to build on what is already present. Add in the elements you know your students need most, such as financial support, increased access to resources, and focus on improvement of specific skills.

To achieve this in our practice, we built a complementary, hybrid, add-on program to the University of Arizona’s well-established and award-winning Undergraduate Research Opportunities Consortium (UROC) experiences. UROC provides the primary coursework and faculty, while we deliver the add-ons that allow for underrepresented minority (URM) student participation. We focused initially on providing additional funding for our students’ experiences. Then, we added a required inclusion-oriented mentor training to bolster the intentionality of those relationships and the quality of this potentially transformative relationship. This often allows us to support underrepresented mentors, as well. Mentoring does not occur in a vacuum though, and the latest research shows that those with a developmental network outperform those without one. Thus, we staffed the program specifically to serve as a supportive developmental network for our students. Finally, we assessed and trained students in academic areas of need (for example, statistics) at both the cohort and individual levels. 

We also suggest that leaders of undergraduate research training programs continuously examine their practices and adjust their models accordingly. We plan to further train our staff in more inclusive and anti-racist practices ensuring that the entire AWARDSS network is informed, intentional, and engaged in supportive practices from day one. 

Pathways Program 

University of Texas at San Antonio (UTSA; HSI)

Leadership: Guadalupe Carmona, Ann Marie Ryan, Francesca Bronder

Our goals for the program are to 1) broaden participation of undergraduates from underrepresented backgrounds in doctoral study, and 2) develop a pipeline of talented interdisciplinary researchers who bring fresh ideas, approaches, and perspectives to addressing the challenges of inequalities that exist in P-20 educational experiences, transitions, and outcomes. 

Through a structured program design, undergraduates can be exposed to research at an early academic stage and discover that through academic and scientific research, they can achieve their passion to systematically improve education and transform their local communities. By learning how research is conducted, closely working with faculty mentors, finding their own research focus, and developing their work, our UTSA Pathways Fellows have gained in academic and personal development, self-confidence, a sense of accomplishment and peer support, independence of work and thought, and have become more academically resilient. For many of them, UTSA Pathways has opened doors and facilitated access to several graduate schools. For others, it has helped them apply their newly acquired research skills to a variety of professional fields and become more marketable in their chosen careers.

We identified three central concepts for UTSA Pathways that we think would be helpful for others who are developing undergraduate research training programs: Empowerment, Transformation, and Inspiration.

• Empowerment. Once unheard student voices are now becoming part of our education research community as fellows actively participate in academic and research activities. Your program should empower students to form their own identities as fellows and help to extend this empowerment to their personal lives.
• Transformation. Our Pathways program has generated change in multiple communities, built new collaborations, recruited new faculty, and obtained supporters devoted to Pathways and its goals of broadening participation of historically underrepresented voices. We suggest that your program identify the critical partners and potential levers of change specific to your program’s model and goals.
• Inspiration. We have found that our fellows’ resilience and commitment has been channeled through their active engagement and dissemination of their research that, for most, begins with UTSA Pathways. And our mentors’ passion and generosity has guided and supported a new generation of scholars in educational research. We encourage you to create an environment of hospitality and engagement that will embrace a passionate group of young scholars to participate in their communities of research and practice.

Produced by Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council. She is also the program officer for the Pathways to the Education Sciences Research Training Program and the new Early Career Mentoring Program for Faculty at Minority Serving Institutions, the two IES training programs for minority serving institutions.  

Formally, I am known as Dr. Lorena Aceves, but you can just call me Lorena. I am a first generation (first in my family to graduate high school and college) Latina scholar. I recently completed my PhD in human development and family studies (HDFS) at the Pennsylvania State University. Currently, I am working as a Society for Research Child Development Federal Postdoctoral Policy Fellow at the Administration for Children and Families’ Office of Head Start. In my current role, I work on all things related to Head Start from issues facing the Head Start workforce to considering the impact that COVID-19 has had on the daily lives of Head Start children. Given that September 15 to October 15 is National Hispanic Heritage Month, I wanted to share my camino (path) with the hopes of inspiring other Latinx students and to demonstrate the beauty of Hispanic excellence in education!

How It Started

My journey into higher education began before I could even remember. My amazing parents are two immigrants from Michoacán, Mexico. Their journey of migration was fueled by the desire to open more doors of opportunity for themselves and their future family. They settled in southern California, where they would start building the foundation of values and motivation that would lead me to my PhD.

In California, my parents had their first interaction with opportunity and education for their daughter with Head Start. As a four-year-old, Head Start gave me a step up in my educational career and connected my parents to resources and services that could increase our familial wellbeing. The journey here was just getting started when my parents decided that it was time to leave southern California for Arizona. In Arizona, my parents were able to earn better wages and purchase a home in a good school district. This opportunity was not as accessible to them in California. This move, as my mom always says, “(as difficult as it was) was the best thing we could have done educationally for you and your younger brother.”

I started my elementary education in the Gilbert Public Schools school district. It was when I got to high school that a major educational opportunity opened for me—the founding of Gilbert Classical Academy (GCA), a public college prep school. GCA was a saving grace for this little Latina who had every aspiration to go to college but had no clue where to even begin! It gave me all the tools and preparation I needed to make it to college, and I did. I was admitted to 80% of the colleges I applied to and continued my higher education at the University of Arizona (UA). 

Moving Away and Embracing My Latina Identity

After high school graduation, I moved two hours south of home to attend the UA. This move was a BIG deal for this eldest Mexican daughter. My parents were not happy about me living in a dorm, but they knew it was necessary to achieve that “American Dream” that we always talked about where I would never have to scrub toilets as my mother had done most of her life.

Life at UA was amazing. I got to embrace my Latina identity because, for once in my life, I was finally surrounded by people who looked like me and had the same familial experiences. I also got to explore all my potential career options. I started college wanting to be a pediatrician but ended up finding my passion in the HDFS major. I loved the idea of studying human beings, especially in the context of their families. I finally was going to be able to understand my family and culture from a scholarly lens.

In my third year of college, I was unsure about what I wanted to do post-graduation. I knew I wanted to pursue graduate education; I just didn’t know for what. And that’s when I stumbled upon research, which ultimately led me to my PhD. Before this moment, I had no idea what a PhD was, but I was sold on pursuing one because being able to use research to support Latinx youth and families seemed like a dream come true.

In my last year of college, I participated in a program to prepare undergraduate students for doctoral studies. The program set me up for success in graduate school. With the program’s help, I was able to gain social capital and academic skills I did not have as a first-generation student, which are critical for successfully pursuing a graduate degree. With the help of the program’s staff and training, I was admitted to 8 out of the 10 doctoral programs that I applied to! I ultimately decided to pursue my PhD at Penn State. This little Arizona girl had no idea what she had signed up for by leaving sunny warm weather for cold, gloomy central Pennsylvania. At Penn State, my research focused on examining the cultural, familial, and individual level factors that contribute to Latinx youth’s academic outcomes.

I started graduate school with the goal of becoming a professor to continue this kind of research and helping other Latinx students like me attain their PhDs. That dream quickly evolved after a few internships at the U.S. Department of Education, including with IES, as well as the White House Initiative on Advancing Educational Equity, Excellence, and Economic Opportunity for Hispanics (formerly known as the White House Hispanic Prosperity Initiative). It was thanks to IES that I was even able to pursue these internships. As a graduate student, I was an IES predoctoral fellow, which afforded me opportunities to do the research I was interested in, as well as pursue these non-traditional graduate experiences. Through these internships, I quickly learned that I could do more. I learned that there is great need for Latinx scholars like myself in federal spaces, where decisions about funding and policies are happening. I became passionate about federal service, which led me to my current postdoctoral position.

Moving Forward

My main goal with my newly blossoming career as a doctora is to be able to work for a federal agency where I can use my skills and training to serve diverse communities, particularly communities of color. Federal leadership is still not reflective of the communities that make up the United States. I hope to serve in a federal leadership position in the future to represent the communities of color and make our leadership more reflective of its citizens. I plan to give voice to Latinx children, youth, and families of this country that need to be heard. Juntos podemos (Together, we can)!

This year, Inside IES Research is publishing a series of interviews (see here and here) showcasing a diverse group of IES-funded education researchers and fellows that are making significant contributions to education research, policy, and practice. As part of our Hispanic Heritage Month blog series, we are focusing on Hispanic researchers and fellows, as well as researchers that focus on the education of Hispanic students.

Lorena Aceves (Lorena.Aceves@acf.hhs.gov) is a Society for Research Child Development Federal Postdoctoral Policy Fellow at the Administration for Children and Families’ Office of Head Start.

This guest blog was produced by Caroline Ebanks (Caroline.Ebanks@ed.gov), Program Officer, National Center for Education Research.

This blog is part of an ongoing series featuring IES training programs as well as our blog series on diversity, equity, inclusion, and accessibility (DEIA) within IES grant programs. For more information, see this DEIA update from Commissioners Elizabeth Albro (National Center for Education Research) and Joan McLaughlin (National Center for Special Education Research).

In 2015, IES launched the Pathways to the Education Sciences Research Training Program to broaden participation in education research. Pathways grants are awarded to minority serving institutions and their partners to provide up to year-long training fellowships to undergraduate, post-baccalaureate, and masters students. Each Pathways program has a specific education theme such as literacy, equity/social justice in education, student success, and education pipelines. Pathways fellows receive an introduction to scientific research methods and their program’s education theme, as well as meaningful opportunities to participate in education research, professional development, and mentoring. Currently, there are seven funded Pathways programs; IES recently launched the newest program focused on learning analytics and data science to the University of California, Irvine. Over 250 students have participated in Pathways, and many (39 at last count) have already started doctoral programs. In honor of HBCU week (September 7-10), Hispanic-Serving Institutions Week (September 13-19), and Hispanic Heritage Month (September 15- October 15), we reached out to six Pathways alumni who are in graduate school to ask them for advice for other students who wish to pursue graduate study related to education research. Here is what they shared with us.

Comfort Abode

RISE Training Program, University of Maryland, College Park/Bowie State University (HBCU)

Doctoral Student, Indiana University

My number one piece of advice for students who want to become education researchers would be to keep in mind the purpose of your research. If nobody understands it, it is not helpful. And in order for people to understand it, you yourself need to understand it. You cannot teach what you do not know. Especially considering that the research is in education, the goal should be to educate teachers, students, faculty, or whomever, about what is being studied and (hopefully) steps that can be taken towards improving that area. You have to keep your audience in mind and while it should not be “dumbed down,” you have to make sure that your point is getting across clearly. In order for that to happen, you have to know what you are talking about. Project RISE was especially helpful in the fact that there were a lot of mentors and people willing to help you understand the scope of the research as well as provide comments and feedback on areas to improve upon.

Jeremy Flood

RISE Training Program, North Carolina Central University (HBCU)/University of North Carolina Wilmington/Pennsylvania State University

Doctoral Student, North Carolina Agricultural and Technical State University

My only advice would be to remember the mission of solving challenges in education. Within the body of education research, there are several ways one can accomplish this—whether it is by policy research, grounded theory, ethnography, or experiments, there are quite a diversity of tools available at a researcher’s disposal, so much so that it may seem overwhelming at first.  Do not stress if you find this true; you are not the first or the last to feel overwhelmed! Instead, use this as an opportunity to rededicate yourself to the mission and allow your dedication to choose a research path that is best for you. Whichever one, two, or three (or more) that you choose, make sure that the end goal seeks to improve the practice of education.

Jessala Grijalva

AWARDSS Training Program, University of Arizona (HSI)/College of Applied Science and Technology at the University of Arizona

Doctoral Student, University of Notre Dame

I advise Pathways fellows to take the time to reflect and internalize the cultural competency components of the program. The Pathways program will not only prepare you with the hard and soft skills that you need to be a successful researcher, but also help you become an all-around culturally competent researcher. Sometimes, we assume that as students of color or students from diverse backgrounds that we are inherently culturally competent; yet, there is so much more to learn and to be aware of. From my experience as a participant in the Pathways program, I’ve learned of ways to extend cultural competency beyond research and into my interactions with other researchers, colleagues, mentors/mentees, and the broader community. To be an effective researcher, it’s not only important to conduct culturally component research, but to also work with people of all walks of life, and to be able to disseminate our research and findings to the public. Training in cultural competency is very rare and very valuable–and something we may not fully appreciate—so take advantage of this opportunity and make cultural competency an important priority in your conduct as a researcher.

Camille Lewis

PURPOSE Program, Florida State University/Florida Agricultural and Mechanical University (HBCU)

Doctoral Student, Florida State University

There is an African proverb that states: “Knowledge is like a garden. If it is not cultivated, it cannot be harvested.” On the quest to become an education researcher, it is easy to get caught up in the hype of being “the expert.”  My #1 piece of advice to anyone who is interested in education research is to remain a student of life. Your journey to becoming an education researcher will be filled with many opportunities to learn, adapt, and understand the process of learning. Embrace these experiences; allow your researcher identity to be shaped and influenced by new discoveries and new interests. Continue to seek new information and allow your knowledge base to be cultivated. My experience as a public-school teacher, PURPOSE fellow, and doctoral student has shown me the importance and necessity of continually seeking advice, experiences, knowledge, and professional development related to learning and education. This pursuit of knowledge has informed and shaped not just my research, but my life outside academia as well. I never allow myself to become a “know it all.” This keeps me humble and allows me to continue to make improvements in every facet of my life.  

Christopher Terrazas, MA

Pathways Program, University of Texas at San Antonio (UTSA; HSI)

Doctoral Student, University of Texas at Austin

UTSA Pathways was instrumental in developing my identity as a researcher and graduate student. The other day, I described my experiences as being in a rocket, and Pathways provided the fuel to take off and get one step closer to my goals as a researcher. During my time, I made it a priority to be curious, always. I did this by attending all seminars offered and asking questions—even questions that I thought were not the right ones to ask at the time. You never know who may share a similar experience or perhaps a differing one to support you in your endeavors. Be bold and use your voice as an instrument to understand the world of research and graduate school during this exciting journey. It is crucial to get into this mindset because this will be your experience, perhaps your first. You will want to make sure that you are well prepared for this process as an aspiring researcher and scholar because this is your future. With that said, my number one piece of advice is to look inward to reflect on your own life experiences. Use these thoughts to feed your inner sense of self because you know more than anyone what you want for your future to be. 

Erica Zamora

Pathways Program, California State University, Sacramento (HSI)

Doctoral Student, University of Arizona

The Pathway Fellows Program had a tremendous impact on my growth as a scholar and education researcher. My advice to students is to engage in research that not only reflect their scholarly interests but also reflect their values as community members and educators. My experience in the program gave me a deeper understanding of the importance of social justice and equity work in research. Education has the potential to transform communities and encourage growth and development while perpetuating various forms of oppression. Engaging in education research that centers the voices of and the issues that historically marginalized groups experience could lead to transformative outcomes at postsecondary institutions.

Written by Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council. She is also the program officer for the Pathways to the Education Sciences Research Training Program and the new Early Career Mentoring Program for Faculty at Minority Serving Institutions, the two IES training programs for minority serving institutions, including Alaska Native and Native Hawaiian-Serving Institutions, American Indian Tribally Controlled Colleges and Universities, Asian American and Native American Pacific Islander-Serving Institutions (AANAPISI), Hispanic-Serving Institutions (HSIs), Historically Black Colleges and Universities (HBCUs), Predominantly Black Institutions, Native American-Serving, Nontribal Institutions, and any other minority-serving institution as specified in request for applications.

As we begin a new school year, NCER and NCSER wanted to share with our community some of the work we have been doing—and are seeking to do more of—in relationship to diversity, equity, inclusion, and accessibility (DEIA). We plan to provide occasional updates via this blog to share progress and keep the conversations going.  

Actions on Diversity

At the end of 2020, IES convened a Technical Working Group (TWG) to get feedback on ways that the research centers could improve our investments focused on DEIA. Under the leadership of Drs. Katina Stapleton and Christina Chhin, we convened a stellar panel that participated in a robust conversation. That conversation and the recommendations from the panel are available in this summary document. We are already implementing some of the recommendations and wanted to share steps that we have taken and our plans for next steps to advance DEIA in IES-funded research.

1. One of the first steps that we took in response to the TWG recommendations was to take a close look at our Requests for Applications (RFAs), identify potential barriers to applicants from underrepresented groups, and revise and/or add language that more clearly articulated our commitment to DEIA, both in terms of those that conduct the research and in the populations studied. These changes were reflected in our FY 2022 RFAs, and we will continue to revise and improve our application materials.
2. IES has been committed to building expertise among a broad range of scholars in the education sciences for nearly two decades. The TWG noted, however, that there is a pressing need to provide funds for early career investigators who may be working at MSIs, teaching-intensive institutions, and/or at institutions with limited opportunities for research mentorship. In response, IES launched an Early Career Mentoring for Faculty at MSIs research program. This new program extends our FY 2016 training investment in MSIs that we recompeted in FY 2021: the Pathways to the Education Sciences Training program. This program is designed to encourage undergraduate, post-baccalaureate, and masters-level students in groups that are historically underrepresented in doctoral education to pursue graduate study relevant to education research. Currently, there are seven IES-funded Pathways training programs in the United States, hosted by minority serving institutions (MSIs) and their partners. We are excited to see who applied in this first round of the Early Career Mentoring program and anticipate investing in this program in FY 2023 and beyond.  
3. The TWG also recommended that IES intentionally reach out to the MSI community to ensure that they know about the opportunities available at IES. We held our first such event since the TWG on September 7, 2021, where IES hosted a virtual listening session at HBCU week. More than 250 scholars attended that session and provided valuable feedback on barriers to HBCU-based researchers applying for research funding from IES. We are in the process of scheduling additional listening sessions with other communities of researchers to provide more opportunities for input from diverse stakeholders and underrepresented groups.
4. The TWG also recommended that IES take a deeper look at the demographic and institutional data of applicants to our grants programs to identify which groups of researchers and institutions are underrepresented. Data indicate that the percentage of applications received from MSIs between 2013 and 2020 was very small—4% of applications to NCER and 1% to NCSER. Of those applications that were funded, 10% of NCER’s awards were made to MSIs and none of NCSER’s awards were made to MSIs. IES reviewed the demographic information FY 2021 NCER and NCSER grant applicants and awardees voluntarily submitted, and among those who reported their demographic information, we found the following:
   • Gender (response rate of approximately 82%) - The majority of the principal investigators that applied for (62%) and received funding (59%) from IES identified as female.
   • Race (response rate of approximately 75%) - The majority of principal investigators that applied for (78%) and received funding (88%) from IES identified as White, while 22% of applicants and 13% of awardees identified as non-White or multi-racial.
   • Ethnicity (response rate of approximately 72%) - The majority of principal investigators that applied for (95%) and received funding (97%) identified as non-Hispanic.
   • Disability (response rate of approximately 70%) - The majority of principal investigators that applied for (97%) and received funding (96%) identified as not having a disability.

These data underscore the need for IES to continue to broaden and diversify the education research pipeline, including institutions and researchers, and better support the needs of underrepresented researchers in the education community. However, tracking our progress has proven to be a challenge. Responding to the demographic survey was voluntary so a significant number of applicants chose not to respond to particular questions. We strongly encourage all our grant applicants to respond to the demographic survey so that we will be better able to track our progress in improving diversity in our grant programs.

Addressing Misconceptions that Limit Diversity in IES Applicants

TWG panel members and attendees at the HBCU session highlighted a series of misconceptions that the education sciences community holds about the funding process at IES and recommended that IES identify communication strategies to address these misconceptions. IES hears that message loud and clear and wants to address at least a few of those misconceptions here.

Myth: IES only funds randomized controlled trials, limiting the range of researchers and institutions that can be competitive for IES grants.

Reality: IES funds a range of research, including measurement work, exploratory research, intervention development and testing, and efficacy and replication studies. We also fund a wide range of methods, including various experimental and quasi-experimental designs and mixed methods that combine quantitative and qualitative methods.

Myth: IES doesn’t support course buyout or summer salary.

Reality: IES supports grant personnel time to carry out research related activities. This can include course buyout and summer salary. Principal investigators on grants coordinate their budget planning with sponsored projects officers to ensure that their budgets comply with institutional guidelines as well as federal guidelines.

Myth: IES program officers are too busy to help novice applicants.

Reality: Because IES program officers are not involved in the peer review of applications, they can provide in-depth technical assistance and advice throughout the application process. They can even review drafts of proposals prior to submission! IES program officers can be your best resource in helping you submit a competitive grant proposal.

If you’d like to learn more about DEIA at IES, please see our Diversity Statement. You can also subscribe to our Newsflash and follow us on Twitter (@IESResearch) for announcements of future listening sessions. Please send any feedback or suggestions to NCER.Commissioner@ed.gov (National Center for Education Research) or NCSER.Commissioner@ed.gov (National Center for Special Education Research). Also, watch this blog over the next few months to read about the wide range of IES grantees and fellows from diverse backgrounds and career paths. Next up is our Hispanic Heritage Month (Sept. 15-Oct. 15, 2021) blog series.

Christina Chhin (Christina.Chhin@ed.gov), Katina Stapleton (Katina.Stapleton@ed.gov), and Katie Taylor (Katherine.Taylor@ed.gov) assisted Commissioners Albro and McLaughlin in writing this blog.

In recent decades, there has been an emphasis on quantitative, causal research in education policy. These methods are best suited for answering questions about the effects of a policy and whether it achieved its intended outcomes. While the question of, “Did it work?” remains critical, there is a need for research that also asks, “Why did it work? For whom? In what contexts?” To answer these types of questions, researchers must incorporate rigorous qualitative methods into their quantitative studies. Education research organizations like the Association for Education Finance and Policy have explicitly invited proposals using qualitative and mixed methodologies in an effort to elevate research addressing a range of critical education policy questions. Funding organizations, including IES, encourage applicants to incorporate qualitative methods into their research process. In this guest blog, Amy Cummings, Craig De Voto, and Katharine Strunk discuss how they are using qualitative methods in their evaluation of a state education policy.

In our IES-funded study, we use qualitative, survey, and administrative data to understand the implementation and impact of Michigan’s early literacy law—the Read by Grade Three Law. Like policies passed in 19 other states, the Read by Grade Three Law aims to improve K-3 student literacy skills and mandates retention for those who do not meet a predetermined benchmark on the state’s third-grade English language arts assessment. Although the retention component of these policies remain controversial, similar laws are under consideration in several other states, including Alaska, Kentucky, and New Mexico. Below are some of the ways that we have integrated qualitative methods in our evaluation study to better understand the policy process in the development of the Read by Grade Three Law.  

Collecting qualitative sources helped us understand how the policy came to be, thereby assisting in the structure of our data collection for examining the law’s implementation and subsequent effects. In our first working paper stemming from this study, we interviewed 24 state-level stakeholders (policymakers, state department of education officials, early literacy leaders) involved in the development of the law and coded state policy documents related to early literacy to assess the similarity between Michigan’s policy and those of other states. Understanding the various components of the Law and how they ended up in the policy led us to ensure that we asked educators about their perceptions and implementation of these components in surveys that are also part of our evaluation. For example, because our interviews made clear the extent to which the inclusion of the retention component of the Law was controversial during its development, we included questions in the survey to assess educators’ perceptions and intended implementation of this component of the Law. In addition, it confirmed the importance of our plan to use state administrative retention and assessment data to evaluate the effect of retention on student literacy outcomes.

To trace the Read by Grade Three Law’s conception, development, and passage, we analyzed these qualitative data using two theories of the policy process: Multiple Streams Framework (MSF) and policy transfer. MSF says that policy issues emerge on government agendas through three streams: problem, policy, and political. When these streams join, a policy window is opened during which there is a greater opportunity for passing legislation. Meanwhile, policy transfer highlights how policies enacted in one place are often used in the development of policies in another.

We found that events in the problem and political streams created conditions ripe for the passage of an early literacy policy in Michigan:

• A national sentiment around improving early literacy, including a retention-based third-grade literacy policy model that had been deemed successful in Florida
• A pressing problem took shape, as evidenced by the state’s consistently below average fourth-grade reading scores on the National Assessment of Educational Progress
• A court case addressing persistently low-test scores in a Detroit-area district
• Previous attempts by the state to improve early literacy

As a result of these events, policy entrepreneurs—those willing to invest resources to get their preferred policy passed—took advantage of political conditions in the state and worked with policymakers to advance a retention-based third-grade literacy policy model. The figure below illustrates interviewee accounts of the Read by Grade Three Law’s development. Our policy document analysis further reveals that Michigan’s and Florida’s policies are very similar, only diverging on nine of the 50 elements on which we coded.

Although this study focuses on the development and passage of Michigan’s early literacy law, our findings highlight both practical and theoretical elements of the policy process that can be useful to researchers and policymakers. To this end, we show how particular conditions, coupled by policy entrepreneurs, spurred Michigan’s consideration of such a policy. It is conceivable that many state education policies beyond early literacy have taken shape under similar circumstances: a national sentiment combined with influential brokers outside government. In this way, our mixed-methods study provides a practical model of what elements might manifest to enact policy change more broadly.

From a theoretical standpoint, this research also extends our understanding of the policy process by showing that MSF and the theory of policy transfer can work together. We learned that policy entrepreneurs can play a vital role in transferring policy from one place to another by capitalizing on conditions in a target location and coming with a specific policy proposal at the ready.

There is, of course, more to be learned about the intersection between different theories of the policy process, as well as how external organizations as opposed to individuals operate as policy entrepreneurs. As the number of education advocacy organizations continues to grow and these groups become increasingly active in shaping policy, this will be an exciting avenue for researchers to continue to explore.

This study is just one example of how qualitative research can be used in education policy research and shows how engaging in such work can be both practically and theoretically valuable. The most comprehensive evaluations will use different methodologies in concert with one another to understand education policies, because ultimately, how policies are conceptualized and developed has important implications for their effectiveness.

Amy Cummings is an education policy PhD student and graduate research assistant at the Education Policy Innovation Collaborative (EPIC) at Michigan State University (MSU).

Craig De Voto is a visiting research assistant professor in the Learning Sciences Research Institute at the University of Illinois at Chicago and an EPIC affiliated researcher.

Katharine O. Strunk is the faculty director of EPIC, the Clifford E. Erickson Distinguished Chair in Education, and a professor of education policy and by courtesy economics at MSU.