Inside IES Research

Notes from NCER & NCSER

Recognizing Our Outstanding Predoctoral Fellows

Each year, the Institute of Education Sciences (IES) recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. For the first time, IES has selected joint recipients for the 2015 award: Meghan McCormick and Eric Taylor. They will receive their awards and present their research at the annual IES Principal Investigators meeting in Washington, D.C. in December 2016.

Meghan completed her Ph.D. in Applied Psychology at New York University and wrote her dissertation on the efficacy of INSIGHTS, a social emotional learning intervention aimed at improving low-income urban students’ academic achievement. She is currently a research associate at MDRC. Eric completed his PhD in the Economics of Education from Stanford University and wrote his dissertation on the contributions of the quality and quantity of classroom instruction to student learning. Eric is currently an assistant professor at the Harvard University Graduate School of Education.

We asked Meghan and Eric how participating in an IES predoctoral training program helped their development as researchers.  For more information about the IES predoctoral training program, visit our website.

Meghan McCormick

Having the opportunity to be part of the IES Predoctoral Training Program helped me to develop a set of theoretical, quantitative, and practical skills that I would not have had the opportunity to develop otherwise. 

I was drawn to attend NYU (New York University) for my doctoral studies specifically because the school of education at NYU offered the (IES) predoctoral training program, in addition to hosting a core set of faculty with research interests very much aligned with my own. In my first year of graduate school, I quickly became aware that being a part of the IES program allowed me the freedom to study with an interdisciplinary set of scholars who could support multiple components of my training through a diverse set of experiences.

For example, in my work with Elise Cappella, Erin O’Connor, and Sandee McClowry, I was able to learn about the logistics of implementing a cluster-randomized trial across a broad set of schools, and conducting impact analyses to evaluate the efficacy of one social-emotional learning program called INSIGHTS. My experience working with Jim Kemple and Lori Nathanson at the Research Alliance for New York City schools showed me how to use research in a way that was responsive to the needs and goals of education policy makers. Quantitative coursework with Jennifer Hill and Sharon Weinberg helped me to apply rigorous quantitative methods to the data that were collected in schools, and to think concretely about the implications of research design for my future work. Coursework with developmental psychologists conducting policy-relevant research, such as Pamela Morris and Larry Aber, helped me to apply comprehensive theoretical framing when examining research questions of interest, and interpreting results. In addition, I have always been primarily interested in conducting interdisciplinary research that is responsive to policy and practice. My dissertation research grew out of my interest in learning about interdisciplinary methods for causal inference and applying them to research questions I had about how, for whom, and under what circumstances the social-emotional learning program I helped to evaluate effected outcomes for low-income students.

Most importantly, perhaps, having been part of the IES program’s collaborative and interdisciplinary community helped me to identify the type of research I wanted to do after finishing graduate school. Primarily, I knew that I wanted to conduct policy-relevant research, using the most rigorous quantitative methods available, with a team of researchers coming from different backgrounds. This realization led me to work at MDRC, where I have been working with JoAnn Hsueh and other colleagues to apply my skills from the predoctoral training program in new research design work that is responsive to critical policy questions in early education policy and practice right now. I feel prepared for this new work given the opportunities that the IES program afforded me across the last five years. 

Eric Taylor

I would emphasize two benefits. First, the IES program at Stanford helped me create and strengthen professional relationships with other education researchers and practitioners. Those relationships provided important opportunities to learn skills in ways that could not happen in the classroom but also complemented the excellent classroom instruction. The new relationships were diverse: other graduate students in different disciplines, Stanford faculty and faculty at other institutions, and, critically, practitioners and policy makers. For example, supported by my fellowship, I joined faculty at (the University of) Michigan and Columbia (University) working with the DC Public Schools to improve teacher applicant screening and hiring.

Second, those relationships combined with the financial support of the fellowship made it possible to work on new and timely research projects. During my time as an IES predoc, with collaborators at Brown, we started a researcher-practitioner partnership with colleges at the Tennessee Department of Education. The resulting work has taught me much about the day-to-day realities of school policy making and management, and how research can and cannot help. The partnership with Tennessee also grew into a five-year grant from IES, which began last year, to study state policy and teacher development through evaluation.

In short, I am certain my career is much further along today than it would have been without the IES predoc fellowship.

By Katina Stapleton, Education Research Analyst, National Center for Education Research 

 

How to Develop Your Career in Education Research: IES Training Opportunities

By Corinne Alfeld, NCER Program Officer

In honor of career development month, we would like to remind you about training opportunities funded by IES. We have invested in training programs since 2004 with the aim of increasing the supply of scientists and researchers in education who are prepared to conduct rigorous education research that advances knowledge within the field and addresses issues important to education policymakers and practitioners. These efforts are intended to lead both to the training of talented education researchers from a variety of backgrounds and to the incorporation of diverse ideas and perspectives in education research.

In this blog we describe five types of training opportunities currently offered through the National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER) that span from undergraduate to the postdoctoral level and beyond.

Training Opportunities for Current or Future Doctoral Students
Are you a current or aspiring doctoral student wondering what training opportunities are available to you? You may be interested in applying to one of 10 training programs funded by NCER’s Predoctoral Interdisciplinary Research Training Program in the Education Sciences that train predoctoral fellows in interdisciplinary programs involving a number of academic disciplines (e.g., economics, education, psychology, public policy, sociology, and statistics, among others). These fellowships can be from 2 to 5 years in length depending on the training program model and typically include tuition and benefits, a $30,000 stipend, and a small research/travel fund. Fellows who complete their training program have the skills necessary to produce research that is rigorous in method as well as relevant and accessible to education stakeholders such as practitioners and policymakers.

If you are interested in becoming a predoctoral fellow, you must apply directly to one of the training programs, not to IES.  Each of the 10 fellowship programs funded in 2014-15 has its own application process and acceptance criteria. For more information on becoming a predoctoral fellow, check out this resource on Applying for a Predoctoral Interdisciplinary Research Training Program Fellowship.

 

Postdoctoral Training Opportunities
Are you finishing up your doctorate and wondering how you can get more experience in education research? Or perhaps you’re looking to return to academia through a postdoctoral position? If so, you may want to apply to one of the our programs funded under the Postdoctoral Research Training Program in the Education Sciences Program (NCER) or the Postdoctoral Research Training Program in Special Education Program (NCSER). Through these two grant programs, IES funds training programs at doctoral-granting institutions to further prepare researchers who have obtained their Ph.D.s or Ed.D.s to become scholars capable of conducting high-quality, independent education or special education research. These postdoctoral training programs provide practical, hands-on experiences; enrichment of theoretical and empirical knowledge; and opportunities for fellows to build professional skills and networks that will support working with other researchers and relevant education research stakeholders.

To inquire about postdoctoral fellowship openings, follow the hyperlinks in this section to search for currently (awarded in 2010-15) funded programs at various universities around the country. For example, here are the 2015 NCER-funded programs and 2012 NCSER-funded programs.

 

Other Upcoming Training Opportunities:

  • Undergraduate, Post-baccalaureate, and Master’s Students. If you are an upper-level undergraduate student, recent graduate, and/or master’s student, especially from a group that is underrepresented in doctoral study (including racial and ethnic minorities, first-generation college students, economically disadvantaged students, veterans, and students with disabilities), you may be interested in the Pathways to the Education Sciences Research Training Program (Pathways). Established in 2015, the Pathways program funds training programs at minority-serving institutions (MSIs) and institutions of higher education that partner with MSIs. These training programs will provide fellows with education research experience and professional development to prepare them to pursue doctoral study in the education sciences or in fields relevant to education research.

Up to five Pathways training programs will be awarded to MSIs (and their partners) in 2016. These new programs will begin recruiting fellows in 2016 and 2017, so keep your eyes and ears open for more information about where and how to apply!

 

  • Early Career Education Researchers. Are you an early career researcher at your first appointment? If so, you may qualify for and be interested in one of our programs that target early career researchers in statistical and research methodology (NCER) and special education (NCSER).

Look for upcoming training opportunities for early career researchers in future Request for Applications for training grants (CFDA 84.305B and 84.324B) and statistical and research methodology grants (CFDA 84.305D).

 

  • Active Researchers Looking to Improve Their Methodological Expertise. Are you a current researcher (e.g., at a university or research firm) who would like to add tools to your methodological toolkit or further refine your skills with such tools? If so, then the Methods Training for Education Researchers Program may be for you.

If you are interested in methodological training, sign up for the IES Newsflash for announcements of upcoming workshops or periodically check our list of IES-funded workshops.

 

If you have questions about our training programs, please contact Corinne.Alfeld@ed.gov.

 

IES Honors Statistician Nathan VanHoudnos as Outstanding Predoctoral Fellow

By Phill Gagne and Katina Stapleton, NCER Program Officers

Each year, IES recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. The 2014 winner, Dr. Nathan VanHoudnos completed his Ph.D. at Carnegie Mellon University and wrote his dissertation on the efficacy of the Hedges Correction for unmodeled clustering. Nathan is currently a postdoctoral fellow at Northwestern University. In this blog, Nathan provides insights on becoming an education researcher and on research study design. 

How did you become interested in education research?

I was born into it. Before he retired, my father was the Director of Research for the Illinois Education Association. Additionally, my grandparents on my mother's side were both teachers. 

 

As a statistician, how do you explain the relevance of your research to education practitioners and policy-makers?

I appeal to the crucial role biostatisticians play in the progress of medical research. Doctors and medical researchers are able to devote their entire intellectual capacity towards the development of new treatments, while biostatisticians are able to think deeply about both how to test these treatments empirically and how to combine the results of many such studies into actionable recommendations for practitioners and policy makers.  I aim to be the education sciences analogue of a biostatistician. Specifically, someone whose career success is decided on (i) the technical merits of the new methodology I have developed and (ii) the usefulness of my new methodology to the field. 

Your research on the Hedges correction suggests that many education researchers mis-specify their analyses for clustered designs. What advice would you give researchers on selecting the right analyses for clustered designs? 

My advice is to focus on the design of the study. If the design is wrong, then the analysis that matches the design will fail, and it is likely that no re-analysis of the collected data will be able to recover from the initial mistake. For example, a common design error is randomizing teachers to experimental conditions, but then assuming that how the school registrar assigned students to classes was equivalent to the experimenter randomizing students to classes. This assumption is false. Registrar based student assignment is a kind of group based, or clustered, random assignment. If this error is not caught at the design stage, the study will necessarily be under powered because the sample size calculations will be off. If the error is not caught at the publication stage, the hypothesis test for the treatment effect will be anti-conservative, i.e. even if the treatment effect is truly zero, the test statistic is still likely to be (incorrectly!) statistically significant. The error will, however, be caught if the What Works Clearinghouse decides to review the study. Their application of the Hedges correction, however, will not fix the design problem. The corrected test statistic will, at best, have low power, just like a re-analysis of the data would. At worst, the corrected test statistic can have nearly zero power. There is no escape from a design error. 


To give a bit of further, perhaps self-serving advice, I would also suggest engaging your local statistician as a collaborator. People like me are always looking to get involved in substantively interesting projects, especially if we can get involved at the planning stage of the project. Additionally, this division of labor is often better for everyone: the statistician gets to focus on interesting methodological challenges and the education researcher gets to focus on the substantive portion of the research. 

How has being an IES predoc and now an IES postdoc helped your development as a researcher?

This is a bit like the joke where one fish asks another "How is the water today?" The other fish responds "What's water?" 

I came to Carnegie Mellon for the joint Ph.D. in Statistics and Public Policy, in part, because the IES predoc program there, the Program for Interdisciplinary Education Research (PIER), would both fund me to become and train me to become an education researcher. The PIER program shaped my entire graduate career. David Klahr (PIER Director) gave me grounding in the education sciences. Brian Junker (PIER Steering committee) taught me how to be both methodologically rigorous and yet still accessible to applied researchers. Sharon Carver (PIER co-Director), who runs the CMU lab school, built in a formal reflection process for the "Field Base Experience" portion of our PIER training. That essay, was, perhaps, the most cathartic thing I have ever written in that it helped to set me on my career path as a statistician who aims to focus on education research. Joel Greenhouse (affiliated PIER faculty), who is himself a biostatistician, chaired my thesis committee. It was his example that refined the direction of my career: I wish to be the education sciences analogue of a biostatistician. 

The IES postdoc program at Northwestern University, where I am advised by Larry Hedges, has been very different. Postdoctoral training is necessarily quite different from graduate school. One thread is common, however, the methodology I develop must be useful to applied education researchers. Larry is, as one might suppose, quite good at focusing my attention on where I need to make technical improvements to my work, but also how I might better communicate my technical results and make them accessible to applied researchers. After only a year at Northwestern, I have grown considerably in both my technical and communication skills.

What career advice would you give to young researchers?

Pick good mentors and heed their advice. To the extent that I am successful, I credit the advice and training of my mentors at Carnegie Mellon and Northwestern. 


Comments? Questions? Please write to us at IESResearch@ed.gov.

Creating Pathways to Doctoral Study in the Education Sciences

By Katina Stapleton, NCER Program Officer 

 

In April 2015, the National Center for Education Research (NCER) launched its newest research training funding opportunity: the Pathways to the Education Sciences Research Training Program. The purpose of the Pathways Training Program is to fund new innovative training programs that promote diversity and prepare underrepresented students for doctoral study in education research. In this blog, I want to provide some background on how the Pathways Training Program was developed and respond to frequently asked questions.

NCER has supported research training programs since 2004, training over 900 pre- and post-doctoral fellows. In 2014, NCER and NCSER sought input from our stakeholders to determine what was going well with our training programs, and to identify areas where we could improve. For example, we held a Technical Working Group meeting, solicited public comment, and engaged with our PIs during our IES Principal Investigators meeting. We also discussed the future of our training programs with the National Board of Education Sciences. As part of these conversations, we asked participants to reflect on whether the needs of the education sciences are the same as when the training programs were established.  One issue that emerged was the demographic shifts taking place across the nation and the need for education researchers to be attuned to an array of social, cultural, and economic issues as they plan and conduct their work. Another issue that emerged was the need to increase the diversity of fellows served by our training programs and in the education research profession as a whole. 

NCER developed the Pathways Training Program in response to this input. The Pathways training program is modeled on efforts by the National Science Foundation and the National Institutes of Health  to increase diversity in the sciences. The Pathways program establishes research training programs at Minority Serving Institutions (and their partners) that will provide students, especially underrepresented students, with an introduction to education research and scientific methods, meaningful opportunities to participate in education research studies, and professional development and mentoring that leads to doctoral study.

The core feature of the Pathways Training Program is a required research apprenticeship, in which fellows gain hands-on research experience under the supervision of faculty mentors. While there are several additional recommended components, the Pathways Training Program Request for Applications (RFA) was purposefully designed to encourage innovation, and therefore provides applicants with wide latitude in how the training program is structured, the student population of interest (i.e. advanced undergrad vs. post-baccalaureate vs. masters), and the training partners involved. Since the request for applications was released, we have received several questions from interested applicants. We have responded to those questions below:     

  • Are individual Pathways programs restricted to minority students? No. The Pathways Training Program, is open to all students, however, it seeks to increase the number of fellows from groups underrepresented in doctoral study, including racial and ethnic minorities, first-generation college students, economically disadvantaged students, veterans, and students with disabilities. We encourage all Pathways applicants to consult the U.S. Department of Education, Office for Civil Right’s Guidance on the Voluntary Use of Race to Achieve Diversity in Postsecondary Education when deciding their student population of interest and developing their proposed program’s recruitment plan.
  • Is my institution eligible to apply? The Pathways Training Program was designed to award training grants to minority-serving institutions (MSIs) and other institutions of higher education in partnership with MSIs. MSIs are institutions of higher education in the United States and its territories enrolling populations with significant percentages of undergraduate minority students or that serve certain populations of minority students under various programs created by Congress or other federal agencies. The Institute chose to focus on MSIs because of their long history as critical stepping stones for underrepresented minority students who pursue doctoral degrees. The RFA defines several categories of MSIs, gives criteria for being considered an eligible MSI, and provides 3 lists that applicants can use to certify that their institution is an eligible MSI for the purpose of this RFA. If your institution does not meet these criteria, then you will have to partner with an eligible MSI in order to apply. If you have any questions, please contact Katina Stapleton, the program officer for the Pathways Program, for assistance. However, ultimately, it will be your institution’s responsibility to demonstrate that it is an eligible MSI.  
  • Can my institution submit more than one Pathways application? No. An institution may submit only one application to the Pathways Training Program. If more than one application from your institution is submitted, IES will only accept one of them for consideration. We recommend that before applying you contact your institution’s sponsored projects’ office to make sure there isn’t another Pathways application already in progress.
  • Can my institution participate in more than one Pathways application? Yes. Please note that the Pathways program is structured so that applications can be submitted by single institutions or by partnerships of two or more institutions. While institutions can only submit one application, it is possible for institutions to serve as a partner on multiple applications.
  • Can I still apply even though I missed the deadline for the Letter of Intent? Yes. Letters of Intent are completely optional. Therefore, even if you missed the deadline, you can still submit an application. If you missed the deadline, but would still like feedback on your proposed training program, please contact Katina Stapleton.

If you would like to learn more about the Pathways Training Program and other potential funding opportunities, please sign up for the Funding Opportunities for Minority Serving Institutions webinar that will be held on Tuesday, June 9th, 2015, 1:00 PM – 2:30 PM ET.