Inside IES Research

Notes from NCER & NCSER

The Month(s) in Review: September and October 2015

November 2, 2015 Blog Editor General, NCER, NCSER Share on TwitterShare on FacebookShare on Google+

By Liz Albro, NCER Associate Commissioner of Teaching and Learning

New Evaluation of State Education Programs and Policies Awards Announced

Congratulations to the recipients of our Evaluation of State Education Programs and Policies awards. These projects examine a range of topics: low-performing schools, college- and career-readiness standards, and teacher effectiveness and evaluation.

Building Strength in Numbers: Friends of IES Briefings

The Friends of IES, a coalition of research organizations working to raise the visibility of IES-funded studies, asked three IES funded researchers to participate in briefings for Department of Education leadership and for the public on Capitol Hill. Sharing findings from their IES-funded studies, the researchers highlighted how providing high quality mathematics instruction to children as young as three-years-old, and providing systematic and sustained opportunities for those children to learn more mathematics in subsequent instructional years, can substantially narrow achievement gaps at the end of preschool and how those gains can persist over time. What to know more? Read our earlier blog post or the AERA news story for additional details.

Congratulations to Patricia Snyder on receiving the 2015 DEC Award for Mentoring

Congratulations to Patricia Snyder, recipient of the 2015 Division for Early Childhood (DEC) Award for Mentoring. DEC, a division of the Council for Exceptional Children, awards this honor to a member who has provided significant training and guidance to students and new practitioners in the field of early childhood special education. Snyder is a professor of special education and early childhood studies and the David Lawrence Jr. Endowed Chair in Early Childhood Studies at the University of Florida. She is also the Principal Investigator (PI) and Training Program Director for a NCSER-funded postdoctoral training grant, Postdoctoral Research Training Fellowships in Early Intervention and Early Learning in Special Education at the University of Florida. She has also served as the PI and co-PI on several other NCSER-funded awards.

Thanks to all of our IES Postdoctoral Fellows: Past, Present and Future!

Did you know that the third week of September was National Postdoc Appreciation Week? While we tweeted our appreciation for the postdocs we support through our NCER and NCSER Postdoctoral Training Programs, we thought you might like to learn a bit more about what some of our postdocs are doing.

Publishing: Postdocs are busy publishing findings from their research. For example, David Braithwaite, a fellow in this Carnegie Mellon postdoctoral training program recently published Effects of Variation and Prior Knowledge on Abstract Concept Learning. Two postdoc fellows, Kimberly Nesbitt and Mary Fuhs, who were trained in this Vanderbilt postdoctoral training program, are co-authors on a recent publication exploring executive function skills and academic achievement in kindergarten.  Josh Polanin, another Vanderbilt postdoc, recently published two methodological papers: one on effect sizes, the other on using a meta-analytic technique to assess the relationship between treatment intensity and program effects.

Receiving Research Funding:  Previous postdoc fellows who trained at the University of Illinois, Urbana-Champaign have recently been awarded research funding. Erin Reid and her colleagues were recently awarded an NSF DRK-12 grant to adapt and study a teacher professional development (PD) intervention, called Collaborative Math (CM), for use in early childhood programs. Former fellow David Purpura was recently awarded a grant from the Kinley Trust to delineate the role of language in early mathematics performance. Dr.  Purpura is also co-PI on a 2015 IES grant, Evaluating the Efficacy of Learning Trajectories in Early Mathematics.

Congratulations and good luck to all of our recently complete postdocs! Sixteen fellows have completed this year with 10 completing in the past two months. These fellows bringing their expertise to the community as full-time faculty, directors of research programs, and research associates at universities, non-profits, government agencies, and other organizations.

What have the Research Centers Funded? Check Out Our New Summary Documents

NCSER has funded research in a variety of topics relevant to special education and early intervention since 2006. Recently, NCSER staff summarized the work on several topics, with more to come in the future.

Research supported by both Centers is also described in our Compendium of Mathematics and Science Research, which was released in October.

Updated IES Research in the News

Curious to know what other IES-funded research projects have gotten media attention? We recently updated our IES Research in the News page, so that’s your quickest way to find out!

IES Honors Statistician Nathan VanHoudnos as Outstanding Predoctoral Fellow

September 17, 2015 Blog Editor General, NCER, Research Training Share on TwitterShare on FacebookShare on Google+

By Phill Gagne and Katina Stapleton, NCER Program Officers

Each year, IES recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. The 2014 winner, Dr. Nathan VanHoudnos completed his Ph.D. at Carnegie Mellon University and wrote his dissertation on the efficacy of the Hedges Correction for unmodeled clustering. Nathan is currently a postdoctoral fellow at Northwestern University. In this blog, Nathan provides insights on becoming an education researcher and on research study design. 

How did you become interested in education research?

I was born into it. Before he retired, my father was the Director of Research for the Illinois Education Association. Additionally, my grandparents on my mother's side were both teachers. 

 

As a statistician, how do you explain the relevance of your research to education practitioners and policy-makers?

I appeal to the crucial role biostatisticians play in the progress of medical research. Doctors and medical researchers are able to devote their entire intellectual capacity towards the development of new treatments, while biostatisticians are able to think deeply about both how to test these treatments empirically and how to combine the results of many such studies into actionable recommendations for practitioners and policy makers.  I aim to be the education sciences analogue of a biostatistician. Specifically, someone whose career success is decided on (i) the technical merits of the new methodology I have developed and (ii) the usefulness of my new methodology to the field. 

Your research on the Hedges correction suggests that many education researchers mis-specify their analyses for clustered designs. What advice would you give researchers on selecting the right analyses for clustered designs? 

My advice is to focus on the design of the study. If the design is wrong, then the analysis that matches the design will fail, and it is likely that no re-analysis of the collected data will be able to recover from the initial mistake. For example, a common design error is randomizing teachers to experimental conditions, but then assuming that how the school registrar assigned students to classes was equivalent to the experimenter randomizing students to classes. This assumption is false. Registrar based student assignment is a kind of group based, or clustered, random assignment. If this error is not caught at the design stage, the study will necessarily be under powered because the sample size calculations will be off. If the error is not caught at the publication stage, the hypothesis test for the treatment effect will be anti-conservative, i.e. even if the treatment effect is truly zero, the test statistic is still likely to be (incorrectly!) statistically significant. The error will, however, be caught if the What Works Clearinghouse decides to review the study. Their application of the Hedges correction, however, will not fix the design problem. The corrected test statistic will, at best, have low power, just like a re-analysis of the data would. At worst, the corrected test statistic can have nearly zero power. There is no escape from a design error. 


To give a bit of further, perhaps self-serving advice, I would also suggest engaging your local statistician as a collaborator. People like me are always looking to get involved in substantively interesting projects, especially if we can get involved at the planning stage of the project. Additionally, this division of labor is often better for everyone: the statistician gets to focus on interesting methodological challenges and the education researcher gets to focus on the substantive portion of the research. 

How has being an IES predoc and now an IES postdoc helped your development as a researcher?

This is a bit like the joke where one fish asks another "How is the water today?" The other fish responds "What's water?" 

I came to Carnegie Mellon for the joint Ph.D. in Statistics and Public Policy, in part, because the IES predoc program there, the Program for Interdisciplinary Education Research (PIER), would both fund me to become and train me to become an education researcher. The PIER program shaped my entire graduate career. David Klahr (PIER Director) gave me grounding in the education sciences. Brian Junker (PIER Steering committee) taught me how to be both methodologically rigorous and yet still accessible to applied researchers. Sharon Carver (PIER co-Director), who runs the CMU lab school, built in a formal reflection process for the "Field Base Experience" portion of our PIER training. That essay, was, perhaps, the most cathartic thing I have ever written in that it helped to set me on my career path as a statistician who aims to focus on education research. Joel Greenhouse (affiliated PIER faculty), who is himself a biostatistician, chaired my thesis committee. It was his example that refined the direction of my career: I wish to be the education sciences analogue of a biostatistician. 

The IES postdoc program at Northwestern University, where I am advised by Larry Hedges, has been very different. Postdoctoral training is necessarily quite different from graduate school. One thread is common, however, the methodology I develop must be useful to applied education researchers. Larry is, as one might suppose, quite good at focusing my attention on where I need to make technical improvements to my work, but also how I might better communicate my technical results and make them accessible to applied researchers. After only a year at Northwestern, I have grown considerably in both my technical and communication skills.

What career advice would you give to young researchers?

Pick good mentors and heed their advice. To the extent that I am successful, I credit the advice and training of my mentors at Carnegie Mellon and Northwestern. 

Comments? Questions? Please write to us at IESResearch@ed.gov.

Investing in Scholars: The NCSER Early Career Development and Mentoring Grant Program

June 29, 2015 Blog Editor General, NCSER Share on TwitterShare on FacebookShare on Google+

Featuring Michael Kennedy, University of Virginia

By Liz Berke, NCSER intern

What do these three individuals have in common:  a former special education teacher in Delaware, a former reading specialist in California, and a former special education teacher in Georgia?  They are the three Principal Investigators of the three inaugural projects funded by NCSER through its Research Training Program in Special Education: Early Career Development and Mentoring grant program.  Through this program, scholars embarking on their research careers in special education and early intervention have the opportunity to work with established mentors as they develop their research skills.   Over the next few months, we will be featuring the current research of our investigators that IES has supported through this and other research programs for early career investigators. We are looking forward to sharing their perspectives with our readers. 

First up in our series is Dr. Michael Kennedy from the University of Virginia.  A former special education teacher in Delaware, Dr. Kennedy is being mentored by Dr. Mary Brownell (University of Florida) and John Lloyd (University of Virginia).  The main aim of his IES funded project is to create valid measures of teacher practices and to work with practitioners to develop effective professional development materials and processes.  The materials are intended to help middle school science and special education teachers improve their delivery of evidence-based vocabulary instruction for students with disabilities.   Dr. Kennedy was recently awarded the Early Career Researcher award from the Instructional Technology Special Interest Group of the American Educational Research Association and University of Virginia’s Alumni Board of Trustees All University Teaching Award.

We had the opportunity to speak with Dr. Kennedy about the challenges and benefits of starting a career in special education research and how IES is helping him reach his professional goals. 

What are some of the biggest challenges that you face as an early career researcher? How do you hope this award will help you overcome those challenges?

I would say the biggest challenge as a young researcher is that my eyes are regularly bigger than my stomach in terms of wanting to take on huge research questions that would require a large interdisciplinary team and access to a plethora of resources.  Being patient and addressing questions that are still important, but actually doable as an Assistant Professor working on a shoestring budget definitely takes discipline.  The Early Career Award from NCSER has helped me assemble a team of Hall of Fame caliber colleagues that is really superb in helping me stay focused on one key component of the larger study at a time, while simultaneously helping me recognize how the initial studies are working toward something greater.  Working on one big project of my own design for four years is something I’ve never done before; I’ve learned it’s very easy to lose sight of the forest because of the pesky trees, so I really rely on my mentors for perspective and guidance.  

What advice would you give to early career researchers?

I think my best advice is to really lock down your niche within your field and go to work on creating new knowledge that people can really connect with in practical ways.  As an example, think about your sub-field’s most successful and well-known researcher and what they are known for (I’ll wait).  Isn’t it remarkable how easy it is to pair that person with the widget, curriculum, or broad body of research they are associated with?  It’s hard to imagine, but they were once Assistant Professors like us trying to get a program of research off the ground.  How did they do it?  Other than being really smart, their currency was, and remains new thinking and ideas that can be translated into materials that help students improve as evaluated by relevant dependent measures.  If you don’t have ideas that other people care about and can make a difference for people, you aren’t going to get very far in our line of work.  So that brings me back to my first comment – you have to become expert in your corner of the world, and then let your creativity take over.

What is your favorite aspect of working with your mentors?

I would say my favorite part of working with my mentors is the access to top notch feedback that simply does not exist post grad school for most people.  As doc students we are constantly receiving feedback from our advisors and other professors, but all of that pretty much goes away as we take our first jobs.  Sure we get feedback from journal editors & reviewers, but that is not regular enough to always make a big difference.  Being able to walk down the hall (or get on the phone in the case of my co-mentor) to have a conversation about a new idea or data specific to a project they are invested in is really a remarkable gift.  Another really important aspect of my relationship with my mentors is how differently they think about things than I do.  They ask questions I never considered and poke holes in my logic that can be frustrating, but I recognize how important it is to consider these perspectives and make sure I address it.  My advice for all considering applying for this project is to think very carefully about who you select as your mentor or co-mentors – think big and don’t settle!  

 

Questions? Comments? Please send them to: IESResearch@ed.gov. 

 

Creating Pathways to Doctoral Study in the Education Sciences

June 4, 2015 Blog Editor Funding Opportunities, NCER, Research Training Share on TwitterShare on FacebookShare on Google+

By Katina Stapleton, NCER Program Officer 

 

In April 2015, the National Center for Education Research (NCER) launched its newest research training funding opportunity: the Pathways to the Education Sciences Research Training Program. The purpose of the Pathways Training Program is to fund new innovative training programs that promote diversity and prepare underrepresented students for doctoral study in education research. In this blog, I want to provide some background on how the Pathways Training Program was developed and respond to frequently asked questions.

NCER has supported research training programs since 2004, training over 900 pre- and post-doctoral fellows. In 2014, NCER and NCSER sought input from our stakeholders to determine what was going well with our training programs, and to identify areas where we could improve. For example, we held a Technical Working Group meeting, solicited public comment, and engaged with our PIs during our IES Principal Investigators meeting. We also discussed the future of our training programs with the National Board of Education Sciences. As part of these conversations, we asked participants to reflect on whether the needs of the education sciences are the same as when the training programs were established.  One issue that emerged was the demographic shifts taking place across the nation and the need for education researchers to be attuned to an array of social, cultural, and economic issues as they plan and conduct their work. Another issue that emerged was the need to increase the diversity of fellows served by our training programs and in the education research profession as a whole. 

NCER developed the Pathways Training Program in response to this input. The Pathways training program is modeled on efforts by the National Science Foundation and the National Institutes of Health  to increase diversity in the sciences. The Pathways program establishes research training programs at Minority Serving Institutions (and their partners) that will provide students, especially underrepresented students, with an introduction to education research and scientific methods, meaningful opportunities to participate in education research studies, and professional development and mentoring that leads to doctoral study.

The core feature of the Pathways Training Program is a required research apprenticeship, in which fellows gain hands-on research experience under the supervision of faculty mentors. While there are several additional recommended components, the Pathways Training Program Request for Applications (RFA) was purposefully designed to encourage innovation, and therefore provides applicants with wide latitude in how the training program is structured, the student population of interest (i.e. advanced undergrad vs. post-baccalaureate vs. masters), and the training partners involved. Since the request for applications was released, we have received several questions from interested applicants. We have responded to those questions below:     

  • Are individual Pathways programs restricted to minority students? No. The Pathways Training Program, is open to all students, however, it seeks to increase the number of fellows from groups underrepresented in doctoral study, including racial and ethnic minorities, first-generation college students, economically disadvantaged students, veterans, and students with disabilities. We encourage all Pathways applicants to consult the U.S. Department of Education, Office for Civil Right’s Guidance on the Voluntary Use of Race to Achieve Diversity in Postsecondary Education when deciding their student population of interest and developing their proposed program’s recruitment plan.
  • Is my institution eligible to apply? The Pathways Training Program was designed to award training grants to minority-serving institutions (MSIs) and other institutions of higher education in partnership with MSIs. MSIs are institutions of higher education in the United States and its territories enrolling populations with significant percentages of undergraduate minority students or that serve certain populations of minority students under various programs created by Congress or other federal agencies. The Institute chose to focus on MSIs because of their long history as critical stepping stones for underrepresented minority students who pursue doctoral degrees. The RFA defines several categories of MSIs, gives criteria for being considered an eligible MSI, and provides 3 lists that applicants can use to certify that their institution is an eligible MSI for the purpose of this RFA. If your institution does not meet these criteria, then you will have to partner with an eligible MSI in order to apply. If you have any questions, please contact Katina Stapleton, the program officer for the Pathways Program, for assistance. However, ultimately, it will be your institution’s responsibility to demonstrate that it is an eligible MSI.  
  • Can my institution submit more than one Pathways application? No. An institution may submit only one application to the Pathways Training Program. If more than one application from your institution is submitted, IES will only accept one of them for consideration. We recommend that before applying you contact your institution’s sponsored projects’ office to make sure there isn’t another Pathways application already in progress.
  • Can my institution participate in more than one Pathways application? Yes. Please note that the Pathways program is structured so that applications can be submitted by single institutions or by partnerships of two or more institutions. While institutions can only submit one application, it is possible for institutions to serve as a partner on multiple applications.
  • Can I still apply even though I missed the deadline for the Letter of Intent? Yes. Letters of Intent are completely optional. Therefore, even if you missed the deadline, you can still submit an application. If you missed the deadline, but would still like feedback on your proposed training program, please contact Katina Stapleton.

If you would like to learn more about the Pathways Training Program and other potential funding opportunities, please sign up for the Funding Opportunities for Minority Serving Institutions webinar that will be held on Tuesday, June 9th, 2015, 1:00 PM – 2:30 PM ET.