Inside IES Research

Notes from NCER & NCSER

The Nexus Between Teaching and Research: What I Learned Working on an IES Grant

 

Samuel Choo is a doctoral student at the dissertation stage in the Department of Early Childhood, Special Education, and Rehabilitation Counseling at the University of Kentucky (UK). In this blog post, he describes how working on an IES grant gave him first-hand experiences in planning and carrying out research in schools. He also discusses how these research experiences helped him understand the important connections between research and teaching.

How did you get started working on this IES research project?

The first I heard of IES was six years ago as a resource room teacher at a middle school. Dr. Brian Bottge, who is now my doctoral adviser, was awarded a NCSER grant to test the effects of Enhanced Anchored Instruction (EAI) on the math performance of middle school students. My school was randomly assigned to the EAI group. The project staff did a good job of teaching us how to implement EAI in our resource rooms. Soon after teaching with the new curriculum, I noticed that my students were much more motivated and engaged than they had been. In fact, they looked like they were actually enjoying math! Posttest scores showed positive results in favor of the new curriculum.

And so this experience as a teacher got you more interested in research?

Yes! The next year I applied to the UK doctoral program. I joined Dr. Bottge’s IES grant team as a research assistant where I learned how classroom-based research is planned and conducted. I had many opportunities to participate in the research experience. In my case, I helped train math and special education teachers, observed classrooms and assessed research fidelity, provided teachers with technical support, assisted in scoring tests, and worked on data entry and analysis. Project leaders also asked me to suggest revisions to the daily lesson plans based on my experiences teaching with EAI the year before.

Can you talk more about your developing research interests related to math education?

After the grant ended and after I finished my doctoral coursework, I went back to teaching in North Carolina, where I taught low performing middle school students in a Title I resource room. I ran my own pilot studies using what I had learned while teaching with EAI as both a research participant and research assistant. To help offset the cost of materials for my first study, I was awarded a $1500 Bright Ideas Grant from the North Carolina’s Electric Cooperatives. Thanks to the company’s generosity, I was able to fully implement all the lesson plans developed by Dr. Bottge’s grant team.

This experience was especially important to me because it was my first try at conducting my own research with a prescribed protocol, which I had learned from working on the IES project. Posttests showed statistically significant improvement of students in the EAI group in both computation and problem solving. Based on these results, the sponsor invited me to participate in a panel discussion in Raleigh, NC. The CEOs of the company attended the event along with policy makers and school administrators from across the state. This whole process, from applying for funding to carrying out the study to reporting the results, helped me make connections between university, classroom, and community.

What have been your big takeaways from these experiences?

From the training I received as a study participant, I have become a better teacher.  From working on an IES-funded grant team, I learned a lot about how to conduct classroom-based studies. I am looking forward to designing new instructional methods and testing their effectiveness. Similar to how my students learned math in a hands-on way, I learned research methods by having the opportunity to use them in practice, and for that I am very grateful. 

How to Develop Your Career in Education Research: IES Training Opportunities

By Corinne Alfeld, NCER Program Officer

In honor of career development month, we would like to remind you about training opportunities funded by IES. We have invested in training programs since 2004 with the aim of increasing the supply of scientists and researchers in education who are prepared to conduct rigorous education research that advances knowledge within the field and addresses issues important to education policymakers and practitioners. These efforts are intended to lead both to the training of talented education researchers from a variety of backgrounds and to the incorporation of diverse ideas and perspectives in education research.

In this blog we describe five types of training opportunities currently offered through the National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER) that span from undergraduate to the postdoctoral level and beyond.

Training Opportunities for Current or Future Doctoral Students
Are you a current or aspiring doctoral student wondering what training opportunities are available to you? You may be interested in applying to one of 10 training programs funded by NCER’s Predoctoral Interdisciplinary Research Training Program in the Education Sciences that train predoctoral fellows in interdisciplinary programs involving a number of academic disciplines (e.g., economics, education, psychology, public policy, sociology, and statistics, among others). These fellowships can be from 2 to 5 years in length depending on the training program model and typically include tuition and benefits, a $30,000 stipend, and a small research/travel fund. Fellows who complete their training program have the skills necessary to produce research that is rigorous in method as well as relevant and accessible to education stakeholders such as practitioners and policymakers.

If you are interested in becoming a predoctoral fellow, you must apply directly to one of the training programs, not to IES.  Each of the 10 fellowship programs funded in 2014-15 has its own application process and acceptance criteria. For more information on becoming a predoctoral fellow, check out this resource on Applying for a Predoctoral Interdisciplinary Research Training Program Fellowship.

 

Postdoctoral Training Opportunities
Are you finishing up your doctorate and wondering how you can get more experience in education research? Or perhaps you’re looking to return to academia through a postdoctoral position? If so, you may want to apply to one of the our programs funded under the Postdoctoral Research Training Program in the Education Sciences Program (NCER) or the Postdoctoral Research Training Program in Special Education Program (NCSER). Through these two grant programs, IES funds training programs at doctoral-granting institutions to further prepare researchers who have obtained their Ph.D.s or Ed.D.s to become scholars capable of conducting high-quality, independent education or special education research. These postdoctoral training programs provide practical, hands-on experiences; enrichment of theoretical and empirical knowledge; and opportunities for fellows to build professional skills and networks that will support working with other researchers and relevant education research stakeholders.

To inquire about postdoctoral fellowship openings, follow the hyperlinks in this section to search for currently (awarded in 2010-15) funded programs at various universities around the country. For example, here are the 2015 NCER-funded programs and 2012 NCSER-funded programs.

 

Other Upcoming Training Opportunities:

  • Undergraduate, Post-baccalaureate, and Master’s Students. If you are an upper-level undergraduate student, recent graduate, and/or master’s student, especially from a group that is underrepresented in doctoral study (including racial and ethnic minorities, first-generation college students, economically disadvantaged students, veterans, and students with disabilities), you may be interested in the Pathways to the Education Sciences Research Training Program (Pathways). Established in 2015, the Pathways program funds training programs at minority-serving institutions (MSIs) and institutions of higher education that partner with MSIs. These training programs will provide fellows with education research experience and professional development to prepare them to pursue doctoral study in the education sciences or in fields relevant to education research.

Up to five Pathways training programs will be awarded to MSIs (and their partners) in 2016. These new programs will begin recruiting fellows in 2016 and 2017, so keep your eyes and ears open for more information about where and how to apply!

 

  • Early Career Education Researchers. Are you an early career researcher at your first appointment? If so, you may qualify for and be interested in one of our programs that target early career researchers in statistical and research methodology (NCER) and special education (NCSER).

Look for upcoming training opportunities for early career researchers in future Request for Applications for training grants (CFDA 84.305B and 84.324B) and statistical and research methodology grants (CFDA 84.305D).

 

  • Active Researchers Looking to Improve Their Methodological Expertise. Are you a current researcher (e.g., at a university or research firm) who would like to add tools to your methodological toolkit or further refine your skills with such tools? If so, then the Methods Training for Education Researchers Program may be for you.

If you are interested in methodological training, sign up for the IES Newsflash for announcements of upcoming workshops or periodically check our list of IES-funded workshops.

 

If you have questions about our training programs, please contact Corinne.Alfeld@ed.gov.

 

The Month(s) in Review: September and October 2015

By Liz Albro, NCER Associate Commissioner of Teaching and Learning

New Evaluation of State Education Programs and Policies Awards Announced

Congratulations to the recipients of our Evaluation of State Education Programs and Policies awards. These projects examine a range of topics: low-performing schools, college- and career-readiness standards, and teacher effectiveness and evaluation.

Building Strength in Numbers: Friends of IES Briefings

The Friends of IES, a coalition of research organizations working to raise the visibility of IES-funded studies, asked three IES funded researchers to participate in briefings for Department of Education leadership and for the public on Capitol Hill. Sharing findings from their IES-funded studies, the researchers highlighted how providing high quality mathematics instruction to children as young as three-years-old, and providing systematic and sustained opportunities for those children to learn more mathematics in subsequent instructional years, can substantially narrow achievement gaps at the end of preschool and how those gains can persist over time. What to know more? Read our earlier blog post or the AERA news story for additional details.

Congratulations to Patricia Snyder on receiving the 2015 DEC Award for Mentoring

Congratulations to Patricia Snyder, recipient of the 2015 Division for Early Childhood (DEC) Award for Mentoring. DEC, a division of the Council for Exceptional Children, awards this honor to a member who has provided significant training and guidance to students and new practitioners in the field of early childhood special education. Snyder is a professor of special education and early childhood studies and the David Lawrence Jr. Endowed Chair in Early Childhood Studies at the University of Florida. She is also the Principal Investigator (PI) and Training Program Director for a NCSER-funded postdoctoral training grant, Postdoctoral Research Training Fellowships in Early Intervention and Early Learning in Special Education at the University of Florida. She has also served as the PI and co-PI on several other NCSER-funded awards.

Thanks to all of our IES Postdoctoral Fellows: Past, Present and Future!

Did you know that the third week of September was National Postdoc Appreciation Week? While we tweeted our appreciation for the postdocs we support through our NCER and NCSER Postdoctoral Training Programs, we thought you might like to learn a bit more about what some of our postdocs are doing.

Publishing: Postdocs are busy publishing findings from their research. For example, David Braithwaite, a fellow in this Carnegie Mellon postdoctoral training program recently published Effects of Variation and Prior Knowledge on Abstract Concept Learning. Two postdoc fellows, Kimberly Nesbitt and Mary Fuhs, who were trained in this Vanderbilt postdoctoral training program, are co-authors on a recent publication exploring executive function skills and academic achievement in kindergarten.  Josh Polanin, another Vanderbilt postdoc, recently published two methodological papers: one on effect sizes, the other on using a meta-analytic technique to assess the relationship between treatment intensity and program effects.

Receiving Research Funding:  Previous postdoc fellows who trained at the University of Illinois, Urbana-Champaign have recently been awarded research funding. Erin Reid and her colleagues were recently awarded an NSF DRK-12 grant to adapt and study a teacher professional development (PD) intervention, called Collaborative Math (CM), for use in early childhood programs. Former fellow David Purpura was recently awarded a grant from the Kinley Trust to delineate the role of language in early mathematics performance. Dr.  Purpura is also co-PI on a 2015 IES grant, Evaluating the Efficacy of Learning Trajectories in Early Mathematics.

Congratulations and good luck to all of our recently complete postdocs! Sixteen fellows have completed this year with 10 completing in the past two months. These fellows bringing their expertise to the community as full-time faculty, directors of research programs, and research associates at universities, non-profits, government agencies, and other organizations.

What have the Research Centers Funded? Check Out Our New Summary Documents

NCSER has funded research in a variety of topics relevant to special education and early intervention since 2006. Recently, NCSER staff summarized the work on several topics, with more to come in the future.

Research supported by both Centers is also described in our Compendium of Mathematics and Science Research, which was released in October.

Updated IES Research in the News

Curious to know what other IES-funded research projects have gotten media attention? We recently updated our IES Research in the News page, so that’s your quickest way to find out!

IES Honors Statistician Nathan VanHoudnos as Outstanding Predoctoral Fellow

By Phill Gagne and Katina Stapleton, NCER Program Officers

Each year, IES recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. The 2014 winner, Dr. Nathan VanHoudnos completed his Ph.D. at Carnegie Mellon University and wrote his dissertation on the efficacy of the Hedges Correction for unmodeled clustering. Nathan is currently a postdoctoral fellow at Northwestern University. In this blog, Nathan provides insights on becoming an education researcher and on research study design. 

How did you become interested in education research?

I was born into it. Before he retired, my father was the Director of Research for the Illinois Education Association. Additionally, my grandparents on my mother's side were both teachers. 

 

As a statistician, how do you explain the relevance of your research to education practitioners and policy-makers?

I appeal to the crucial role biostatisticians play in the progress of medical research. Doctors and medical researchers are able to devote their entire intellectual capacity towards the development of new treatments, while biostatisticians are able to think deeply about both how to test these treatments empirically and how to combine the results of many such studies into actionable recommendations for practitioners and policy makers.  I aim to be the education sciences analogue of a biostatistician. Specifically, someone whose career success is decided on (i) the technical merits of the new methodology I have developed and (ii) the usefulness of my new methodology to the field. 

Your research on the Hedges correction suggests that many education researchers mis-specify their analyses for clustered designs. What advice would you give researchers on selecting the right analyses for clustered designs? 

My advice is to focus on the design of the study. If the design is wrong, then the analysis that matches the design will fail, and it is likely that no re-analysis of the collected data will be able to recover from the initial mistake. For example, a common design error is randomizing teachers to experimental conditions, but then assuming that how the school registrar assigned students to classes was equivalent to the experimenter randomizing students to classes. This assumption is false. Registrar based student assignment is a kind of group based, or clustered, random assignment. If this error is not caught at the design stage, the study will necessarily be under powered because the sample size calculations will be off. If the error is not caught at the publication stage, the hypothesis test for the treatment effect will be anti-conservative, i.e. even if the treatment effect is truly zero, the test statistic is still likely to be (incorrectly!) statistically significant. The error will, however, be caught if the What Works Clearinghouse decides to review the study. Their application of the Hedges correction, however, will not fix the design problem. The corrected test statistic will, at best, have low power, just like a re-analysis of the data would. At worst, the corrected test statistic can have nearly zero power. There is no escape from a design error. 


To give a bit of further, perhaps self-serving advice, I would also suggest engaging your local statistician as a collaborator. People like me are always looking to get involved in substantively interesting projects, especially if we can get involved at the planning stage of the project. Additionally, this division of labor is often better for everyone: the statistician gets to focus on interesting methodological challenges and the education researcher gets to focus on the substantive portion of the research. 

How has being an IES predoc and now an IES postdoc helped your development as a researcher?

This is a bit like the joke where one fish asks another "How is the water today?" The other fish responds "What's water?" 

I came to Carnegie Mellon for the joint Ph.D. in Statistics and Public Policy, in part, because the IES predoc program there, the Program for Interdisciplinary Education Research (PIER), would both fund me to become and train me to become an education researcher. The PIER program shaped my entire graduate career. David Klahr (PIER Director) gave me grounding in the education sciences. Brian Junker (PIER Steering committee) taught me how to be both methodologically rigorous and yet still accessible to applied researchers. Sharon Carver (PIER co-Director), who runs the CMU lab school, built in a formal reflection process for the "Field Base Experience" portion of our PIER training. That essay, was, perhaps, the most cathartic thing I have ever written in that it helped to set me on my career path as a statistician who aims to focus on education research. Joel Greenhouse (affiliated PIER faculty), who is himself a biostatistician, chaired my thesis committee. It was his example that refined the direction of my career: I wish to be the education sciences analogue of a biostatistician. 

The IES postdoc program at Northwestern University, where I am advised by Larry Hedges, has been very different. Postdoctoral training is necessarily quite different from graduate school. One thread is common, however, the methodology I develop must be useful to applied education researchers. Larry is, as one might suppose, quite good at focusing my attention on where I need to make technical improvements to my work, but also how I might better communicate my technical results and make them accessible to applied researchers. After only a year at Northwestern, I have grown considerably in both my technical and communication skills.

What career advice would you give to young researchers?

Pick good mentors and heed their advice. To the extent that I am successful, I credit the advice and training of my mentors at Carnegie Mellon and Northwestern. 


Comments? Questions? Please write to us at IESResearch@ed.gov.

Investing in Scholars: The NCSER Early Career Development and Mentoring Grant Program

Featuring Michael Kennedy, University of Virginia

By Liz Berke, NCSER intern

What do these three individuals have in common:  a former special education teacher in Delaware, a former reading specialist in California, and a former special education teacher in Georgia?  They are the three Principal Investigators of the three inaugural projects funded by NCSER through its Research Training Program in Special Education: Early Career Development and Mentoring grant program.  Through this program, scholars embarking on their research careers in special education and early intervention have the opportunity to work with established mentors as they develop their research skills.   Over the next few months, we will be featuring the current research of our investigators that IES has supported through this and other research programs for early career investigators. We are looking forward to sharing their perspectives with our readers. 

First up in our series is Dr. Michael Kennedy from the University of Virginia.  A former special education teacher in Delaware, Dr. Kennedy is being mentored by Dr. Mary Brownell (University of Florida) and John Lloyd (University of Virginia).  The main aim of his IES funded project is to create valid measures of teacher practices and to work with practitioners to develop effective professional development materials and processes.  The materials are intended to help middle school science and special education teachers improve their delivery of evidence-based vocabulary instruction for students with disabilities.   Dr. Kennedy was recently awarded the Early Career Researcher award from the Instructional Technology Special Interest Group of the American Educational Research Association and University of Virginia’s Alumni Board of Trustees All University Teaching Award.

We had the opportunity to speak with Dr. Kennedy about the challenges and benefits of starting a career in special education research and how IES is helping him reach his professional goals. 

What are some of the biggest challenges that you face as an early career researcher? How do you hope this award will help you overcome those challenges?

I would say the biggest challenge as a young researcher is that my eyes are regularly bigger than my stomach in terms of wanting to take on huge research questions that would require a large interdisciplinary team and access to a plethora of resources.  Being patient and addressing questions that are still important, but actually doable as an Assistant Professor working on a shoestring budget definitely takes discipline.  The Early Career Award from NCSER has helped me assemble a team of Hall of Fame caliber colleagues that is really superb in helping me stay focused on one key component of the larger study at a time, while simultaneously helping me recognize how the initial studies are working toward something greater.  Working on one big project of my own design for four years is something I’ve never done before; I’ve learned it’s very easy to lose sight of the forest because of the pesky trees, so I really rely on my mentors for perspective and guidance.  

What advice would you give to early career researchers?

I think my best advice is to really lock down your niche within your field and go to work on creating new knowledge that people can really connect with in practical ways.  As an example, think about your sub-field’s most successful and well-known researcher and what they are known for (I’ll wait).  Isn’t it remarkable how easy it is to pair that person with the widget, curriculum, or broad body of research they are associated with?  It’s hard to imagine, but they were once Assistant Professors like us trying to get a program of research off the ground.  How did they do it?  Other than being really smart, their currency was, and remains new thinking and ideas that can be translated into materials that help students improve as evaluated by relevant dependent measures.  If you don’t have ideas that other people care about and can make a difference for people, you aren’t going to get very far in our line of work.  So that brings me back to my first comment – you have to become expert in your corner of the world, and then let your creativity take over.

What is your favorite aspect of working with your mentors?

I would say my favorite part of working with my mentors is the access to top notch feedback that simply does not exist post grad school for most people.  As doc students we are constantly receiving feedback from our advisors and other professors, but all of that pretty much goes away as we take our first jobs.  Sure we get feedback from journal editors & reviewers, but that is not regular enough to always make a big difference.  Being able to walk down the hall (or get on the phone in the case of my co-mentor) to have a conversation about a new idea or data specific to a project they are invested in is really a remarkable gift.  Another really important aspect of my relationship with my mentors is how differently they think about things than I do.  They ask questions I never considered and poke holes in my logic that can be frustrating, but I recognize how important it is to consider these perspectives and make sure I address it.  My advice for all considering applying for this project is to think very carefully about who you select as your mentor or co-mentors – think big and don’t settle!  

 

Questions? Comments? Please send them to: IESResearch@ed.gov