IES Blog

Institute of Education Sciences

RCT-YES: Supporting a Culture of Research Use in Education

By Ruth Curran Neild, Delegated Director, IES

The mission of the Institute of Education Sciences (IES), at its core, is to create a culture in which independent, rigorous research and statistics are used to improve education. But sometimes research is seen by practitioners and policymakers as something that is done for them or to them, but not by them. And that’s something we’re hoping to change.

IES is always looking for new ways to involve educators in producing and learning about high-quality, useful research. We believe that if state and school district staff see themselves as full participants in scientific investigation, they will be more likely to make research a part of their routine practice. Simply put, we want to make it easier for educators to learn what works in their context and to contribute to the general knowledge of effective practices in education.    

That’s why we’re so pleased to add the RCT-YESTM software to the IES-funded toolkit of free, user-friendly resources for conducting research. Peter Schochet of Mathematica Policy Research, Inc. led the development of the software, as part of a contract with IES held by Decision Information Resources, Inc.

RCT-YES has a straightforward interface that allows the user to specify the analyses for data from a randomized controlled trial (RCT) or a quasi-experiment. Definitions and tips in the software help guide the user and accompanying documentation includes a mini-course on RCTs. When the user enters information about the data set and study design, RCT-YES produces a program to run the specified analyses (in either R or Stata) and provide a set of formatted tables.

The target users are those who have a basic knowledge of statistics and research design but do not have advanced training in conducting or analyzing data from impact studies. But we expect that even experienced researchers will like the simplicity and convenience of RCT-YES and benefit from some of its novel features, such as how it reports results.

When used properly, RCT-YES provides all of the statistics needed by the What Works ClearinghouseTM (WWC) to conduct a study review.  This is an important feature because the WWC often needs to contact authors—even experienced ones—to obtain additional statistics to make a determination of study quality.  RCT-YES could help advance the field by increasing the completeness of study reports.

Another unique feature of the software is that it defaults to practices recommended by IES’ National Center for Education Statistics for the protection of personally identifiable information. For example, the program suppresses reporting on small-size subgroups.

While the user sees only the simplicity of the interface, the underlying estimation methods and code required painstaking and sophisticated work.  RCT-YES relies on design-based estimation methods, and the development, articulation, peer review, and publication of this approach in the context of RCT-YES was the first careful step. Design-based methods make fewer assumptions about the statistical model than methods traditionally used in education (such as hierarchical linear modeling), making this approach especially appropriate for software designed with educators in mind.

The software is available for download from the RCT-YES website, where you can also find support videos, documentation, a user guide, and links to other helpful resources. The videos below, which are also hosted on the RCT-YES website, give a quick overview of the software.

There are many other ways that IES fosters a culture of research use in education. For instance, our 10 Regional Educational Laboratories (RELs) have research alliances that work with states and districts to develop research agendas. The RELs also host events to share best practices for putting research into action, such as the year-long series of webinars and training sessions on building, implementing, and effectively using Early Warning Systems to reduce dropping out.

IES also offers grants to states and districts to do quick evaluations of programs and policies that have been implemented in their schools. The low-cost, short-duration evaluations not only help the grantees discover what is working, but can help others who might use the same program or implement a similar policy. (We’ll announce the first round of grant recipients in the coming weeks).

Visit the IES website to learn more about our work. You can also stay on top of news and information from IES by following us on Facebook and Twitter

 

 

 

The Institute of Education Sciences at AERA

The American Educational Research Association (AERA) will hold its annual meeting April 8 through April 12 in Washington, D.C.—the largest educational research gathering in the nation. This will be a special meeting for AERA, as it is celebrating 100 years of advocating for the development and use of research in education. The program includes hundreds of sessions, including opportunities to learn about cutting edge education research and opportunities to broaden and deepen the field. 

About 30 sessions will feature staff from the Institute of Education Sciences (IES) discussing IES-funded research, evaluation, and statistics, as well as training and funding opportunities.

On Saturday, April 9, at 10:35 a.m., attendees will have a chance to meet the Institute’s leadership and hear about the areas of work that IES will be focusing on in the coming year. Speakers include Ruth Curran Neild, IES’ delegated director, and the leaders of the four centers in IES: Thomas Brock, commissioner of the National Center for Education Research (NCER); Peggy Carr, acting commissioner of the National Center for Educational Statistics (NCES); Joy Lesnick, acting commissioner of the National Center for Education Evaluation and Regional Assistance (NCEE), and Joan McLaughlin, commissioner of the National Center for Special Education Research (NCSER).

On Monday, April 11, at 9:45 a.m., attendees can speak to one of several IES staffers who will be available at the Research Funding Opportunities—Meet Your Program Officers session. Program officers from NCER, NCSER, and NCEE will be on hand to answer questions about programs and grant funding opportunities. Several IES representatives will also be on hand Monday afternoon, at 4:15 p.m. for the Federally Funded Data Resources: Opportunities for Research session to discuss the myriad datasets and resources that are available to researchers.

NCES staff will lead sessions and present on a variety of topics, from The Role of School Finance in the Pursuit of Equity (Saturday, 12:25 p.m.) to Understanding Federal Education Policies and Data about English Learners (Sunday, April 10, 8:15 a.m.) and what we can learn from the results of PIAAC, a survey of adult skills (also Sunday, 8:15 a.m.). Dr. Carr will be a part of several sessions, including one on Sunday morning (10:35 a.m.) about future directions for NCES longitudinal studies and another on Monday morning (10 a.m.) entitled Issues and Challenges in the Fair and Valid Assessment of Diverse Populations in the 21st Century

On Monday, at 11:45 a.m., you can also learn about an IES-supported tool, called RCT-YES, that is designed to reduce barriers to rigorous impact studies by simplifying estimation and reporting of study results (Dr. Lesnick will be among those presenting). And a team from the IES research centers (NCER/NCSER) will present Sunday morning (10:35 a.m.) on communication strategies for disseminating education research (which includes this blog!).

IES staff will also participate in a number of other roundtables and poster sessions. For instance, on Tuesday, April 12, at 8:15 a.m., grab a cup of coffee and attend the structured poster session with the Institute’s 10 Regional Educational Laboratories (RELs). This session will focus on building partnerships to improve data use in education.  REL work will also be featured at several other AERA sessions.  

Did you know that the National Library of Education (NLE) is a component of IES? On Friday and Monday afternoon, attendees will have a unique opportunity to go on a site visit to the library. You’ll learn about the library’s current and historical resources – including its collection of more than 20,000 textbooks dating from the mid-19th century. The Library offers information, statistical, and referral services to the Department of Education and other government agencies and institutions, and to the public.

If you are going to AERA, follow us on Twitter to learn more about our sessions and our work.  And if you are tweeting during one of our sessions, please include @IESResearch in your tweet. 

By Dana Tofig, Communications Director, IES

The PI Meeting in 140 Characters

By Wendy Wei, Program Assistant, National Center for Education Research

How can practitioners and policymakers apply education research to their everyday work if they never hear about it or do not understand it? Communicating and disseminating research findings plays an integral role in promoting the education sciences and advancing the field.

That is why we made communication and dissemination a major theme at the IES Principal Investigators’ Meeting held earlier this month (December 10-11). The two-day meeting in Washington, D.C., featured five sessions that focused on communications – ranging from data visualization techniques to effective dissemination strategies to hearing journalists’ perspectives on how to share scientific results with the general public.

There was a lot of talk about social media during the meeting and plenty of tweeting about the presentations. We used the Twitter hashtag, #IESPIMtg, to foster an ongoing conversation for meeting attendees and to share findings that emerged from sessions.  Any tweet that included #IESPIMtg was automatically pooled together, generating a live Twitter feed that was on display in the lobby throughout the meeting.

 You can see all of the #IESPImtg tweets online, but here are some highlights:

"There is a tremendous sense of urgency to bridge the gap between research and practice..." --John B King #IESPIMtg

— Leah Wisdom (@lifelnglearner) December 10, 2015

.@StanfordEd's Sean Reardon: Good partnership work can lead to new knowledge, change policy+practice, improve data quality #IESPIMtg

— Bill Penuel (@bpenuel) December 11, 2015

#IESPIMtg Practitioner partners play a critical role in making sense of data and analyses in RPPs.

— Jennifer Russell (@Jenn_L_Russell) December 10, 2015

And we can get a little bit meta now…communicating about how to communicate:

Hirsh-Pasek & Golinkoff urges researchers to create "'edible science' that is accessible, digestible and usable." #IESPIMtg

— Tomoko Wakabayashi (@twakabayashi264) December 10, 2015

Awesome presentation on #DataVisualization by @jschwabish: Show the data, reduce the clutter, stop distracting attention. #IESPIMtg

— Rudy Ruiz (@RudyRuiz_BMore) December 10, 2015

.@KavithaCardoza Explaining your research--Don't think of it as "dumbing down." Think of it as simplifying. #IESPIMtg

— Dana Tofig (@dtofig) December 11, 2015

And, of course, what's Twitter without a little fun? When we tweeted this picture...

The poster session is going strong. Principal investigators present findings from #iesfunded research. #IESPIMtg

— IES Research (@IESResearch) December 10, 2015

...Chris Magnuson, Director of Innovation for Live It, Learn It, posted this reply: 

@IESResearch careful...photo looks like it was taken on Death Star! May the force be with all grantees! #SBIR #IES

— Chris Magnuson (@cromagnuson) December 10, 2015

The National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER) have made a commitment to be active contributors in communicating with and engaging the general public in the exciting findings of NCER- and NCSER-funded work. Over the past few years, we have been active on Twitter (you can follow us @IESResearch), and this past year, we launched our blog (the very one you are reading!). These two platforms have provided us with an outlet to share research findings, provide updates about events and deadlines, and connect with audiences we otherwise might not reach.

For those of you who could not make the PI meeting, videos will be posted on the conference website in about a month. So stay tuned!

We hope you’ll continue the conversation started at the PI meeting by following us on Twitter at @IESResearch or sharing your thoughts with us at IESResearch@ed.gov.

 

The Nexus Between Teaching and Research: What I Learned Working on an IES Grant

 

Samuel Choo is a doctoral student at the dissertation stage in the Department of Early Childhood, Special Education, and Rehabilitation Counseling at the University of Kentucky (UK). In this blog post, he describes how working on an IES grant gave him first-hand experiences in planning and carrying out research in schools. He also discusses how these research experiences helped him understand the important connections between research and teaching.

How did you get started working on this IES research project?

The first I heard of IES was six years ago as a resource room teacher at a middle school. Dr. Brian Bottge, who is now my doctoral adviser, was awarded a NCSER grant to test the effects of Enhanced Anchored Instruction (EAI) on the math performance of middle school students. My school was randomly assigned to the EAI group. The project staff did a good job of teaching us how to implement EAI in our resource rooms. Soon after teaching with the new curriculum, I noticed that my students were much more motivated and engaged than they had been. In fact, they looked like they were actually enjoying math! Posttest scores showed positive results in favor of the new curriculum.

And so this experience as a teacher got you more interested in research?

Yes! The next year I applied to the UK doctoral program. I joined Dr. Bottge’s IES grant team as a research assistant where I learned how classroom-based research is planned and conducted. I had many opportunities to participate in the research experience. In my case, I helped train math and special education teachers, observed classrooms and assessed research fidelity, provided teachers with technical support, assisted in scoring tests, and worked on data entry and analysis. Project leaders also asked me to suggest revisions to the daily lesson plans based on my experiences teaching with EAI the year before.

Can you talk more about your developing research interests related to math education?

After the grant ended and after I finished my doctoral coursework, I went back to teaching in North Carolina, where I taught low performing middle school students in a Title I resource room. I ran my own pilot studies using what I had learned while teaching with EAI as both a research participant and research assistant. To help offset the cost of materials for my first study, I was awarded a $1500 Bright Ideas Grant from the North Carolina’s Electric Cooperatives. Thanks to the company’s generosity, I was able to fully implement all the lesson plans developed by Dr. Bottge’s grant team.

This experience was especially important to me because it was my first try at conducting my own research with a prescribed protocol, which I had learned from working on the IES project. Posttests showed statistically significant improvement of students in the EAI group in both computation and problem solving. Based on these results, the sponsor invited me to participate in a panel discussion in Raleigh, NC. The CEOs of the company attended the event along with policy makers and school administrators from across the state. This whole process, from applying for funding to carrying out the study to reporting the results, helped me make connections between university, classroom, and community.

What have been your big takeaways from these experiences?

From the training I received as a study participant, I have become a better teacher.  From working on an IES-funded grant team, I learned a lot about how to conduct classroom-based studies. I am looking forward to designing new instructional methods and testing their effectiveness. Similar to how my students learned math in a hands-on way, I learned research methods by having the opportunity to use them in practice, and for that I am very grateful. 

How to Develop Your Career in Education Research: IES Training Opportunities

By Corinne Alfeld, NCER Program Officer

In honor of career development month, we would like to remind you about training opportunities funded by IES. We have invested in training programs since 2004 with the aim of increasing the supply of scientists and researchers in education who are prepared to conduct rigorous education research that advances knowledge within the field and addresses issues important to education policymakers and practitioners. These efforts are intended to lead both to the training of talented education researchers from a variety of backgrounds and to the incorporation of diverse ideas and perspectives in education research.

In this blog we describe five types of training opportunities currently offered through the National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER) that span from undergraduate to the postdoctoral level and beyond.

Training Opportunities for Current or Future Doctoral Students
Are you a current or aspiring doctoral student wondering what training opportunities are available to you? You may be interested in applying to one of 10 training programs funded by NCER’s Predoctoral Interdisciplinary Research Training Program in the Education Sciences that train predoctoral fellows in interdisciplinary programs involving a number of academic disciplines (e.g., economics, education, psychology, public policy, sociology, and statistics, among others). These fellowships can be from 2 to 5 years in length depending on the training program model and typically include tuition and benefits, a $30,000 stipend, and a small research/travel fund. Fellows who complete their training program have the skills necessary to produce research that is rigorous in method as well as relevant and accessible to education stakeholders such as practitioners and policymakers.

If you are interested in becoming a predoctoral fellow, you must apply directly to one of the training programs, not to IES.  Each of the 10 fellowship programs funded in 2014-15 has its own application process and acceptance criteria. For more information on becoming a predoctoral fellow, check out this resource on Applying for a Predoctoral Interdisciplinary Research Training Program Fellowship.

 

Postdoctoral Training Opportunities
Are you finishing up your doctorate and wondering how you can get more experience in education research? Or perhaps you’re looking to return to academia through a postdoctoral position? If so, you may want to apply to one of the our programs funded under the Postdoctoral Research Training Program in the Education Sciences Program (NCER) or the Postdoctoral Research Training Program in Special Education Program (NCSER). Through these two grant programs, IES funds training programs at doctoral-granting institutions to further prepare researchers who have obtained their Ph.D.s or Ed.D.s to become scholars capable of conducting high-quality, independent education or special education research. These postdoctoral training programs provide practical, hands-on experiences; enrichment of theoretical and empirical knowledge; and opportunities for fellows to build professional skills and networks that will support working with other researchers and relevant education research stakeholders.

To inquire about postdoctoral fellowship openings, follow the hyperlinks in this section to search for currently (awarded in 2010-15) funded programs at various universities around the country. For example, here are the 2015 NCER-funded programs and 2012 NCSER-funded programs.

 

Other Upcoming Training Opportunities:

  • Undergraduate, Post-baccalaureate, and Master’s Students. If you are an upper-level undergraduate student, recent graduate, and/or master’s student, especially from a group that is underrepresented in doctoral study (including racial and ethnic minorities, first-generation college students, economically disadvantaged students, veterans, and students with disabilities), you may be interested in the Pathways to the Education Sciences Research Training Program (Pathways). Established in 2015, the Pathways program funds training programs at minority-serving institutions (MSIs) and institutions of higher education that partner with MSIs. These training programs will provide fellows with education research experience and professional development to prepare them to pursue doctoral study in the education sciences or in fields relevant to education research.

Up to five Pathways training programs will be awarded to MSIs (and their partners) in 2016. These new programs will begin recruiting fellows in 2016 and 2017, so keep your eyes and ears open for more information about where and how to apply!

 

  • Early Career Education Researchers. Are you an early career researcher at your first appointment? If so, you may qualify for and be interested in one of our programs that target early career researchers in statistical and research methodology (NCER) and special education (NCSER).

Look for upcoming training opportunities for early career researchers in future Request for Applications for training grants (CFDA 84.305B and 84.324B) and statistical and research methodology grants (CFDA 84.305D).

 

  • Active Researchers Looking to Improve Their Methodological Expertise. Are you a current researcher (e.g., at a university or research firm) who would like to add tools to your methodological toolkit or further refine your skills with such tools? If so, then the Methods Training for Education Researchers Program may be for you.

If you are interested in methodological training, sign up for the IES Newsflash for announcements of upcoming workshops or periodically check our list of IES-funded workshops.

 

If you have questions about our training programs, please contact Corinne.Alfeld@ed.gov.