IES Blog

Institute of Education Sciences

Teach a Researcher to Fish: Training to Build Capacity for IES Data Analysis

The Institute of Education Sciences is pleased to announce upcoming training opportunities to help researchers study the state of adult skills and competencies. Training Researchers to Use PIAAC to Further Multidisciplinary Research is a hands-on, interactive training to build the field’s capacity for conducting research using data from the OECD Program for the International Assessment of Adult Competencies (PIAAC).

Picture of students participating in trainingThe training, conducted by the Educational Testing Service (ETS), aims to teach researchers how to use IES data and data tools for further, independent research beyond the training so that they can meet the emerging needs of policymakers and practitioners needs for years to come.

This program is an example of the various ways that IES is building the evidence base in education. The training is supported by a Methods Research Training grant from the National Center for Education Research. It uses PIAAC data, which in the U.S. were collected by the National Center for Education Statistics (NCES). The training also uses data tools that are available through NCES.

Beginning this August, ETS is holding 3-day and 1-day PIAAC trainings in cities throughout the U.S. These trainings will bring together researchers from various organizations and institutions to learn not only about the data and tools but also about how to use them to address important questions about policy-related research from a wide host of fields including education, gerontology, sociology, public health, economics, workforce development, and criminal justice and corrections education. These trainings will culminate with an IES/ETS-sponsored conference in Washington, D.C. in December 2018, during which participants will have an opportunity to present their research.

Who is Eligible?

Researchers from universities, research firms, or other organizations (e.g., advocacy groups, local governments) and who have a doctoral degree or are graduate students in a doctoral programs, experience with statistical packages (e.g., SAS, SPSS) and with secondary data analysis, and an interest in adult learning, skills, and competencies.

What Does it Cost?

The training itself is free for participants, and participants who are U.S. citizens or U.S. permanent residents will receive assistance to cover housing and per diem during the training. Visit the training website for more information about possible finical assistance.

When is the Training? How do I Apply?

The training will take place several times in the coming months:

  • August 30-Sept. 1, 2017 in Chicago;
  • October 2-4, 2017 in Atlanta; 
  • December 4-6, 2017 in Houston;
  • April 13, 2018 in New York City (at the AERA Annual Conference)
  • Culminating Conference: December 1-3, 2018, in Washington, DC

Visit the ETS training website for more information about the program and the most up-to-date schedule. Registration is open and can be completed online.

Written by Meredith Larson, Program Officer, National Center for Education Research

 

Recognizing Our Outstanding Predoctoral Fellows

Each year, the Institute of Education Sciences (IES) recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. For the first time, IES has selected joint recipients for the 2015 award: Meghan McCormick and Eric Taylor. They will receive their awards and present their research at the annual IES Principal Investigators meeting in Washington, D.C. in December 2016.

Meghan completed her Ph.D. in Applied Psychology at New York University and wrote her dissertation on the efficacy of INSIGHTS, a social emotional learning intervention aimed at improving low-income urban students’ academic achievement. She is currently a research associate at MDRC. Eric completed his PhD in the Economics of Education from Stanford University and wrote his dissertation on the contributions of the quality and quantity of classroom instruction to student learning. Eric is currently an assistant professor at the Harvard University Graduate School of Education.

We asked Meghan and Eric how participating in an IES predoctoral training program helped their development as researchers.  For more information about the IES predoctoral training program, visit our website.

Meghan McCormick

Having the opportunity to be part of the IES Predoctoral Training Program helped me to develop a set of theoretical, quantitative, and practical skills that I would not have had the opportunity to develop otherwise. 

I was drawn to attend NYU (New York University) for my doctoral studies specifically because the school of education at NYU offered the (IES) predoctoral training program, in addition to hosting a core set of faculty with research interests very much aligned with my own. In my first year of graduate school, I quickly became aware that being a part of the IES program allowed me the freedom to study with an interdisciplinary set of scholars who could support multiple components of my training through a diverse set of experiences.

For example, in my work with Elise Cappella, Erin O’Connor, and Sandee McClowry, I was able to learn about the logistics of implementing a cluster-randomized trial across a broad set of schools, and conducting impact analyses to evaluate the efficacy of one social-emotional learning program called INSIGHTS. My experience working with Jim Kemple and Lori Nathanson at the Research Alliance for New York City schools showed me how to use research in a way that was responsive to the needs and goals of education policy makers. Quantitative coursework with Jennifer Hill and Sharon Weinberg helped me to apply rigorous quantitative methods to the data that were collected in schools, and to think concretely about the implications of research design for my future work. Coursework with developmental psychologists conducting policy-relevant research, such as Pamela Morris and Larry Aber, helped me to apply comprehensive theoretical framing when examining research questions of interest, and interpreting results. In addition, I have always been primarily interested in conducting interdisciplinary research that is responsive to policy and practice. My dissertation research grew out of my interest in learning about interdisciplinary methods for causal inference and applying them to research questions I had about how, for whom, and under what circumstances the social-emotional learning program I helped to evaluate effected outcomes for low-income students.

Most importantly, perhaps, having been part of the IES program’s collaborative and interdisciplinary community helped me to identify the type of research I wanted to do after finishing graduate school. Primarily, I knew that I wanted to conduct policy-relevant research, using the most rigorous quantitative methods available, with a team of researchers coming from different backgrounds. This realization led me to work at MDRC, where I have been working with JoAnn Hsueh and other colleagues to apply my skills from the predoctoral training program in new research design work that is responsive to critical policy questions in early education policy and practice right now. I feel prepared for this new work given the opportunities that the IES program afforded me across the last five years. 

Eric Taylor

I would emphasize two benefits. First, the IES program at Stanford helped me create and strengthen professional relationships with other education researchers and practitioners. Those relationships provided important opportunities to learn skills in ways that could not happen in the classroom but also complemented the excellent classroom instruction. The new relationships were diverse: other graduate students in different disciplines, Stanford faculty and faculty at other institutions, and, critically, practitioners and policy makers. For example, supported by my fellowship, I joined faculty at (the University of) Michigan and Columbia (University) working with the DC Public Schools to improve teacher applicant screening and hiring.

Second, those relationships combined with the financial support of the fellowship made it possible to work on new and timely research projects. During my time as an IES predoc, with collaborators at Brown, we started a researcher-practitioner partnership with colleges at the Tennessee Department of Education. The resulting work has taught me much about the day-to-day realities of school policy making and management, and how research can and cannot help. The partnership with Tennessee also grew into a five-year grant from IES, which began last year, to study state policy and teacher development through evaluation.

In short, I am certain my career is much further along today than it would have been without the IES predoc fellowship.

By Katina Stapleton, Education Research Analyst, National Center for Education Research 

 

RCT-YES: Supporting a Culture of Research Use in Education

By Ruth Curran Neild, Delegated Director, IES

The mission of the Institute of Education Sciences (IES), at its core, is to create a culture in which independent, rigorous research and statistics are used to improve education. But sometimes research is seen by practitioners and policymakers as something that is done for them or to them, but not by them. And that’s something we’re hoping to change.

IES is always looking for new ways to involve educators in producing and learning about high-quality, useful research. We believe that if state and school district staff see themselves as full participants in scientific investigation, they will be more likely to make research a part of their routine practice. Simply put, we want to make it easier for educators to learn what works in their context and to contribute to the general knowledge of effective practices in education.    

That’s why we’re so pleased to add the RCT-YESTM software to the IES-funded toolkit of free, user-friendly resources for conducting research. Peter Schochet of Mathematica Policy Research, Inc. led the development of the software, as part of a contract with IES held by Decision Information Resources, Inc.

RCT-YES has a straightforward interface that allows the user to specify the analyses for data from a randomized controlled trial (RCT) or a quasi-experiment. Definitions and tips in the software help guide the user and accompanying documentation includes a mini-course on RCTs. When the user enters information about the data set and study design, RCT-YES produces a program to run the specified analyses (in either R or Stata) and provide a set of formatted tables.

The target users are those who have a basic knowledge of statistics and research design but do not have advanced training in conducting or analyzing data from impact studies. But we expect that even experienced researchers will like the simplicity and convenience of RCT-YES and benefit from some of its novel features, such as how it reports results.

When used properly, RCT-YES provides all of the statistics needed by the What Works ClearinghouseTM (WWC) to conduct a study review.  This is an important feature because the WWC often needs to contact authors—even experienced ones—to obtain additional statistics to make a determination of study quality.  RCT-YES could help advance the field by increasing the completeness of study reports.

Another unique feature of the software is that it defaults to practices recommended by IES’ National Center for Education Statistics for the protection of personally identifiable information. For example, the program suppresses reporting on small-size subgroups.

While the user sees only the simplicity of the interface, the underlying estimation methods and code required painstaking and sophisticated work.  RCT-YES relies on design-based estimation methods, and the development, articulation, peer review, and publication of this approach in the context of RCT-YES was the first careful step. Design-based methods make fewer assumptions about the statistical model than methods traditionally used in education (such as hierarchical linear modeling), making this approach especially appropriate for software designed with educators in mind.

The software is available for download from the RCT-YES website, where you can also find support videos, documentation, a user guide, and links to other helpful resources. The videos below, which are also hosted on the RCT-YES website, give a quick overview of the software.

There are many other ways that IES fosters a culture of research use in education. For instance, our 10 Regional Educational Laboratories (RELs) have research alliances that work with states and districts to develop research agendas. The RELs also host events to share best practices for putting research into action, such as the year-long series of webinars and training sessions on building, implementing, and effectively using Early Warning Systems to reduce dropping out.

IES also offers grants to states and districts to do quick evaluations of programs and policies that have been implemented in their schools. The low-cost, short-duration evaluations not only help the grantees discover what is working, but can help others who might use the same program or implement a similar policy. (We’ll announce the first round of grant recipients in the coming weeks).

Visit the IES website to learn more about our work. You can also stay on top of news and information from IES by following us on Facebook and Twitter

 

 

 

The Institute of Education Sciences at AERA

The American Educational Research Association (AERA) will hold its annual meeting April 8 through April 12 in Washington, D.C.—the largest educational research gathering in the nation. This will be a special meeting for AERA, as it is celebrating 100 years of advocating for the development and use of research in education. The program includes hundreds of sessions, including opportunities to learn about cutting edge education research and opportunities to broaden and deepen the field. 

About 30 sessions will feature staff from the Institute of Education Sciences (IES) discussing IES-funded research, evaluation, and statistics, as well as training and funding opportunities.

On Saturday, April 9, at 10:35 a.m., attendees will have a chance to meet the Institute’s leadership and hear about the areas of work that IES will be focusing on in the coming year. Speakers include Ruth Curran Neild, IES’ delegated director, and the leaders of the four centers in IES: Thomas Brock, commissioner of the National Center for Education Research (NCER); Peggy Carr, acting commissioner of the National Center for Educational Statistics (NCES); Joy Lesnick, acting commissioner of the National Center for Education Evaluation and Regional Assistance (NCEE), and Joan McLaughlin, commissioner of the National Center for Special Education Research (NCSER).

On Monday, April 11, at 9:45 a.m., attendees can speak to one of several IES staffers who will be available at the Research Funding Opportunities—Meet Your Program Officers session. Program officers from NCER, NCSER, and NCEE will be on hand to answer questions about programs and grant funding opportunities. Several IES representatives will also be on hand Monday afternoon, at 4:15 p.m. for the Federally Funded Data Resources: Opportunities for Research session to discuss the myriad datasets and resources that are available to researchers.

NCES staff will lead sessions and present on a variety of topics, from The Role of School Finance in the Pursuit of Equity (Saturday, 12:25 p.m.) to Understanding Federal Education Policies and Data about English Learners (Sunday, April 10, 8:15 a.m.) and what we can learn from the results of PIAAC, a survey of adult skills (also Sunday, 8:15 a.m.). Dr. Carr will be a part of several sessions, including one on Sunday morning (10:35 a.m.) about future directions for NCES longitudinal studies and another on Monday morning (10 a.m.) entitled Issues and Challenges in the Fair and Valid Assessment of Diverse Populations in the 21st Century

On Monday, at 11:45 a.m., you can also learn about an IES-supported tool, called RCT-YES, that is designed to reduce barriers to rigorous impact studies by simplifying estimation and reporting of study results (Dr. Lesnick will be among those presenting). And a team from the IES research centers (NCER/NCSER) will present Sunday morning (10:35 a.m.) on communication strategies for disseminating education research (which includes this blog!).

IES staff will also participate in a number of other roundtables and poster sessions. For instance, on Tuesday, April 12, at 8:15 a.m., grab a cup of coffee and attend the structured poster session with the Institute’s 10 Regional Educational Laboratories (RELs). This session will focus on building partnerships to improve data use in education.  REL work will also be featured at several other AERA sessions.  

Did you know that the National Library of Education (NLE) is a component of IES? On Friday and Monday afternoon, attendees will have a unique opportunity to go on a site visit to the library. You’ll learn about the library’s current and historical resources – including its collection of more than 20,000 textbooks dating from the mid-19th century. The Library offers information, statistical, and referral services to the Department of Education and other government agencies and institutions, and to the public.

If you are going to AERA, follow us on Twitter to learn more about our sessions and our work.  And if you are tweeting during one of our sessions, please include @IESResearch in your tweet. 

By Dana Tofig, Communications Director, IES

The PI Meeting in 140 Characters

By Wendy Wei, Program Assistant, National Center for Education Research

How can practitioners and policymakers apply education research to their everyday work if they never hear about it or do not understand it? Communicating and disseminating research findings plays an integral role in promoting the education sciences and advancing the field.

That is why we made communication and dissemination a major theme at the IES Principal Investigators’ Meeting held earlier this month (December 10-11). The two-day meeting in Washington, D.C., featured five sessions that focused on communications – ranging from data visualization techniques to effective dissemination strategies to hearing journalists’ perspectives on how to share scientific results with the general public.

There was a lot of talk about social media during the meeting and plenty of tweeting about the presentations. We used the Twitter hashtag, #IESPIMtg, to foster an ongoing conversation for meeting attendees and to share findings that emerged from sessions.  Any tweet that included #IESPIMtg was automatically pooled together, generating a live Twitter feed that was on display in the lobby throughout the meeting.

 You can see all of the #IESPImtg tweets online, but here are some highlights:

"There is a tremendous sense of urgency to bridge the gap between research and practice..." --John B King #IESPIMtg

— Leah Wisdom (@lifelnglearner) December 10, 2015

.@StanfordEd's Sean Reardon: Good partnership work can lead to new knowledge, change policy+practice, improve data quality #IESPIMtg

— Bill Penuel (@bpenuel) December 11, 2015

#IESPIMtg Practitioner partners play a critical role in making sense of data and analyses in RPPs.

— Jennifer Russell (@Jenn_L_Russell) December 10, 2015

And we can get a little bit meta now…communicating about how to communicate:

Hirsh-Pasek & Golinkoff urges researchers to create "'edible science' that is accessible, digestible and usable." #IESPIMtg

— Tomoko Wakabayashi (@twakabayashi264) December 10, 2015

Awesome presentation on #DataVisualization by @jschwabish: Show the data, reduce the clutter, stop distracting attention. #IESPIMtg

— Rudy Ruiz (@RudyRuiz_BMore) December 10, 2015

.@KavithaCardoza Explaining your research--Don't think of it as "dumbing down." Think of it as simplifying. #IESPIMtg

— Dana Tofig (@dtofig) December 11, 2015

And, of course, what's Twitter without a little fun? When we tweeted this picture...

The poster session is going strong. Principal investigators present findings from #iesfunded research. #IESPIMtg

— IES Research (@IESResearch) December 10, 2015

...Chris Magnuson, Director of Innovation for Live It, Learn It, posted this reply: 

@IESResearch careful...photo looks like it was taken on Death Star! May the force be with all grantees! #SBIR #IES

— Chris Magnuson (@cromagnuson) December 10, 2015

The National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER) have made a commitment to be active contributors in communicating with and engaging the general public in the exciting findings of NCER- and NCSER-funded work. Over the past few years, we have been active on Twitter (you can follow us @IESResearch), and this past year, we launched our blog (the very one you are reading!). These two platforms have provided us with an outlet to share research findings, provide updates about events and deadlines, and connect with audiences we otherwise might not reach.

For those of you who could not make the PI meeting, videos will be posted on the conference website in about a month. So stay tuned!

We hope you’ll continue the conversation started at the PI meeting by following us on Twitter at @IESResearch or sharing your thoughts with us at IESResearch@ed.gov.