Inside IES Research

Notes from NCER & NCSER

Celebrating the Launch of the Registry of Efficacy and Effectiveness Studies (REES)

The Registry of Efficacy and Effectiveness Studies (REES) is now ready for use! REES is a registry of causal impact studies in education developed with a grant from the National Center for Education Research (NCER). REES will increase transparency, improve the replicability of studies, and provide easy access to information about completed and ongoing studies.

The release of REES aligns with recent IES efforts to promote study registration. In the FY 2019 Requests for Applications (RFAs) for the Education Research and Special Education Research Grants Programs, IES recommended that applicants describe a plan for pre-registering their studies in both the project narrative (as part of the research) as well as the data management plan. IES is also developing the Standards for Excellence in Education Research (SEER) and has identified study registration as an important dimension of high value education research.

We asked the REES team to tell us more about how the registry works.

How can researchers access REES? REES can be accessed through the SREE website at www.sreereg.org. Over the next year, REES will transition to a permanent home at the Inter-university Consortium for Political and Social Research (ICPSR) at the University of Michigan, but it will still be accessible through the SREE link.

Screenshot of REESWhat kinds of studies can be registered? REES is a registry of causal impact studies. It accommodates a range of study designs including randomized controlled trials, quasi-experimental designs, regression discontinuity designs, and single case designs.

What information should be included in a study entry? A REES entry includes basic study information and a pre-analysis plan. The checklist of required information for a registry entry provides detailed information for each of the different design options. All of the information for a REES entry should be easily found in a grant application.

How long does it take to register a study? For a study with a complete grant application, completing a REES entry should be straightforward and take approximately one hour.

What if a study entry needs to be changed? Principal investigators (PI) or other authorized research team members should update a REES entry as changes occur. All updates to an entry will be time-stamped. Original entries and updated entries will be publically available.

Are registered studies searchable by the public? Yes! When a PI or authorized research team member is ready to make the study available in the public domain, they click on the publish option. This will time stamp the entry and make it publically available. REES entries that are published are available on the search page. A pdf of individual entries can be downloaded from the search page or an Excel file of multiple entries can be exported.

What will happen to studies that were entered in the pilot phase of REES? A REES entry that was started and/or completed during the pilot phase is a part of the REES database. To make the study publically available and a part of the searchable database, the PI or other authorized research team member needs to click on the publish option for the entry.

Over the next two years, the REES team will be working to ensure the sustainability and visibility of REES with a grant from the National Center for Special Education Research (NCSER). To do this, the team will transfer REES to its permanent location on the ICPSR website and disseminate information about REES within the education research community, as well as with funders, publishers, and users of education research, through meetings, conferences, websites, social media, and targeted outreach.

So, what are you waiting for? Go check it out!

If you have questions about REES, please email contact@sreereg.org.

 

 

Recognizing Our Outstanding IES Predoctoral Fellows

Each year, the Institute of Education Sciences (IES) recognizes an outstanding fellow from our Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. This year, IES has selected joint recipients for the 2017 award: Rachel Abenavoli and Callie Little. They will receive their awards and present their research at the annual IES Principal Investigators meeting in Washington, D.C. in January 2019.

Dr. Abenavoli received her doctorate in Human Development and Family Studies from Pennsylvania State University (Penn State). She is currently a postdoctoral research scientist at New York University’s Steinhardt’s Institute of Human Development and Social Change, and is working in the area of early learning and social-emotional development.  Dr. Little received her doctorate in Developmental Psychology from Florida State University (FSU). She is currently a research fellow in the School of Behavioural, Cognitive and Social Sciences at the University of New England in Australia, where she specializes in understanding the development of cognitive and academic skills for individuals from early childhood through college.

We asked our Outstanding Fellows how participating in an IES-funded predoctoral training program helped their development as researchers.

Rachel Abenavoli

The IES-funded predoctoral training program at Penn State provided me with the resources, opportunities, and skills needed to begin building my own program of education research. I began my graduate program with broad interests in young childrenRachel Abenavoli’s social-emotional functioning. The fellowship helped me shape and narrow the focus of my research. By participating in regular seminars, attending talks, and meeting with external invited speakers, I began applying my developmental lens and focus on early social-emotional skills to the study of educational contexts and educationally-relevant outcomes. With generous fellowship funding that gave me the freedom and flexibility to pursue my own research questions, I was able to focus my work in graduate school on the interplay between children’s social-emotional and academic skills as they make the transition to school.

The IES fellowship also provided me with opportunities to learn, practice, and hone my methodological skills. Courses and seminars in program evaluation, causal inference, and multilevel modeling were particularly critical in building my capacity for conducting rigorous school-based research. Invited speakers complemented these core learning experiences by highlighting best practices and innovative approaches in education science. I came away from this training equipped with a range of analytic and methodological tools that are necessary to address the diversity and complexity of education research questions, settings, and designs.

Being an IES fellow has also connected me to a network of established and early career education researchers. Regular meetings with faculty mentors and other IES fellows from different home departments provided a space to discuss new ideas, experiences in the field, and possible collaborations. Conference funding enabled me to attend education research conferences and expand my professional network beyond Penn State. Engaging with this community solidified for me the value of a collaborative and interdisciplinary approach to understanding children’s learning and development in context.

My work is more rigorous and more relevant to education practice and policy because of the IES predoctoral fellowship, and I’m so grateful for the experiences, mentors, and other fellows who are critical to the success of the program at Penn State.

Callie Little

First and foremost, the support and training I received through the IES pre-doctoral fellowship at FSU provided me with an intense and philosophical appreciation of construct validity: whether an investigation can accurately measure what it was designeCallie Littled to measure. This appreciation will continue to guide how I develop and implement research studies.

The rigorous statistical courses included in FSU’s core curriculum provided a solid foundation for building my quantitative skills. Additionally, the combined focus on study-design and methodological training supplied me with a comprehensive knowledge base and the skills to investigate the complex associations among reading skills, and between reading skills and behavioral outcomes. The exposure to multiple advanced statistical methods coupled with the opportunity to directly apply these methods to relevant data so early in my career prepared me for my current projects which use large-scale twin data to conduct high-quality research on individual differences in the development of cognitive and academic skills.

The multidisciplinary environment at FSU, with its rich and diverse range of research programs and faculty, exposed me to a series of analytic techniques and content-area expertise that helped to shape an open-minded and creative approach to formulating research questions. This unique environment was one of the greatest advantages to the IES fellowship, providing the opportunity for strong mentorship, collaboration, and feedback. Most importantly, I used this opportunity to develop ongoing projects with colleagues where we innovatively combine evidence-based methods from several fields.

Fellowship funding enabled me to attend conferences and workshops with other IES fellowship teams, and provided me with access to research resources. I gained new insights into science communication, learned new techniques, and broadened my network of collaborators. I was able to recruit participants and purchase standardized assessment materials, and design and implement several of my own studies during my Ph.D. The resulting rich and unique data sets form the foundation of my current independent research.  From my experiences through the IES fellowship at FSU, I stand well-prepared to continue to conduct innovative and high-quality research into the complex mechanisms underlying achievement.

Katina Stapleton is the program officer for the Predoctoral Interdisciplinary Research Training Programs in the Education Sciences.

Recognizing Our Outstanding IES Fellows

Each year, the Institute of Education Sciences recognizes some of its fellows for their academic accomplishments and contributions to education research. This year, IES has selected Rachel Baker as the 2016 Outstanding Fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences.

Dr. Baker (pictured right) received her doctorate in Higher Education Policy and the Economics of Education from the Stanford Graduate School of Education. She is currently an Assistant Professor of Educational Policy at the University of California, Irvine, where she studies inequalities in postsecondary access and success using behavioral economic models of decision making and quasi-experimental and experimental methods. Dr. Baker will receive her award and present her research at the annual IES Principal Investigators meeting in Washington, D.C. in January 2018.

For the first time this year, IES is also recognizing two finalists for the outstanding fellow award, Dr. Elizabeth Tighe and Dr. Karrie E. Godwin.

Dr. Tighe received her doctorate in Cognitive Psychology from Florida State University. She is currently an Assistant Professor of Educational Psychology at Georgia State University where she focuses on advancing our understanding of the literacy skills and instructional needs of struggling adult readers who attend Adult Basic and Secondary Education programs.

Dr. Godwin received her doctorate in Developmental Psychology from Carnegie Mellon University. She is currently an Assistant Professor of Educational Psychology in the School of Lifespan Development and Educational Sciences at Kent State University. Her research examines how cognitive and environmental factors shape children’s development and learning in the laboratory and in the classroom.

We asked all three awardees how participating in an IES predoctoral training program helped their development as researchers.  For more information about the IES predoctoral training program, visit our website.

Rachel Baker, Fellow in Stanford University Predoctoral Training Program in Quantitative Education Policy Analysis

With full acknowledgement that it is impossible to know for certain how my development as a researcher has been shaped by participating in the IES pre-doctoral fellowship (where’s the counterfactual?), I can point to three factors that I think have been critical:  (1) the community of Stanford’s IES pre-doctoral fellows and  associated faculty, (2) the tightly structured curriculum and frequent opportunities to engage with high quality research, and (3) the freedom, within this structure, to engage with my own research questions.

From my first day at the Center for Education Policy Analysis (CEPA) at Stanford’s Graduate School of Education, I was exposed to a pervasive culture of intellectual rigor and the pursuit of high-quality research. But this culture of exactitude was paired with diversity of thought, a true investment in the ideas and work of students, and a sense of collegiality and collaboration. Every day, I felt both challenged and supported by the Stanford IES group. This professional network has been essential to my growth.

CEPA’s core curriculum prepared me well to conduct high-quality, policy-focused research. In particular, the classes on designing and implementing quasi-experimental studies has influenced my work tremendously – from the large, obvious ways, such as how I conceptualize and design research, to the small details of implementation that can make or break a study. The series of required classes was complemented by frequent, less formal engagement with the practice of research in the form of weekly seminars in which students presented work in progress and an unparalleled seminar series with speakers from other institutions.

But within this tight community and academic structure, a real benefit of the IES fellowship was my ability to engage with the research questions that I was most interested in. The financial security of the fellowship meant that I could work on projects, directed by faculty or of my own design, that I thought were timely, important, and interesting. In my five years, I worked closely with four CEPA faculty members, each of whom influenced the way I ask and answer questions in essential and unique ways.

I am grateful to the IES pre-doctoral fellowship, and especially the Stanford IES group, for the five years of opportunities, resources, and professional community.

Elizabeth Tighe, Fellow in Florida State University Program to Increase Research Capacity in Educational Science

The IES predoctoral training fellowship provided multiple avenues for me to work with interdisciplinary research teams and take courses that developed my quantitative skills within the realm of education research. Three benefits in particular were integral in shaping my development as a researcher: rigorous quantitative training; generous financial support for designing and implementing my own studies as well as providing networking opportunities at conferences; and the interdisciplinary and collaborative nature of the coursework and research.

For my graduate studies, I chose Florida State University for the innovative and interdisciplinary research conducted through the Florida Center for Reading Research. By assisting on various projects, I honed my quantitative skills and learned new theoretical perspectives from multiple disciplines. For example, I gained experience with eye-tracking equipment, assessment and measurement of reading-related constructs, and evaluation of classroom curricular materials. The flexibility of the fellowship allowed me to develop my own program of research, which focused on struggling adult readers enrolled in adult literacy programs. In addition, I utilized my quantitative skills on large-scale, existing datasets of students enrolled in K-12 education. These experiences provided ample opportunities to bridge my interests in adult literacy and quantitative methodology and to publish and present at different outlets.    

The financial support for conferences afforded me extensive networking opportunities with colleagues. This helped me to establish grant-writing collaborations, provide statistical consulting for projects, participate in cross-university symposia, and form professional friendships from which I continue to reap benefits today. I attended practitioner-oriented conferences in adult education and a training workshop for using a large-scale dataset on adults’ literacy, numeracy, and digital problem-solving skills. In conjunction with a colleague at the University of Iowa, my current lab received a grant to use this large-scale dataset to examine the literacy skills of United States inmates. 

My accrued training, experiences, and interdisciplinary collaborations are directly applicable to my current role as an Assistant Professor of Psychology and Assistant Director of the Adult Literacy Research Center (ALRC) at Georgia State University. I work alongside an interdisciplinary team of scholars within the Language and Literacy Initiative, which affords me the opportunity to work with professors and to mentor graduate students in multiple departments. My lab is currently is working with the ALRC and Applied Linguistics Department to investigate the comprehension monitoring skills of struggling adult readers using eye-tracking equipment. As a result of my IES training, it was imperative for me to find a position that supported my multidisciplinary research interests.

Karrie E. Godwin, Assistant Professor, Kent State University, Fellow in Carnegie Mellon Program in Interdisciplinary Education Research (PIER)

My participation as an IES predoctoral fellow in the PIER program at Carnegie Mellon University was seminal in guiding my development as a researcher. For me, there are three aspects of the program which were especially formative.

First, I was fortunate to be surrounded by thoughtful, caring, and passionate mentors who embody the goals of the PIER program. I am especially grateful to Anna Fisher, David Klahr, and Sharon Carver, who provided incredible guidance, advanced my critical thinking skills, taught me the importance of good experimental design and research hygiene, and how to effectively disseminate my work—not only to other scientists, but also to practitioners and stakeholders.

Second, PIER is committed to training scholars in rigorous research methodology that bridges theory and practice. My training has enabled me to conduct disciplined research in which I examine children’s cognitive development and the underlying mechanisms of change which inform educational practices by identifying causal and malleable factors that can be leveraged to promote better learning outcomes for children.

Lastly, PIER exposed me to a diverse set of scholars affording unique and dynamic opportunities for interdisciplinary research collaborations that would have been highly improbable in more traditional and siloed environments. For example, in addition to working with other psychologists, I formed productive collaborations with colleagues in robotics, human computer interaction, and statistics. This was a unique and powerful component of the program. Additionally, this feature of the program encouraged and developed my communication skills to groups and communities outside of my specific domain where traditional jargon is typically ineffective. This skill has been incredibly important in helping to communicate my research to other scientists from different disciplines but also to practitioners and the media.

In my new role as an assistant professor at Kent State University, I am drawing upon my experience in PIER and using the skills I gained during my fellowship to build my program of research. I am continuing to investigate children’s cognitive development in order to create more optimal learning environments and instructional materials that aim to enhance children’s learning outcomes. In addition, my new position allows me to help train students to become producers of high-quality research and to help future educational practitioners be thoughtful consumers of research. I am certain the skills I gained as an IES fellow at Carnegie Mellon will enable me to fulfill my commitment to improve children's learning outcomes through disciplined research.

Compiled by Katina Stapleton, National Center for Education Research