Location
Institute of Education Sciences (IES) Board Room
80 F Street NW
Washington, DC 20001

Participants

National Board for Education Sciences (NBES) Members Present
Bridget Terry Long, Ph.D., Chair
Anthony S. Bryk, Ed.D.
David Chard, Ph.D.
Darryl J. Ford, Ph.D.
Adam Gamoran, Ph.D.
Robert Granger, Ed.D. (by phone)
Larry V. Hedges, Ph.D.
Susanna Loeb, Ph.D.
Judith Singer, Ph.D.
Robert A. Underwood, Ed.D.
Hirokazu Yoshikawa, Ph.D.

Ex-Officio Members Present
John Q. Easton, Ph.D., Director, IES, U.S. Department of Education (ED)
Thomas Brock, Ph.D., Commissioner, National Center for Education Research (NCER)
Sean P. "Jack" Buckley, Ph.D., Commissioner, National Center for Education Statistics (NCES)
Ruth Curran Neild, Ph.D., Commissioner, National Center for Education Evaluation and Regional Assistance (NCEE)
Joan Ferrini-Mundy, Ph.D., Assistant Director, National Science Foundation (NSF), Directorate for Education and Human Resources
Robert Kominski, Ph. D., Assistant Chief, Social and Demographic Statistics, Social, Economic, & Housing Statistics Division, Bureau of the Census, U.S. Department of Commerce
Peggy McCardle, Ph.D. M.P.H., Branch Chief, Child Development & Behavior Branch, Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), National Institutes of Health (NIH)
Deborah Speece, Ph.D., Commissioner, National Center for Special Education Research (NCSER)

NBES Staff
Ellie Pelaez, Designated Federal Official
Rebecca McGill-Wilkinson, Ph.D., Associate Research Scientist, NCER

ED and Other Federal Staff
Elizabeth Albro, NCER
Corinne Alfeld, IES
Lucas Alvarez‚ Government Accountability Office (GAO)
James Benson‚ IES
Sue Betka, IES
Nora Boretti‚ GAO
Lisa Bridges, IES
Karen Douglas, NCER
Erica Johnson, IES
Diana McCallum, IES
Brett Miller, NICHD, NIH
Audrey Pendleton, IES
Anne Ricciuti, IES
Allen Ruby, NCER
Scott Spicer‚ GAO
Bill Ward, IES

Invited Presenters
Carmel Martin‚ J.D., Assistant Secretary for Planning, Evaluation, and Policy Development, ED
Richard Laine‚ M.P.P., M.B.A., Education Division Director, National Governors Association Center for Best Practices
Marcia Sprague‚ J.D., Attorney, Office of General Counsel, ED

Members of the Public
Jean Gossman, Education Daily
Kim Hymes, Council for Exceptional Children
Carla Jacobs, Lewis-Burke Associates, LLC
Jim Kohlmoos, National Association of State Boards of Education
Sarah Mancoll, Society for Research in Child Development
Melissa Mayville, National Education Association
Deborah Mitchell, National Education Association
Kate Nielson, National Governors Association
LaTosha Plavnik, Consortium of Social Science Associations
Sarah Spreitzer, Lewis-Burke Associates, LLC
Gerald Sroufe, American Educational Research Association
Karen Studwell, American Psychological Association
Alison Thompson, Lewis-Burke Associates, LLC
John Waters, Knowledge Alliance

Meeting Summary

Call to Order, Chair's Remarks
Adam Gamoran, Ph.D., NBES Member

Dr. Gamoran called the meeting to order at 8:34 a.m. and called the roll. NBES members unanimously approved the minutes of the October 5, 2012, NBES meeting.

Swearing-In of New and Reappointed Members
John Q. Easton, Ph.D., IES Director

Dr. Easton swore in reappointed members Bridget Terry Long, Ph.D., Kris D. Gutierrez, Ph.D., and Robert A. Underwood, Ed.D., and new member Darryl J. Ford, Ph.D.

Chair's Remarks
Bridget Terry Long, Ph.D., NBES Chair

Dr. Long welcomed Dr. Ford to the Board. She congratulated Dr. Gamoran on being named the next president of the William T. Grant Foundation and Susanna Loeb, Ph.D., on being elected to the National Academy of Education. Dr. Long said that Deborah Loewenberg Ball, Ph.D., was named to the NSF board and had to resign from NBES; she thanked Dr. Ball for her service. Dr. Long noted that the Board is close to its full complement; two more candidates are under review. She pointed out that Monica Herk, Ph.D., former Executive Director of NBES, has moved on to another position, and she expressed gratitude for Dr. Herk's contributions to NBES. In the absence of an Executive Director, Rebecca McGill-Wilkinson, Ph.D., Associate Research Scientist for NCER, is assisting the NBES with research and crafting the meeting agenda. Dr. Long welcomed Thomas Brock, Ph.D., recently appointed Commissioner of NCER.

Dr. Long commented that she has organized her term around three themes on which the Board is continually attempting to move forward: (1) IES research funding; (2) impact through disseminating and scaling up promising practices; and (3) advocating for support and use of research in policymaking. In regards to research funding, Dr. Long reminded the board that they had discussed recruitment and training of peer reviewers in recent meetings. Recent meetings also including discussion of new IES research programs, and a new research program would be a topic later in the meeting. The second theme on impact and dissemination had been addressed in previous meetings by discussing the What Works Clearinghouse (WWC), the Regional Education Laboratories (RELs), and design-based implementation. Dr. Long said the Board would discuss the role of the researcher in dissemination later in the day. Dr. Long indicated that Amber Winkler, Ph.D., research director of the Thomas B. Fordham Institute, had planned to participate in the discussion on the role of the researcher in dissemination, but was unable to attend due to a family emergency. Regarding the theme of advocating for the support and use of research, Dr. Long reminded the Board that past meetings have included briefings about what has been going on with the Hill. Dr. Long indicated that today's discussion of the Common Core State Standards would include consideration of how research is being used to form the standards, determine implementation, and study impact.

Update: Recent Developments at IES
John Q. Easton, Ph.D., IES Director

Dr. Easton welcomed the Board members, particularly Dr. Ford, who brings a new perspective to the table. In addition to having nearly-full Board membership, IES now has four permanent commissioners in place. Dr. Easton welcomed Dr. Brock and thanked Dr. Albro for her superb work as Acting Commissioner of NCER for 1 year and a half while maintaining her responsibilities as Associate Commissioner. Dr. Easton also thanked Peggy McCardle, Ph.D. M.P.H., who is retiring from NICHD and is attending her last meeting as an ex-officio member of the Board. Brett Miller, Ph.D., of NICHD will join the Board at the next meeting. Dr. Easton also congratulated Dr. Gamoran on his new position.

Dr. Easton noted that ED and other Departments are operating under a continuing resolution through March, and the fiscal year 2013 budget is not fully available. If the sequester takes place on March 1, all federal agencies will be required to reduce their budgets further. Dr. Easton explained that IES conducts two grant competitions each year, reviewing and awarding one group in the fall and a second in the spring. Because of the uncertainty around federal funding available for awards, IES is deferring decision-making about round-one applications until round-two applications have been reviewed. Applicants have been notified of the delay. IES hopes to determine awards in April, pending a clearer picture of the status of federal funding. Dr. Easton stressed that IES aims to ensure equity across the competition; when the amount of money available for funding is known, IES will award whatever grants it can.

Dr. McCardle said NICHD is taking a similar approach in an effort to treat all of its applicants fairly. Dr. Easton explained that the IES currently allocates about $200 million to research, development, and dissemination, but some of that money is already committed to paying for multi-year grants underway. Only a small sliver of the budget will be available for new research. It was noted that the uncertainty affects decision-making at every level (e.g., hiring research staff). Dr. Easton said IES is discussing whether the delay will affect the timing of grant applications for the following year.

Dr. Easton said there appears to be interest in Congress about reauthorization of the Education Sciences Reform Act (ESRA), and The Aspen Institute held a 2-day event on the issue. Anthony S. Bryk, Ed.D., attended the event and said Congressional staff members from both parties participated. The general consensus was that the IES has had a tremendous impact on the field by bringing rigor to empirical inquiry in education, fostering the development of new research professionals in the field, creating a new infrastructure for education research, and establishing independent education research capacity within the federal government. Dr. Bryk said some discussion addressed whether IES could become more relevant—that is, could it better inform practice improvement in the field. There was, however, no clear resolution about the language of the legislation. In addition, some discussion addressed whether NCES should be a separate entity, but ultimately there was a sense that it should remain within IES.

Dr. Bryk said the meeting participants also discussed the RELs and the potential for overlap with the Comprehensive Technical Assistance Centers and the Content Centers. He made the case for ensuring that IES remains connected to practice through the RELs, but he anticipated there may be an effort to address possible overlap. Further discussion addressed the relationship of IES to other entities that support applied research, such as Advanced Research Projects Agency for Education (ARPA-ED), an advanced research effort within ED similar to the Defense Advanced Research Programs Agency (DARPA).

Dr. Bryk said no timeline for ESRA consideration was offered at the meeting, but he suggested NBES remain vigilant and express its perspective, because the issue is on the minds of members of Congress.

Commissioner Updates

National Center for Education Evaluation and Regional Assistance (NCEE)
Ruth Curran Neild, Ph.D., NCEE Commissioner

Dr. Neild said the honest and thoughtful comments from the October 2012 Board meeting about the WWC spurred conversations internally about the nature of the practice guides and options for making them more targeted and useful. In June, she hopes to present some new models to the Board for consideration. NCEE plans to announce a new 5-year contract award for the overall operation of WWC by the end of February. Dr. Neild explained that NCEE also awards several smaller contracts to support the WWC that bring more scholars and more new ideas into the process.

The Education Resources Information Center (ERIC) fulfills the NCEE requirement to host a searchable database of education references. Efforts are underway to include more grey (i.e., informally published) literature into ERIC, including information derived from WWC reviews. IES now has a new cloud-based server for ERIC, which means that the agency can host ERIC in-house and consider new ways that ERIC can support the IES mission. IES released a request for proposals (RFP) to redesign the ERIC website to highlight IES publications, better tag resources to distinguish IES publications and funded research, and improve usability and search capabilities.

The RELs will submit about 50 reports to be published this year, and various products are in the pipeline. NCEE is hosting a REL Professional Development Day on March 6 that includes sessions on writing, research development, and measurement.

NCEE has released several new reports, including an implementation evaluation of the School Improvement Grants. The agency is focusing on disseminating more information and doing so more quickly. Dr. Neild added that the results of 24 randomized, controlled trials have been published that represent work completed by the RELs from 2006 to 2011.

Discussion

Dr. Long noted that interested Board members may wish to attend the March 6 REL Professional Development Day. Dr. Gamoran praised the ambition of the new RELs, as evidenced by the number of publications under development. He also asked about the WWC's mechanisms for developing standards, especially given the fact that there are separate contracts to manage the K–12 division and the post-secondary division. He sees the development of standards as being one of the most important contributions of the WWC. Dr. Neild replied that the two new contractors have some overlap between their statistics teams (the teams of experts that consult on methodological issues). Additionally, both contractors are required to coordinate and collaborate with each other. Finally, the additional WWC staff recently hired will manage and monitor the collaboration and coordination between the contractors.

Dr. Yoshikawa suggested that the WWC consider a pre-K practice guide or review, as pre-K has recently become a hot political topic. Dr. Neild replied that the start of a new contract offers an ideal opportunity for the WWC to consider what new topics should be addressed. She explained the process of the What Works Clearinghouse: a topic will be opened and a lot of work will be done on that topic, and then the topic will be closed. Multiple topics are open at once. When the new contract begins in March, they will discuss which topics to open.

Dr. Singer suggested the two methods of dissemination (the WWC and ERIC) are disconnected. She said ERIC is basically a repository but that is not as useful as ERIC being a dissemination machine. She wondered if there are opportunities to build an infrastructure to capitalize on the timing of the new WWC contract and the forthcoming ERIC contract. She suggested that IES consider capitalizing on the current momentum to transform these resources into something better than Google's search engine, which most people use now to find information. IES can play a role in curating the available resources of both the WWC and ERIC. The medical community's widely used database, PubMed, established a standardized set of terms that facilitate meaningful searches; it could be a model for the future of ERIC in the 21st century. Dr. Neild said one step is to ensure that all IES products are included in ERIC and that ERIC offers links to recommended resources. She said ERIC can do better in collecting the grey literature.

Dr. Bryk noted that relevance was a central theme at The Aspen Institute's meeting. The WWC could be vulnerable to funding cuts if its products are not perceived as meaningful and up to date. IES should consider how its programs demonstrate its connections with the field and provide real (meaningful/useful) resources for the improvement of practice.

National Center for Education Research (NCER)
Thomas Brock, Ph.D., NCER Commissioner

NCER received more than 700 grant applications in 2012, mostly for its education research grant program. Among those, 74 applications were for the new Research-Practitioner Partnerships in Education Research program, which supports partnering around issues and problems of practice identified by the state and local education agencies. Although most of the proposed work involves secondary data analysis, the ultimate goal is to support new and existing partnerships in pursuing continuous improvement models. Dr. Brock called the response encouraging.

NCER is working actively with grantees to bring their research to fruition and disseminate their findings. In the past year, 15 NCER-funded studies met the standards for inclusion in the WWC. Of about 200 NCER-funded studies, 74 have met the WWC standards since 2007. NCER recognizes the value of disseminating all research results in ERIC, even if they do not yield positive findings. NCEE, NCER, and NCSER are discussing how to collaborate in making more of their funded research available. Dr. Brock said NCER has contracted with Westat to catalogue all NCER-funded studies and hopes to create an internal search engine that ultimately becomes open to the public.

Dr. Brock described the Reading for Understanding Initiative (RfU), which funds six research teams (five university-based teams and the Educational Testing Service [ETS]) to advance theories and develop interventions to improve reading comprehension from pre-K through grade 12. NCER serves as a hub to facilitate collaboration among a diverse, interdisciplinary group of researchers who meet three times a year to share findings. The five university-based teams are testing interventions to improve reading comprehension through a variety of curricula, supplemental materials, and professional development opportunities. Through RfU, ETS is developing a tool—the Global, Integrated, Scenario-Based Assessments (GISA)—that measures deep reading comprehension for students at all grade levels. A goal of this RfU initiative is to develop a framework and design principles for an integrated, multilevel assessment system that assesses reading components as well as global comprehension.

Discussion

Dr. Bryk, returning again to the theme of relevance, asked for more details on what individual practitioners gain from the RfU research and how the program can serve as a model for future efforts. Dr. Brock noted that the collaborative structure of the initiative was noteworthy and was something that IES was trying to promote in its grantmaking. Karen Douglas, Ph.D., added that the RfU grantees are already working closely with schools; teachers and administrators have been involved with the research from the beginning. Some research is already being disseminated among schools across the country. All of the grantee organizations have relationships with schools, Dr. Douglas said. The program will provide a lot of useful information about organizing large, interdisciplinary teams around a research goal, she added. Dr. Bryk noted that IES will likely be asked to demonstrate how a large research program focusing on a single topic helps those outside the research community do their work better.

National Center for Special Education Research (NCSER)
Deborah Speece, Ph.D., NCSER Commissioner

In November, NCSER, the National Institute on Disability and Rehabilitation Research (NIDRR), and the Office of Special Education Programs (OSEP), both part of ED's Office of Special Education and Rehabilitative Services (OSERS), convened a technical work group on improving outcomes for adolescents with disabilities. Both scholars and practitioners participated in the work group and their expertise represented the full range of disabilities from adolescents through adulthood; they discussed which components of a school model could be tested to improve outcomes for adolescents, as well as which methodology would be most appropriate. The audience included staff from the Departments of Labor and Justice and the Office of Management and Budget, among others. NCSER continues to work in collaboration with NIDRR and OSEP to develop ideas for research on improving outcomes for the most vulnerable members of this group: adolescents with emotional and behavioral disorders.

NCSER released Mark Lipsey's report on making research more understandable, and it will be presented at the March 6 principal investigators (PIs) meeting. NCSER also published an online video training series to help researchers learn about accessing and using data from the Pre-Elementary Education Longitudinal Study. NCSER will again convene the Single-Case Design Summer Institute, hosted by the University of Wisconsin, and recently published video training modules from the 2011 Institute.

NCSER and NCER staff are working together to organize the March 2013 PIs meeting, which this year will encourage more information-sharing among PIs. Instead of poster presentations, an exhibition area will allow PIs to present their findings using laptops. In addition, PIs will be encouraged to share products they've developed, such as software, curricula, or tests. Dr. Speece said the meeting also will include seminars and sessions as usual.

National Center for Education Statistics (NCES)
Sean "Jack" Buckley, Ph.D., NCES Commissioner

The National Assessment of Education Programs' (NAEP's) latest assessment, Technology and Engineering Literacy (TEL), aims to address Technology and Engineering, the components often overlooked by STEM (science, technology, engineering, and mathematics) assessment efforts that traditionally focus on science and mathematics. The National Assessment Governing Board (NAGB) has been working on measuring TEL since the mid-2000s, following recommendations from the National Research Council and the National Academy of Engineering. NAEP has since partnered with numerous organizations (the American Association for the Advancement of Science, the National Institute of Standards and Technology, the National Aeronautics and Space Administration, the NSF, the high-tech sector, etc.) and gathered input from the field to determine what the assessment should cover. Dr. Buckley said NAEP already assesses science achievement; the complexity lies in differentiating, for example, between ideal scientific solutions and real-life engineering solutions, including the tradeoffs made to ensure that an engineering solution works.

The TEL defines technology as "any modification of the natural world done to fulfill human needs or desires," and engineering as "a systematic and often iterative approach to designing objects, processes, and systems to meet human needs and wants." Literacy is defined as the capacity to use, understand, and evaluate technology, as well as understand technology principles and strategies around engineering and technology more broadly. The TEL measures literacy in three categories: technology and society, design and systems, and information and communication technology. Within these three categories, the TEL looks at students' ability to understand technological principles, develop solutions and achieve goals, and communicate and collaborate. Dr. Buckley pointed out that other large standardized tests have assessed collaboration, and such measures are difficult to put in place. Practical problems include, for example, determining whether the elegance of the solution matters—that is, is achieving the goal through "brute force" a good solution?

Dr. Buckley presented to the Board a video created by NCES to explain the new assessment. The video clearly describes what the assessment covers and gives examples. It emphasizes the goal of learning whether students have the skills needed to address the challenges of our evolving society. Dr. Buckley noted that the computer-based assessment tests students' higher order thinking and ability to communicate their understanding of results (e.g., of research) in an informative way. NCES is pilot-testing the TEL among eighth-graders now. About 20 percent of the test questions follow traditional formats (e.g., multiple choice) and 80 percent of the test is scenario-based (the pilot scenarios are 10, 20, and 30 minutes long). An online tutorial (on redesigning a toaster) allows users to get a sense of the test; the theme of the tutorial highlights understanding the lifecycle of a product.

NAEP will also take a new approach to reporting results with the TEL, using both composite scores and scores by content area. The NAGB will develop achievement levels to help put these scale scores in context. The TEL assessment will also include student achievement results by practices (i.e., understanding technological principles, developing solutions and achieving goals, and communicating and collaborating). With more complex tasks, there is more interest in tracing the processes used by students to determine, for example, whether the characteristics of a solution are as important as the solutions themselves. Dr. Buckley pointed out that a recent computer-based writing test collected a lot of data about how students used the computer (e.g., editing their work, using spellcheck) but did not have a clear framework to determine what constitutes an "elegant" use of word processing. NCES hopes to look at and evaluate the processes used by students taking the TEL.

NAEP is on track to begin data collection with the TEL in January 2014, and the first results will be reported about 1 year later. Because the test items are so complicated, NAEP is piloting its use among 15,000 students, so that it has a very large sample to assess. (By comparison, the operational data collection for a single grade will have a sample size of about 20,000 students.)

Discussion

Dr. Gamoran commented that the new assessment may provide some insight into the effects of the digital divide, if they exist. Dr. Buckley said large gaps in access to technology would increase the difficulty of collecting data; however, he was optimistic, because collection of computer-based writing and math data among students in earlier grades indicates that some technology is so pervasive that virtually all students use it. Dr. Gutierrez noted that The Pew Research Center surveys find that students may have access to technology in schools, but few low-income students have computers at home. She added that those who do have computers at home rarely have broadband service. However, smartphones are increasingly available in these homes.

Dr. Gamoran asked whether the assessments are related to what is actually happening in schools. Dr. Buckley responded that it is not entirely clear whether schools are teaching the kind of technology and engineering approaches that the TEL measures. The International Baccalaureate curriculum includes design and technology in science starting in grade 6. Schools cover some engineering in their science classes, and information and communication technology comes up frequently in classwork. Dr. Buckley hoped that the assessment would be sensitive to instruction, but it may measure situational or general intelligence—that is, understanding problems not seen before. He added that for most NAEP items, there is limited work assessing their validity with respect to sensitivity to instruction.

Dr. Loeb asked whether there are underlying skills and knowledge the TEL attempts to assess. Dr. Buckley explained that in developing the framework, the NAGB develops detailed lists of what to measure. However, the topics tend to be broad, making it challenging to assess. Computer-based tests cover more of the framework than previous tests and allow for use of new types of test items. Dr. Buckley said the success of the new approach will not be known until results are available, but newer test designs are better than older designs.

Dr. Ford asked whether the culture of the schools has been considered in designing the TEL. Dr. Buckley answered that the TEL includes a detailed survey of students and teachers to understand students' access to technology at home and the use of technology by teachers. It may be possible to distinguish those schools where technology is infused into content across the whole school experience from those that have designated technology teachers. The results of the survey will inform redesign of the assessment. The video about the TEL could be a good model for communicating the content of WWC practice guides to practitioners.

New IES Research Programs
John Q. Easton, Ph.D., Director, IES

Dr. Easton said relevance and usability are clear themes of interest to the Board and IES, but it is difficult to judge whether our work actually has an impact on practice. In his experience, fostering partnerships among researchers, practitioners, and policymakers is one mechanism to ensure that research findings make an impact. IES is encouraging partnerships in its Research-Practitioner Partnerships in the Education Research grant program; next year it hopes to encourage exploratory research and further development of partner relationships. The same concept is being applied in the RELs and their research alliances. Where promising practices have been identified, the new grant program fosters close collaboration around questions of successful implementation, scaling up, adapting findings to individual settings, and making continuous improvements. The model is new, said Dr. Easton, so IES needs new ways to think about measuring outcomes.

IES believes that its impact should be judged on the uptake of effective practices, tools, etc., but little is known about where such uptake occurs, under what conditions, by whom, or how. Therefore, IES proposes to create a research and development (R&D) center focused on research use to encourage systematic inquiry into the application of research findings and evidence in education decision-making. The W. T. Grant Foundation offers some small grants on the topic, but as the federal home of education research, IES wants to be the locus of such research. Grantees of the center would not be required to study IES-funded research entities (e.g., RELs, RfU grantees, research alliances), but it is hoped they would. The goal, in part, is to evaluate IES' work alongside that of others, said Dr. Easton. Plans are not firm, but Dr. Easton envisioned a center smaller than the Institute's typical R&D centers that conduct case studies or smaller scale analyses of existing data to lay the groundwork for funding of larger efforts to test models, for example.

Dr. Easton said an R&D center on research use corresponds with questions about relevance and usability. We want our investments in research to make a difference, he said, resulting in better schools, stronger organizations, and better systems. Such a center would be part of IES' broader mission and provide a better understanding of how to build capacity to support the goals of research investment. He asked for Board input.

Discussion

Numerous Board members expressed enthusiasm for this new research program, and made suggestions regarding the focus of the program. Dr. Loeb noted that IES should consider what constitutes "relevance" in both the short term and medium term. She added it would be helpful to know whether research has identified interventions that work and, if so, whether those interventions are implemented. To better understand researcher/practitioner collaborations, it is useful to identify the specific issues that affect uptake or the success of interventions, such as from where they receive their information and why they are making decisions. Dr. Loeb added that IES should cast a wide net and capture the theory of action behind research, and should also seek to learn where decision-makers get the data that inform their choices.

Dr. Gamoran reminded the Board that research for practice differs from research for policy, and that IES may want to think about which to emphasize or at least push grant applicants to recognize and consider the distinction. In the context of an R&D center on research, Dr. Gamoran suggested that it would be better for IES if grantees did not focus on evaluating IES' own research and that IES should not encourage them to do so. By focusing only on the RELs, for example, grantees could overlook important information from other sources and could limit their focus to questions in which IES is not interested.

Dr. Granger indicated the W. T. Grant Foundation has focused their work on this topic on the practitioner side of the translation of research into practice; as a result, it has focused on understanding how decision-making and policies shape behavior. This includes understanding the decision-makers themselves, how organizational and policy settings shape their behavior, and the kind of information they need. He commented that because there is so little data about the use of research, anything IES does will help. Additionally, he suggested focusing on practice rather than policy.

Dr. Gutierrez recommended future calls for research should pay attention to equity not just in the language of proposals but in the actual research design and implementation. Dr. Gutierrez had a few suggestions for making sure equity is appropriately handled: (1) applicants should be required to provide evidence of need in the context of the proposed intervention as well as evidence of potential impact; (2) applicants should have a history of involvement with the target community that demonstrates they understand the community's needs; (3) a lot of RFAs address fidelity of implementation, but IES should also encourage productive adaptation of interventions; and (4) the issue of opportunity as an outcome should be built into discussions. Dr. Easton noted that IES' new topic, Continuous Improvement Research in Education, addresses adaptation.

Dr. Singer stated that much literature from the past 50 years focuses on the disconnect between federal-level decision-making and real-world implementation, because people are not rational actors. Dr. Singer also suggested IES should create a framework for setting priorities; otherwise, the results of focusing on research use will be so vast they provide no information. Such a framework should consider the levers of interest—e.g., the chief state school officers, superintendents, teachers—and the forces that affect them—e.g., private-sector developers of curricula, textbooks, tests, and other materials. Dr. Singer recommended IES not focus on all of these groups and forces as it would be too many to provide detailed information on any individual group or force. She suggested that IES consider with whom it can partner and where it can have more influence.

Dr. Yoshikawa recommended all of the four topics put forth by IES in the background materials (the Research-Practitioner Partnerships in Education Research grant program, the R&D center for research use, the Continuous Improvement Research in Education topic, and the Evaluation of State and Local Programs and Policies grants program) should include a focus on measurement, including timescales. Dr. Yoshikawa suggested IES could establish measurement criteria, and should set measurement priorities because this is a huge area of research. He also said it may be useful to include a measure(s) that crosses all four topics.

Dr. Long agreed with and reiterated Dr. Singer's point that IES will need to set priorities for this research. She also emphasized the importance of considering the educational context and that generalizing findings from one context to another may be difficult. Dr. Long recommended that IES think about choosing reviewers to evaluate applications who have a clear understanding of educational context and particular partnerships. Dr. Long suggested other actors may be part of the forces that affect education decision-makers—e.g., bloggers and tweeters—and that IES should consider how to counter bad information that is widely available.

Dr. Chard said the WWC demonstrates that many gaps remain in education research, so it may not be the right time to tell people they should make better use of research that does not exist. Dr. Gamoran later countered that there are sufficient data in several domains, and the focus on research use could answer the questions of for whom and in what settings interventions are effective.

By way of analogy, Dr. McCardle noted that more consumers are demanding that doctors be better informed about research related to their field, but the reliability of available research can be questionable. The U.S. Food and Drug Administration requires a certain threshold of evidence before products can be used by humans, but the medical community does not always have sufficient research on which to make good recommendations for changing practice. The concept of evidence-based medicine has been promulgated for decades, but even when research is available, it is still not clear how to disseminate it, encourage its use, and assess the impact. Dr. McCardle indicated IES is the ideal organization to take on such a role for education.

Dr. Ferrini-Mundy said that the NSF created implementation centers that focus on translating research to assist teachers with implementing the evidence-based curricula that NSF created. However, these efforts at translation were not seen as a good use of funds and were cut. Dr. Ferrini-Mundy also said NSF is also concerned about the uptake of research findings. NSF has a new initiative called the Innovation Corps, which promotes translation of basic research into practical uses. The Innovation Corps approach is very prescriptive and includes a course on entrepreneurship to help researchers take their work beyond the laboratory. It could be a model for an ED program. Dr. McCardle suggested if such an approach were considered, IES should determine whether the Small Business Innovation Research (SBIR) program could fund it. Later in the discussion, Dr. Chard reiterated that IES should consider using SBIR funds because education researchers often do not consider the private market.

Dr. Bryk suggested the influence of research on practice is different from the influence of research on decision-making and policy. The proposal for the R&D center on research use seems to focus more on decision-making. Dr. Bryk recommended IES should broaden its focus to consider how organizations use evidence to change their internal culture.

Dr. Gamoran said that researchers are producing good data but they are not being used, so it is important to consider new approaches and to better understand the "client." Dr. Underwood countered that researchers are prone to respond to every problem by calling for yet more research, and the study of research use seems to move us further from the real issues of control, management, and motivation. Practitioners must be more involved in structuring research so that it meets their needs, not the aims of researchers. Professional development is key to helping practitioners incorporate research. Dr. Granger added that he hopes that the focus on research use would not produce more "navel-gazing" but rather would feed the notion that it is fundamental that any enterprise identify and serve its clients.

Dr. Hedges suggested it is important to get some perspective on what the reasonable effects of research might be, because unrealistically high expectations have harmed the field in the past. Again using the analogy of the health care system, Dr. Hedges said surveys indicate that despite new research about what works, health care workers are very slow to change practice.

There was consensus among the Board members that the focus on research use should be oriented more toward practice than policy. Because we do not know where teachers get their information, Dr. Singer suggested that IES look at other fields to determine the mechanisms that people use to find information and how they use it.

What Do the Common Core State Standards Mean for Education and IES?
Richard Laine‚ Education Division Director, National Governors Association Center for Best Practices
Carmel Martin‚ J.D., Assistant Secretary for Planning, Evaluation, and Policy Development, ED

Mr. Laine described the rationale behind the Common Core State Standards, which aim to streamline existing standards and increase their rigor. Ideally, states identify the end-goal—i.e., successful transition to college or workforce training—and work backwards, determining what students need to know at each grade level to achieve the goal. The Standards make the connections between education and career preparation more transparent and, it is hoped, will stem the tide of money spent on remedial education, said Mr. Laine.

The Standards differ significantly from existing approaches, so much so that states will see large drops in the percentage of students who achieve proficiency as measured by current state tests. Thus, Mr. Laine, explained, when the results of the new assessment of students using the Common Core State Standards become available in 2015, states will find that far fewer students are considered proficient than they had been under state assessments.

At the same time, there has been a lot of turnover among state governors. Twenty-six current governors were not in office when their states adopted the Common Core State Standards, and 39 are up for reelection in 2014. Numerous states have legislation pending to eliminate the Standards. Still, most states are moving forward with implementation, Mr. Laine said.

Governors are also facing economic challenges compounded by decreasing federal dollars. The new Standards are being implemented at the same time that more stringent evaluation of teacher effectiveness is taking root. Mr. Laine anticipated a lot of resistance from states when the results of the new assessment of the Standards are released. Waivers provide some relief, but they are short-term solutions. All of these new approaches (Common Core State Standards, teacher evaluation) increase the focus on accountability and results yet fail to recognize the current lack of capacity, said Mr. Laine.

The National Governors Association believes the following topics should be the focus of research, policymaking, and practice:

• Improving early education, particularly because schools do poorly at remediation
• Rebalancing assessment and accountability to focus less on punishment and more on improving practice
• Cultivating talent, because few believe that teachers are prepared to teach at the level of the Common Core State Standards, that principals have the leadership skills to implement the Standards, or that the current professional development programs are sufficient. There is even less confidence in teacher preparation for future generations.
• Identifying the needs of institutes of higher education to serve increasing numbers of students seeking workforce education through mechanisms other than 4-year institutions
• Reallocating resources to more meaningful efforts, bridging the gap between what researchers deem important and what parents value

Mr. Laine urged IES to look more closely at what questions practitioners, teachers, leaders, and policymakers need to answer and how to get the data to answer them. Most research efforts take too long to help practitioners or policymakers make day-to-day decisions. The public sector could learn from the private sector about shortening research cycles. IES should consider adding development considerations to research, so that investigators are thinking about implementation and research at the same time. More focus is needed on differentiated interventions and scaling up solutions, given the need to move large groups of students forward.

Despite a wealth of knowledge, we do a poor job of implementing research, Mr. Laine noted, particularly about effective leadership. Individual decision-makers select the policies they wish to implement, and policies change with every change in personnel. Mr. Laine hoped that once the Standards are implemented, the system remains stable for 4 to 5 years, so that data from assessments can be used to determine what works and inform future efforts. He raised concerns about the unintended consequences when the new assessments show significant drops in proficiency scores; for example, low-income people might feel their school systems are moving the goalposts, and suburban families might take their children out of public schools. Mr. Laine urged ED to consider the possibilities and build a public education system that supports individuals and the economy.

Ms. Martin said that the Common Core State Standards are "a gamechanger"—the first widespread program to put into practice higher learning standards. With the need to use money more wisely, the commonality of the Standards offers incredible potential for the education community to pool resources and create more sophisticated tools for implementation. The Standards do not just align with learning content but also with skills needed to succeed in college and in careers.

From the federal perspective, high levels of college and career readiness are the primary goal. Ms. Martin said federal policies have not required commonality, and state-led efforts to move to the common Standards are the right thing. ED is optimistic, she noted, because the state and local leaders who have already implemented the Standards have exceeded expectations.

ED has considered how it can support researchers and policymakers in ensuring that the Standards are implemented effectively by teachers. Grant initiatives, statewide longitudinal data systems, Race to the Top (RTT), the school turnaround competition, and Elementary and Secondary Education Act (ESEA) waivers are some of the mechanisms in place to help support implementation. Some state-led efforts demonstrate early success:

• New Jersey developed a model curriculum that includes specific supports for English language learners (ELLs) and students with disabilities.
• Kentucky and Louisiana are restructuring teacher and leadership programs to align with the Standards.
• Nevada created a graduate course for teacher development.
• Tennessee hired 200 teachers to staff regional centers for professional development through in-service opportunities.
• New York has 13 local education agency-developed curriculum modules, with resources across the state and a website.
• Massachusetts created a digital resource library of curricula and interim assessments.

In addition, 34 states have ESEA waivers, and each is committed to aligning existing state standards with the Common Core State Standards to aid the transition. All have professional development programs around the Standards for teachers and leaders and are developing instructional materials. The RTT efforts are also helping, said Ms. Martin. ED is challenging states to ramp up access to college courses at the secondary level and to look at teacher/leader preparation for the new Standards.

The Standards have significant policy and research implications. As we move forward, states need a lot of information:

• Long-term validation of the Standards to determine students' college and career readiness
• Short-term implications of the Standards for ELLs and in relation to other linguistic demands and the English language proficiency standards
• Accommodations needed to ensure that the Standards are appropriately applied to boost the success of students with disabilities
• Research and development programs, such as Investing in Innovation, to go beyond evaluation of effectiveness to validation and scale-up
• Ways to improve the technology infrastructure and research on which technology applications can be levers for productivity

From a policy perspective, the biggest challenge is implementing the Standards to ensure they are a tool for equity and civil rights. The Common Core State Standards should benefit not only students in wealthy suburban school districts who were likely to succeed in college anyway but also disadvantaged students. Ms. Martin concluded that she agreed we need to know how to differentiate interventions to meet a range of education needs.

Discussion

Dr. Long asked about the status of implementation and said she is concerned that implementation would not happen in the way it was intended. In response, Mr. Laine said there are already some powerful examples of changes at school systems, among some universities, and in state policies to implement the Standards. Mr. Laine indicated RTT provided enough money for states to try some new things, but not enough to invest in translating the successes. He also said approximately six states are focusing on fixing the disconnects across policies. Some states are thinking about regional structures to build capacity, such as Tennessee, which recognizes the need to create an infrastructure for professional development. Mr. Laine said the issue of how to translate the successes and lessons learned to a large scale is one they are still trying to solve. He said the Secretary may focus on those schools performing at the lowest 5 percent, but most of us would not be willing to send our children to schools that rank even as high as 75 percent. Although the focus on the lowest-performing schools is appropriate, the scale of necessary change is much larger.

Ms. Martin was optimistic, noting that every waiver state is working on implementation as well as other efforts around college/career readiness outside of the Common Core. She is concerned about constant leadership turnover at most levels, because education reforms require years or decades to take root. She recommended the Office for Planning, Evaluation, and Policy Development and IES should work together to disseminate best practices and facilitate a community of practice across states, building on efforts that began under RTT. New tools developed under RTT are now being made more broadly available, addressing topics such as setting up teacher evaluations and reaching different student groups. With persistence, there is good potential for success, Ms. Martin said. She added that low-performing schools are the Secretary's priority, but ED is not only focused on the bottom 5 percent. Waivers allow schools to use their capacity but also create accountability for continuous improvement.

Dr. Gamoran noted that Wisconsin anticipated the drop in proficiency scores. The state recalibrated its assessments and showed a drop of 30 to 40 percentage points. Communicating the results went well, as people recognized that the new scores were more realistic, and the results did not dampen the enthusiasm for assessment. Mr. Laine added that other states (Kentucky, Tennessee) have done the same, incorporating a strategy for informing the public about the reason for the drop in scores and how the state is addressing the matter. Florida, however, faced a huge backlash when it raised the cutoffs, so the state reversed the changes, said Mr. Laine. He emphasized the importance of encouraging cooperation now, but said some states will wait until 2015.

In response to a question from Dr. Gamoran, Mr. Laine said the Next Generation Science Standards are still not finalized, but they help combat the argument that the education policy is narrowing the curriculum to English language arts and math. In reality, said Mr. Laine, systems can only change so much at once. He believes the Next Generation Science Standards will build capacity to implement the Common Core State Standards, and both efforts require similar policies. In response to an additional question from Dr. Gamoran, Ms. Martin was unsure whether federal funding would support the development of assessments aligned with the Next Generation Science Standards.

Dr. Bryk suggested that the implementation of the Common Core State Standards and the new teacher evaluation efforts are on a collision course. MetLife's survey of teachers and principals found that morale is at an all-time low, following a long decline, and it is unrealistic to believe that teachers are going to individualize the Standards to meet individual needs. Schools are being asked to make an incredible transformation, and from the ground up, not from the top down. Dr. Bryk commented that it is not feasible to ask for massive changes while also asking for high accountability in the context of new requirements.

Mr. Laine said teachers lack a sense of urgency around the Standards, because most believe their teaching already aligns with the rigor of the Standards. Policymakers, however, have a heightened sense of urgency. Mr. Laine believes the new Standards push teachers out of their comfort zone and move them in the right direction.

Ms. Martin said that it is important to recognize teachers' anxiety, and policymakers should make teachers part of the process, not victims of change. ED has held roundtables with hundreds of groups of teachers to get their input and to empower them during policy shifts. Ms. Martin emphasized that the changes around teacher/leader evaluation should be made in parallel with changing Standards. She said evaluation should not be seen as a way to identify bad teachers but rather an essential tool for continuous improvement. Although including student outcomes in teacher evaluation may be controversial, evaluation also includes multiple measures, and these measures must be used to inform professional development and supports for teachers. Ms. Martin said we cannot implement new, higher Standards without real teacher evaluation.

Mr. Laine suggested more research and interventions aimed at leaders such as principals in addition to teachers. Many tools coming from higher education are focused on professional development, but there is little research on them. Dr. Singer suggested massive open online courses (MOOCs) could provide a lot of data in a short period and could play a role in professional development around Common Core State Standards. Mr. Laine's concern about using MOOCs for professional development is that underprepared teachers will not benefit as much from them. He suggested, though, that for the majority of teachers and principals, marrying MOOCs with professional learning communities would be beneficial.

Dr. Chard said that given the declining number of people pursuing teaching and the lack of capacity in schools, more efforts should address feeding the pipeline. Also, Ms. Martin said more research is needed on what works to ensure that schools have sufficient numbers of qualified teachers, especially in high-demand schools. Mr. Laine added that current salary guides focus on a teacher's degree(s) and number of years of experience, which is not enough to bring in better teachers. Even good teachers struggle with the constraints of a bad system, said Mr. Laine. A lot of funding is locked into mechanisms that prevent systemic changes, he continued. The Secretary is looking at such connections, articulating where we want the system to be, and encouraging people to unlock the constraints.

Dr. Ferrini-Mundy shared that it is hoped that the Standards will prepare more students for college-level math courses. It is important, she said, to monitor the transition from high school to college to assess the success of the Standards. If higher education policies addressing readiness at the entry level are not well crafted, they could trigger a potential disaster. Mr. Laine commented that institutes of higher education have not been as engaged in the Standards as they need to be. However, many institutes of higher education plan to wait for the assessment results before changing their criteria for placement and entrance exams and using 11th grade assessment results for making entrance decisions. Mr. Laine pointed out that assessment takes place during 11th grade, and those students do not start college for another 16 months. Therefore, efforts in grade 12 around remediation or acceleration are crucial to the transition from high school to college, and governors can play a role in addressing that middle stage. Ms. Martin said ED wants to tackle the issue more aggressively. RTT challenged institutes of higher education to reconsider how they operate and to make systematic changes that would "push down" into K-12 efforts. Some successful models have emerged as a result.

Dr. Bryk commented that the enormous changes on the horizon should lead to a new primary IES research focus and new research funding approaches. Instead of rolling out large programs and evaluating the impact later, more efforts should focus on the scaffolding needed at all levels to ensure that the goals of the Standards are met. As the implementation occurs, research should be funded to inform ongoing improvement. In addition to providing technical assistance, we also need to better understand how people can learn faster to improve. In IES parlance, learning about a project as it's being put in place is referred to as implementation evaluation, Dr. Easton pointed out.

Dr. Neild noted that NCEE has been encouraging the RELs to provide more support for Common Core implementation and more virtual education. Dr. Bryk suggested that IES fund more analysis of work underway instead of waiting until programs are fully implemented and that such analysis be undertaken by a research agency, not the body that provides technical assistance.

Dr. Gutierrez raised skepticism about the notion that using Common Core State Standards translates to equity unless all systems are saturated with capacity. She questioned what constitutes remediation in an environment promoting new ways of thinking about engagement, practice, and knowledge. Dr. Long agreed that IES can do some more research in the area of remediation in the context of the new Standards. Mr. Laine suggested increasing focus on acceleration of learning to help those students who are behind and doing so in new ways and on a large scale. Such efforts require not only growing the pool of teacher talent but taking into account the distribution of talent. Mr. Laine also suggested technology may allow for acceleration of learning. Ms. Martin suggested learning from charter schools about models to accelerate learning beyond the school day and pairing the best teachers with the students who need the most help.

Dr. Gamoran commented that implementation of the new Standards will affect a lot of research already underway, such as studies using existing state assessments as an outcome variable. Assessments of the Standards should be deeper and richer than current state assessments, but results will show a break from one set of tests to the next. On the other hand, with nearly all states using the same Standards, research could be more applicable to other states.

Speaking to the call for more rapid cycle analysis and mechanisms for continuous improvement, Mr. Laine said the Gates Foundation's Measures of Effective Teaching (MET) study has short-term data points and more school districts are using multiple measures; schools could gather information about teacher effectiveness weekly that feeds into improving teacher practices.

Dr. Ford commented that private schools are facing the same questions about what constitutes rigor and how to remediate. Private schools focus exclusively on college preparation, and they do not have the same organizational constraints as public schools, yet there are lessons that public schools can learn from private schools. The new Standards pose unique challenges, and even with the best professional development for all current teachers, schools would still need a new set of development opportunities for incoming teachers, who face higher demands.

Dr. Loeb asked about the coordination of the Governors and ED regarding leveraging the rollout of the Standards to allow evaluation. Ms. Martin said ED is building evaluation components into some of its programs and identifying the most important research questions at the outset. Efforts are underway to ensure that researchers collect meaningful data, including data that can provide immediate information. Sometimes, these efforts are constrained by what data schools can and will collect. The Office of Planning, Evaluation, and Policy Development is working more closely with IES and its centers to improve evaluation, including collecting appropriate data. Dr. Easton reminded the group that the Common Core State Standards are amenable to federal evaluation but are not part of federal policy. ED is reaching out to states to identify discrete components of implementation that it could evaluate in a rigorous way, such as professional development or modules focused on ELLs. Audrey Pendleton of NCEE noted that there are challenges to setting up research projects because there is no federal requirement.

Dr. Long concluded the discussion by noting that IES can set the research agenda and encourage others to take on the work. Many researchers may not have thought about the impact on their studies of moving from state assessments to the Common Core State Standards assessments, she noted.

Lunch
Participants adjourned for lunch at 1:04 p.m. During the lunch break, NBES members participated in ethics training delivered by Marcia Sprague of the Ethics Division of ED's Office of General Counsel. The meeting resumed at 2:14 p.m.

The Role of the Researcher in Dissemination
Ruth Curran Neild, Ph.D., NCEE Commissioner

Dr. Gurierrez indicated that one presenter for this session, Dr. Amber Winkler, had to cancel at the last minute. Dr. Neild said that to prepare researchers to interact with practitioners, we need to understand their skills as well as the incentives and disincentives they have to create products for practitioners. The RELs, for example, build capacity for state and local education agencies to access, interpret, and apply data, while also building capacity among researchers to engage more fully with practitioners in investigation and present work in ways that practitioners understand.

Researchers are not accustomed to thinking about audiences or products. They need help learning to write for different audiences and for different contexts or products. Not only do most researchers lack training in diverse writing styles but they have no incentive to develop such skills. Such efforts do not count toward tenure and therefore have no value.

Dr. Neild said the RELs offer researchers some brief training workshops with professional writers, but more effort is needed to raise awareness on how to improve writing skills and to demonstrate that formal training in writing is effective. In addition, researchers often lack skills for partnering with or even listening to practitioners, so the ideas of exchange and collaboration are foreign concepts that should be explained.

Discussion

Dr. Singer commented that professional development opportunities for researchers are available, but the researchers must be convinced of the value of such efforts. She said many do see the value of media training (e.g., giving interviews to the media), so training could focus on writing for new media (e.g., blogs, Twitter). Packaging such training should emphasize the value to the user and vary depending on the audience and incentives/motivation. Partnering with the Education Writers Association, for example, may be effective. Education about using other media, such as video, to present research could be appealing.

Dr. Hedges pointed out that NAEP has created engaging media presentations for other products, not just for the TEL assessments. It also has experimented with writing to make complicated material accessible. The international assessments (Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study) have taken some interesting approaches to communicate information to various audiences. Dr. Hedges recommended that IES take advantage of some of these activities.

Dr. Granger commented that some efforts have been made to help researchers better disseminate their findings, which are consistent with the model of moving research into practice. Better translation of materials into practitioner terms, media training, etc., have been useful but not that productive in forging a synergistic relationship between researchers and practitioners. Dr. Granger also said the W. T. Grant Foundation has been seeking to strengthen that relationship by offering mentored fellowships, embedding researchers in practice settings, and creating researcher-practitioner partnerships.

Dr. Gutierrez said researchers may not be interested in learning about writing better, but they may be interested in learning about new genres and the use of social media. It may be helpful to use new media to facilitate the kind of ongoing conversations among partners that build strong relationships.

Dr. Long commented that it may be necessary to define more clearly the terms under discussion—e.g., partnership and exchange—so that researchers can better define what skills they need to build. She added that the field needs to break out of thinking of a one-way path from researcher to practitioner and consider what a true partnership and a true exchange means.

Dr. Loeb said making research engaging takes a skilled writer. Rather than trying to teach a lot of researchers to become better writers, it may be more worthwhile to seek organizations that can help translate research into information useful for policymakers (as California has) or individuals who excel at writing about research. Dr. Buckley added that a good team effort allows all the participants to do what they are good at and learn from one another. Creating such a dynamic between communicators (e.g., writers, videographers) and researchers can take a lot of back and forth discussion to achieve a high-quality product.

Dr. Gamoran commented that skilled communicators can take complex research findings and distill them into materials for practitioners and lay audiences. For example, thanks to grant funding, reports written by the Board on Science Education at the National Research Council were translated into companion documents for practitioners with real-world examples of improving science education. Reaching other audiences requires developing targeted materials.

Dr. Yoshikawa indicated some organizations may have the capacity to serve as matchmakers, pairing researchers with influential advocates, for example. Senior-level staff can suggest matches or provide models on how to write effective editorials or tailor presentations to different audiences. Organizations can host gatherings designed to break down silos and foster relationships. Some research organizations require that practitioners be represented in research panels.

Dr. Bryk noted that the conversation is addressing both how researchers can better disseminate their research into practice (a one-way conversation) and the need for researchers to listen to and better understand practitioners in an effort to improve scholarship. The components of constructive partnerships are fertile ground for research. It would be helpful to better understand what works in what circumstances.

Dr. Ford commented that many researchers perceive marketing and communications as branding. Ideally, researchers would see the value of using professional communicators to disseminate their findings and consider such steps as they are setting up their studies.

Dr. Chard added that building productive relationships between researchers and practitioners can take years of developing trust and becoming comfortable talking openly about concerns and flaws. It is important to think about the long-term, because once trust is created, partnerships can be incredibly productive. The thinking about communication and dissemination is evolving from calling on researchers to disseminate their findings to encouraging collaboration and exchange to understanding that true partnership involves a series of exchanges.

Dr. Hedges indicated that it is becoming apparent that most problems require a multi-disciplinary approach. Such efforts touch on all the topics raised so far: engaging practitioners and policymakers, amassing a team whose members together can provide a range of skills, and building trust and confidence among partners. Dr. Hedges also commented that funding agencies may have insight into how to encourage multi-disciplinary research and how and when it is effective.

Dr. Loeb reiterated her point that organizations exist can facilitate dissemination for researchers. Dr. Singer commented that education research still takes a very traditional approach to disseminating findings (e.g., written reports that sit on shelves). It is important to ask whether disseminating results should be required and whether funding should include alternative mechanisms for disseminating findings (which require other skills and oversight). It is also important to determine whether research is looking at issues of concern to practitioners, policymakers, and the public. Finally, Dr. Singer wondered, if disseminating research is not valued by academic institutions as a good use of researchers' time, should IES promote such efforts?

Dr. Loeb said some grant programs already require applicants to propose a dissemination plan for their findings. Dr. Yoshikawa suggested it may be helpful to contact those funding agencies to determine what works and what does not and what efforts have been made to build grantees' skills. Dr. Easton indicated that the quality of many written materials is poor. Dr. Gamoran suggested perhaps grants could take a sequenced approach, so that grantees focus on dissemination once it becomes clear that the findings are worth communicating broadly. Drs. McCardle and Miller noted that grant applicants often provide very general dissemination plans, so the criteria should be specific. A detailed publication plan can be requested from applicants, which allows them to publish important findings as they arise.

Dr. Singer suggested IES could convene a workshop that brings together various constituencies, such as editors of research journals and broader professional publications, bloggers, journalists, and others to discuss better dissemination of research.

Dr. Bryk commented that individual institutions could offer targeted academic writing workshops. Dr. McCardle added that K-12 education could teach better writing skills. Dr. Long also added that technology allows individuals to complete exercises in editing and writing.

Dr. Granger commented that personal relationships remain vital to communication. Building a network of professionals in various areas helps people gather a variety of perspectives and sort out complex issues, which leads to better communication with target audiences.

IES and the Major Education Research Associations

Dr. Long canceled discussion of this topic on the basis that it may pose perceived conflicts of interest. She noted that many NBES members are officers or serve on committees of the major research associations.

Common Core State Standards: Follow-Up Discussion

Dr. Singer asked whether IES should convene representatives of various constituencies to discuss how evaluation and process improvement can be combined with the implementation of the Common Core State Standards. Dr. Loeb noted that states rarely implement a large program at the same time, so IES could take advantage of the opportunity to build in some evaluation and get data back quickly. Dr. Easton said the stakeholders should drive the determination of the research questions. Dr. Hedges commented that the implementation is a good natural experiment. Gathering data on the Standards from the outset is important.

Dr. Loeb suggested that states could be randomized—for example, on the basis of their efforts at professional development. Dr. Chard pointed out that some states have been rolling out the Standards for years through professional development and other means, so randomization may not be feasible.

Board members agreed that IES can play a role in coordinating research. Dr. Bryk commented that even though the Standards are not a federal initiative, it is the federal government's responsibility to provide research of value to states and districts about such a large program. Dr. Easton suggested the NSF, NICHD, and private foundations may be willing to partner with IES to convene such a meeting. Dr. Loeb emphasized that larger district representatives should probably be included. Dr. Long suggested that IES act quickly. Dr. Chard also suggested that research pay particular attention to the impact of the Standards on students in low-income communities, and those with disabilities. Dr. Yoshikawa added that it is also important to learn whether underlying skills are affected by the new approach of the Standards.

Board Business: Executive Director Position and 2013 Annual Report: Roundtable Discussion

Dr. Long reminded Board members about the new structure of the annual report, first used in 2012, that summarizes the Board's efforts and organizes them thematically. The report is due June 1, and so the Board will have a public meeting by conference call to review and approve the draft report before its scheduled June meeting. Dr. Long would like to have a draft ready for review by mid-April. Last year's report focused on the Board's discussions, ideas, and feedback to commissioners; it demonstrated the Board's contributions to IES and signaled to the field the Board's areas of interest.

Dr. Long described the role of the Executive Director, a position that is currently open. Dr. Easton described the constraints that IES faces in filling the position. He explained that Dr. McGill-Wilkinson is providing some Board support, and IES is considering how it can offer additional staff support. Dr. Long noted that the current arrangement puts more burden on her, because Dr. McGill-Wilkinson is prohibited from offering advice or counsel to the Board (about policies, strategies, etc.). It was noted that an external consultant cannot act on behalf of a government entity. It was not clear whether the Board Chair can seek staff support from her own institution.

Dr. Long pointed out that the Board's mandate is broader than that of a typical federal advisory committee. Much discussion ensued about the Board's needs, roles of support staff, and options for support. It was agreed that the Board will assess the current arrangement after some time to determine further needs. It was noted that Dr. Long brings a great deal of energy and institutional memory to the role of chair, but the current approach to managing the work of the Board through a patchwork support system is not sustainable in the long term. It is possible that reauthorization of ESRA could give the Board hiring authority with some flexibility.

Adjournment

Bridget Terry Long, Ph.D., NBES Chair

Dr. Long adjourned the meeting at 3:40 p.m.

View, download, and print the full meeting minutes as a PDF file (1.2 MB)

The National Board for Education Sciences is a Federal advisory committee chartered by Congress, operating under the Federal Advisory Committee Act (FACA); 5 U.S.C., App. 2). The Board provides advice to the Director on the policies of the Institute of Education Sciences. The findings and recommendations of the Board do not represent the views of the Agency, and this document does not represent information approved or disseminated by the Department of Education.