Creating a Culture of Science
In striving to enhance the scientific quality of our work, we have focused on people. It is people who have the responsibility for conceptualizing and coordinating research programs. The recent National Research Council report on scientific inquiry in education concluded that building a scientific culture within the Department's research agency is a prerequisite for all else, and this reflects our approach as well. I think it is critically important to understand that successful research agencies, such as the NIH, embody a scientific culture because the people in the principal program management roles share the dispositions and training that characterize scientists. It is this shared culture, much more than statutes, rules, and regulations, that supports high-quality research. Scattering a few scientists among a large number of employees without the training and dispositions of scientists does not work. Several months ago I identified the relatively small number of accomplished scientists in OERI and asked them to meet regularly as a group to move our new programs forward. The day after the first meeting, I received an email from one of the older hands in attendance. He wrote that it was the first time he had actually felt like he was working in a research agency. Creating a culture in which those experiences are routine is essential.
My experience in trying to increase the number of qualified scientists at OERI highlights an area in which new legislation can be useful. We need to be able to hire scientists on excepted service positions outside the regular civil service. OERI currently has this authority, and we want to see it continued. One of the people I have recruited, a very senior distinguished scientist, was hired on an excepted service line, and it took us only two weeks. Had we not had that authority, she would not be here yet, and the critical work that she has done over the last six months would not have been accomplished. In hiring scientists, we need to move quickly and flexibly. We also need to hire scientists for limited terms, so that we will be continuously able to bring new scientists into the agency as others return to their institutions or move on to other positions. Other science agencies find this strategy invaluable.
Building a scientific culture at the Department's research agency also requires stability in leadership and the shared sense that the organization can pursue its agenda over the long term. The Office of Educational Research and Improvement has had more Assistant Secretaries and Acting Assistant Secretaries than it has had years of existence. That is not a recipe for building a strong organization. In making appointments under the new legislation, the Department intends to emphasize the scientific and management qualifications of the Director and Commissioners of the principal centers of operation and, in particular, their willingness to serve for a substantial period of time, so as to encourage stability and continuity in leadership and management of the centers.
With highly qualified scientists throughout the agency and in leadership roles, we can address the quality issues that arise from inadequate peer review, which has been a chronic problem. A 1999 study of peer review in OERI by the National Educational Research Policy and Priorities Board found three panels of reviewers for the field-initiated studies competition that did not have a single member with research training and experience in the subject area of the competition. I will state the obvious: If the reviewers don't know anything about what they are reviewing, they aren't going to be able to separate the scientifically strong applications from the weak.
We hope new legislation will make it easier for us to set up standing review panels comprised of experts on particular topics, instead of using panels that can only meet once and that consist of members who are forced to be jacks of all trades. That structural feature, when combined with a selection process for peer reviewers that is carried out by staff in the agency who are themselves accomplished scientists, will have more effect in raising the quality standards than anything else we can do.