Co-Principal Investigator: Mark A. McDaniel, Kathleen B. McDermott

Purpose: This project builds on earlier work on Test-Enhanced Learning (TEL), a method for improving students' retention of knowledge in many different subject matters. When students are given a quiz that requires active retrieval of information, they remember that information better on a later test relative to information not previously tested (or even relative to information that is restudied but not initially tested). Active retrieval and feedback during the test are the critical processes that produce this positive effect. Quizzing also induces greater metacognition during student learning (informing students about the state of their knowledge) and recent evidence indicates that retrieval also fosters transfer of knowledge to new settings. This TEL approach has been developed by this team in two previous IES grants, which have generated strong evidence for test-enhanced learning in both laboratory experiments and in actual school settings. Evidence to date indicates that TEL greatly enhances student learning in science, social studies, and vocabulary (English and Spanish) in the middle school curriculum. Researchers propose to create a TEL teacher manual for middle and high school teachers to help them integrate TEL into their classroom practices.

Project Activities: The primary focus of the current proposal is to develop a manual for teachers to use in adapting the TEL approach for their middle school and high school classrooms. During the first year and a half of the grant, researchers will conduct several experiments to address fundamental issues about implementing TEL (e.g., How many quizzes should be given? What subject matters can be used?). Researchers will develop a beta version of the manual during the second year of the grant and then will assess whether it can be used well by teachers who did not participate in the development of the manual during the third year. This assessment will occur both through observations in the classroom and via focus groups with teachers and students. Researchers will continue conducting research in an iterative manner within classroom settings to answer new questions for the manual as they arise. Researchers will also conduct surveys of teacher and student opinion about the TEL approach.

Products: Products include a fully developed manual that teachers can use in adapting the TEL approach in middle and high school classrooms. Peer reviewed publications will also be produced.

Structured Abstract

Setting: This study will take place in southern Illinois.

Population: Middle and high school students and teachers will serve as participants in this project. The population of students participating are socioeconomically diverse (mostly middle and working class), but are not ethnically diverse (97 percent Caucasian). At least 9 teachers and their students will participate in the study.

Intervention: The TEL approach is grounded in at least three theoretical processes that augment learning and retention. One key process in active retrieval of information from memory, which is shown to have powerful positive effects on later retention and which promotes the use of information in other situations. A second key process is learning from feedback. Basic memory research suggests that failing to answer a test question can potentiate learning for the correct answer when it later provided. Accordingly, a component of the TEL intervention is to provide correct answer feedback for all quizzes. A third key process is improvement in metacognition. Basic research suggests that learners generally cannot judge how well they will remember previously studied information, leading ineffective allocation of study time. However, when learned study under conditions that increase the accuracy of metacognitive judgments (judgments of learning), they show an advantage in learning when they are allowed to restudy the texts. Thus, TEL incorporates metacognition.

The TEL approach has already been developed. However, it lacks the supporting materials for teachers to implement it. The focus of the development will be on creating a teacher manual with five major sections: Introduction, Background, Classroom Use (with subsections), Frequently Asked Questions, and Further Resources. The aim is for the entire manual to be about 30 pages long with both a published form of the manual and website where it can be downloaded for free.

Research Design and Methods: The development process is composed of three components: classroom experiments, developing a complete manual, and conducting a pilot study. The classroom experiments, which will be run primarily in the first 1.5 years, use only within-student manipulations, such that some materials will receive normal classroom exposure (i.e., without TEL) and other materials will be assigned to the treatment conditions, with materials counterbalanced across students. The majority of these experiments will be conducted in high school classrooms because there is no available experimental work documenting the positive effects of quizzing in high school classes. The research will target courses in three content areas: biology, chemistry, and history. The materials and quizzes used for these experiments will be constructed directly from the existing curricula, from both existing materials as well as materials created specifically for research purposes (that is, low-stakes quizzes given in class). Results of these experiments will help determine the necessary parameters for teacher-implemented TEL and, in particular, the teacher manual.

During Year 2, the components of the teacher manual will be developed and tested in an iterative, user-centered design and development process. Focus groups, observations, and student and teacher surveys will be used. In the final year of the project, a pilot study will be conducted and a group of six teachers who have not participated in the development of the TEL manual will use the TEL manual and implement TEL in half of their class sections. The other half of the teachers' class sections will be assigned to the control condition. Pilot data on student outcomes will be collected.

Control Condition: For the experiments during the first phase of the study, students will serve as their own controls due to the within-subjects design. During the pilot study phase of the project, the class sections assigned to the control condition will be taught using their normal teaching methods.

Key Measures: Data collected from the experiments will include students' performances on quizzes and exams in the content areas (i.e., biology, chemistry, and history) through multiple-choice tests, short-answers, and essay tests, as well as solving problems (in chemistry). Researchers will also use observations, focus groups, and student and teacher surveys to assess the development and usability of the TEL manual.

Data Analytic Strategy: Qualitative analyses of information collected in observations, focus groups, and teacher and student surveys will be used to verify feasibility.

Project Website: http://www.retrievalpractice.org/

Related IES Projects: Test-enhanced Learning (R305H030339) and Test-Enhanced Learning in the Classroom (R305H060080)

Products and Publications

Book chapter

Pyc, M.A., Agarwal, P.K., and Roediger, H.L. (2014). Test-Enhanced Learning. In V.A. Benassi, C.E. Overson, and C M. Hakala (Eds.), Applying the Science of Learning in Education: Infusing Psychological Science Into the Curriculum (pp. 78–90). e-book: Society for the Teaching of Psychology, American Psychological Association.

Book chapter, edition specified

Roediger, H.L., McDermott, K.B., and McDaniel, M.A. (2011). Using Testing to Improve Learning and Memory. In M.A. Gernsbacher, R. Pew, L. Hough, and J.R. Pomerantz (Eds.), Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society (2nd ed., pp. 65–75). New York: Worth Publishing Company.

Journal article, monograph, or newsletter

Agarwal, P.K., Bain, P.M., and Chamberlain, R.W. (2012). The Value of Applied Research: Retrieval Practice Improves Classroom Learning and Recommendations From a Teacher, a Principal, and a Scientist. Educational Psychology Review, 24(3): 437–448.

Agarwal, P.K., D'Antonio, L., Roediger, H.L., McDermott, K.B., and McDaniel, M.A. (2014). Classroom-Based Programs of Retrieval Practice Reduce Middle School and High School Students' Test Anxiety. Journal of Applied Research in Memory and Cognition, 3(3): 131–139.

McDaniel, M.A., Agarwal, P.K., Huelser, B.J., McDermott, K.B., and Roediger, H.L. (2011). Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement. Journal of Educational Psychology, 103(2): 399–414.

McDaniel, M.A., Thomas, R.C., Agarwal, P.K., McDermott, K.B., and Roediger, H.L. III (2013). Quizzing in the Middle-School Science: Successful Transfer Performance on Classrooom Exams. Applied Cognitive Psychology, 27(3): 360–372.

McDermott, K.B., Agarwal, P.K., D'Antonio, L., Roediger, H.L., and McDaniel, M.A. (2014). Both Multiple-Choice and Short-Answer Quizzes Enhance Later Exam Performance in Middle and High School Classes. Journal of Experimental Psychology: Applied, 20(1): 3–21.

Roediger, H.L. (2013). Applying Cognitive Psychology to Education: Translational Educational Science. Psychological Science in the Public Interest, 14(1): 1–3.

Roediger, H.L., Agarwal, P., McDaniel, M.A., and McDermott, K.B. (2011). Test-Enhanced Learning in the Classroom: Long-Term Improvements From Quizzing. Journal of Experimental Psychology: Applied, 17(4): 382–395.

Wooldridge, C.L., Bugg, J.M., McDaniel, M.A., and Liu, Y. (2014). The Testing Effect With Authentic Educational Materials: A Cautionary Note. Journal of Applied Research in Memory and Cognition, 3(3): 214–221.