WWC review of this study

Do Judgments of Learning Directly Enhance Learning of Educational Materials?

Ariel, Robert, Karpicke, Jeffrey D., Witherby, Amber E., Tauber, Sarah K. (2021). Educational Psychology Review v33 n2 p693-712. . Retrieved from: https://eric.ed.gov/?id=EJ1295971

  • Randomized Controlled Trial
     examining 
    514
     Students
    , grade
    PS

Reviewed: January 2023

No statistically significant positive
findings
Meets WWC standards without reservations
Comprehension outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Researcher-developed reading comprehension test

Judgements of learning - Ariel et al. (2021) vs. Business as usual

0 Days

Full sample: Aggregate and term-specific Judgments of Learning;
514 students

49.29

47.34

No

--
More Outcomes
Show Supplemental Findings

Researcher-developed reading comprehension test

Judgements of learning - Ariel et al. (2021) vs. Business as usual

0 Days

Experiment 4: Term-specific Judgments of Learning;
129 students

47.41

41.00

No

--

Researcher-developed reading comprehension test

Judgements of learning - Ariel et al. (2021) vs. Business as usual

0 Days

Experiment 2a: Term-specific Judgments of Learning;
90 students

49.29

44.44

No

--

Researcher-developed reading comprehension test

Judgements of learning - Ariel et al. (2021) vs. Business as usual

0 Days

Experiment 1: Aggregate Judgments of Learning;
80 students

48.27

45.87

No

--

Researcher-developed reading comprehension test

Judgements of learning - Ariel et al. (2021) vs. Business as usual

0 Days

Experiment 3: Term-specific Judgments of Learning;
116 students

40.31

42.37

No

--

Researcher-developed reading comprehension test

Judgements of learning - Ariel et al. (2021) vs. Business as usual

0 Days

Experiment 2b: Term-specific Judgments of Learning;
99 students

63.13

65.36

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 45%
    Male: 55%

Setting

The study took place using an online task accessed through Amazon Mechanical Turk (mturk.com).

Study sample

A total of 514 participants in the United States were recruited online on the Amazon Mechanical Turk website to participate in the study. Participants were paid a small amount (about $1.00 to $1.50) to complete the task required for the study. All participants were high school graduates and 45 percent were female. The authors do not provide further information on the study participants.

Intervention Group

Judgements of Learning (JOLs) ask learners to rate their confidence in their ability to later recall or answer questions about recently studied material. In this study, participants read an educational text and were instructed to make aggregate or term-specific JOLs immediately after reading the text. Aggregate JOLs involved asking participants to rate their confidence in understanding the text as a whole, while term-specific JOLs involved asking participants how confident they were in their understanding by prompting about specific content in the passage. For example, considering an educational passage about how minerals are made, an aggregate JOL might be “How confident are you that you understand the information in the passage?”, while a term-specific JOL might be “How confident are you that you understand how minerals are made?” Each task lasted 10 to 15 minutes.

Comparison Group

Participants in the comparison group read the educational text but did not receive a JOL prompt after reading the text.

Support for implementation

The intervention was implemented by researchers online and hosted as a Human Intelligence Task on Amazon Mechanical Turk. A Human Intelligence task is one where individuals are paid a small amount of money to complete a discrete task. In this case, the task was to read the educational passage and respond to a short quiz about the content. The study does not describe any additional support for implementation.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top