WWC review of this study

Can feedback, correct, and incorrect worked examples improve numerical magnitude estimation precision? [Undergraduate students]

Fitzsimmons, C. J., Morehead, K. T., Thompson, C. A., Buerke, M., & Dunlosky, J. (2021). The Journal of Experimental Education, 1-26. https://doi.org/10.1080/00220973.2021.1891009.

  • Randomized Controlled Trial
     examining 
    49
     Students
    , grade
    PS

Reviewed: September 2022

No statistically significant positive
findings
Meets WWC standards without reservations
Numbers and Operations outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021) vs. Business as usual

0 Days

Full sample;
49 students

25.64

26.16

No

--
More Outcomes

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021) vs. Business as usual

0 Days

Full sample;
49 students

17.52

19.66

No

--
Show Supplemental Findings

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
49 students

14.91

21.28

Yes

-26
 
 

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
49 students

4.90

14.01

Yes

-33
 
 

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with feedback – Fitzsimmons et al., (2021)

0 Days

Full sample;
47 students

8.50

17.30

Yes

-34
 
 

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with feedback – Fitzsimmons et al., (2021)

0 Days

Full sample;
47 students

-4.40

8.31

Yes

-44
 
 


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 89%
    Male: 11%

  • Suburban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Ohio
  • Race
    Other or unknown
    22%
    White
    78%
  • Ethnicity
    Other or unknown    
    100%
  • Eligible for Free and Reduced Price Lunch
    Other or unknown    
    100%

Setting

The study took place with psychology students at Kent State University.

Study sample

This manuscript examines three different versions of the magnitude estimation practice intervention with undergraduates. This review focuses on one version called magnitude estimation practice with incorrect worked examples. The other two versions of the intervention, and a description of their differences, can be accessed through the drop-down menu. The manuscript also describes an experiment using versions of this intervention with elementary school students. The details of that experiment can be found on a separate study page. All 49 participating students were undergraduates in the psychology department of Kent State University; 89% of the participants were female. The authors did not collect other demographic information. However, all undergraduate students in the same psychology department from which the participants were recruited were an average of 19 years old and 78% White.

Intervention Group

The intervention provides students with practice in magnitude estimation which is intended to improve students’ accuracy placing whole numbers on a number line. The intervention was provided by members of the study team and delivered to individual students. The intervention was administered in a single 30-minute session. During that 30-minute session, students first took the pretest, then iteratively practiced number line placement problems with the support of the study team member who administered the session, followed by the student taking 10 of the 30 posttest questions at a time. The number line ranged from 1 thousand to 1 billion. Participants placed each number on the number line and rated their confidence in their placement. Students completed the number line practice problems and test problems individually on computers. In the magnitude estimation practice with incorrect worked examples group, the study team member showed students an incorrect number line placement and then asked the student to explain why the answer was wrong. If the student did not respond, the study team member continued to prompt the students until they provided an answer. The student responses were recorded by the study team member.

Comparison Group

The business-as-usual comparison was to a group of students that engaged in magnitude estimation practice without guidance or feedback in a single 30-minute session. Students completed number line practice problems and test problems individually on computers. In this group, the study team members did not provide students with any prompts for explanations and students were not asked to judge the confidence of their answers. The intervention group was compared to other versions of the intervention as well: In the magnitude estimation practice with correct worked examples group, the study team member showed students a correct number line placement and then asked the student to explain why the answer was correct. In the magnitude estimation practice with feedback group, the study team member asked the student to place the number on the number line, after which the study team member showed the student where the number really belonged and asked the student to explain why their placement was similar, higher, or lower than the correct placement.

Support for implementation

The study team administered the activities in the intervention and comparison conditions. No information about training or preparation of the study team is provided.

Reviewed: September 2022

At least one finding shows promising evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
Numbers and Operations outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with feedback – Fitzsimmons et al., (2021) vs. Business as usual

0 Days

Full sample;
48 students

30.23

19.66

Yes

 
 
40
 
More Outcomes

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with feedback – Fitzsimmons et al., (2021) vs. Business as usual

0 Days

Full sample;
48 students

34.56

26.16

Yes

 
 
33
 
Show Supplemental Findings

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with feedback – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
47 students

33.15

20.44

Yes

 
 
44
 

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with feedback – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
47 students

34.55

25.75

Yes

 
 
34
 

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with feedback – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
48 students

17.61

14.01

No

--

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with feedback – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
48 students

24.02

21.28

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 89%
    Male: 11%

  • Suburban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Ohio
  • Race
    Other or unknown
    22%
    White
    78%
  • Ethnicity
    Other or unknown    
    100%
  • Eligible for Free and Reduced Price Lunch
    Other or unknown    
    100%

Setting

The study took place with psychology students at Kent State University.

Study sample

This manuscript examines three different versions of the magnitude estimation practice intervention with undergraduates. This review focuses on one version called magnitude estimation practice with feedback. The other two versions of the intervention, and a description of their differences, can be accessed through the drop-down menu. The manuscript also describes an experiment using versions of this intervention with elementary school students. The details of that experiment can be found on a separate study page. All 48 participating students were undergraduates in the psychology department of Kent State University; 89% of the participants were female. The authors did not collect other demographic information. However, all undergraduate students in the same psychology department from which the participants were recruited were an average of 19 years old and 78% White.

Intervention Group

The intervention provides students with practice in magnitude estimation which is intended to improve students’ accuracy placing whole numbers on a number line. The intervention was provided by members of the study team and delivered to individual students. The intervention was administered in a single 30-minute session. During that 30-minute session, students first took the pretest, then iteratively practiced number line placement problems with the support of the study team member who administered the session, followed by the student taking 10 of the 30 posttest questions at a time. The number line ranged from 1 thousand to 1 billion. Participants placed each number on the number line and rated their confidence in their placement. Students completed the number line practice problems and test problems individually on computers. In the magnitude estimation practice with feedback group, the study team member asked the student to place the number on the number line, after which the study team member showed the student where the number really belonged and asked the student to explain why their placement was similar, higher, or lower than the correct placement. If the student did not respond, the study team member continued to prompt the students until they provided an answer. The student responses were recorded by the study team member.

Comparison Group

The business-as-usual comparison was to a group of students that engaged in magnitude estimation practice without guidance or feedback in a single 30-minute session. Students completed number line practice problems and test problems individually on computers. In this group, the study team members did not provide students with any prompts for explanations and students were not asked to judge the confidence of their answers. The intervention group was compared to other versions of the intervention as well: In the magnitude estimation practice with incorrect worked examples group, the study team member showed students an incorrect number line placement and then asked the student to explain why the answer was wrong. In the magnitude estimation practice with correct worked examples group, the study team member showed students a correct number line placement and then asked the student to explain why the answer was correct.

Support for implementation

The study team administered the activities in the intervention and comparison conditions. No information about training or preparation of the study team is provided.

Reviewed: September 2022

At least one finding shows promising evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
Numbers and Operations outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021) vs. Business as usual

0 Days

Full sample;
50 students

26.63

19.66

Yes

 
 
27
 
More Outcomes

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021) vs. Business as usual

0 Days

Full sample;
50 students

31.95

26.16

Yes

 
 
25
 
Show Supplemental Findings

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
49 students

29.55

20.44

Yes

 
 
33
 

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with incorrect worked examples – Fitzsimmons et al., (2021)

0 Days

Full sample;
49 students

32.12

25.75

Yes

 
 
26
 

Researcher-developed percent absolute error in categorization

Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with feedback – Fitzsimmons et al., (2021)

0 Days

Full sample;
48 students

14.56

17.30

No

--

Researcher-developed percent absolute error in number line estimation

Magnitude estimation practice with correct worked examples – Fitzsimmons et al., (2021) vs. Magnitude estimation practice with feedback – Fitzsimmons et al., (2021)

0 Days

Full sample;
48 students

4.71

8.31

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 89%
    Male: 11%

  • Suburban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Ohio
  • Race
    Other or unknown
    22%
    White
    78%
  • Ethnicity
    Other or unknown    
    100%
  • Eligible for Free and Reduced Price Lunch
    Other or unknown    
    100%

Setting

The study took place with psychology students at Kent State University.

Study sample

This manuscript examines three different versions of the magnitude estimation practice intervention with undergraduates. This review focuses on one version called magnitude estimation practice with correct worked examples. The other two versions of the intervention, and a description of their differences, can be accessed through the drop-down menu. The manuscript also describes an experiment using versions of this intervention with elementary school students. The details of that experiment can be found on a separate study page. All 50 participating students were undergraduates in the psychology department of Kent State University; 89% of the participants were female. The authors did not collect other demographic information. However, all undergraduate students in the same psychology department from which the participants were recruited were an average of 19 years old and 78% White.

Intervention Group

The intervention provides students with practice in magnitude estimation which is intended to improve students’ accuracy placing whole numbers on a number line. The intervention was provided by members of the study team and delivered to individual students. The intervention was administered in a single 30-minute session. During that 30-minute session, students first took the pretest, then iteratively practiced number line placement problems with the support of the study team member who administered the session, followed by the student taking 10 of the 30 posttest questions at a time. The number line ranged from 1 thousand to 1 billion. Participants placed each number on the number line and rated their confidence in their placement. Students completed the number line practice problems and test problems individually on computers. In the magnitude estimation practice with correct worked examples group, the study team member showed students a correct number line placement and then asked the student to explain why the answer was correct. If the student did not respond, the study team member continued to prompt the students until they provided an answer. The student responses were recorded by the study team member.

Comparison Group

The business-as-usual comparison was to a group of students that engaged in magnitude estimation practice without guidance or feedback in a single 30-minute session. Students completed number line practice problems and test problems individually on computers. In this group, the study team members did not provide students with any prompts for explanations and students were not asked to judge the confidence of their answers. The intervention group was compared to other versions of the intervention as well: In the magnitude estimation practice with incorrect worked examples group, the study team member showed students an incorrect number line placement and then asked the student to explain why the answer was wrong. In the magnitude estimation practice with feedback group, the study team member asked the student to place the number on the number line, after which the study team member showed the student where the number really belonged and asked the student to explain why their placement was similar, higher, or lower than the correct placement.

Support for implementation

The study team administered the activities in the intervention and comparison conditions. No information about training or preparation of the study team is provided.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top