WWC review of this study

Who should take college-level courses? Impact findings from an evaluation of a multiple measures assessment strategy.

Barnett, E. A., Kopko, E., Cullinan, D., Belfield, & C. R. (2020). Center for the Analysis of Postsecondary Readiness. Retrieved from: https://eric.ed.gov/?id=ED609173

  • Randomized Controlled Trial
     examining 
    12,971
     Students
    , grade
    PS

Reviewed: August 2021

At least one finding shows strong evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
College Degree Attainment outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Attainment of Associate's Degree after 3 terms

Multiple measures and assessment and placement vs. Business as usual

3 Semesters

Full sample;
12,971 students

3.20

3.50

No

--
Postsecondary Academic Achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Completed college-level English course

Multiple measures and assessment and placement vs. Business as usual

1 Semester

Full sample for students that received an English placement;
10,719 students

34.30

28.00

Yes

 
 
7
 
More Outcomes

Completed college-level math course

Multiple measures and assessment and placement vs. Business as usual

1 Semester

Full sample for students that received a math placement;
9,693 students

17.00

15.00

Yes

 
 
4
 
Show Supplemental Findings

Completed college-level English course

Multiple measures and assessment and placement vs. Business as usual

2 Semesters

Full sample for students that received an English placement;
10,719 students

42.90

39.60

Yes

 
 
3

Completed college-level English course

Multiple measures and assessment and placement vs. Business as usual

3 Semesters

Full sample for students that received an English placement;
10,719 students

47.10

44.20

Yes

 
 
3

Completed college-level math course

Multiple measures and assessment and placement vs. Business as usual

2 Semesters

Full sample for students that received a math placement;
9,693 students

24.00

23.10

No

--

Completed college-level math course

Multiple measures and assessment and placement vs. Business as usual

3 Semesters

Full sample for students that received a math placement;
9,693 students

29.50

29.10

No

--
Progressing in college outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

College credits earned: Semester 1

Multiple measures and assessment and placement vs. Business as usual

1 Semester

Full sample;
12,971 students

5.76

5.42

Yes

 
 
3
 
More Outcomes

Continuous enrollment for 1 term

Multiple measures and assessment and placement vs. Business as usual

1 Semester

Full sample;
12,971 students

80.40

81.10

No

--
Show Supplemental Findings

College Level Credits Earned: After 2 Semesters

Multiple measures and assessment and placement vs. Business as usual

2 Semesters

Full sample;
12,971 students

10.76

10.45

No

--

College Level Credits Earned: After 3 Semesters

Multiple measures and assessment and placement vs. Business as usual

3 Semesters

Full sample;
12,971 students

15.26

15.02

No

--

Continuous enrollment for 2 terms

Multiple measures and assessment and placement vs. Business as usual

2 Semesters

Full sample;
12,971 students

61.20

61.40

No

--

Continuous enrollment for 3 terms

Multiple measures and assessment and placement vs. Business as usual

3 Semesters

Full sample;
12,971 students

43.40

44.00

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 48%
    Male: 47%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    New York
  • Race
    Asian
    3%
    Black
    20%
    Native American
    1%
    Other or unknown
    33%
    White
    43%
  • Ethnicity
    Hispanic    
    20%
    Not Hispanic or Latino    
    80%

Setting

Seven SUNY (The State University of New York) community colleges in New York participated in the study: Cayuga Community College, Jefferson Community College, Niagara Community College, Onondaga Community College, Rockland Community College, Schenectady Community College, and Westchester Community College. SUNY campuses are located in urban, suburban, and rural areas and serve diverse populations. All colleges that participated in the study offered both developmental-level and college-level courses.

Study sample

The sample included 12,971 prospective first-year students from seven community colleges: 6,589 in the intervention condition and 6,382 in the comparison condition. Of the students in the study sample: 48 percent were female, 20 percent were Black, 43 percent were White, 3 percent were Asian, and 1 percent American Indian/Native American. Twenty percent of students were Hispanic. About 43 percent of students were Pell Grant recipients.

Intervention Group

The intervention is an alternative, multiple measures, data analytics placement system implemented with entering first-year college students in order to place them in either remedial or college-level math and English courses as needed. Predictive algorithms were developed separately at each college using historical data that weighted different factors such as placement test scores, high school GPAs, and time since high school graduation according to how well they predicted success in college-level math and English courses. The algorithms were varied across campuses in terms of the weighting of each factor as well as the threshold to determine placement in remedial or college-level courses. As part of the development of the algorithms, each college calculated misplacement rates and used them to establish the cut points to determine students’ placement into remedial or college-level math and English courses.

Comparison Group

Students in the comparison condition were assessed for course placement (remedial or college-level) using the colleges' standard procedures, most often relying on the ACCUPLACER assessment and its' corresponding cut points.

Support for implementation

No further details were provided.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top