WWC review of this study

Multiple measures placement using data analytics: An implementation and early impacts report.

Barnett, E., A., Bergman, P., Kopko, E., Reddy, V., Belfield, C., & Roy, S. (2018). Center for the Analysis of Postsecondary Readiness. Retrieved from: https://eric.ed.gov/?id=ED588752

  • Randomized Controlled Trial
     examining 
    4,729
     Students
    , grade
    PS

Reviewed: August 2019

At least one finding shows strong evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
Academic achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

Course pass rate

Multiple measures placement using data analytics vs. Business as usual

1 Semester

Full sample;
4,729 students

65.80

61.60

Yes

 
 
4
 
Access and enrollment outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

College enrollment: First semester

Multiple measures placement using data analytics vs. Business as usual

0 Semesters

Full sample;
4,729 students

81.60

80.70

Yes

 
 
1
 
Credit accumulation and persistence outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

Credits earned

Multiple measures placement using data analytics vs. Business as usual

1 Semester

Full sample;
4,729 students

5.77

5.17

Yes

N/A

Characteristics of study sample as reported by study author.


  • Female: 48%
    Male: 52%
  • Race
    Asian
    3%
    Black
    18%
    Native American
    1%
    White
    43%
  • Ethnicity
    Hispanic
    22%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    New York

Setting

The analytic sample includes 4,729 students who took a placement test at one of five State University of New York colleges at the time of fall 2016 entry. This is an interim report and there are seven colleges involved in the overall project (only five colleges were involved in the interim analyses). All students in the sample were at risk of being placed in either developmental English or math.

Study sample

Fifty-two percent of enrolled community college students in the sample were male. Forty-three percent of the sample were White, 18 percent were Black, and 22 percent were Hispanic. Forty-nine percent of all sample students enrolling in at least one course during the fall 2016 term received a federal Pell Grant for that term. The intervention sample demographics are as follows: 48 percent female, 19 percent Black, 44 percent White, 22 percent Hispanic. The comparison group demographics are: 48 percent female, 17 percent Black, 44 percent White, 22 percent Hispanic.

Intervention Group

The intervention entailed use of multiple measures to determine whether students should be placed into remedial math or English programs. That is, the use of the multiple measures system is conceptualized as the intervention. Students were placed into developmental or college-level courses using this system. The intervention involved applying a predictive algorithm based on alternative data (i.e., SAT scores and high school performance, college outcomes, placement tests) to predict students' eligibility for enrollment in college-level or developmental courses (in math and English).

Comparison Group

The status quo placement system uses an established cut score to place students.

Support for implementation

Implementation of the multiple measures placement system required a year of planning time with the research team to develop new procedures and processes. Participating colleges were initially required to help develop placement score algorithms using historical data, develop new plans for obtaining high school transcripts and other high school data, and enter these data into IT systems. Implementation efforts also required development of new placement reports and training testing staff.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top