Project Activities
The researchers will conduct a randomized control trial (RCT) across three institutions. The researchers will compare the outcomes for students who receive DAACS to those who receive business as usual to determine whether DAACS improves postsecondary outcomes. They will also conduct a cost-analysis and cost-effectiveness study.
Structured Abstract
Setting
This project will take place at three postsecondary institutions: one a traditional brick-and-mortar institution in New York one blended (online and in-person), and one fully online.
Sample
Approximately 36,000 newly enrolled undergraduates will participate over the course of the study.
DAACS is a comprehensive intervention that aims to help students transition into postsecondary education by providing them and their advisors valuable feedback to navigate the early demands of postsecondary education. DAACS has five components:
- Diagnostic assessments of students' readiness for college in terms of self-regulated learning, reading, writing, and mathematics
- Feedback, strategies, and links to open educational resources that help students address deficiencies
- Automated nudges to encourage students to utilize DAACS resources
- Information and support that enables academic advisors to help students address deficiencies identified by the assessments, including a DAACS dashboard and professional development on DAACS and SRL
- Predictive models that identify students at risk, as well as the specific risk factors.
Newly starting postsecondary students learn about DAAC during orientation. The students take four online assessments of college readiness, including mathematics, reading, writing, and self-regulated learning. The DAACS gives them immediate feedback, along with suggestions and links to websites they can use to boost their knowledge and skills. Students' academic advisors use their DAACS results to inform advising conversations and guide students' self-directed learning. Institutional leaders use DAACS data for early identification of students who are at-risk of academic failure and to direct them to individualized supports.
Research design and methods
The researchers will conduct a randomized controlled trial at the three institutions, randomly assigning students at each institution to a treatment or control group. The researchers will track students' early credit acquisition, retention, academic achievement, and DAACS use (treatment only) for the duration of the grant. They will also monitor implementation and conduct interviews with advisors, students, and institutional leaders to gather information about their perceptions of DAACS. To determine the impact on students, they will conduct an intention-to-treat (ITT) analyses to whether being assigned to the intervention had an impact on students, and they will also investigate the effect of actually receiving the intervention by estimating the average effect of taking up the intervention among only the students who actually complied with the assignment to condition.
Control condition
Students in the control condition will receive the business-as-usual student orientation at their institution, without DAACS.
Key measures
Short-term outcomes include engagement, as measured by students' and advisors' use of DAACS (clicks on assessment results, feedback, resources, or advisor dashboards). Intermediate academic outcomes include time to first credit completion, ratio of credits earned to attempted, and term-to-term retention. Long-term outcomes include one-year retention, grade point average, and graduate rate. Moderating variables include amount and of DAACS use and the components of DAACS used, student characteristics (baseline academic achievement, socio-economic status, demographics, student status), and type of institution (traditional, blended, online).
Data analytic strategy
The researchers will use regression analyses to test moderator effects and predictive models to examine the effects of DAACS results on the accuracy of predictions of students' academic success.
Cost analysis strategy
For the cost analysis, the researchers will use the ingredients method to identify and compute all costs and will use incremental cost-effectiveness ratios to determine the cost effectiveness of implementing DAACS.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The researchers will produce evidence of the efficacy, or lack thereof, of DAACS for improving student outcomes as well as the cost and cost effectiveness of DAACS. They will share their findings through scholarly publications and presentations to practitioners and postsecondary policymakers.
Supplemental information
Co-Principal Investigators: Andrade, Heidi; Cleary, Timothy
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
