Co-Principal Investigators: Klipple, Karon; Matlen, Bryan J.; Mortrude, Judy; Walters, Kirk

Related Network Teams: This project will serve as the network research team for the CREATE Adult Skills Network, which conducts research and dissemination to support the millions of U.S. adults who have low foundational skills and includes the following other projects: Adult Skills Network Lead(R305N210014), Teaching Skills That Matter (TSTM)–SkillBlox Research Team(R305N210025), Writing in Adult Secondary Education Classes (W-ASE) (R305N210030), Adult Skills Assessment Program: Actionable Assessments for Adult Learners (R305N210031), Content-Integrated Language Instruction for Adults with Technology Support(R305N210032), Developing and Implementing a Technology-Based Reading Comprehension Instruction System for Adult Literacy Students (R305A200413)
Kick-off Webinar available for viewing here

Purpose: This research team will develop and pilot test a technology-enabled adult numeracy course and corresponding professional development designed for adult basic education learners. Nearly a third of the adult population in the United States does not have sufficient numeracy skills to manage critical quantitative tasks in their daily lives and over 16 percent have low digital literacy skills. Modern technology has the potential to significantly improve access to and the quality of adult education. Building upon the strong evidence-based instructional model and technological infrastructure of the Carnegie Math Pathways Quantway courses, the proposed digital course will provide a personalized and adaptive learning experience, designed with specific applied contexts relevant for adult learners, that supports development of numeracy and digital literacy. It will also integrate routines and interventions targeting academic mindsets and habits that promote engagement and persistence and support for English language learners. In addition, the project will develop professional development to support course instructors.

Project Activities: The team will begin by developing the course and professional development activities in collaboration with adult numeracy instructors, adapting the existing Quantway course for learners in adult basic education contexts, and they will test its usability testing in a small number of classes. They will then conduct feasibility and fidelity testing and iterative improvement of the course and professional development. Finally, they will conduct a quasi-experimental study to examine the promise of the intervention.

Products: The products of this project will include a fully developed adult numeracy course and professional development program, as well as various dissemination products, such as reports and journal articles on the implementation and results of the impact study; white papers on course design principles, core components, guidelines for implementation, and implications for interventions in adult numeracy more broadly; presentations at national conferences for both researchers and practitioners; and customized presentations for study partners in Pennsylvania.

Structured Abstract

Setting: The proposed project will take place in a variety of adult basic numeracy sites across the state of Pennsylvania including a mix of urban and rural sites.

Sample: Over the course of the project, approximately 45 adult numeracy instructors and more than 1,600 adult basic education learners will participate.

Intervention: The Carnegie Math Pathways Quantway curriculum focuses on number and operations, proportional reasoning, algebraic reasoning, mathematical modeling, and statistical thinking. In addition, Quantway addresses outcomes in multiple literacies, including quantitative literacy, reading and communication, information literacy, visual/graphical literacy, and technology literacy. The instructional design employs research-based instructional practices that support learners who are similar to those enrolled in adult basic education courses, such as collaborative learning and the use of contextualized mathematics tasks. The final ABE-ready version of this curriculum will be a 10-week course, implemented in online or blended formats. It will support development of numeracy and digital literacy through personalized and adaptive learning experiences, designed with applied contexts relevant for adult learners. The course will target the core mathematical content of the middle grades levels and support basic digital literacy skills and competencies related to the use of digital information. The course will include teacher supports and integrated video conferencing to support learner collaboration. It will also include interventions targeting academic mindsets and habits that promote engagement and persistence and supports for English language learners.

Research Design and Methods: The research team will leverage small-scale user testing as part of their iterative development. The user testing will consist of a series of small-scale, single-group studies, where adult learners will engage in targeted portions of the intervention. The researchers will leverage this feedback to further refine the curriculum. Following the development phase, they will conduct an implementation study and then a quasi-experimental matched comparison study to test the promise of the full intervention. As part of the pilot study, the researchers will collect data to run a cost analysis of the intervention.

Control Condition: The comparison condition includes teachers matched by factors such as age, education level, certification type (if any), and adult education teaching experience, using their business-as-usual course.

Key Measures: In the development phase, data will include instructor interviews, adult learner focus groups, and online course usage data. Data collected in the implementation study include instructor background characteristics and various quantitative and qualitative measures, such interviews and focus groups with instructors and adult learners, course observations, online course usage data. The implementation study will also collect preliminary evidence of impact of the course on instructor and adult learner outcomes, including leaners' opportunities to learn, mathematics achievement, digital literacy, self-efficacy, growth mindset, and course persistence, and instructors' mathematical knowledge for teaching, and self-efficacy. The pilot study will utilize these same outcomes measures.

Data Analytic Strategy: The researchers will leverage descriptive and qualitative analysis of learner and instructor feedback, as well as from observational data for the development phase. The researchers will examine learner outcome data through regression models. The researchers' primary analytic approaches include hierarchical linear modeling, regressing an adult learner or instructor outcome on condition assignment and controlling for learner, class, instructor, and agency covariates, including random effects for clustering. The analysis will be repeated for each outcome and post hoc adjustments will be applied for outcomes within the same domain to account for multiple comparisons.

Cost Analysis: Cost analysis will consist of the ingredients method and will include all expenditures on personnel, facilities, equipment, materials, and training in the pilot study. The researchers will also compute cost-effectiveness ratios to determine the amount of additional money spent on the treatment (as compared to the control group) in relation to the amount of additional achievement of the treatment group, where achievement is measured in standard deviation units.