Project Activities
The researchers will conduct exploratory work to understand the context of implementation of TSTM to date. They will leverage this information to guide the development of digital educational resources by refining the online SkillBlox platform and working with instructors to increase its content and usability while also aligning content to TSTM. They will then pilot test the fully developed platform and conduct a cost analysis of TSTM-aligned SkillBlox.
Structured Abstract
Setting
The study will focus on adult education programs in 10 states—5 states where the TSTM initiative was first piloted (Arizona, Arkansas, Illinois, Pennsylvania, and Virginia) and 5 to be sampled from the remaining 18 TSTM states.
Sample
During the exploratory phase, 40 master teachers and 200 classroom teachers will participate. During the development phase of the SkillBlox, up to 200 teachers and 1,000 students will participate. The pilot test of the TSTM-aligned version of SkillBlox will include up to 150 programs, 1000 adult education teachers, and more than 25,000 students in the study states.
Intervention
The TSTM-aligned version of SkillBlox will build off an emerging SkillBlox platform that organizes instructional content according to adult education topics and skill levels as defined by the College and Career Readiness Standards for Adult Education. Additional updates to SkillBlox are underway that provide expanded functionality for both instructors and students. Instructors will be able to locate, organize, share, and assign content by TSTM skills and topic areas, and students will be able to access and save their work on assigned skill blocks. This project will add extensive TSTM content to SkillBlox and make enhancements based on usability testing with instructors. The resulting pilot version of TSTM-aligned SkillBlox will be designed to allow instructors to easily locate a TSTM-aligned skill they want to teach; select relevant lessons and high quality OERs, such as articles, videos, games, simulations, and assessments they wish to use to teach that skill; and then assign skill blocks to students through the platform.
Research design and methods
In the first phase of work, the researchers will explore the current rollout of TSTM in 10 study states, using a mixed-methods design that combines (a) qualitative data from interviews and focus groups with (b) quantitative, state-provided administrative data on program participation and learner outcomes used for OCTAE's National Reporting System (NRS). In the second phase, the researchers will collaborate with practitioners and students to iteratively develop the TSTM-aligned SkillBlox platform and associated OER content and supports and conduct usability and feasibility testing. For the iterative development, they will use a design-based process to refine the initial SkillBlox design by testing and revising design assumptions through iterative sprint cycles. During the third phase, the researchers will test the promise of the intervention using a randomized control trial (RCT) in multiple states and program sites. The RCT will also include an implementation and cost study.
Control condition
The control group will have access to regular professional development opportunities and supports available in the 10 study states, including those related to TSTM, as well as publicly available OER. However, they will not be able to access the TSTM-aligned version of SkillBlox.
Key measures
The exploratory and pilot study measures include both teacher and student outcomes. Teacher outcomes include measures of instructional practices, use of instructional materials (including OER), self-efficacy, student expectations, and familiarity with the principles of the TSTM initiative. Student outcomes include course completion; measurable skill gains; and post-program credential attainment, postsecondary enrollment, and employment. Measures for the design and development effort include OER usage, teacher feedback on utility and effectiveness of SkillBlox features and OER, student feedback on utility and relevance of TSTM-aligned OER developed via SkillBlox, and teacher and student feedback on SkillBlox content and iterative prototypes.
Data analytic strategy
For the exploratory mixed-methods study, the researchers will use a number of approaches, including thematic analysis, summary statistics, and simple regression analyses. For the iterative development, they will rely primarily on qualitative, inductive, open coding and thematic analysis. For the pilot study analysis, the researchers will use multilevel regression methods to measure the impact of TSTM-aligned SkillBlox, controlling for contextual program-, teacher-, and student-level background variables, along with qualitative thematic methods to measure implementation and identify potential barriers in using TSTM-aligned SkillBlox.
Cost analysis strategy
The researchers will use the "ingredients" method to estimate the cost of using TSTM-aligned SkillBlox and related OER, such as the cost for the time teachers spend learning how to use these digital supports for TSTM.
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project will include an analysis of and reports on the TSTM rollout; a developed platform and open educational resources (OERs) to support adult education learners, educators, and programs; initial evidence of the promise of the TSTM-aligned OER platform including analysis of its costs; and peer-reviewed publications.
ERIC Citations: Find available citations in ERIC for this award here.
Kick-off Webinar available for viewing here
Related projects
Supplemental information
Co-Principal Investigators: Vanek, Jenifer
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
