Description:

Co-Principal Investigators: Clarke, Ben; Shanley, Lina

Purpose: The purpose of this project was to evaluate the efficacy of KinderTEK, a fully developed, tablet-based math program for young learners. Many teachers turn to technology in the hopes that online programs, applications (apps), and virtual manipulatives can provide differentiated instruction, independent practice, and engaging content for students within their classrooms. KinderTEK is a product of a prior IES development grant (R324A110286) and an OSEP Stepping Up Technology grant that can function both as a standard curriculum addressing whole number concepts and as an individualized, preventative intervention with many opportunities for differentiated instruction. The adaptive KinderTEK program blends focused whole number content and evidence-based instructional design principles with the affordances of iPads to differentiate instruction for each student.

Project Activities: The researchers evaluated the efficacy of a fully developed, tablet-based math intervention targeting kindergarten students' conceptual understanding of whole number concepts and procedural fluency in solving whole number problems. During this randomized control trial, the researchers randomly assigned classrooms to use KinderTEK (intervention) or continue with business-as-usual math practice (control).

Key Outcomes: The main findings of this study will be shared once they are reported in publicly available peer-reviewed publications.

Structured Abstract

Setting: This study took place in urban, suburban, and rural elementary schools in Oregon and Pennsylvania.

Sample: The study included 1,227 students and 70 teachers. The schools participating in the study were from economically and racially diverse communities with 14.3 percent of student families having income below the poverty level, 22.6 percent were eligible for food stamps/Snap, 18 percent received special education services, 4.7 percent were English language learners, and 17.5 percent represented minority populations.

Intervention: The KinderTEK intervention is an iPad application that is aligned with the Common Core State Standards to provide systematic and explicit mathematics instruction to kindergarten students. KinderTEK specifically targets students' conceptual understanding of whole number concepts and procedural fluency in solving whole number problems. It is an adaptive program that provides individualized instruction with learning scaffolds and formative assessments to support students with wide-ranging needs. The KinderTEK iPad app includes 51 instructional activities, as well as rewards and games for students. KinderTEK is designed to be used in 15-minute sessions, and students' progress is automatically saved from session to session. Through each version of the app (Pro Connected, Offline, Offline-Lite), teachers can track student progress and adjust students' settings. With Pro Connected, teachers can also access a website for student management, reporting, and parent communications. The KinderTEK.com website offers implementation support to all users.

Research Design and Methods: The researchers conducted a cluster randomized control trial with classrooms within schools randomly assigned to implement KinderTEK or to continue with their business-as-usual mathematics instruction and practices. Across five cohorts, they recruited kindergarten classrooms and randomly assigned to treatment and control conditions. The implementation of the intervention with cohort 1 took place in study year 1 (2017–18), cohorts 2 and 3 in year 2 (2018–19), and cohorts 4 and 5 in year 3 (2019–20). The kindergarten year of cohorts 4 and 5 was interrupted by COVID-19 school shut-downs, and neither implementation nor data collection reached levels appropriate for inclusion in the planned study. Thus, analysis was restricted to Cohorts 1 through 3, leading to 70 classrooms (43 in Oregon, 27 in Pennsylvania) in 12 districts. Prior to implementation, teachers in the KinderTEK condition completed a series of online training modules (90 minutes, required) and were encouraged to spend additional time exploring the program as both a teacher and student. Throughout implementation, the research staff monitored student and teacher use and provided implementation support, as needed.

Students in the treatment condition were expected to use KinderTEK individually at their desks or at a station in the classroom during their core math block (but not during teacher-led instruction of new content) to control for math time. Teachers had the ability to change settings (e.g., mode, minutes, on-screen student supports, and reward timing) but were encouraged to use the default sequenced mode throughout the study. Treatment teachers were asked to implement KinderTEK Pro Connected with all their students for 22 weeks with a minimum usage of 18 instructional weeks. Understanding the constraints of school schedules, student absences, and busy kindergarten classrooms, the researchers set a usage goal of 15 minutes per day, 4 days per week.

Teachers were encouraged to monitor their students' use and progress through the KinderTEK curriculum using the embedded teacher app or the accompanying teacher website. Throughout implementation, the research team monitored and provided feedback about fidelity to the recommended usage schedule and conducted classroom observations. The researchers measured outcomes primarily measured through math assessments administered to students before and after KinderTEK implementation in kindergarten, and they administered grade 1 follow-up assessments to most students in cohorts 1 through 3.

Control Condition: Kindergarten classrooms in the comparison condition employed their business-as-usual mathematics instruction and practices. KinderTEK log data confirm no treatment diffusion occurred. No comparison students used KinderTEK.

Key Measures: Key outcome measures included the researcher-developed KinderTEK Proximal Assessment and the standardized Assessing Student Proficiency in Early Number Sense (ASPENS), EasyCBM, and Measures of Academic Progress Growth K–2 (MAP). In addition, the researchers gathered qualitative and quantitative data regarding implementation fidelity, student engagement, and user perceptions through KinderTEK analytics (detailed user-data collected by the KinderTEK program), researcher-designed observations of KinderTEK use, and researcher-designed surveys of teacher and student perceptions.

Data Analytic Strategy: The researchers analyzed the effects of the KinderTEK intervention using with a random coefficients analysis that accounts for students nested in classrooms and schools.

Cost Analysis: The cost analysis estimates included all licensing fees; staff time needed for establishing Pro Connected accounts (for teachers, classroom aides/volunteers, students and parents), completing optional professional development, communicating with and supporting parents, managing devices, regularly checking student progress using the data dashboard, technical assistance; and costs for optional supplementary materials.

For a class that already owns iPads, the estimated first-year cost to implement KinderTEK Pro Connected with the parent portal/hybrid learning features activated (i.e., parents can view data and students can use school KinderTEK accounts at home) is $1,946 ($65 per student). Subsequent year costs are $1,650 ($55 per student).

For a class that purchases iPads in order to implement KinderTEK, the estimated first-year cost to implement KinderTEK Pro Connected with the parent portal/hybrid learning features activated (i.e., parents can view data and students can use school KinderTEK accounts at home) is $7,224 ($241 per student) for a 1 iPad: 2 students or $12,502 ($417 per student) for a 1 iPad: 1 student. Subsequent year costs are $2,729 (i.e., $1,650 license renewal and PD refreshers, plus $1,079 equipment replacement costs) or $91 per student for 1:2 implementations or $3,807 (i.e., $1,650 license renewal and PD refreshers, plus $2,157 equipment replacement costs) or $127 per student for 1:1 implementations.

Related IES Project: KinderTEK: Teaching Early Knowledge of Whole Number Concepts Through Technology (R324A110286)

Products and Publications

ERIC Citations:  Find available citations in ERIC for this award here.

Publicly Available Data: Data for published studies related to this project will be available from the Center on Teaching and Learning at the University of Oregon (https://ctl.uoregon.edu/research/projects/evaluation-kindertek-ipad-math-program)

Additional Online Resources and Information:
Intervention website: https://kindertek.com/.