Skip Navigation

icon of glasses and a book Early Learning Programs and Policies

Grantees

- OR -

Investigator

- OR -

Goals

- OR -

FY Awards

- OR -

Development and Validation of the Systematic Assessment of Book Reading

Year: 2014
Name of Institution:
American Institutes for Research (AIR)
Goal: Measurement
Principal Investigator:
Pentimonti, Jill
Award Amount: $1,598,455
Award Period: 4 Years (7/1/2014-6/30/2018)
Award Number: R305A150587

Description:

Co-Principal Investigators: Ryan Bowles (Michigan State University), Tricia Zucker (University of Texas Health Science Center, Houston), Laura Justice and Mileidis Gort (Ohio State University)

Previous Award Number: R305A140430

Previous Institution: Ohio State University

Purpose: The purpose of this project is to expand and validate a new version of an observational measure called the Systematic Assessment of Book Reading (SABR), developed with funding from previous studies funded by the Institute of Education Sciences. Measuring the quality of interactive shared book reading within the early childhood classroom represents a specific dimension of teacher-child interactions that is of great interest to researchers and practitioners. However, there are very few standardized tools available for quantifying or describing young children's reading experiences.  SABR measures the quality of teacher behaviors during shared book-reading sessions. In this study, the tool will be revised for greater ease of use and accuracyby improving training materials and scoring protocols that will be disseminated at no cost via the internet. In addition, a short version will be iteratively designed for use by practitioners (e.g., coaches and other education professionals), and for use to support professional development.

Project Activities: The aims of the study are to develop the SABR-2.0, establish its psychometric properties, and develop final, online dissemination tools for a range of end users, to include researchers and practitioners. As part of this work, the researchers will add components to the existing version of the SABR (SABR-1.0) to standardize the observational protocol, broaden the relevant age range, and ensure a more comprehensive focus on teacher and child behaviors. Capitalizing on archival data from 100 classrooms coded with the SABR-1.0, the researchers will develop and pilot test a large pool of new items and iteratively test alternative coding procedures. After developing one narrative and one informational text to administer the SABR-2.0 standardized observation protocol for narrative and informational text elicitations, they will collect videos of interactive shared book-reading sessions in 300 classrooms. They will code these videos to generate a large pool of potential test items. From the initial item pool, they will identify a final item pool that can be reliably scored and demonstrates an optimal internal structure for the long form and short form. Test-retest and alternate form reliability for the narrative and informational texts will be examined. For construct validity, they will examine the relationship between SABR-2.0 scores and two existing measures of shared book-reading quality, which were not designed around the same principles of usability and scalability. Criterion-referenced validity will be established by examining predictive relations of the SABR-2.0 and children's language and literacy skills. They will develop and pilot test training materials for researchers and practitioners to ensure: (a) standardized elicitation protocols, (b) certification of observers who achieve acceptable inter-rater reliability, and (c) easy to use scoring materials. To promote widespread use, they will disseminate the long form to targeted groups of researchers and they will disseminate the short form to educational practitioners with video exemplars of possible professional development applications.

Products: The products of this project will be a fully developed and validated measure of teachers' and children's book reading behaviors, associations with children's language and literacy skills and online access to the measurement tool and training materials.  The final products for the revised SABR (SABR-2.0) will include: (a) a SABR-2.0 long form designed for video coding by researchers that will be disseminated at no cost via the researchers' websites; (b) a SABR-2.0 short form designed for coding (requiring less coding time) by educational practitioners that will also disseminated at no cost online; (c) a SABR-2.0 assessment kit including one narrative and one informational text required for standardized scoring that is available at a low cost; (d) online reliability training materials available for free download and certification to use either the long form or short form; and (e) online training materials with video exemplars demonstrating possible PD applications of the short form. Peer reviewed publications will also be produced.

Structured Abstract

Setting: This project will take place in preschool classrooms in Ohio and Texas.

Sample: The proposed research will involve 300 early childhood teachers and 900 children in their classrooms recruited from across the states of Ohio and Texas. In Ohio, 68 percent of students will meet federal criteria for free and reduced lunch. The Ohio sample will include approximately 59 percent African American, 26 percent White, 7 percent Hispanic or Latino, 2 percent Asian, and 6 percent other or biracial. In Texas, approximately 80 percent of students will meet federal criteria for free and reduced lunch.  The Texas sample will include approximately 41 percent Hispanic or Latino, 26 percent White, 24 percent African American, 6 percent Asian, and 3 percent other or biracial.

Assessment: The Systematic Assessment of Book Reading, version 1 (SABR-1.0) was developed exclusively for researchers to be able to compare results of shared book-reading interventions across studies and conduct more nuanced assessments of shared book-reading qualities across conditions. The current tool provides information on specific teacher behaviors during shared book reading that might be used in developing effective professional development, examining adult behaviors within the shared-reading context that appear to provide instructional support to children's print-related and phonological awareness skills, vocabulary and oral language skills, abstract thinking skills, and elaborative responses to the text. The SABR-1.0 also captures more general features of the reading session, including adult behaviors that create a warm, supportive setting for shared reading.

In this study, the researchers will extend the range of behaviors measured by the tool to capture child engagement and interaction. In addition, the SABR-2.0 will utilize the design feature of utterance coding, thus eliminating the need for time-intensive transcription, and will improve upon the original tool by including both long and short versions of the tool. The revised tool will add one narrative and one informational text to be used in conjunction with the tool. The addition of these two texts will improve standardized scoring and minimize the potential impact of text effects, while providing valuable information on the quality of shared book-reading sessions when different genre of texts are utilized. The end product will be a tool with four key features: (1) a more comprehensive measure of interactive shared book reading (e.g., capture child engagement and interaction with a broader age range of children and measure a range of teacher behaviors); (2) standardization of read aloud elicitation materials to reduce text effects by creating two texts to be used as part of the SABR-2.0 tool (one narrative and one informational book); (3) creation of long/research form and short practitioner form; and (4) greater ease of use and accuracy by improving training materials and scoring protocols that will be disseminated at no cost via the internet.

Research Design and Methods: The researchers will to revise and improve the SABR-1. In Year 1, they will develop items to expand and refine the measure. The team will recode archival data to support more efficient examination of new SABR-2.0 data. They will also create two SABR-2.0 companion texts (one narrative book and one informational book).  In Years 2-3, they will collect new data using the SABR-2.0 and evaluate the psychometric properties of the measure. They will collect multiple videotaped shared book reading sessions in the 300 classrooms as well as directly assess language and literacy skills of three students within each classroom (n= 900 children) in the fall and spring of the academic year.

At the end of the psychometric evaluation, they will have an assessment tool with good psychometric properties for both a long form and a short form, designed for a range of end users. The researchers will utilize information from Years 1-3 to develop final materials and inform their training development activities. Year 4 activities will focus on development of final online training and certification materials, practical guidelines for diverse end users, and product dissemination activities. They will develop and finalize an SABR-2.0 long and short form, an assessment kit, online reliability training materials and videos that demonstrate professional development application of the SABR-2.0.  The researchers will pilot test these materials with target end users (researchers and practitioners) to ensure that materials are usable and that coding reliability can be achieved.

Control Condition: Due to the nature of the research design, there is no control condition.

Key Measures: The researchers will collect data using two alternative measures of shared book reading for the purposes of establishing criterion validity.  They will use the Quality of Book Reading scale from the Early Language and Literacy Classroom Observation Pre-K Toolkit and a coding scheme developed by Hindman and colleagues. A battery of measures will be used to assess children's language and literacy skills. The Sentence Structure, Word Structure, and Expressive Vocabulary subtests from the Clinical Evaluation of Language Fundamentals-Preschool:2 (CELF-P:2)  will be used to measure children's language ability.  The Inferences Making Task will be used to measure children's inferential language skills.  The Print Knowledge and Phonological Awareness subtests of the Test of Preschool Early Literacy will be used to measure children's literacy skills.

Data Analytic Strategy:  The research team will conduct item level analyses to identify a set of items that effectively measure all important aspects of the book reading interaction. They will use a two-stage approach, an exploratory factor analysis approach followed by a Rasch measurement approach. The researchers will conduct analyses to test equivalence of the narrative and information books as elicitation materials for the shared book reading interaction.  They will use a Differential Item Functioning analysis approach with common-person linking to test whether the items behave similarly regardless of book. Additional analyses will be conducted to examine the criterion, construct, and predictive validity of the measure.