Co-Principal Investigator(s): Dole, Janice A.

Purpose: In this project, the researchers propsed to develop and pilot a reliable, comprehensive, and practical primary-grade measure called the Evidence-based Reading/Writing Teacher Knowledge Assessment System (ERWTKAS). ERWTKAS was intended to measure teachers' knowledge about effective, evidence-based reading and writing instruction. By the end of this project, the researchers aimed to have completed the development of ERWTKAS as well as validated the measure against students' reading and writing achievement.

Structured Abstract

THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING

Setting: The project takes place in elementary schools in Utah.

Sample: In this study, 45 schools, 135 classroom teachers, and 900 students (grades 1-3) in Utah will participate.

Assessment: ERWTKAS will be designed to comprehensively measure teachers' knowledge about effective, evidence-based reading and writing instruction by focusing on the knowledge teachers possess and how that knowledge is enacted within a classroom setting.

Research Design and Methods: ERWTKAS will be designed across three phases. In the first phase, the researchers will identify, code, and organize essential information about what teachers need to know about effective, evidence-based reading and writing instruction. They will use a systematic literature review, classroom observations and interviews of primary grade teachers, teacher focus groups, and an expert audit trail review of these other data to verify interpretation, coding, and categorization. In phase 2, finalized categories, themes, and items from phase 1 will be used in conjunction with modern test development theory to determine the format, content, subscales, and items to be included in a two-subscale assessment system. The two subscales are an objective test of knowledge about effective, evidence-based reading and writing instruction and a classroom observation rating scale to assess enacted knowledge. In phase 3, a predictive validity study will be conducted to determine how well teacher ERWTKAS scores predict student reading and writing achievement.

Key Measures: The teacher measure will be the ERWTKAS scales. The student measures will include the following standardized reading and writing tests and subtests: Dynamic Indicators of Basic Early Literacy Skills (DIBELS) test, Iowa Test of Basic Skills (reading subtest), Stanford Achievement Test – 10 (reading subtest), Gates-MacGinitie Reading Test, IPT Early Literacy-English Writing Test, and the Test of Early Written Language – 2, Form A.

Data Analytic Strategy: The ERWTKAS and subscale scores will be analyzed using traditional test and measurement approaches to test-retest, internal consistency, rater objectivity for reliability, and face and content validity. Confirmatory factor analysis will be used to examine the construct validity of the ERWTKAS. Generalizability and multi-facet Rasch measurement analyses on the classroom observation ERWTKAS subscale scores will be conducted to further strengthen the reliability of the observation ratings. Multi-level statistical modeling techniques will be used to evaluate the predictive validity of the primary grade teacher ERWTKAS subscales and total student scores on selected nationally standardized reading and writing tests.

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Select Publications:

Journal articles

Reutzel, D. R., Dole, J. A., Read, S., Fawson, P., Herman, K., Jones, C. D., Sudweeks, R., and Fargo, J. (2011). Conceptually and Methodologically Vexing Issues in Teacher Knowledge Assessment. Reading and Writing Quarterly, 27(3): 183–211.

** This project was submitted to and funded under Teacher Quality: Reading and Writing in FY 2005.