Previous Award Number: R305M060057
Previous Awardee: LessonLab, Inc

Purpose: To address teacher assessment needs in the early 2000s, the research team proposed a new assessment approach to measure teachers' knowledge of teaching mathematics. Many of the assessments of teacher knowledge that were available at the time of this project assessed low-level or marginally relevant knowledge and not teachers' deep knowledge of subject matter and actual teaching skills. In addition, many assessments of teacher knowledge were criticized for technical shortcomings related to the instruments' reliability, inter-rater reliability, or validity.

Project Activities: In this project, the researchers proposed to develop video-analysis assessments for three pre-algebra topic areas: fractions, ratio and proportion, and equations. They were also to examine the reliability and validity of the novel assessments. Their approach drew on findings from the expert-novice literature in cognitive psychology and education and used teacher lesson analysis ability as a proxy for their knowledge of teaching mathematics. They proposed to use video-clips of classroom instruction as item prompts. Each video-analysis assessment (administered on-line) would consist of a set of video clips and an analysis task, and teachers would analyze each clip and recorded their responses in text fields.

Structured Abstract

THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING

Setting: The schools are in California.

Sample: Participants are 200 elementary and middle school teachers.

Intervention: The approach draws on findings from the expert-novice literature in cognitive psychology and education and uses teacher lesson analysis ability as a proxy for their knowledge of teaching mathematics. Video-clips of classroom instruction are used as item prompts. Each video-analysis assessment (administered on-line) consists of a set of video clips and an analysis task. Teachers will analyze each clip and record their responses in text fields.

Research Design and Methods: The research follows a mixed methods approach. The researchers will collect video-analysis assessment data and criterion measures of teacher knowledge from approximately 200 teachers and records of teaching practice and student data from approximately 40 of those teachers. The project consists of two phases. Phase 1 is designed to function as a proof of concept and consists of the development of a video-analysis assessment on fraction concepts and operations, including an estimate of the instrument's internal consistency and inter-rater reliability. Finally, phase 1 includes a comprehensive validity study, which uses the following three sources:

1. a criterion measure to assess teachers' pedagogical content knowledge on fractions
2. data from student performance assessment on fractions and student achievement data from standardized tests
3. a record of teaching practice on the topic of fractions, consisting of a lesson plan, a videotaped lesson, and a teacher post-interview.

Phase 2 is the scale-up phase of the project and two additional video-analysis assessments will be developed. One will be on ratio and proportion and one on variables, expressions, and equations. Again, the instruments' internal consistency and inter-rater reliability will be estimated. For these assessments, validity will be evaluated based on criterion measures of pedagogical content knowledge for the respective topic areas.

Key Measures: Validity will be evaluated through a criterion measure of pedagogical content knowledge, records of teaching practice, and student learning outcomes.

Data Analytic Strategy: Analyses of video-analysis data, criterion measures, and student data will be quantitative. Data will be analyzed using classical test theory and item response theory. For the validation study, video-analysis scores will be correlated with scores on the criterion measures and will be used to predict teaching practice and student outcomes. Analysis of the teaching practice records will follow a qualitative approach through the development of a coding system.

Products AND Publications

ERIC Citations: Find available citations in ERIC for this award here.

Select Publications:

Journal articles Kersting, N. (2008). Using Video Clips of Mathematics Classroom Instruction as Item Prompts to Measure Teachers' Knowledge of Teaching Mathematics. Educational and Psychological Measurement, 68(5): 845–886.

Kersting, N.B., Givvin, K., Sotelo, F., and Stigler, J. (2010). Teacher's Analysis of Classroom Video Predicts Student Learning of Mathematics: Further Explorations of a Novel Measure of Teacher Knowledge. Journal of Teacher Education, 61(1): 172–181.

Kersting, N.B., Givvin, K.B., Thompson, B., Santagata, R., and Stigler, J. (2012). Measuring Usable Knowledge: Teachers' Analyses of Mathematics Classroom Videos Predict Teaching Quality and Student Learning. American Educational Research Journal, 49(3): 568–590.

Kersting, N.B., Sherin, B., and Stigler, J.W. (2014). Automated Scoring of Teachers' Open-Ended Responses to Video Prompts: Bringing the Classroom-Video-Analysis (CVA) Assessment to Scale. Educational and Psychological Measurement, 74(6): 950–974.

** This project was submitted to and funded under Teacher Quality: Mathematics and Science Education in FY 2006.