|Title:||Latent Variable Regression Four-Level/Five-Level Hierarchical Models for Experimental/Quasi-Experimental Studies, Evaluation Studies, and Teacher and/or School Accountability|
|Principal Investigator:||Baker, Eva||Awardee:||University of California, Los Angeles|
|Program:||Statistical and Research Methodology in Education [Program Details]|
|Award Period:||2 years||Award Amount:||$398,886|
|Goal:||Methodological Innovation||Award Number:||R305U070004|
Purpose: In the era of intervention-based experimental studies, it is not uncommon to encounter higher level, hierarchically nested data. Furthermore, a higher level data structure is, in fact, rather typical in areas of teacher and school accountability. The primary purpose of this project is to examine new extensions of commonly used hierarchical models to provide researchers with new sets of statistical tools that can help to answer important questions that could not be previously studied due to a lack of appropriate statistical methods.
Project Activities: The researchers will extend current hierarchical models (HMs) to include higher levels of nesting (e.g., four or five levels), such as occur when students are nested within classrooms, within schools, within districts. In addition, the researchers will incorporate a latent variable regression (LVR) feature into the higher level HMs (i.e., LVR-HM4s, LVR-HM5s) that can be used for experimental and quasi-experimental studies, evaluation studies, and growth modeling studies, potentially providing a new approach to estimating "value added" for purposes of evaluating teacher and school accountability. To illustrate the LVR-HM4s/HM5s models, secondary data analysis of two longitudinal data sets from large urban schools districts will be conducted. The researchers will also develop two illustrative statistical programs: annotated statistical programs of LVR-HM4s/HM5s and a user-friendly power analysis module customized to four-level/five-level HMs.
Products: The outcomes of this research will include new sets of statistical tools for researchers to use in studying the student achievement gap over time. Published reports of this research will also become available.
Purpose: The purpose of this project is to examine new extensions of commonly used hierarchical models to provide researchers with new sets of statistical tools that can help to answer important questions that could not be studied previously due to a lack of appropriate statistical methods.
Research Design and Methods: The researchers will illustrate LVR-HM4, the Multi-Site Multi-Cohort Change Model, using a multi-site, multiple-cohort longitudinal dataset from an urban district school located in a northwestern state. This dataset includes five different cohorts, and each cohort consists of longitudinal two-time points measures between the third and fifth grade in 74 elementary schools. The outcomes of interest are reading scale scores in the Iowa Test of Basic Skills (ITBS) test. These scale scores are vertically equated developmental scores. For each student in each cohort, ITBS reading scores for third and fifth grades provide the basis of estimating gains in reading achievement between two grades. The total number of students in the sample is 11,530, and the average number of students per cohort is approximately 2,306. The researchers will illustrate the LVR-HM5, Teacher Effect Change Model, using a large-scale district dataset from an urban school district in Southern California. This data set contains five years of data from 1999 to 2003 for five different cohorts from 77 elementary schools. The students are from first to fifth grades. The total number of students and teachers in this sample are 14,974 and 1,037, respectively. The outcomes of interest are Stanford Achievement Test-9 reading scores.
Data Analytic Strategy: A fully Bayesian approach using the Markov Chain Monte Carlo method will be employed to estimate the LVR-HM4 and LVR-HM5 models. Results from both analyses will be compared with those from current three-level HMs and four-level HMs, respectively, in order to directly explore the implications and value of modeling, rather than ignoring key aspects of existing data structures.
Journal article, monograph, or newsletter
Choi, K., and Seltzer, M. (2010). Modeling Heterogeneity in Relationships Between Initial Status and Rates of Change: Treating Latent Variable Regression Coefficients as Random Coefficients in a Three-Level Hierarchical Model. Journal of Educational and Behavioral Statistics, 35(1): 54–91.
Goldschmidt, P., Choi, K., Martinez, F., and Novak, J. (2010). Using Growth Models to Monitor School Performance: Comparing the Effect of the Metric and the Assessment. School Effectiveness and School Improvement, 21(3): 337–357.
** This project was submitted to and funded as an Unsolicited application in FY 2007.