Project Activities
The research team aimed to conduct a series of studies using a national representative sample of schools and several subsamples to determine VAL-ED's construct validity. They also were to conduct other studies to determine the stability of VAL-ED ratings over time and to learn about how principals, teachers, and supervisors use VAL-ED results.
Structured Abstract
Setting
The participating schools constitute a nationally representative sample.
Sample
The 300 schools participating in the VAL-ED national field test are sampled via a 2-stage cluster sampling design. The resultant sample is designed to be nationally representative. To accomplish this, a census sampling frame of school districts based upon the Common Core of Data is stratified according to geographic region (Northeast, South, Midwest, and West) and locale (urban, suburban, and rural). The sampling approach will yield 300 schools and their principals, at least 300 supervisors, and approximately 15,000 teachers.
The Vanderbilt Assessment of Leadership in Education (VAL-ED) is a paper and online assessment that utilizes a multi-rater, evidence-based approach to measure the effectiveness of principal leadership behaviors known to influence teacher performance and student learning. Teachers, the principal, and the principal's supervisor respond to the behavior inventory. The VAL-ED measures core components and key processes. Core components refer to characteristics of schools that support the learning of students and enhance the ability of teachers to teach. Key processes refer to how leaders create and manage those core components. Effective learning-centered leadership is at the intersection of the two dimensions: core components created through key processes.
Research design and methods
The research team is conducting a series of studies using a national representative sample of schools and several subsamples. Using extant data from the national sample of schools, researchers will conduct an item bias study using differential item functioning analyses and a confirmatory factor analysis study to evaluate the construct validity of VAL-ED. Researchers will use a known group study to validate VAL-ED scores against identification of top- and low- performing schools by district administrators. They will use an evidence-sampling study to validate principal, supervisor, and teacher ratings of principal portfolios against trained raters. A correlation study will test the predictive validity of VAL-ED using value-added analyses. Other studies include a retest/stability study that correlates initial VAL-ED ratings against those made at a later date by the same raters; an equivalent forms study that tests for the equivalence of different forms used to administer VAL-ED; and a consequences study that surveys a sub-set of principals, teachers, and supervisors to determine how they will use the results of VAL-ED.
Key measures
The key components of the VAL-ED assessment will include a behavior inventory or profile, interpretable from both norm-referenced and standards-referenced perspectives.
Data analytic strategy
Data analyses include differential item functioning analyses, confirmatory factor analysis, Bayesian analyses, sensitivity/specificity analyses, receiver operating characteristic analyses, and value-added modeling of student achievement.
People and institutions involved
IES program contact(s)
Products and publications
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Books
Elliott, S.N., Murphy, J., Goldring, E., and Porter, A. (2008). Handbook: Implementation and Interpretation of the Vanderbilt Assessment of Leadership in Education. Nashville, TN: Discovery Education Assessments.
Porter, A.C., Goldring, E.B., Elliott, S.N., Murphy, J., Polikoff, M., and Cravens, X. (2008). Setting Performance Standards for the VAL-ED Assessment of Principal Leadership. New York: Wallace Foundation.
Porter, A.C., Murphy, J., Goldring, E.B., Elliott, S.N., Polikoff, M.S., and May, H. (2008). VAL-ED: Technical Manual, Version 1.0. Nashville, TN: Discovery Education Assessments.
Book chapters
Murphy, J., Elliott, S.N., Goldring, E., and Porter, A.C. (2010). Leaders for Productive Schools. In E. Baker, P. Peterson, and B. McGaw (Eds.), International Encyclopedia of Education (3rd ed., pp. 746-751). Oxford, UK: Elsevier Limited.
Porter, A.C., Murphy, J., Goldring, E., Elliott, S.N., and Cravens, X.C. (2011). Vanderbilt Assessment of Leadership in Education: A New Tool for Principal Evaluation and Professional Growth. In J. Shen (Ed.), Tools for Improving Principalship (pp. 13 - 46). Berne, Switzerland: Peter Lang Publishing Group.
Journal articles
Cravens, X.C., Goldring, E., Porter, A., Polikoff, M.S., Murphy, J., and Elliott, S. (2013). Setting Proficiency Standards for School Leadership Assessment: An Examination of Cut Score Decision Making. Educational Administration Quarterly, 49(1): 124-160.
Goldring, E., Cravens, X., Murphy, J. Porter, A., Elliott, S., and Carson, B. (2009). The Evaluation of Principals: What and how do States and Urban Districts Assess Leadership?. Elementary School Journal, 110(1): 19-39.
Goldring, E., Porter, A., Murphy, J., Elliott, S. N., and Cravens, X. (2009). Assessing Learning-Centered Leadership: Connections to Research, Professional Standards, and Current Practices. Leadership and Policy in Schools, 8(1): 1-36.
Minor, E. C., Porter, A. C., Murphy, J., Goldring, E. B., Cravens, X., and Elloitt, S. N. (2014). A Known Group Analysis Validity Study of the Vanderbilt Assessment of Leadership in Education in US Elementary and Secondary Schools. Educational Assessment, Evaluation and Accountability, 26(1): 29-48.
Murphy, J.F., Goldring, E.B., Cravens, X.C., Elliott, S.N., and Porter, A.C. (2011). The Vanderbilt Assessment of Leadership in Education: Measuring Learning-Centered Leadership. Journal of East China Normal University, 29(1): 1-10.
Polikoff, M.S., May, H., Porter, A.C., Elliott, S.N., Goldring, E., and Murphy, J. (2009). An Examination of Differential Item Functioning in the Vanderbilt Assessment of Leadership in Education. Journal of School Leadership, 19(6): 661-679.
Porter, A.C., Polikoff, M., Goldring, E.B., Murphy, J., Elliott, S. N., and May, H. (2010). Developing a Psychometrically Sound Assessment of School Leadership: The VAL-ED as a Case Study. Educational Administration Quarterly, 46(2): 135-173.
Porter, A.C., Polikoff, M.S., Goldring, E., Murphy, J., Elliott, S.N., and May, H. (2010). Investigating the Validity and Reliability of the Vanderbilt Assessment of Leadership in Education. Elementary School Journal, 111(2): 282-313.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
