IES Blog

Institute of Education Sciences

Developing the Vanderbilt Assessment of Leadership in Education (VALED)

As education accountability policies continue to hold school leaders responsible for the success of their schools, it is crucial to assess and develop leadership throughout the school year. In honor of the IES 20th Anniversary, we are highlighting NCER’s investment in leadership measures. This guest blog discusses the Vanderbilt Assessment of Leadership in Education (VALED). The VALED team was led by Andy Porter and included Ellen Goldring, Joseph Murphy and Steve Elliott, all at Vanderbilt University at the time. Other important contributors to the work are Xiu Cravens, Morgan Polikoff, Beth Minor Covay, and Henry May. The VALED was initially developed with funding from the Wallace Foundation and then further developed and validated with funding from IES.

What motivated your team to develop VALED?

There is currently widespread agreement that school principals have a major impact on schools and student achievement. However, at the time we developed VALED, we noticed that there were limited research-based instruments to measure principal leadership effectiveness aligned to both licensure standards and rooted in the evidence base. Prior to the VALED, principal leadership evaluation focused primarily on managerial tasks. However, we believed that principal leadership centered on improving teaching and learning, school culture, and community and parent engagement (often called learning-centered leadership) is at the core of leadership effectiveness.

What does VALED measure?

The VALED is a multi-rater assessment of learning-centered leadership behaviors. The principal, his/her supervisor, and teachers in the school complete it, which is why VALED is sometimes referred to as a 360 assessment or multi-source feedback.

VALED measures six core components and six key processes that define learning-centered leadership. The core components are high standards for student learning, rigorous curriculum, quality instruction, culture of learning and professional behavior, connections to external communities, and performance accountability. The key processes are planning, implementing, supporting, communicating, monitoring, and advocating.

How is the VALED different from other school leadership assessments?

The VALED is unique because it focuses on school leadership behaviors aligned to school improvement and school effectiveness, incorporates feedback and input from those who collaborate closely with the principal, includes a self- assessment, acknowledges the distributed work of leadership in a school, and has strong psychometric properties. We think there are several elements that contribute to the uniqueness of the instrument.

First, VALED is based on what we have learned from scholarship and academic research rather than less robust frameworks such as personal opinions and or unrepresentative samples. The VALED was crafted from concepts identified as important in that knowledge and understanding. The VALED model is based upon knowledge about connections between leadership and learning and provides a good deal of the required support for the accuracy, viability, and stability of the instrument.

Second, principals rarely receive data-based feedback, even though feedback is essential for growth and improvement. The rationale behind multi-source or 360-degree feedback is that information regarding leadership efficacy resides within the shared experiences of teachers and supervisors, collaborating with the principal, rather than from any one source alone. Data that pinpoint gaps between principal’s own self-assessment, and their teachers’ and supervisors’ ratings of their leadership effectiveness can serve as powerful motivators for change.

Finally, in contrast to some other leadership measures, VALED has undergone extensive psychometric development and testing. We conducted a sorting study to investigate content validity and a pilot study where we addressed ceiling effects, and cognitive interviews to refine wording. We also conducted a known group study that showed the tool’s ability to reliably distinguish principals, test-retest reliability, convergent-divergent validity, and principal value-added to student achievement. As part of this testing, we identified several key properties of VALED. The measure—  

  • Works well in a variety of settings and circumstances
  • Is construct valid
  • Is reliable
  • Is feasible for widespread use
  • Provides accurate and useful reporting of results
  • Is unbiased
  • Yields a diagnostic profile for summative and formative purposes
  • Can be used to measure progress over time in the development of leadership
  • Predicts important outcomes
  • Is part of a comprehensive assessment of the effectiveness of a leader's behaviors

What is the influence of VALED on education leadership research and practice?

VALED is used in schools and districts across the US and internationally for both formative and evaluative purposes to support school leadership development. For example, Baltimore City Public Schools uses VALED as a component of their School Leader Evaluations. VALED has also spurred studies on principal evaluation, including the association between evaluation, feedback and important school outcomes, the implementation of principal evaluation, and its uses to support principal growth and development. In addition, it provides a reliable and valid instrument for scholars to use in their studies as a measure of leadership effectiveness.


Andy Porter is professor emeritus of education at the Pennsylvania State University. He has published widely on psychometrics, student assessment, education indicators, and research on teaching.

Ellen Goldring is Patricia and Rodes Hart Chair, professor of education and leadership at Vanderbilt University. Her research interests focus on the intersection of education policy and school improvement with emphases on education leadership.

Joseph Murphy is an emeritus professor of education and the former Frank W. Mayborn Chair of Education at Peabody College, Vanderbilt University. He has published widely on school improvement, with special emphasis on leadership and policy and has been led national efforts to develop leadership standards. 

Produced by Katina Stapleton (Katina.Stapleton@ed.gov), program officer for NCER’s education leadership portfolio.

 
 
 

NCES Celebrates LGBTQ+ Pride Month

June is LGBTQ+ Pride Month, and NCES is proud to share some of the work we have undertaken to collect data on the characteristics and well-being of sexual and gender minority populations.

Inclusion of questions about sexual orientation and gender identity on federal surveys allows for better understanding of sexual and gender minority populations relative to the general population. These sexual orientation and gender identity (SOGI) data meet a critical need for information to understand trends within larger population groups, and insights gained from analysis of the data can lead to potential resources and needed interventions being provided to better serve the community. Giving respondents the opportunity to describe themselves and bring their “whole self” to a questionnaire helps them to be seen and heard by researchers and policymakers.

Sometimes, NCES is asked why questions like this appear on an education survey. They can be sensitive questions for some people, after all. NCES asks these questions to be able to understand the different experiences, equity, and outcomes related to education for sexual and gender minorities, just as NCES does for groups identified by other demographic characteristics like race, ethnicity, household income, and what part of the country someone lives in. By sexual minorities, we mean people who report their sexual orientation to be something other than straight or heterosexual, and by gender minorities, we mean people whose sex as recorded at birth is different from their gender.

Over the past 10 years, NCES has researched how to best ask respondents about their sexual orientation and gender identity, how respondents react to these questions, and the quality of data that NCES has collected on these characteristics.

At NCES, several studies include background questions for adults about their sexual orientation and gender identity. These are the High School Longitudinal Study: 2009 (HSLS:09) Second Follow-up in 2016, the Baccalaureate and Beyond Longitudinal Study (B&B) 08/18 and 16/21 collections, the National Postsecondary Student Aid Study (NPSAS) in 2020, and the Beginning Postsecondary Students Longitudinal Study (BPS) 2020/22 (see table below for more details about these surveys).


 


The collection of these data allows NCES to describe the experiences of gender and sexual minority individuals. For example:

  • In 2020, postsecondary students who identified as genderqueer, gender nonconforming, or a different identity had difficulty finding safe and stable housing at three times the rate (9 percent) of students who identified as male or female (3 percent each).1
     
  • In 2018, about 10 years after completing a 2007–08 bachelor’s degree, graduates who were gender minorities2 described their financial situations. Graduates who were gender minorities were less likely to own a home (31 percent) or hold a retirement account (74 percent) than graduates who were not gender minorities (63 percent and 87 percent, respectively) (figure 1).3  

Figure 1. Percentage of 2007–08 bachelor’s degree recipients who owned a home, had a retirement account, reported negative net worth, and did not meet essential expenses in the past 12 months, by gender minority status in 2018

NOTE: “Retirement account” includes both employer-based retirement accounts such as 401(k), 403(b), and pensions, and non-employer-based retirement accounts such as individual retirement accounts. Respondents are considered to have negative net worth if they would still be in debt after selling all their major possessions, turning all their investments and other assets into cash, and paying off as many debts as they could. “Did not meet essential expenses” refers to being unable to meet essential living expenses such as mortgage or rent payments, utility bills, or important medical care. “Past 12 months” refers to any of the 12 months preceding the interview. Gender minority indicates whether the respondent’s gender identity differed from the sex assigned at birth. Gender identity categories include male; female; transgender, male-to-female; transgender, female-to-male; genderqueer or gender nonconforming; a different gender identity; and more than one gender identity.
SOURCE: U.S. Department of Education, National Center for Education Statistics, 2008/18 Baccalaureate and Beyond Longitudinal Study (B&B:08/18).


  • In the 2017–18 school year, 18 percent of public schools had a recognized student group that promoted the acceptance of students’ sexual orientation and gender identity, such as a Gay-Straight Alliance (GSA). This was an increase from the 2015–16 school year, in which 12 percent of schools reported having a GSA.4
     
  • For 2008 bachelor’s degree graduates with a full-time job in 2018, straight people reported higher average salaries than either lesbian/gay or bisexual people.  

NCES is committed to collecting data about equity in education and describing the experiences of SGM students, graduates, and educators.

To learn more about the research conducted at NCES and across the federal statistical system on the measurement of SOGI, please visit the Federal Committee on Statistical Methodology (FCSM) website and check out these two presentations from the FCSM 2022 Research and Policy Conference: How do you Describe Yourself in the Workplace? Asking Teachers about their Sexual Orientation and Gender Identity in a School Survey and Assessing Open-Ended Self-Reports of Sexual Orientation and Gender Identity: Is There Room For Improvement?.

 

By Maura Spiegelman and Elise Christopher, NCES


[1] U.S. Department of Education, National Center for Education Statistics, 2019–20 National Postsecondary Student Aid Study (NPSAS:20, preliminary data).

[2] On the NCES surveys mentioned above, gender identity categories include male; female; transgender, male-to-female; transgender, female-to-male; genderqueer or gender nonconforming; a different gender identity; and more than one gender identity.

[3] U.S. Department of Education, National Center for Education Statistics, 2008/18 Baccalaureate and Beyond Longitudinal Study (B&B:08/18).

[4] U.S. Department of Education, National Center for Education Statistics, 2015–16 and 2017–18 School Survey on Crime and Safety (SSOCS).

Intersecting Identities: Advancing Research for Racialized English Learners

This year, Inside IES Research is publishing a series of blogs showcasing a diverse group of IES-funded education researchers and fellows that are making significant contributions to education research, policy, and practice. In recognition of Asian American and Pacific Islander Heritage Month, in this interview blog we asked Ben Le, an IES Predoctoral Fellow at New York University and a team member of the IES-funded R&D Center on the Success of English Learners (CSEL), to discuss his career journey and research interests.

How have your background and experiences shaped your scholarship and career in studying diversity, equity, and inclusion in education?

My research interests center around how race/ethnicity and language intersect to create unique privileges and discrimination. I hope my research can explore different ways we can support racially and linguistically marginalized students in schools, allowing them to bring their complete selves into the classroom and to help them thrive without having to give up their familial and communal languages.

Growing up in the United States as a Vietnamese-Mexican man has motivated me to look for new ways that we can conceptualize barriers for linguistically and racially marginalized students. While English learners (ELs) are currently the primary focus of my research, I’d like to recognize that I have never been classified as an EL.

I have been fortunate enough to be part of the IES-funded NYU Predoctoral Interdisciplinary Research Training (IES-PIRT) program, which has provided me the opportunity to  further explore and better understand racialized ELs’ access and opportunity in the classroom. My hope is that my IES-PIRT training will prepare me to work closely with local communities and organizations to enact change in our school systems. Ideally, we can build systems that truly support linguistically and racially marginalized students while offering them both access and opportunity that prepares them for life after school.

Can you tell us about your current IES-funded project?

As part of the CSEL R&D Center work, I am using a quantitative intersectional lens to highlight the importance of race/ethnicity for the diverse group of ELs in New York City public schools. I am particularly interested in how patterns of high school and college outcomes for current and former ELs vary based on race/ethnicity and gender. Focusing on 6-year graduation rates, I disaggregated my sample by race/ethnicity, gender, and ever-EL status (whether the student has ever been classified as an EL) to compare the probabilities across these subgroups and look for differential probabilities of being an ever-EL and a specific race/ethnicity. I focused on the two largest racial/ethnic groups of ELs in New York City, Asian Pacific Islander (PI) and Latine. For example, I compared the probability to graduate within 4 years between never-EL Asian/ (PI) young women to ever-EL Latino young men.

Interestingly, results, which were presented at the 2022 American Educational Research Association Annual Meeting, show that student probabilities for 6-year graduation are primarily organized by race/ethnicity, with Asian/PI students outperforming Latine students. Additionally, young women tend to outperform young men of their same racial/ethnic group, and in general, ever-EL status seems to matter even more for young men than young women. But these patterns do not explain away the racial/ethnic disparities seen in this New York City data. While ever-EL status matters, on aggregate, the ever-EL and never-EL differences primarily exist within racial/ethnic and gender subgroups. For example, never-EL Asian/PI young men outperform ever-EL Asian/PI young men, but ever-EL Asian/PI young men still outperform never-EL Latina young women.

Through my research, I hope to highlight the diversity and nuance within this ever-EL population, not to argue that ever-EL status does not matter. Instead, these findings have only motivated me to continue centering race/ethnicity and gender in future analyses for ELs.

What do you see as the greatest research needs to improve the relevance of education research for diverse communities of students and families?

From my perspective, we need to center the voices and concerns of these communities, families, and students in our data collection and analysis. I think it is essential to be involved with the families and meet them where they are to find effective solutions that benefit the communities we strive to serve. We need to make sure we are uplifting underserved families’ voices instead of talking over them. Relatedly, we need data and data collection to reflect the nuances and intricacies we are trying to discuss. Hopefully, future data collection can more accurately reflect the identities of the students we study. For example, I hope we can move away from collecting data as “male/female” and have a more expansive understanding of gender identities and not reify the gender binary.

What advice would you give to graduate students from underrepresented, minoritized groups that are pursuing a career in education research?

My first piece of advice would be to remember your own lived experiences and try to remind yourself that you do deserve to be in your graduate program. It’s easy to feel imposter syndrome—I think a lot of us do. Historically, academia and these programs were not made for us, and sadly, there is still a lot of work to be done, so that we don’t need to change to fit into these spaces. Still, these institutions and research fields benefit from our voices and perspectives. Remembering that these programs need us and that our experiences matter may be easier said than done, but I find it helpful to surround myself with fellow critical scholars and peers both within and outside of academia.

Secondly, finding community and support from peers and mentors has been absolutely crucial for my research and mental health. Doctoral programs aren’t easy; you are constantly being challenged intellectually and then you have to put your ideas and work out to be judged and critiqued. Being able to lean on friends and mentors for emotional support and to challenge and refine your research ideas is key to having a good and productive experience. I am super fortunate at NYU, through my sociology of education program and the IES-PIRT program, to have found such a caring community and supportive mentors, while also being pushed and challenged to pursue better and more critical work.


Produced by Helyn Kim (Helyn.Kim@ed.gov), program officer for the English Learners portfolio, NCER.

 

Releasing CCD Nonfiscal Data

The Common Core of Data (CCD) contains basic information on public elementary and secondary schools, local education agencies (LEAs), and state education agencies (SEAs) in the United States. The CCD collects fiscal and nonfiscal data about all public schools, public school districts, and state education agencies in the United States. Both IPEDS and CCD provide a sampling frame to many survey collections, including many conducted by NCES and the Department of Education. This blog post, one in a series of posts about CCD nonfiscal data, focuses on CCD’s two major releases and their corresponding components. For information on how to access and use CCD data, read the blog post Accessing the Common Core of Data (CCD).
 

Data Releases

CCD nonfiscal data are published in two releases every school year—as preliminary files and as provisional data files—within the CCD Data File tool. Understanding the differences between the two releases is important to understand how CCD nonfiscal data are released.

  • The preliminary files contain basic information about schools and districts, such as name, address, phone number, status, and NCES ID number. Many schools and districts utilize information from the directory file, such as the NCES ID, to apply for grants or other opportunities for their schools. Therefore, it is important that these files are released first, even if the data are still preliminary. 
     
  • The provisional data files are the full release of the CCD nonfiscal data. These data files provide school-, district-, and state-level data on topics like enrollment, staffing, and free or reduced-price lunch. These files are much more detailed and include data that are broken down by characteristics such as grade, race/ethnicity, and gender—as well as by combinations of these characteristics. These files are not updated unless there is a significant change to the data.

Each file release includes a version that indicates the type of release. The first preliminary files have “0a” in the file names, and revised preliminary files include “0b,” “0c,” and so on. The first provisional files have “1a,” in the file names, and revised provisional files include “1b,” “1c,” and so on. Note, however, that releasing revised files is rare.
 

Components of a Release

It is important to utilize the various components that accompany each release to find additional information that is specific to the file and can help you better understand the data. In addition, there are other resources available that provide more ways to access and understand the data.
 

Documentation Components

Every data file will have documentation files that provide information about the data. These include the following:

  • release notes—basic information about the data release, including details about any changes to the files, such as a change in a variable’s description or a variable that was added to the file; summary tables that include national totals and tables with selected frequencies are also included.
     
  • state data notes—information on data anomalies that are discovered during NCES’s collaboration with the states; broken down by state and by file type, these notes describe things like changes to how data were collected by the state.
     
  • companion files—included in each data file component, these files include a list of all the variables in the data file—including a brief description—and frequency tables; you should start with the companion files to better understand what variables are in each data file.


Resources and Tools

Along with the release of the CCD nonfiscal data files, additional resources are also updated to improve access to the data.

  • Summary Tables: Released with the provisional data files, Summary Tables provide a national-level look at the data. These tables show the operational status of schools and districts by type as well as the number of schools, students, and teachers by state.
     
  • Locators and ElSi: There are two primary tools that can be used to access CCD data: the Locators (School Locator and District Locator) and the Elementary/Secondary Information System (ElSi). These tools are updated as the data files are released. The Locators are updated with each release, while ElSi is updated with the release of the provisional data files. Learn more about these tools.
     
  • Online Documentation: The online documentation provides some general information about CCD. This information is not year specific, but it provides a detailed explanation about how the data are collected, processed, and reviewed.
  • Reference Library: The reference library includes detailed documentation on various components of the CCD files that applies to multiple years, levels, and components of the data collection. The library includes crosswalks, documentation describing changes to the collection, and guidance for utilizing the data files, such as how to aggregate free or reduced-price lunch data.

Be sure to follow NCES on TwitterFacebookLinkedIn, and YouTube and subscribe to the NCES News Flash to stay up-to-date on future CCD releases and resources.

 

By Patrick Keaton, NCES

Delivering Mental Health Supports to Adolescents in School Settings

Mental Health Awareness Month was established in May of 1949 to increase awareness of the importance of mental health and wellness in Americans’ lives and to celebrate improvements in mental health treatment. Nearly 75 years later as we honor mental health awareness in this country, we are confronted with an alarming rise in mental health needs among America’s youth that the U.S. Surgeon General advises could be helped in part with school-based prevention and intervention supports.

One approach to school-based mental health is being tested in an IES-funded efficacy replication study called School Adolescent Mood Project: Efficacy of IPT-AST in Schools. This project is being led by Dr. Jami Young, associate chair of research in the Department of Child and Adolescent Psychiatry and Behavioral Sciences, faculty member in the PolicyLab at the Children’s Hospital of Philadelphia (CHOP), and professor of psychiatry at the University of Pennsylvania Perelman School of Medicine. Dr. Young was interviewed by Rebecca Sun, IES Intern, to help us learn more about conditions on the ground in her study and how telehealth-delivered Interpersonal Psychotherapy-Adolescent Skills Training (IPT-AST) is working in these schools.

Tell us about the schools you are working in and the different mental health needs you are seeing. How do you think the pandemic influenced what you are seeing now? 

In the School Adolescent Mood Project (SAM project), we have been working closely with 16 schools in Pennsylvania and New Jersey that serve 9th and 10th grade students. Over the past three years, what we are seeing in these 16 schools is very different than what we are used to seeing in our past two decades of school-based depression prevention work. Compared to our earlier studies (see this one for example), in the current SAM project, the rate of teens with elevated scores on our depression screening has more than doubled, as has the percentage of students with scores indicating more significant depression.

The increases we have seen in the SAM project mirror increases in depression and anxiety that have been reported by others during the pandemic, including the most recent findings reported by the CDC. Our still unpublished screening data from the SAM project suggest that there are a substantial number of adolescents who are experiencing mental health difficulties and would benefit from services in schools and/or the community. There is much work that needs to be done, including increases in funding and workforce expansion, to grow and sustain school-based services to support these students. Additionally, establishing connections between schools and mental health agencies is essential, so students who need more intensive services can receive timely care.

What do the youth in your study say are their biggest concerns? 

We are seeing many adolescents who are reporting significant issues with depression and anxiety. Unfortunately, we have also seen an increase in adolescents who report suicidal thoughts and behaviors. The pandemic was difficult for many adolescents as they experienced school closures, loss of loved ones to COVID-19, and feelings of loneliness and social isolation. Even with the return to in-person learning and afterschool activities in the 2021-2022 school year that allowed many students to regain a sense of normalcy, other adolescents found the return to school to be a challenging adjustment. In the SAM project, we saw an increase in the number of adolescents who reported serious thoughts of wanting to kill themselves or who had made a suicide attempt from the 2020-2021 school year when most schools were virtual or hybrid to the 2021-2022 school year when schools were back to in person learning. Unfortunately, we have continued to see an increase in the percentage of adolescents reporting thoughts of wanting to kill themselves this academic year.      

Our school partners have also noted a significant increase in students with suicide-related concerns. One school counselor shared, “I think we have done more suicide risk assessments this year than the past 5 years combined.” This points to a critical need to fund and implement evidence-based screening, assessment, and prevention initiatives for youth suicide.

What have you learned so far about the feasibility and effectiveness of providing the IPT-AST intervention in a telehealth format?

We needed to think creatively about how to ensure that students received the support that they needed and to consider how we could promote continuity of care if schools needed to close again. We made the decision to deliver IPT-AST through telehealth. We thought the COVID pandemic was an important opportunity to capitalize on the recent growth and innovations in digital health and to study the acceptability and efficacy of IPT-AST when delivered through telehealth.

We’ve interviewed a subset of adolescents who participated in our IPT-AST groups to get their feedback about the program, including how they felt about IPT-AST being delivered through telehealth. We are still analyzing these interviews and looking at our other data on acceptability and feasibility, including attendance and satisfaction data. Anecdotally, we have noticed a change in the acceptability of telehealth over the course of our study. In our first year, providing our group online meant that students could receive services and interact with their peers even when school was closed. As we have moved further from school closures, more adolescents have expressed a desire to return to in-person groups. Several adolescents have suggested we consider hybrid models that incorporate both in-person and virtual sessions. One positive benefit of running IPT-AST groups online is that we have been able to include students from different schools in the same group. If we find positive benefits of telehealth-delivered IPT-AST, it will be interesting to see whether some schools and districts prefer to continue this approach as it will enable providers to deliver these services outside of the school day.

Do you think the pandemic has eased the stigma surrounding seeking mental health?

We do not measure stigma specifically in our study but hope that all the attention around mental health has been beneficial. As May is Mental Health Awareness Month, we think it is important to emphasize that mental health is health. If we can continue to spread this message, stigma around mental health will continue to decrease. We are hopeful that the increased focus on youth mental health, decreases in stigma, and proposed policy solutions to increase access to mental services will mean that more adolescents will be able to get the care they need.


This blog was produced by Emily Doolittle (Emily.Doolittle@ed.gov), NCER team lead for social behavioral research.