Co-Principal Investigator: Gresham, Frank; Cook, Clayton R.

Purpose: The purpose of this project is to develop, refine, and validate the Psychosocial and Educational Difficulties and Strengths Scales (PEDS) to evaluate student behavior from multiple sources (e.g., teachers, parents, students) and to produce results that accurately indicate how student behavior may vary across contexts. Collecting multiple informant ratings of children's psychosocial strengths and difficulties is a common practice during comprehensive special education evaluations. Discrepancies often exist, however, among different informants' ratings of child social and emotional functioning, presenting practical challenges for school personnel when interpreting and using this information. Ratings scales typically do not include guidelines for interpreting discrepancies among informants, resulting in practitioners making subjective decisions based on discrepant reports. Historically, differences in ratings have been attributed to measurement error (e.g., reliability of ratings). Students, however, often behave differently in different contexts (e.g., home versus school) so differences in ratings may be expected. Identifying the contexts in which children experience psychosocial strengths and difficulties, and providing practitioners with information regarding how to understand and address those differences, may facilitate the use of individualized interventions to address a child's specific pattern of behavior, thereby boosting the efficacy of interventions. To address this need, the researchers will develop PEDS, a freely available assessment designed to assess contextual variations and similarities in strengths and difficulties. PEDS will be accompanied by guidance for school personnel to interpret and use the measure to provide individualized, context-relevant interventions and services to meet a student's needs.

Project Activities:  The research will be conducted in two phases. The Diverging Operations Phase (Year 1) will involve cross-validating previous findings demonstrating that informant discrepancies are due largely to real variation in students' behavior and not measurement error. In the Assessment Development and Validation Phase (Years 2 and 3), the research team will develop and validate discrepant and nondiscrepant scales of psychosocial strengths and difficulties as well as develop guidelines to enhance the feasibility and usability of the developed measures for educational decision making and programming. This phase involves a series of studies focused on the following: content validity (two studies), exploratory analyses to examine factor structure and identify discrepant and nondiscrepant items, confirmatory analyses, convergent/divergent validity, and feasibility/usability.

Products: The products of this project will include a developed and validated PEDS measure, consisting of discrepant and nondiscrepant scales of psychosocial strengths and difficulties, along with supporting materials and resources to enhance feasibility and use of PEDS in the school setting. The products will also include peer-reviewed publications and presentations.

Structured Abstract

Setting: The project will take place in general and special education classrooms in middle schools in an urban district in Minnesota. Observations will also take place in the students' homes.

Sample: Approximately 1200 middle school students (with and without disabilities), their teachers, and their parents will participate across all years of the study. A group of nine experts in school-based assessment, behavior, measurement construction, and data-based decision making will also provide feedback during item development.

Assessment: The PEDS will be a freely available instrument that is specifically designed to assess contextual variations and similarities in strengths and difficulties. The PEDS will also provide guidance for school personnel to interpret and use the rating scales to provide individualized, context-relevant services to meet a student's needs. PEDS will include multiple strength-based scales in addition to scales focused on problems/difficulties. Scales will include items that have been classified as low agreement between informants (i.e., discrepant) and high agreement between informants (i.e., nondiscrepant). The proposed items will adhere to a 4-point rating scale (Never/Rarely, Sometimes, Often, Very Often).

Research Design and Methods: In Year 1, the research team will conduct observations of students in their homes and classroom, and the results will be compared with teacher and parent ratings of student behavior across these same contexts. These comparisons aim to gather evidence supporting whether informant discrepancies (e.g., teacher ratings of high problem behavior and parent ratings of low problem behavior) reflect contextual variation in the display of children's psychosocial strengths and difficulties, and not merely measurement error. In Years 2 and 3, the assessment will be developed and validated. The project team will review the theoretical foundation of psychosocial strengths and difficulties and then systematically generate items that capture each of the constructs. The initial pool of items will be expanded, refined, and verified based on feedback from an external panel of experts. The items will then be administered to students, their parents, and their teachers to determine how well the items fit the underlying theoretical model. These participants will also complete additional self-report measures of behavior for the investigators to examine whether results can be explained by factors that may be related to rater bias, such as high levels of stress or depression. In addition, a series of multitrait-multimethod studies will investigate the convergent and discriminant validity and temporal stability of the scales. A series of focus groups with teachers (special/general education), school psychologists, and administrators will provide feedback regarding the feasibility of the measure and information that can be used to develop relevant guidelines to support the usability of the data to inform data-driven decision making.

Control Condition: Due to the nature of the research design, there is no control condition.

Key Measures: In addition to the PEDS, the assessment being developed, key measures include the observation system and behavior rating scales used during the development and validation studies. More specifically, during the Year 1 cross-validation study, the Stage Observation System and Direct Observation Form will be used for observations of student behavior. Behavior rating scales used in Years 1 to 3 include the Social Skills Improvement System (SSIS) and the Achenbach Scales of Empirically Based Assessment. In addition, to examine convergent and divergent relationships of the PEDS with existing measures, the multitrait-multimethod studies will use the Strengths and Difficulties Questionnaire to assess convergent relationships, and curriculum-based measures of student performance in oral reading and math calculation and the Children's Academic Intrinsic Motivation Inventory to assess divergent relationships. Administrative data including, for example, standardized academic test scores, attendance rates, and student disciplinary incidents (office discipline referrals, suspensions, expulsions) will be collected to represent methods of data collection that are different from informant ratings of student performance. A series of semi-structured focus groups with teachers (special/general education), school psychologists, and administrators will provide information regarding stakeholder understanding of the content of the PEDS scales, potential use of the resulting data in schools for decision making about student behavior, materials or resources that would support the effective use of PEDS in school, and effective strategies that could be linked to the resulting data to facilitate an assessment-to-intervention link.

Data Analytic Strategy: Multiple statistical and measurement techniques will be used to develop and assess the PEDS. Latent class analysis will be used in the first phase to demonstrate that informant discrepancies reflect contextual variations in the expression of behavior. In the second phase, principal axis analyses will be performed to examine initial factor structure of the scales and informant agreement metrics will be used to classify items as discrepant/nondiscrepant. Confirmatory factor analysis combined with item response theory will be used as confirmatory analyses to validate the underlying factor structure and examine relationships between the measures. Multilevel modeling will be used to conduct convergent and divergent validity analyses. Finally, structured qualitative data on the content, feasibility, and usability of the scales will be analyzed using content analysis to identify themes.