|Title:||Development and Validation of a Screener for Behavioral and Emotional Problems in Elementary and Middle School|
|Principal Investigator:||Kamphaus, Randy||Awardee:||Georgia State University|
|Program:||Social and Behavioral Outcomes to Support Learning [Program Details]|
|Award Period:||5/1/2006 to 4/30/2010||Award Amount:||$941,141|
Previous Award Number: R324B060033
Purpose: The purpose of this project is to develop and validate a five-minute screening assessment to be used by teachers to identify elementary and middle school children with behavioral or emotional problems that predispose them to academic failure. The study will build upon pilot research to develop a screener from the longer form of the Behavioral Assessment System for Children, second edition (BASC-2; Reynolds & Kamphaus, 2004), currently used widely in the U.S. and abroad.
Setting: This project will be implemented in large, linguistically, socio-economically, and ethnically diverse school districts in California and Maryland.
Population: Students between the ages of 5 and 13 in the kindergarten through seventh grades are targeted for this study. It is expected that teacher ratings and parent questionnaire information will be obtained from approximately 1400 students in each of three cohorts for a total of 3,600 students over three years. Two boys and two girls will be selected at random for participation in each class at each grade level within each school. Within each school, information will be gathered from teachers as well as collected from archival data on students (e.g., grades, number of absences, standardized test scores).
Project Activities: The research investigation will be conducted sequentially, in three phases: (1) develop the screener; (2) assess the psychometric quality of the screener, resulting in multiple measures of reliability and validity; (3) estimate the screener's long-term predictive validity using an accelerated longitudinal design. The outcome is a screening instrument that will take the teacher five minutes or less per child, and have adequate reliability and predictive validity as judged by standards set by measurement science research precedents. Statistical methods such as principal components analysis, reliability and item analysis, Receiver Operating Characteristic Curve analysis, and multiple regression will be employed to measure the reliability and validity, and estimate the predictive validity of the developed screener.
Products: The expected outcomes from this study include: (a) A developed and validated teacher-rated screener to accurately identify children with behavioral or emotional problems; (b) published reports on the results from the validity studies and the psychometric quality of the screener for identifying children with behavioral and/or emotional problems; (c) published reports on the longer-term predictive validity of the developed screener based on the analysis of the longitudinal investigation; and (d) Screener/Assessment documentation and forms to be used for special education classification.
Research Design and Methods: The research investigation will be conducted sequentially, in three phases: (1) develop the screener; (2) assess the psychometric quality of the screener, resulting in multiple measures of reliability and validity; (3) estimate the screener's long-term predictive validity using an accelerated longitudinal design. The outcome is a screening instrument that will take the teacher five minutes or less per child, and have adequate reliability and predictive validity as judged by standards set by measurement science research precedents. Statistical methods such as principal components analysis, reliability and item analysis, Receiver Operating Characteristic Curve analysis, and multiple regression will be employed to measure the reliability, validity, and estimate the predictive validity of the developed screener.
Key Measures: The screener is based on widely used comprehensive measure of children's behavioral and emotional adjustment, the Behavioral Assessment System for Children-2 (BASC), of which the PI is a co-developer. The BASC-2 consists of the Self-Report of Personality (SRP), Parent Rating Scales (PRS), and Teacher Rating Scales (TRS), a Student Observation System (BASC SOS), and a structured developmental history (BASC SDH). The items on the TRS are rated on a four-point response scale of frequency ranging from "Never" to "Almost Always." The full TRS forms can be completed in about 20 minutes, and scales can cover the age range of 2 ½ through 18. The proposed investigation will include larger samples than included on previous work with the BASC, create a screener that spans ages 5-13, draws items from a modern, broader, and higher quality item pool, assess incremental validity, and assess predictive validity over an eight year period among other improvements over the BASC.
Data Analytic Strategy: The data analysis takes place in three phases: substantive, structural and external. The first stage, the substantive stage, is concerned with how the construct(s) is defined, both theoretically and empirically. For this stage, principal components factor analysis will be used to develop the screener items and the results will be compared to factors of child adjustment derived from the literature. The structural component provides assurances that the results can be replicated if the instrument was given to the same individuals under similar conditions. Three reliability coefficients will be estimated: internal consistency, short and long term test-retest, and inter-rater. In addition, criterion and predictive validity evidence will provide support that the inferences made from the screener scores are trustworthy (e.g., usefulness of screener scores to make decisions about child behavior). The external phase will examine the relationships between screener scores and additional instruments to assess the usefulness of the screener for identifying children at risk for behavioral, emotional, and academic adjustment in schools. To measure the accuracy of discrimination between children with emotional, behavioral, or academic problems and those without, Receiver Operating Curve (ROC) analysis will be conducted. Known group validity analyses will also be performed to examine the ability of the screener to identify children with disabilities from those who are developing normally.
Intervention: Due to the nature of the research, there is no intervention.
Control Condition: Due to the nature of the research, there is no control condition.
Publications from this project:
DiStefano, C. & Morgan, G. (2010). Investigating the BESS TRS-CA using Rasch Rating Scale Methodology. School Psychology Quarterly, 25 (4), 202–212. doi: 10.1037/a0021509.
Kamphaus, R.W., DiStefano, C., Dowdy, E., Eklund, K., & Dunn, A. (2010). Determining the presence of a problem: Comparing two approaches for detecting youth behavioral risk. School Psychology Review, 39, 395–407.
DiStefano, C. & Morgan, G. (2011). Examining classification criteria: A comparison of three methods for creating cut scores. Psychological Assessment, 23 (2), 354–363.doi: 10.1037/a0021745.
Dowdy, E., Chin, J., Twyford, J., & Dever, B. (2011). A factor analytic investigation of the BASC-2 Behavioral and Emotional Screening System Parent Form: Psychometric properties, practical implications, and future directions. Journal of School Psychology, 49, 265–280. doi: 10.1016/j.jsp.2011.03.005
Dowdy, E., Dever, B., DiStefano, C., & Chin, J. (2011). Screening for emotional and behavioral risk among students with limited English proficiency. School Psychology Quarterly, 26, 14–26. doi: 10.1037/a0022072
Dowdy, E., Doane, K., Eklund, K., & Dever, B. (2011). A comparison of teacher nomination and screening to identify behavioral and emotional risk. Journal of Emotional and Behavioral Disorders. doi: 10.1177/1063426611417627
Dowdy, E., Twyford, J., Chin, J., Kamphaus, R., & Mays, K. (2011). Factor structure of the BASC2 Behavioral and Emotional Screening System student form. Psychological Assessment, 23, 379–387. doi: 10.1037/a0021843
Dever, B., Mays, K., Kamphaus, R. W., & Dowdy, E. (2012). The factor structure of the BASC-2 Behavioral and Emotional Screening System Teacher Form, Child/Adolescent. Journal of Psychoeducational Assessment, 30 (5), 488–495. doi: 10.1177/0734282912438869
Dowdy, E., Kamphaus, R. W., Abdou, A. S., & Twyford, J. (2012). Detection of symptoms of prevalent mental health disorders of childhood with the parent form of the Behavioral and Emotional Screening System. Assessment for Effective Intervention. doi:10.1177/1534508412447009
Chin, J., Dowdy, E., & Quirk, M. (in press). Universal screening in middle school: Examining the Behavioral and Emotional Screening System. Journal of Psychoeducational Assessment.