|Title:||Validation of the Behavioral and Emotional Screening System for Early Identification for Social-Emotional and Behavioral Problems in Preschoolers|
|Principal Investigator:||DiStefano, Christine||Awardee:||University of South Carolina|
|Program:||Early Intervention and Early Learning [Program Details]|
|Award Period:||7/1/2010 to 6/30/2014||Award Amount:||$1,372,484|
Purpose: During the last four decades, the number of preschoolers served in center-based programs (e.g., childcare centers, Head Start Programs, publicly funded pre-kindergarten programs, private preschools) has increased dramatically. Concurrent with the expansion of services to young children has been the fact that many young children who are enrolling in preschools exhibit emerging behavioral and social emotional difficulties that are severe enough to impede their social development and educational progress. The purpose of this study is to validate the Behavioral and Emotional Screening System (BESS) teacher rating form for preschool-aged children to determine the plausibility of the instrument as part of a universal screening system. There is limited information on screening systems for use with preschoolers. This project will examine the BESS to determine its: (a) psychometric properties, (b) accuracy in predicting children’s behavior over time, and (c) implementation feasibility.
Project Activities: As a first step, researchers will review the pool of items from which the BESS was developed to make sure that those chosen are the ones that are most likely to identify children at-risk for poor social-emotional development. Data from children will be collected twice yearly over the course of three years to provide information on various aspects of reliability and validity. Data will also be collected from teachers and assistant teachers to assess the feasibility of the BESS.
Products: The main product will be a fully developed and validated assessment for wide-scale use in preschools to identify early those who at risk for social-emotional disorders. Data will be made available on the BESS’ feasibility, efficiency, acceptability, validity, and reliability.
Setting: The research will be conducted in school districts across South Carolina.
Population: Children aged 4-5 will be included in this study.
Assessment: This study will validate the BESS teacher rating form for preschoolers (BESS TRS-P). The BESS was developed from the longer preschool form of the Behavioral Assessment System for Children (BASC 2nd Ed.; Reynolds & Kamphaus, 2004) which is widely used across the U.S. for behavioral assessment and intervention for children, especially those with behavioral and emotional problems.
Research Design and Methods: The research team will first conduct an in-depth analysis of the norming data base from which the BESS items were selected, to ensure that the screener is using the best performing items on children’s behavioral and emotional status. The reliability and validity of the BESS will then be tested in four stages. Stage 1 focuses on establishing construct stability, which will include determining if scores on the BESS are internally consistent, stable across raters, and stable across time. In addition, the research team will determine whether the time requirements for assessing a student are reasonable for use by preschool teachers. Stage 2 focuses on concurrent validity, to determine whether the BESS relates to other screening instruments measuring similar constructs. In Stage 2, researchers will assess the trade-off between the time saved using the short form of the BESS compared to the information gained from using the longer assessment the BESS was developed from, the Behavioral Assessment System for Children. Stage 3 focuses on the evidence of predictive validity of the BESS over a longer time period than one academic year. In this stage, the research team will also use a teacher questionnaire to assess the acceptability, feasibility, and usability of the BESS as a screening tool. In Stage 4, the research team will assess the differential validity of the BESS to determine whether this measure can distinguish between preschoolers who possess a disorder and those who are developing normally. The cut scores for the screener must have an acceptable balance of sensitivity (the proportion of children with disorders that are correctly identified) and specificity (the proportion of students without a disorder that are not flagged at risk by the screener). In Stage 4, researchers will also address feedback from teachers on any weaknesses of the BESS screener.
Data will be collected twice yearly (fall and spring) over the course of three school years. Cross-sectional and longitudinal data will be available for three cohorts of children over these three years from pre-kindergarten through first grade. Data will also be collected from teachers and assistant teachers over this time.
Control Condition: Due to the nature of the design of this project, there is no control group.
Key Measures: In addition to the BESS, other social-emotional, school readiness, and academic assessment measures will be included in this study. Questionnaires will also be used to assess the acceptability, feasibility, and usability of the BESS, and to document referrals for significant behavior problems, special services, and problematic classroom behavior.
Data Analytic Strategy: Correlational analyses will be the primary analytic method used in Stages 1 through 3. ROC (Receiver Operating Characteristic Curve) analyses and Item Response Theory procedures will be used in Stage 4 to determine differential validity of the BESS.
Horn, E.M., Palmer, S.B., Butera, G.D., and Lieber, J.A (2016). Six Steps to Inclusive Preschool Curriculum: A UDL-Based Framework for Children's School Success. Baltimore, MD: Brookes Publishing.
Butera, G., Horn, E., Palmer, S., and Lieber, J. (in press). Science and Math for Children Within Early Childhood Special Education. In S. Odom, B. Reichow, E. Barton, and B. Boyd (Eds.), Handbook of Early Childhood Special Education. New York: Springer.
DiStefano, C. (2016). Examining Fit With Structural Equation Models. In K. Schweizer, and C. DiStefano (Eds.), Principles and Methods of Test Construction: Standards and Recent Advances (pp. 166–196). Germany: Hogrefe Publishers.
Greer, F., and Liu, J. (2016). Creating Short Forms and Screening Measures. In K. Schweizer, and C. DiStefano (Eds.), Principles and Methods of Test Construction: Standards and Recent Advancements (pp. 272–287). Germany: Hogrefe Publishers.
Journal article, monograph, or newsletter
DiStefano, C., Ene, M., and Leighton, E. (2016). Teacher Ratings of Child Behavior in Preschool: A MIMIC Investigation of the BESS TRS-P. Psychological Assessment, 28(8): 1015–1019. doi:10.1037/pas0000218
DiStefano, C., Greer, F.W., and Kamphaus, R.W. (2013). Multifactor Modeling of Emotional and Behavioral Risk of Preschool-Age Children. Psychological Assessment, 25(2): 467–476. doi:10.1037/a003139
DiStefano, C., Greer, F.W., Kamphaus, R.W., and Brown , W.H. (2014). Using Rasch Rating Scale Methodology to Examine a Behavioral Screener for Preschoolers at Risk. Journal of Early Intervention, 36(3): 192–211. doi:10.1177/1053815115573078
Greer, F.W., DiStefano, C., Liu, J., and Cain, L. (2015). Preliminary Psychometric Evidence of the Behavioral and Emotional Screening System Teacher Rating Scale–Preschool. Assessment for Effective Education, 40(4): 240–246. doi:10.1177/1534508415571594
Greer, F.W., Wilson, B.S., DiStefano, C., and Liu, J. (2012). Considering Social Validity in the Context of Emotional and Behavioral Screening. School Psychology Forum, 6(4): 148–159.
Mihai, A., Friesen, A., Butera, G., Horn, E., Lieber, J., and Palmer, S. (2015). Teaching Phonological Awareness to all Children Through Storybook Reading. Young Exceptional Children, 18(4): 3–18. doi:10.1177/1096250614535221