Grant Closed

Validation of the Behavioral and Emotional Screening System for Early Identification for Social-Emotional and Behavioral Problems in Preschoolers

NCSER

Program: Special Education Research Grants

Program topic(s): Early Intervention and Early Learning

Award amount: $1,372,484

Principal investigator: Christine DiStefano

Awardee:

University of South Carolina

Year: 2010

Project type:

Measurement

Award number: R324A100104

Purpose

During the last four decades, the number of preschoolers served in center-based programs (e.g., childcare centers, Head Start Programs, publicly funded pre-kindergarten programs, private preschools) has increased dramatically. Concurrent with the expansion of services to young children has been the fact that many young children who are enrolling in preschools exhibit emerging behavioral and social emotional difficulties that are severe enough to impede their social development and educational progress. The purpose of this study is to validate the Behavioral and Emotional Screening System (BESS) teacher rating form for preschool-aged children to determine the plausibility of the instrument as part of a universal screening system. There is limited information on screening systems for use with preschoolers. This project will examine the BESS to determine its: (a) psychometric properties, (b) accuracy in predicting children's behavior over time, and (c) implementation feasibility.

Project Activities

As a first step, researchers will review the pool of items from which the BESS was developed to make sure that those chosen are the ones that are most likely to identify children at-risk for poor social-emotional development. Data from children will be collected twice yearly over the course of three years to provide information on various aspects of reliability and validity. Data will also be collected from teachers and assistant teachers to assess the feasibility of the BESS.

Structured Abstract

Setting

The research will be conducted in school districts across South Carolina.

Sample

Children aged 4-5 will be included in this study.

Assessment

This study will validate the BESS teacher rating form for preschoolers (BESS TRS-P). The BESS was developed from the longer preschool form of the Behavioral Assessment System for Children (BASC 2nd Ed.; Reynolds & Kamphaus, 2004) which is widely used across the U.S. for behavioral assessment and intervention for children, especially those with behavioral and emotional problems.

Research design and methods

The research team will first conduct an in-depth analysis of the norming data base from which the BESS items were selected, to ensure that the screener is using the best performing items on children's behavioral and emotional status. The reliability and validity of the BESS will then be tested in four stages. Stage 1 focuses on establishing construct stability, which will include determining if scores on the BESS are internally consistent, stable across raters, and stable across time. In addition, the research team will determine whether the time requirements for assessing a student are reasonable for use by preschool teachers. Stage 2 focuses on concurrent validity, to determine whether the BESS relates to other screening instruments measuring similar constructs. In Stage 2, researchers will assess the trade-off between the time saved using the short form of the BESS compared to the information gained from using the longer assessment the BESS was developed from, the Behavioral Assessment System for Children. Stage 3 focuses on the evidence of predictive validity of the BESS over a longer time period than one academic year. In this stage, the research team will also use a teacher questionnaire to assess the acceptability, feasibility, and usability of the BESS as a screening tool. In Stage 4, the research team will assess the differential validity of the BESS to determine whether this measure can distinguish between preschoolers who possess a disorder and those who are developing normally. The cut scores for the screener must have an acceptable balance of sensitivity (the proportion of children with disorders that are correctly identified) and specificity (the proportion of students without a disorder that are not flagged at risk by the screener). In Stage 4, researchers will also address feedback from teachers on any weaknesses of the BESS screener.

Control condition

Due to the nature of the design of this project, there is no control group.

Key measures

In addition to the BESS, other social-emotional, school readiness, and academic assessment measures will be included in this study. Questionnaires will also be used to assess the acceptability, feasibility, and usability of the BESS, and to document referrals for significant behavior problems, special services, and problematic classroom behavior.

Data analytic strategy

Correlational analyses will be the primary analytic method used in Stages 1 through 3. ROC (Receiver Operating Characteristic Curve) analyses and Item Response Theory procedures will be used in Stage 4 to determine differential validity of the BESS.

People and institutions involved

IES program contact(s)

Amy Sussman

Education Research Analyst

NCSER

Products and publications

Products: The main product will be a fully developed and validated assessment for wide-scale use in preschools to identify early those who at risk for social-emotional disorders. Data will be made available on the BESS' feasibility, efficiency, acceptability, validity, and reliability.

Book

Horn, E.M., Palmer, S.B., Butera, G.D., and Lieber, J.A (2016). Six Steps to Inclusive Preschool Curriculum: A UDL-Based Framework for Children's School Success. Baltimore, MD: Brookes Publishing.

Book chapter

Butera, G., Horn, E., Palmer, S., and Lieber, J. (in press). Science and Math for Children Within Early Childhood Special Education. In S. Odom, B. Reichow, E. Barton, and B. Boyd (Eds.), Handbook of Early Childhood Special Education. New York: Springer.

DiStefano, C. (2016). Examining Fit With Structural Equation Models. In K. Schweizer, and C. DiStefano (Eds.), Principles and Methods of Test Construction: Standards and Recent Advances (pp. 166-196). Germany: Hogrefe Publishers.

Greer, F., and Liu, J. (2016). Creating Short Forms and Screening Measures. In K. Schweizer, and C. DiStefano (Eds.), Principles and Methods of Test Construction: Standards and Recent Advancements (pp. 272-287). Germany: Hogrefe Publishers.

Journal article, monograph, or newsletter

DiStefano, C., Ene, M., and Leighton, E. (2016). Teacher Ratings of Child Behavior in Preschool: A MIMIC Investigation of the BESS TRS-P. Psychological Assessment, 28(8): 1015-1019. doi:10.1037/pas0000218

DiStefano, C., Greer, F.W., and Kamphaus, R.W. (2013). Multifactor Modeling of Emotional and Behavioral Risk of Preschool-Age Children. Psychological Assessment, 25(2): 467-476. doi:10.1037/a003139

DiStefano, C., Greer, F.W., Kamphaus, R.W., and Brown , W.H. (2014). Using Rasch Rating Scale Methodology to Examine a Behavioral Screener for Preschoolers at Risk. Journal of Early Intervention, 36(3): 192-211. doi:10.1177/1053815115573078

Greer, F.W., DiStefano, C., Liu, J., and Cain, L. (2015). Preliminary Psychometric Evidence of the Behavioral and Emotional Screening System Teacher Rating Scale-Preschool. Assessment for Effective Education, 40(4): 240-246. doi:10.1177/1534508415571594

Greer, F.W., Wilson, B.S., DiStefano, C., and Liu, J. (2012). Considering Social Validity in the Context of Emotional and Behavioral Screening. School Psychology Forum, 6(4): 148-159.

Mihai, A., Friesen, A., Butera, G., Horn, E., Lieber, J., and Palmer, S. (2015). Teaching Phonological Awareness to all Children Through Storybook Reading. Young Exceptional Children, 18(4): 3-18. doi:10.1177/1096250614535221

Supplemental information

Data will be collected twice yearly (fall and spring) over the course of three school years. Cross-sectional and longitudinal data will be available for three cohorts of children over these three years from pre-kindergarten through first grade. Data will also be collected from teachers and assistant teachers over this time.

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

Validation of the Behavioral and Emotional Screening System for Early Identification for Social-Emotional and Behavioral Problems in Preschoolers

Purpose

Project Activities

Structured Abstract

Setting

Sample

Research design and methods

Control condition

Key measures

Data analytic strategy

People and institutions involved

IES program contact(s)

Amy Sussman

Products and publications

Supplemental information

Questions about this project?

Questions about this project?

You may also like

Data Science Methods for Digital Learning Platform...

Meta-Analysis Training Institute (MATI)

Bayesian Longitudinal Data Modeling in Education S...