Grant Closed

Enhancing Data-based Decision-Making in Schools

NCSER
Program: Special Education Research Grants
Program topic(s): Social, Emotional, and Behavioral Competence
Award amount: $1,689,910
Principal investigator: Robert Horner
Awardee:
University of Oregon
Year: 2007
Project type:
Development and Innovation
Award number: R324A070226

Purpose

Schoolwide Positive Behavior Support is a frequently used systems-level intervention that involves school teams to actively engage in assessment, decision-making, and implementation of behavior supports. These teams use data to identify rates and patterns of problem behavior. The data are expected to influence decisions by the team about what interventions to implement. Using data effectively and efficiently however, can be a difficult task. Although there are an increasing number of educational data-management systems being adopted by districts across the country, little information is available about which features of these systems will be of greatest value for improving data-based decision-making. One such data-management system available to schools implementing schoolwide positive behavior support is the Schoolwide Information System. Learning more about how school teams use this information system provides an opportunity to better understand the role that data may play in influencing team decisions, and the impact on student outcomes.

Project Activities

The research team will develop the direct observation instrument in collaboration with school teams in elementary schools that use the Schoolwide Information System. First, the researchers will observe decision-making among school teams and use the information they gather to revise the instrument and establish its reliability and validity. The research team will then use the instrument in two research studies. First, the researchers will observe two different types of teams—teams that use the Schoolwide Information System and teams that do not use the system. The researchers will compare the quality of decision-making by these teams and explore the impact of those decisions on student outcomes. The second study will provide teams not currently using data management systems with training in the use of the Schoolwide Information System. The purpose is to understand the relationship between introduction of the data-based decision-making procedures and the quality of decision-making by school teams.

Structured Abstract

Setting

Elementary schools (grades K–5) in Oregon and North Carolina.

Sample

School teams (i.e., Behavior Support Teams) in participating schools. A heterogeneous sample of teams will be purposefully chosen. The variables accounted for in selecting the sample include the number of students served by the school, the number of team members (minimum of five), the interdisciplinary composition of the teams (e.g., administrator, teacher, behavior specialist, school psychologist, etc.), the average years of experience of a team's members, the frequency with which the team meets to discuss student social behavior issues (minimum of once a month), and the average number of social behavior issues addressed at a meeting.
Intervention
The intervention in this study is the implementation of the data-management system (i.e., Schoolwide Information System) of Schoolwide Positive Behavior Support. As part of team decision-making meetings, team members identify rates of problem behavior and patterns of problem behavior by type of problem behavior, location of discipline events, time of day of discipline events, and individual students involved in discipline events. These data are expected to influence decisions by the team about what interventions to implement, the scale (whole school, specific location, specific student) of implementation, and adaptations needed to existing interventions. The observation instrument being developed will be used to record a team's decision-making behaviors about primary- and secondary-level social behaviors and interventions only.

Research design and methods

For the development phase, the instrument will be developed with collaboration from school teams in 10 elementary schools currently implementing the data-management system. A draft of the instrument will be created with an accompanying protocol that lists behaviors frequently appearing in published descriptions of decision-making and problem-solving models. Two project observers will simultaneously observe meetings of the school teams and use the instrument to independently rate decision-making behaviors, and the instrument will be revised as necessary. The instrument will then be used within two research studies. The first will involve a post-test only quasi-experimental design with 30 elementary schools (15 using the data-management system, and 15 not using the system). Data will be collected both on the decision-making by school teams and the relationship between decisions and student outcomes (office discipline referrals, out-of-school suspensions, and third-grade reading scores). The second study will employ a single-subject, multiple-baseline design that provides training to five elementary school teams not using the data-management system and will examine the functional relationship between the data-based decision-making procedures and the decision-making by school teams.

Control condition

Behavior Support Teams in schools not using the Schoolwide Information System schools will serve as the controls.

Key measures

Criterion validity of the instrument will be established with the Social Problem-Solving Inventory—Revised. Information about rate of office disciplinary referrals, student out-of-school suspensions, and third-grade reading scores will be collected. Reading scores will include the proportion of students who achieve the state-level reading standard, and students' mean scores on the Dynamic Indicators of Basic Early Literacy Skills.

Data analytic strategy

To establish psychometric properties of the observation tool, the observation data collected on the school teams will be reviewed, any discrepancies will be discussed, and revisions will be made to the instrument as necessary. The cycle of "observe, review, and revise" is repeated until reliability and validity of the instrument's scores are established. For the first study, data analysis of differences in decision-making and the relationship between decisions and student outcomes will be completed via tests comparing the differences between the independent mean scores of the two groups (i.e., schools that use the Schoolwide Information System and teams that do not) on the instrument's decision-making measures, the measures of student social behavior, and the measures of student academic achievement. The single subject study will be analyzed using visual analysis including change in level and trend as well as percentage of non-overlapping of data points. Effect size will also be computed.

People and institutions involved

IES program contact(s)

Jacquelyn Buckley

Associate Commissioner
NCSER

Products and publications

Products: The products from this study include a fully developed and validated direct observation assessment tool that can be used in research on data-based decision-making. Published reports of the research will be available.

Book chapter

Newton, S.J., Horner, R.H., Algozzine, R.F., Todd, A.W., and Algozzine, K.M. (2009). Using a Problem-Solving Model to Enhance Data-Based Decision-Making in Education. In Sailor, Dunlap, G. Sugai, and R.H. Horner (Eds.), Handbook of Positive Behavior Support (pp. 551-580). New York: Springer.

Book chapter, edition specified

Todd, A., Algozzine, B., Horner, R., and Algozzine K. (in press). Data-Based Decision-Making. In C. Reynolds, K. Vannest, and E. Fletcher-Janzen (Eds.), Encyclopedia of Special Education: A Reference for the Education of Children, Adolescents, and Adults With Disabilities and Other Exceptional Individuals (4th ed.). Hoboken, NJ: John Wiley and Sons.

Journal article, monograph, or newsletter

Algozzine, B., Horner, R.H., Todd, A.W., Newton, J.S., Algozzine, K., and Cusumano, D. (2015). Measuring the Process and Outcomes of Team Problem Solving. Journal of Psychoeducational Assessment, 34(3): 211-229. doi:10.1177/0734282915592535

Algozzine, B., Newton, J.S., Horner, R.H., Todd, A.W., and Algozzine, K.M. (2012). Development and Technical Characteristics of a Team Decision-Making Assessment Tool: Decision Observation, Recording and Analysis (DORA). Journal of Psychoeducational Assessment, 30(3): 237-249. doi:10.1177/0734282911423884

Newton, J.S., Algozzine, B., Algozzine, K., Horner, R.H., and Todd, A.W. (2011). Building Local Capacity for Training and Coaching Data-Based Problem Solving With Positive Behavior Intervention and Support Teams. Journal of Applied School Psychology, 27(3): 228-245. doi:10.1080/15377903.2011.590104

Newton, J.S., Horner, R.H., Algozzine, B., Todd, A.W., and Algozzine, K.M. (2012). A Randomized Wait-List Controlled Analysis of the Implementation Integrity of Team-Initiated Problem Solving. Journal of School Psychology, 50(4): 421-441. doi:10.1016/j.jsp.2012.04.002

Newton, J.S., Horner, R.H., Todd, A.W., Algozzine, B., and Algozzine, K.M. (2012). A Pilot Study of a Problem-Solving Model for Team Decision Making. Education and Treatment of Children, 35(1): 25-49.

Todd, A.W., Horner, R.H., Berry, D., Sanders, C., Bugni, M., Currier, A., Potts, N., Newton, J.S., Algozzine, B., and Algozzine, K. (2012). A Case Study of Team-Initiated Problem Solving Addressing Student Behavior in One Elementary School. Journal of Special Education Leadership, 25(2): 81-89.

Todd, A.W., Horner, R.H., Newton, J.S., Algozzine, R.F., Algozzine, K.M., and Frank, J.L. (2011). Effects of Team-Initiated Problem Solving on Decision-Making by Schoolwide Behavior Support Teams. Journal of Applied School Psychology, 27(1): 42-59. doi:1080/15377903.2011.540510

Nongovernment report, issue brief, or practice guide

Newton, J.S., Todd, A.W., Algozzine, K.M., Horner, R.H., and Algozzine, B. (2009). Team-Initiated Problem Solving Training Manual. Eugene, OR: Educational and Community Supports, University of Oregon.

Todd, A.W., Newton, J.S., Horner, R.H., Algozzine, B., and Algozzine, K. (2010). Team-Initiated Problem-Solving Trainer-of-Trainers Manual. Eugene, OR: Educational and Community Supports, University of Oregon.

Todd, A.W., Newton, J.S., Horner, R.H., Algozzine, B., and Algozzine, K.M (2009). Decision Observation, Recording, and Analysis (DORA) Training Manual. Eugene, OR: Educational and Community Supports, University of Oregon.

Supplemental information

To that end, the researchers will use current models of decision theory to build and validate a direct observation instrument, the Decision Observation, Recording, and Analysis tool, for assessing the decision-making skills of schoolwide positive behavior support teams. In addition, the researchers will examine whether the use of the Schoolwide Information System improves the quality of decision-making by school teams. The researchers will also investigate the impact of team decisions on student outcomes (e.g., office discipline referrals, reading scores).

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More