|Title:||Improving the Assessment Capability of Standardized Tests: How High-Stakes Testing Environments Compromise Performance|
|Principal Investigator:||Beilock, Sian||Awardee:||University of Chicago|
|Program:||Cognition and Student Learning [Program Details]|
|Award Period:||3 years||Award Amount:||$427,786|
|Type:||Development and Innovation||Award Number:||R305H050004|
Purpose: There is substantial concern over the ability of high-stakes tests to assess competence and predict future academic performance. This research team proposes that some aspects of test-taking environments reduce cognitive resources available for completing test problems, resulting in less-than-optimal performance. This may be particularly true for members of groups that have traditionally not done well when tested in particular domains. For example, when aware of the stereotype that "women are bad at math," women do not perform as well on math tests as they do when they complete the same problems in a non-test taking situation. This phenomenon has been called "stereotype threat" by social psychologists.
The purpose of this project is twofold. First, these researchers are examining how stereotype threat undermines women's expression of math skills by examining how testing situations impact performance. Second, building on this set of findings, the researchers are developing and testing, in an education delivery setting, new assessment tools that are designed to reduce the negative effects of stereotype threat. The results of this project should be able to provide new guidance on how standardized assessments can be developed that accurately reflect all students' abilities and potential.
Setting: The experimental research is being conducted in a medium-sized Midwestern town and a large Midwestern city.
Population: Approximately 900 female undergraduates and women in a residential program for female students with interests in math and science are participating. Because stereotype threat is most likely to occur for those highly invested in performing well, women pursuing math and science careers provide an ideal population to examine in a real-world education assessment setting.
Research Design and Methods: The first component of this work explores how stereotype threat interferes with women's math performance. Studies 1 and 2 test the idea that stereotype threat induces verbal worries that consume phonological resources needed for solving certain types of math problems. Studies 3 and 4 test the counterintuitive predictions that individuals with greater working memory capacity are more susceptible to stereotype threat. Studies 5 and 6 examine if stereotype threat carries over and impairs subsequent tasks that rely on working memory but are not implicated by stereotype threat.
The second component of this work investigates techniques to reduce the negative impact of high-stakes testing situations in real-world education assessment by modifying the major test used in college admissions (i.e., SAT) based on the above findings and administering it to students in the residential program for women in math and science. Studies 7 and 8 modify the Quantitative portion of the SAT using principles (tested above) designed to diminish stereotype threat effects. It is anticipated that these new tests will produce better overall math performance and reveal greater predictive validity for academic achievement (e.g., college math grades) than those currently used. Study 9 examines SAT performance following training designed to inoculate students against the negative consequences of high-stakes situations.
Students are randomly assigned to experimental or control conditions in every experiment described above.
Control Condition: Students in the control conditions are not provided with framing information that precipitates stereotype threat.
Key Measures: Student performance on math problems is the primary data being collected.
Data Analytic Strategy: Analysis of variance techniques are being used to examine student performance on mathematical problems as a function of participation in the experimental or control condition.
Project Website: http://hpl.uchicago.edu/Projects/Projects.html
Beilock, S.L. (2007). Choking Under Pressure. In R. Baumeister, and K. Vohs (Eds.), Encyclopedia of Social Psychology (pp. 141–142). Los Angeles: Sage Publications.
Beilock, S.L., and Lyons, I.M. (2009). Expertise and the Mental Simulation of Action. In K.D. Markman, W.P. Klein, and J.A. Suhr (Eds.), Handbook of Imagination and Mental Simulation (pp. 21–34). New York: Psychology Press.
Beilock, S.L., and Ramirez, G. (2011). On the Interplay of Emotion and Cognitive Control: Implications for Enhancing Academic Achievement. In J.P. Mestre, and B.H Ross (Eds.), The Psychology of Learning and Motivation, Volume 55 (pp. 137–169). San Diego: Elsevier, Inc.
DeCaro, M.S., and Beilock, S.L. (2013). The Benefits and Perils of Attentional Control. In M. Csikszentmihalyi, and B. Bruya (Eds.), Effortless Attention: A New Perspective in the Cognitive Science of Attention and Action (pp. 51–73). Cambridge, MA: MIT Press.
Schmader, T., and Beilock, S.L. (2011). An Integration of Processes That Underlie Stereotype Threat. In T. Schmader, and M. Inzlicht (Eds.), Stereotype Threat: Theory, Process, and Application (pp. 34–50). New York: Oxford University Press.
Journal article, monograph, or newsletter
Beilock, S.L. (2008). Math Performance in Stressful Situations. Current Directions in Psychological Science, 17(5): 339–343.
Beilock, S.L., and Decaro, M.S. (2007). From Poor Performance to Success Under Stress: Working Memory, Strategy Selection, and Mathematical Problem Solving Under Pressure. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(6): 983–998.
Beilock, S.L., and Gonso, S. (2008). Putting in the Mind Versus Putting on the Green: Expertise, Performance Time, and the Linking of Imagery and Action. The Quarterly Journal of Experimental Psychology, 61(6): 920–932.
Beilock, S.L., Jellison, W.A., Rydell, R.J., Mcconnell, A.R., and Carr, T.H. (2006). On the Causal Mechanisms of Stereotype Threat: Can Skills That Don't Rely Heavily on Working Memory Still be Threatened?. Personality and Social Psychology Bulletin, 32(8): 1059–1071.
Beilock, S.L., Lyons, I.M., Mattarella-Micke, A., Nusbaum, H.C., and Small, S.L. (2008). Sports Experience Changes the Neural Processing of Action Language. PNAS Proceedings of the National Academy of Sciences of the United States of America, 105(36): 13269–13273.
Beilock, S.L., Rydell, R.J., and McConnell, A.R. (2007). Stereotype Threat and Working Memory: Mechanisms, Alleviation, and Spillover. Journal of Experimental Psychology: General, 136(2): 256–276.
Decaro, M.S., Thomas, R., and Beilock, S.L. (2008). Individual Differences in Category Learning: Sometimes Less Working Memory Capacity Is Better Than More. Cognition, 107(1): 284–294.
Decaro, M.S., Wieth, M., and Beilock, S.L. (2007). Methodologies for Examining Problem Solving Success and Failure. Methods, 42(1): 58–67.
Ping, R.M., Dhillon, S., and Beilock, S.L. (2009). Reach for What You Like: The Body's Role in Shaping Preferences. Emotion Review, 1(2): 140–150.
Rydell, B.J., McConnell, A.R., and Beilock, S.L. (2009). Multiple Social Identities and Stereotype Threat: Imbalance, Accessibility, and Working Memory. Journal of Personality and Social Psychology, 96(5): 949–966.
Sibley, B.A., and Beilock, S.L. (2007). Exercise and Working Memory: An Individual Differences Investigation. Journal of Sport and Exercise Psychology, 29(6): 783–791.