Research References

Balestreri, K., Sambolt, M., Duhon, C., Smerdon, B., & Harris, J. (2014). The college and career readiness and success organizer. Washington, DC: College & Career Readiness & Success Center at American Institutes for Research (AIR). Retrieved from https://eric.ed.gov/?id=ED555672.

From the abstract:

The ‘College and Career Readiness and Success Organizer’ is a graphic that displays a consolidated overview of the many elements that impact a learner's ability to succeed in college and careers at both the institutional and individual levels. The ‘Organizer’ is intended to be a comprehensive and visual representation of the complexities of the college and career readiness and success universe and is a composite of essential considerations that are equal in importance and interconnected. Each of the four strands presents a distinctive topic area essential to college and career readiness and success and can be used to facilitate discussions and inform collaboration within and across various stakeholder communities. Additionally, it can contribute to strategic planning, conceptualization, and decision making as well as alignment of strategies and initiatives to ensure that all learners achieve college and career readiness and success. The College and Career Readiness and Success Center recommends using the ‘Organizer’ in its entirety to frame conversations and develop college and career readiness and success priorities to ensure alignment and cohesion.

Camara, W., & Quenomoen, R. (2012). Defining and measuring college and career readiness and informing the development of performance level descriptors (PLDs). The College Board and National Center on Educational Outcomes. Retrieved from https://parcc-assessment.org/content/uploads/2017/11/Defining-Measuring-CCR-Camara-Quenemoen.pdf.

From the executive summary:

The purpose of this paper is to assist PARCC in developing a working definition of college and career readiness (CCR) which can be used to: (a) establish an interpretative argument for the primary inferences that will be made from test scores; (b) determine CCR in high school and ascertain whether students are ‘on track’ toward CCR at lower grades; (c) aid in collecting validation evidence of CCR metrics, PLDs and cut scores; (d) determine the criterion associated with CCR; (e) guide the development of performance level descriptors (PLDs) early in the design of assessments; and (e) clarify expectations for item and test development.

Center on Standards and Assessments Implementation. (2018). Selected alternatives for assessing college and career readiness: CSAI update. San Francisco, CA: WestEd. Retrieved from https://eric.ed.gov/?id=ED588469.

From the introduction:

States often use summative assessments as a measure to determine students' readiness for college and career tracks. Some states administer the Smarter Balanced Assessment Consortium (Smarter Balanced) or Partnership for Assessment of Readiness for College and Careers (PARCC) assessments to measure high school student performance on college and career readiness content. Other states administer their own state-developed assessments, or administer the SAT or ACT assessments as part of their state assessment portfolios, as a measure of college and career readiness. States often use summative assessments to measure student performance against academic standards designed to represent college and career ready knowledge and skills. However, a summative assessment may not always be the best method of measuring students' college and career readiness. Due to differences in learning needs, or differences in state definitions of college and career readiness, students may require alternative methods of properly and accurately demonstrating this knowledge and these skills. Three examples of these alternatives are utilizing local assessments, utilizing performance-based assessments (such as a portfolio), or offering dual enrollment programs. This brief discusses each of these three examples.

Conley, D. (2012). A complete definition of college and career readiness. Eugene, OR: Education Policy Improvement Center (NJ1). Retrieved from https://eric.ed.gov/?id=ED537876.

From the introduction:

In many circles, efforts are under way to develop definitions of college readiness, career readiness, or both. This brief contains a definition that is the culmination of 18 years of study and research on this topic. This definition, then, is based on both empirical evidence gathered via multiple research studies and on-the-ground interactions with practitioners attempting to improve programs that affect a wide range of students, particularly those who would be first in family to attend college. Postsecondary, in this definition, refers to any formal setting in which an individual pursues additional instruction beyond high school. This might include two- or four-year degree programs, certificate or licensure programs, apprenticeships, or training programs in the military. Furthermore, this definition differs from indices or ‘cut scores’ that use a single cut score alone or in combination with another element, such as high school grade point average, to predict college success. This definition is designed to be actionable. It describes what students should know and be able to do in general terms that can then be defined and addressed with increasing detail and measured in a variety of ways using appropriately complex measures.

Kyllonnon, P. (2012). Measurement of 21st century skills within the Common Core State Standards. Princeton, NJ: Educational Testing Service (ETS). Retrieved from http://www.ets.org/Media/Research/pdf/session5-kyllonen-paper-tea2012.pdf.

From the introduction:

For the past 10 years or so, Educational Testing Service has supported the Center for New Constructs, which has had as its mission the identification and evaluation of constructs and measures other than those typically measured by content tests, such as mathematics, English language arts, and science achievement tests. Since then the center has been renamed the Center for Academic and Workforce Readiness and Success (CAWRS) to highlight ETS's recent foray into the workplace, but the work of the center has remained the same—to explore the foundations, rationale, and feasibility of measuring personal characteristics that are important for educational success but that are typically ignored in educational assessment. During this period CAWRS staff have explored this area through literature reviews, collaborations, and data collections ranging from a few students to more than 100,000, including some in K–12 education settings, in community college and college settings, in graduate and professional school, and more recently, in the workplace. CAWRS staff have explored a wide array of constructs, ranging from ones that overlap considerably with what is already tested on standardized tests, such as critical thinking, to ones that are quite distinct, such as teamwork and work ethic. CAWRS also has explored a wide variety of measurement methods, such as self-rating and others' rating scales, situational judgment tests, anchoring vignettes, conditional reasoning, the implicit association test, and others. The purpose of this paper is to review some of this work, particularly as it pertains to the measurement of 21st century skills in K–12 education. I will touch on other areas, but that is the focus. This paper begins with a brief history and justification for why there is interest especially now in 21st century skills, and what is meant by that term.

McMurrer, J., & Frizzell, M. (2013). Career readiness assessments across states: A summary of survey findings. Washington, DC: Center on Education Policy, Graduate School of Education and Human Development, The George Washington University. Retrieved from https://eric.ed.gov/?id=ED554578.

From the abstract:

The notion of what it means for a student to be ‘career-ready’ is changing as a result of the recent push by the federal and state governments to ensure that all students are prepared for college and careers by the time they graduate from high school. While much attention has been paid to the ‘college-ready’ aspect of college and career readiness, the term ‘career-ready’ still means different things to different people. Although many state departments of education are currently engaged in defining career readiness and determining how best to measure it, the differences in the scope and complexity of those definitions are significant. Unfortunately, the information available about how states and school districts are defining career readiness and which assessments they are using to measure a varied set of career-related skills is scant and often confusing. It is equally difficult to obtain a coherent understanding of which kinds of career-related skills each assessment measures and how states and districts are using the results of the assessments to evaluate student readiness. This report by the Center on Education Policy (CEP) describes how states are defining career readiness and which assessments states and districts are using to measure this attribute. The report is based on a survey administered in the summer of 2013 to state directors of career and technical education (CTE) or their designees about career readiness assessments. A total of 46 states completed the survey. (The District of Columbia is counted as a state in the tallies in this report.) Key findings included: (1) Only 14 of the 46 states responding to the survey have a statewide definition of what it means for high school students to be career- or work-ready; (2) States and their school districts are using various assessments to gauge career readiness; (3) In many states, school districts or students, not the state, pay the costs associated with taking CTE exams; (4) More states use student results on career readiness assessments to meet federal reporting requirements than use them to make school accountability decisions; (5) Nearly all (45) of the survey states reported facing challenges in assessing high school students' career education or their level of career readiness; and (6) The Common Core State Standards (CCSS) have had little impact thus far on the way that career and technical education skills are assessed. An appendix discusses study methods.

Mishkind, A. (2014). Overview: State definitions of college and career readiness. Washington, DC: College & Career Readiness & Success Center at American Institutes for Research (AIR). Retrieved from https://eric.ed.gov/?id=ED555670.

From the abstract:

The purpose of this overview is to highlight select common elements of state definitions of ‘college and career readiness.’ Although preparing students for postgraduation opportunities has long been a priority for states, districts, and schools, a burgeoning global economy and ongoing labor market shifts call for renewed attention to the readiness requirements for student success in this changing postsecondary landscape. The current domestic workforce demands employees who can fill middle- and high-skills jobs that require a combination of technical and academic skills (Carnevale, Smith, & Strohl, 2010). However, employers struggle to find such qualified workers (Organization for Economic Co-operation and Development, 2013). A definition of college and career readiness can help build an understanding of what these skills are, by highlighting the knowledge, skills, and dispositions of learners who are prepared for postsecondary success. In addition, these definitions can guide educators' efforts to identify which students are on track to succeed in the economy of the 21st century. Recent widespread adoption of the Common Core State Standards or other similar standards in English language arts and mathematics has furthered the conversation about what prepared students look like. The adoption of these standards and the Next Generation Science Standards are designed to set higher expectations for critical thinking, problem-solving, and collaborative skills across a range of academic subjects. As these standards create a set of academic expectations for college and career readiness, definitions of college and career readiness can reinforce these goals and set similar expectations in other areas of student performance. In addition, definitions can play an essential role in guiding states' efforts to achieve their visions for college and career readiness to ensure that students receive the multidimensional preparation necessary for success in the global economy. An appendix entitled, ‘College and Career Readiness: Actionable State Definitions,’ is included.

REL Mid-Atlantic. (2019). Research-based instruments for measuring students' soft skills. Princeton, NJ: Mathematica Policy Research. Retrieved from https://ies.ed.gov/ncee/edlabs/regions/midatlantic/pdf/ RELMA_Soft_Skills_FactSheet_020719.pdf.

From the introduction:

Educators are increasingly emphasizing development of students' soft skills to promote career readiness. Soft skills (sometimes called socioemotional skills) are interpersonal or intrapersonal competencies such as ability to work in a team, taking responsibility for completion of a task, and communication. Education researchers and business leaders agree that soft skills are a critical component of college and career success. To help educators identify and improve their students' soft skills, research organizations such as the RAND Corporation and the Collaborative for Academic, Social, and Emotional Learning (CASEL) have compiled information about measures of key soft skills. This fact sheet provides a sample of instruments for assessing soft skills that can help educators identify students' strengths or challenges with different soft skills, such as teamwork or conscientiousness.

Additional Ask A REL Responses to Consult

Ask A REL Appalachia at SRI International. (2018). How are soft skills defined and measured in education and in the workplace? Retrieved from https://ies.ed.gov/ncee/edlabs/regions/appalachia/askarel/aar30.asp.

Ask A REL Mid-Atlantic at Mathematica Policy Research. (2017). What are the knowledge, skills, and experiences high school students need to transition successfully into careers? Retrieved from https://ies.ed.gov/ncee/edlabs/regions/midatlantic/askarel_14.asp.

Additional Organizations to Consult

College & Career Readiness & Success Center at American Institutes for Research (AIR): https://ccrscenter.org/

From the website:

The College and Career Readiness and Success Center (CCRS Center) is dedicated to ensuring all students graduate high school ready for college and career success. The mission of the CCRS Center is to serve Regional Comprehensive Centers in building the capacity of states to effectively implement initiatives for college and career readiness and success. Through technical assistance delivery and supporting resources, the CCRS Center provides customized support that facilitates the continuous design, implementation, and improvement of college and career readiness priorities.

Council of Chief State School Officers (CCSSO: https://ccsso.org/

From the website:

The Council of Chief State School Officers (CCSSO) is a nonpartisan, nationwide, nonprofit organization of public officials who head departments of elementary and secondary education in the states, the District of Columbia, the Department of Defense Education Activity, the Bureau of Indian Education and the five U.S. extra-state jurisdictions. As an organization, we are committed to ensuring that all students participating in our public education system—regardless of background—graduate prepared for college, careers, and life. To realize this, we bring together dedicated leaders and exceptional ideas to achieve measurable progress for every student.

• Resources on Career Readiness: https://ccsso.org/topics/career-readiness

Partnership for Assessment of Readiness for College and Careers (PARCC): https://parcc-assessment.org/

From the website:

The Partnership for the Assessment of Readiness for College and Career (PARCC) is a collaboration of states that share a commitment to developing new-era assessments that measure students' readiness for college and career. This includes readiness to master rigorous academic content at each grade level, think critically and apply knowledge to solve problems, and conduct research to develop and communicate a point of view. The PARCC states make many of their high-quality resources available to the public through this Partner Resource Center.

U.S. Department of Education, Office of Career, Technical, and Adult Education, Division of Academic and Technical Education: https://cte.ed.gov/

From the website:

The Office of Career, Technical, and Adult Education (OCTAE), Division of Academic and Technical Education (DATE) is responsible for helping all students acquire challenging academic and technical skills and be prepared for high-skill, high-wage, or high-demand occupations in the 21st century global economy. OCTAE's career and technical education initiatives are designed to:

Administer state formula and discretionary grant programs under the Carl D. Perkins Career and Technical Education Act (Perkins).
• Provide assistance to states to improve program quality, implementation, and accountability.
• Establish national initiatives that help states implement rigorous career and technical education programs.
• Employability Skills Initiative: https://cte.ed.gov/initiatives/employability-skills-framework

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

• (“career readiness” OR “career ready” OR “workforce readiness” OR “workforce ready”) AND (framework OR construct OR model OR measur* OR eval*)

Databases and Resources

We searched ERIC, a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES), for relevant resources. Additionally, we searched the academic database ProQuest, Google Scholar, and the commercial search engine Google.

Reference Search and Selection Criteria

In reviewing resources, Reference Desk researchers consider—among other things—these four factors:

• Date of the publication: Searches cover information available within the last 10 years, except in the case of nationally known seminal resources.
• Reference sources: IES, nationally funded, and certain other vetted sources known for strict attention to research protocols receive highest priority. Applicable resources must be publicly available online and in English.
• Methodology: The following methodological priorities/considerations guide the review and selection of the references: (a) study types—randomized controlled trials, quasi experiments, surveys, descriptive data analyses, literature reviews, policy briefs, etc., generally in this order; (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected), study duration, etc.; (c) limitations, generalizability of the findings and conclusions, etc.
• Existing knowledge base: Vetted resources (e.g., peer-reviewed research journals) are the primary focus, but the research base is occasionally slim or nonexistent. In those cases, the best resources available may include, for example, reports, white papers, guides, reviews in non-peer-reviewed journals, newspaper articles, interviews with content specialists, and organization website.

Resources included in this document were last accessed on March 20, 2019. URLs, descriptions, and content included here were current at that time.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Appalachian Region (Kentucky, Tennessee, Virginia, and West Virginia), which is served by the Regional Educational Laboratory Appalachia (REL AP) at SRI International. This Ask A REL response was developed by REL AP under Contract ED-IES-17-C-0004 from the U.S. Department of Education, Institute of Education Sciences, administered by SRI International. The content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government.