Documenting the Value of Non-Degree Credentials: The Potential Role of the Multistate Longitudinal Data Exchange

Written by Heather McKay, Sara Haviland, and Suzanne Michael, Rutgers, The State University of New Jersey, Education and Employment Research Center and Patrick Lane, Western Interstate Commission for Higher Education

 

 

Introduction

In the light of the changing nature of work and education, state leaders across the nation are considering the role of non-degree credentials (NDCs) in postsecondary education. In fact, many states have included non-degree credentials – credit and noncredit certificates, industry certifications, licensures, and apprenticeships – in their state’s postsecondary goals.[ii] Non-degree credentials are also at the forefront of states’ discussions about reducing equity gaps in education and the labor market, filling labor market demands, and preparing their workforces for the future. Essential to these conversations is good data on the value of NDCs in the labor market and their potential with respect to educational and career pathways.

Current data indicate that non-degree credentials have become more prevalent over the last 15 years, with more than one-quarter of Americans now holding an NDC.[iii] However, those data are not robust in terms of who those Americans are or where they live, the fields or skills covered by the credentials they earned, the quality of the NDCs conferred, or the value of any given NDC to the individual who earned it.[iv] While the need for good data on NDCs is clear, systems for collecting and analyzing them are neither well developed nor utilized. Some states have begun to use State Longitudinal Data Systems (SLDS) to gather and aggregate data on NDCs.[v] Other states are employing existent data to better understand the value or quality of NDCs, thereby helping both policymakers and consumers to make more informed decisions about the use of state and personal resources.[vi]

These activities are all important steps but remain limited by state lines, providing only partial answers about national patterns and possible directions for policy and program activities. Another factor limiting such analyses is the rise of nontraditional private providers, such as bootcamps and other for-profit opportunities, which have become an increasing source of NDCs. Few of these private providers collect the student data (particularly Social Security numbers) necessary for researchers, state agencies, or the providers themselves to evaluate how those who participate in these opportunities fare in the workplace. Today, to truly understand the value and potential of NDCs, we need multistate outcomes data that recognize the realities of migration in and out of states – of individuals crossing state borders to study, to work, to deal with family circumstances, or for other reasons. We also need information about how the different types of educational opportunities that are becoming more prevalent in the modern age can benefit students and workers while helping provide the skilled talent today’s economy requires. The Multistate Longitudinal Data Exchange (MLDE) is a nascent system that has the potential to respond to these needs.

The MLDE was piloted in 2010 and expanded shortly after as a collaboration between the Western Interstate Commission for Higher Education (WICHE) and agencies within participating states that collect data related to education and employment (e.g., K-12 and higher education, workforce agencies). The MLDE facilitates the exchange of data between states to provide practitioners, policymakers, and researchers with a comprehensive data system that can enhance their understanding of 21st-century educational and career pipelines and trajectories. The MLDE is well suited to provide information on NDCs because it can document the movement of people who hold or are in pursuit of NDCs both within and across state lines as they move between education and careers. This brief explores the various ways that multistate data and the MLDE might be able to help states better answer many lingering questions about NDC policy and practice.

Over the past five years, the MLDE’s third-party evaluator, the Education & Employment Research Center (EERC) at Rutgers, the State University of New Jersey, has engaged in a mixed-methods research evaluation of the MLDE implementation process. This brief draws on EERC’s qualitative research, which included 40 interviews with state leaders and WICHE staff, observations of user group meetings, surveys, and MLDE document analysis.

In the following sections, we present the reasons why states have joined the MLDE or considered joining the MLDE. We then discuss states’ interest in data and the ways data can be used to evaluate NDCs. We include in our discussion ideas heard from state policymakers during our study as well as ideas found in current literature and media on the topic. We conclude with the recommendation that as states and territories continue to develop and build their own longitudinal data systems, they should consider the value of both adding data on NDCs and sharing that data with other states.

How might cross-state data sharing improve policymakers’, practitioners’, and researchers’ understanding of non-degree credentials?

Over the past five years, while studying the implementation of the MLDE, EERC has accumulated information on both the need for and potential uses of multistate longitudinal education and workforce data. Although EERC’s interviews with state leaders did not specifically focus on NDCs, many of the leaders’ comments about the need for more and better data, and how they would use such data if it were available, can be interpreted to refer at least in part to the relative lack of data on NDCs. NDC data are therefore a natural extension of the MLDE’s current infrastructure that could yield rich results.

The addition of NDC data to state agency datasets and SLDS is an important step in the pursuit of answers about the prevalence and value of NDCs. A recent report by the Workforce Data Quality Campaign (WDQC) looked at the extent to which states are already collecting NDC data. Although just beginning, infrastructures are being developed for higher quality data: 36 states reported collecting data on most for-credit certificates from their public two-year institutions; 27 states reported collecting data about most registered apprenticeship certificates; and 22 states reported having access to most licensing data.[vii] However, WDQC and others have identified a number of significant issues with respect to collecting and aggregating NDC data. For example, it is often unclear what forms of credentials qualify as an NDC in a given dataset. Are all types of NDCs receiving equal attention by data collectors, or is there a privileging of some over others? Naturally, NDCs from postsecondary institutions and providers that participate in workforce training programs – when they are already incorporated into state data infrastructures – are most likely to be linked to earnings information, but states that rely on those data sources could be missing information about a large swath of NDCs that may (or may not) have value in the workforce. Nonetheless, the WDQC report showcased growing awareness of the need for good data on NDCs and the progress that has already been made in securing that information.

In addition to collecting data, some states have begun to use available data to implement quality-assurance processes that examine either the content of the credential or its outcomes, typically measured in terms of educational or labor market achievements.[viii] In fact, 30 states have already developed or are in the process of developing a list of “credentials of value” to help guide the allocation of public dollars for financial aid, workforce development, or other programs.[ix] In addition, some states are making data available to the public to assist consumers with educational and career decision making. These important efforts will benefit if more states join the MLDE and actively share their data on NDCs.[x]

Further, standardization of analysis of shared data sets would help answer questions about the pathways of state-to-state migrants and the future of work, and it would expand the value of existent SLDS for state policymakers and their constituents.

The Pathways of State-to-State Migrants

One of the most prominent reasons state leaders gave for wanting to collect and exchange data with other states was to enhance their understanding of the movement of workers and students: those who migrate in and out of the state, those who commute daily out of state for school or work. Tracking the movement of students and workers would enable enhanced analysis of the alignment of state educational investments. It would also allow the exploration of questions regarding the extent to which the state draws in migrants from other states vs. the extent to which its own students, once they complete their education or training, move out of state for better job opportunities. Analysis of education and employment pipelines could also inform discussions of human capital and the “balance of trade” between states,[xi] thereby informing state policy and practice decisions with respect to educational resources and workforce development. Data show that just over 2 percent of the U.S. population moved to a new state during the period of 2016 through 2018.[xii] An analysis of cross-state commuting data from 2011 shows that an additional 4 percent of workers crossed state lines for employment as commuters, with some states posting substantially higher numbers.[xiii]

Significance for NDCs:

Multistate data on NDCs could help states track long-term educational and labor market outcomes of individuals who have earned one or more NDCs and who have migrated into or out of a state. Such tracking could enhance understanding of the effects of NDCs on educational and career pathways and financial well-being. These data could help inform decisions regarding which NDCs should be supported and promoted by shedding light on the relative quality and value of NDCs to the people who earn them.

Future of Work

Some of the state leaders with whom EERC spoke reflected on the changing value of the MLDE for their state. Many commented on the impact of the global economy on job opportunities and how new technologies were transforming the nature of work. These changes required more real-time data to facilitate decision making about educational and economic policies and programs.

Significance for NDCs:

NDCs are frequently mentioned in the context of the future of work, where they are most often described as valuable tools for training and retraining workers so they have the requisite knowledge and skills needed for new technologies. Tracking the pathways of NDC seekers and holders could help policymakers respond more strategically to the changing needs of industry sectors and employers and better prepare the next generation of workers. Workers, as well as employers, will continue to cross state borders, including some who will work remotely in states other than the one in which they live. Multistate data will therefore become increasingly useful to regional planners and state policymakers involved with the allocation of economic resources, including tax incentives, as well as educators as they develop or refine their curriculum and program offerings.

Expanding the Value of SLDS Data for States and Constituents

Some state leaders talked about using the MLDE to provide more comprehensive data to their constituents, thereby fostering more informed decision making. As one respondent said, “I think that’s what ultimately our goal is: to be transparent and to provide data that is useful to the public, not just state agencies.”

Significance for NDCs:

Improving the data available to constituents aligns well with the work cited above and the fact that some 30 states have already developed lists of NDC “credentials of value.” It also aligns well and would build on work currently being performed by the Credential Engine to document and count credentials.[xiv] Access to multistate data through the MLDE would enable a more comprehensive analysis of the impact of NDCs on educational and labor market outcomes. This would help individuals, including incumbent workers, to make more informed choices about pursuing specific NDCs or other academic credentials. It would also help industry, unions, and professional organizations make more informed decisions about the viability of NDCs and their provision of such credentials.

Filling Other Data Gaps on NDCs

An increasing number of NDCs come from private providers who may be under no mandate to report on their students. These credentials include formal certifications in computer coding through private bootcamps, digital badges provided by employers, and numerous other types of training and education. While some of these programs may look and feel like formal education programs, they can exist in a regulatory gray area where they claim not to be subject to state laws on the provision of education programs and avoid reporting requirements (if any) that would allow policymakers and the public to evaluate how the credential they offer benefits participants in the workplace.

Even states that regulate these providers may not require them to submit accountability or evaluation data to state data systems; states that do not regulate them certainly have no such requirements. Additionally, providers of many of these newer credentials are unlikely to collect or share Social Security numbers, which are usually a key data element required to access earnings information.

For providers that have a strong interest in understanding more about how those that earn their credentials fare in the workforce, an approach that borrows from the MLDE has the potential to help them get the information they need. The data infrastructure supporting the MLDE was designed to provide a state with earnings information about its students in cases where that state did not have students’ Social Security numbers, but other states did. To grossly simplify the approach, the system allowed states without Social Security numbers to request that other states provide those numbers to participating earnings data repositories.

The MLDE approach mimics efforts from Idaho, where the state obtained earnings data on high school graduates who did not enter the state’s postsecondary system (and thus the state never had access to the students’ Social Security numbers).[xv]

---

Conceptually, the solution is straightforward:

• Develop partnerships between NDC providers and the sources of the desired outcomes data as well as the state’s higher education data repository. This allows the training provider to submit individual-level data to the state’s higher education data repository, which can then use this information to match with its own records.
• Have the postsecondary education data source request earnings data from the state’s Unemployment Insurance system and return aggregated data to the provider, while carrying out any needed state research or analysis on its own.
• Use the data and information for evaluation of education and training programs or consumer information.

---

In many cases, the state agency holding earnings data for evaluation purposes also has the postsecondary data that is necessary in the matching phase, therefore significantly easing the burden of developing the necessary partnerships. The analysis of these data points would provide information on how the participants in these new NDC programs fare in the workforce. As a proof-of-concept, responsible and ethical providers would most likely be interested in participating on a voluntary basis in exchange for data that could help significantly improve their programs.

Although such tracking and analysis would be limited to only those individuals who had participated in a states’ formal public postsecondary system, it would provide a starting point for evaluation purposes. States participating in multistate data initiatives like the MLDE could broaden the scope of such an effort to include data from across the country.

Conclusion

As NDCs become more prominent in today’s labor market and educational landscape, data on their outcomes will be increasingly important for the public sector, industry, and workers as they make decisions about preparing the workforce and distributing resources. While some states have begun to look at NDC data using their SLDS, to truly understand the value and potential of non-degree credentials we need multistate outcomes on a variety of NDCs. Students and workers migrate into and out of states for a variety of reasons, and it is important that these realities are captured in data systems and analyses where possible. This may be even more true now with the education and employment impacts of COVID-19. With the future of work being so uncertain, and with budgets being severely constrained by the pandemic, it will be crucial for states to better understand the outcomes of their investments. Understanding the quality and value of NDCs in educational and career pathways requires cross-state data access now more than ever.

 

*****

 

For More Information

For further information about the MLDE, please contact Patrick Lane, WICHE vice president of policy analysis and research, at plane@wiche.edu, or visit the project website: https://www.wiche.edu/key-initiatives/multistate-longitudinal-data-exchange/. The other briefs in this series include: Building Trust for Inter-Organizational Data Sharing: The Case of the MLDE, Diffusion of an Innovation: Lessons from the Multistate Longitudinal Data Exchange, Designing the Architecture of a Multistate Data Sharing Model, and Documenting the Value of Non-Degree Credentials: The Potential Role of the Multistate Longitudinal Data Exchange.

For further information about evaluation research in data sharing or workforce development, please contact Heather McKay, director of Rutgers’ Education & Employment Research Center, at hmckay@rutgers.edu, or visit the Center’s website: http://smlr.rutgers.edu/eerc.

Endnotes

[i] The other briefs in this series include: Building a Multistate Governance System, Building Trust for Inter-Organizational Data Sharing: The Case of the MLDE, Designing the Architecture of a Multistate Data Sharing Model, and Documenting the Value of Non-Degree Credentials: The Potential Role of the Multi-State Longitudinal Data Exchange. Available from the WICHE website at https://www.wiche.edu/key-initiatives/multistate-longitudinal-data-exchange/.

[ii] HCM Strategists, States with Higher Education Attainment Goals (Washington, DC: HCM Strategists, 2019), accessed on 10 June 2020 at http://strongernation.luminafoundation.org/report/2019/media/Attainment_Goal_state%20rundown_021519.pdf.

[iii] Michelle Van Noy, Heather McKay, and Suzanne Michael, Non-Degree Credential Quality: A Conceptual Framework to Guide Measurement (Piscataway, NJ: Education and Employment Research Center, 2019), accessed on 10 June 2020 at https://smlr.rutgers.edu/NDCQuality.

[iv] Van Noy, McKay, and Michael, Non-Degree Credential Quality.

[v] Jenna Leventoff, Measuring Non-Degree Credential Attainment (Washington, DC: Workforce Data Quality Campaign, 2018), accessed on 10 June 2020 at https://www.nationalskillscoalition.org/resources/publications/file/Measuring-Non-Degree-Credential-Attainment-50-State-Scan.pdf.

[vi] David Ramsay, Garden State Uses Credential Data to Help State Economy and Education Bloom (Trenton, NJ: Division of Workforce Research and Analytics, Department of Labor and Workforce Development, 2018), accessed on 10 June 2020 at https://credentialengine.org/2018/08/22/garden-state-uses-credential-data-to-help-state-economy-and-education-bloom/. Amy Ellen Duke-Benfield, “Expanding Opportunities: Defining Quality Non-Degree Credentials for States,” PowerPoint presentation, Denver, CO, 3 December 2019, accessed on 10 June 2020 at  https://postsecondarydata.sheeo.org/wp-content/uploads/2019/12/145_DukeBenfield_DefiningQuality.pdf.

[vii] Leventoff, Measuring Non-Degree.

[viii] Duke-Benfield, “Expanding Opportunities.” Van Noy, McKay, and Michael, Non-Degree Credential.

[ix] Laura A Scione, E-Campus News, “More states are recognizing the importance of non-degree credentials,” accessed on 10 June 2020 at https://www.ecampusnews.com/2019/02/12/more-states-are-recognizing-the-importance-of-non-degree-credentials/. Leventoff, Measuring Non-Degree.

[x] Ramsay, Garden State. Duke-Benfield, “Expanding Opportunities.”

[xi] Brian T. Prescott, Beyond Borders: Understanding the Development and Mobility of Human Capital in an Age of Data-Driven Accountability. A Report on WICHE’s Multistate Longitudinal Data Exchange Pilot Project (Boulder, Colorado: Western Interstate Commission for Higher Education, 2014), accessed on 10 June 2020 at https://www.wiche.edu/longitudinalDataExchange.

[xii] Author calculations from three tables from 2016, 2017, 2018 from the United States Census Bureau, State-to-State Migration Flows data tables, accessed on 10 June 2020 at https://www.census.gov/data/tables/time-series/demo/geographic-mobility/state-to-state-migration.html.

[xiii] Brian McKenzie, Out-of-State and Long Commutes: 2011, (Washington, DC: United States Census Bureau, 2013), accessed on 10 June 2020 at https://www.census.gov/library/publications/2013/acs/acs-20.html.

[xiv] Carrie Samson, Counting U.S. Secondary and Postsecondary Credentials (Washington, DC: Credential Engine, 2018), accessed on 10 June 2020 at https://credentialengine.org/2018/04/05/counting-u-s-secondary-and-postsecondary-credentials-april-2018-report/.

[xv] National Center for Education Statistics, SLDS Topical Webinar Summary: Using DMV Records to Access Social Security Numbers (Washington, DC: Institution for Education Sciences, 2013), accessed on 10 June 2020 at https://nces.ed.gov/programs/slds/pdf/Using_DMV_Records_to_Access_SSNs_Webinar_Nov2013.pdf.

References

Duke-Benfield, Amy Ellen. “Expanding Opportunities: Defining Quality Non-Degree Credentials for States.” PowerPoint presentation, SHEEO Communities of Practice meeting, December 3, 2019. Denver, CO. Accessed on 10 June 2020 at https://postsecondarydata.sheeo.org/wp-content/uploads/2019/12/145_DukeBenfield_DefiningQuality.pdf.

HCM Strategists. States with Higher Education Attainment Goals. Washington, DC: HCM Strategists, 2019. Accessed on 10 June 2020 at http://strongernation.luminafoundation.org/report/2019/media/Attainment_Goal_state%20rundown_021519.pdf.

Leventoff, Jenna. Measuring Non-Degree Credential Attainment. Washington, DC: Workforce Data Quality Campaign, 2018. Accessed on 10 June 2020 at https://www.nationalskillscoalition.org/resources/publications/file/Measuring-Non-Degree-Credential-Attainment-50-State-Scan.pdf.

McKenzie, Brian. Out-of-State and Long Commutes: 2011. Washington, DC: United States Census Bureau, 2013. Accessed on 10 June 2020 at https://www.census.gov/library/publications/2013/acs/acs-20.html.

National Center for Education Statistics. SLDS Topical Webinar Summary: Using DMV Records to Access Social Security Numbers. Washington, DC: Institution for Education Sciences, 2013. Accessed on 10 June 2020 at https://nces.ed.gov/programs/slds/pdf/Using_DMV_Records_to_Access_SSNs_Webinar_Nov2013.pdf.

Prescott, Brian T. Beyond Borders: Understanding the Development and Mobility of Human Capital in an Age of Data-Driven Accountability. A Report on WICHE’s Multistate Longitudinal Data Exchange Pilot Project. Boulder, Colorado: Western Interstate Commission for Higher Education, 2014. Accessed on 10 June 2020 at https://www.wiche.edu/longitudinalDataExchange.

Ramsay, David. Garden State Uses Credential Data to Help State Economy and Education Bloom. Trenton, NJ: Division of Workforce Research and Analytics, Department of Labor and Workforce Development, 2018. Accessed on 10 June 2020 at https://credentialengine.org/2018/08/22/garden-state-uses-credential-data-to-help-state-economy-and-education-bloom/.

Samson, Carrie. Counting U.S. Secondary and Postsecondary Credentials. Washington, DC: Credential Engine, 2018. Accessed on 10 June 2020 at https://credentialengine.org/2018/04/05/counting-u-s-secondary-and-postsecondary-credentials-april-2018-report/.

Scione, Laura A, E-Campus News. “More states are recognizing the importance of non-degree credentials.” Accessed on 10 June 2020 at https://www.ecampusnews.com/2019/02/12/more-states-are-recognizing-the-importance-of-non-degree-credentials/.

United States Census Bureau. “State-to-State Migration Flows.” Accessed on 10 June 2020 at https://www.census.gov/data/tables/time-series/demo/geographic-mobility/state-to-state-migration.html.

Van Noy, Michelle, Heather McKay, and Suzanne Michael. Non-Degree Credential Quality: A Conceptual Framework to Guide Measurement. Piscataway, NJ: Education and Employment Research Center, 2019. Accessed on 10 June 2020 at https://smlr.rutgers.edu/NDCQuality.