Data Sources and Limitations


for OP4T Methodology

Occupational Prioritization for Training

Methodology

Background

The Employment Department's Research Division is often asked to provide data and information to guide decisions relating to workforce education and training. Division leadership and staff are often cautious about "over-selling" their data, perhaps seeking to be overly sure that policy-makers fully understand the limitations of the data.

This paper outlines an approach which we believe will be useful in helping policy-makers prioritize the use of scarce training and education resources, taking advantage of every possible item of useful information, while minimizing the impact of weak or non-existent data sources.

The Approach in Outline Form

The proposed approach has four main steps, each one building upon the ones before it, and each having the ability to contribute to data-based decision-making even if subsequent steps cannot be completed.

  • Focus on Oregon's high-wage, high-demand occupations, building on the principles established by Oregon's Workforce Investment Board in 2007.
  • Build a foundation on strong data: occupational demand (growth and replacement openings), occupational wages, occupational vacancies (overall number and share reported as difficult to fill), and geographic diversity of occupational need.
  • Add nuance based on more subjective analysis: impact of immigration, perceived supply / pipeline shortages, business affirmation of the workforce need, links to key industry groups (i.e., those already identified as Governor's or other priorities). Provide prioritized lists of occupations to policy-makers.
  • Invite subject matter experts to "dig deeper" into key occupations identified in steps one through three, determining which should become Oregon's priorities for workforce-related training.

The Research Division's primary role would be in steps one through three; planning, policy-making, and other entities would likely play the lead role in step four.

The Detailed Plan

Step One:

Start with all occupations on Oregon's statewide high-wage (and) high-demand list.  These lists were based on concepts and definitions approved by Oregon's Workforce Investment Board in 2007. In a nutshell, occupations on this list:

  • Pay more than the median wage ($17.60 per hour for the statewide list in 2013, varies for regional lists) for all occupations. AND
  • Are projected to have more than the median number of openings (286 in 10 years for the statewide list in 2012, varies for regional lists) for all occupations in Oregon.

This approach immediately narrows the focus from all occupations (714 in Oregon) to 214 that pay reasonably high wages and are expected to be in reasonably high demand.

Step Two:

Rate the occupations based on the following characteristics:

  • Total (growth plus replacement) openings – list the occupations in a sort from most openings to fewest openings; identify the quintiles; give those occupations in the top quintile (those with the most openings) a value of 5, those in the next group a value of 4, and so on down to the last quintile, which will get a value of 1. Source: Oregon Employment Department (OED) 2012-2022 Employment Projections
  • Relative occupational need – list the occupations in a sort from highest relative need (defined as the total number of projected openings [growth and replacement] compared with the current size of the occupation) to lowest relative need; give those occupations in the top quintile (those with the highest share of openings relative to current employment) a value of 5, those in the next group a value of 4, and so on down to the last quintile, which will get a value of 1. Source: OED 2012-2022 Employment Projections
  • Wage – list the occupations in a sort from highest median wage to lowest median wage; give those occupations in the top quintile (those with the highest wages) a value of 5, those in the next group a value of 4, and so on down to the last quintile, which will get a value of 1. Source: OED 2014 Oregon Wage Information
  • Number of vacancies – list the occupations in a sort from most vacancies to fewest vacancies; give those occupations in the top quintile (those with the most reported vacancies) a value of 5, those in the next group a value of 4, and so on down to the last quintile, which will get a value of 1. Source: OED 2013 Vacancy Survey
  • Difficult to fill share of vacancies – list the occupations in a sort from the highest share of difficult to fill vacancies to the lowest share of difficult to fill vacancies; give those occupations in the top quintile (those with the longest average duration) a value of 5, those in the next group a value of 4, and so on down to the last quintile, which will get a value of 1. Source: OED 2013 Vacancy Survey
  • Geographic diversity of need – determine how many of Oregon's regional high-wage, high-demand lists each occupation appears on; list the occupations in a sort from those appearing on the most regional lists (maximum 15) to those appearing on the fewest; give those occupations in the top quintile (those on the most high-wage, high-demand lists) a value of 5, those in the next group a value of 4, and so on down to the last quintile, which will get a value of 1. Source: OED High-Wage, High-Demand Occupational Lists, 2013

At the end of this step, Oregon's 214 high-wage, high-demand occupations each have a composite (additive) score ranging from 30 (for an occupation that was in the top quintile for all of the above measures) to 6 (for an occupation in the lowest quintile for all measures). For regional prioritizations, the final component – geographic diversity of need – does not apply, so regional scores range from 5 to 25.

Step Three:

Add other insights and nuances to the discussion, based on weaker, more subjective, but still potentially valuable factors.

 (Note that each of these factors would be applied in a multiplicative manner, so a value of 1 would have no impact on the prioritization developed in Step Two. Only in cases where specific knowledge suggested that a factor should impact the initial prioritizations would the value be raised above or reduced below 1.)

  • Impact of immigration – start each occupation with a default value of 1, meaning that the impact of immigration was not known. Change the factor to as low as 0.9 (for occupations most likely to recruit across state or national boundaries) and as high as 1.1 (for occupations least likely to recruit across state or national lines). This factor would place higher priority on training for occupations where the local workforce is likely to be the primary source of workers. Source: OED Research staff review of U.S. Census Bureau American Community Survey data on out-of-state movers, by occupation.
  • Impact of supply / pipeline shortages – start each occupation with a default value of 1, meaning that the supply of workers was not known. In occupations where supply can be measured with at least moderate precision, change the factor to as low as 0.7 (for occupations which appear to have a surplus of individuals completing relevant training) and as high as 1.3 (for occupations which appear to have a shortage of individuals completing relevant training). This factor would place higher priority on training for occupations where supply that can be measured with reasonable accuracy falls short of projected demand. This factor is potentially the most significant of the four Step Three factors, but its increased weight in the final ranking is only used in the relatively few cases were demand and supply data are both believed to be very reliable and where evidence suggests almost all individuals completing certain programs of training move into a specific occupation. Source: OED Research staff review of estimated supply of education program completers; analyst judgment.
  • Business affirmation of the workforce need – start each occupation with a default value of 1, meaning that the extent to which businesses felt there was a shortage of workers in the occupation was not known. Change the factor to as high as 1.1 (if businesses strongly confirm the need for more trained individuals). Decision-making on this factor was based primarily on input from Oregon community colleges and Business Oregon, through special surveys conducted by OED staff. Both groups were able to report on specific business conversations or actions demonstrating business commitment to the training of workers in certain occupations. This factor defaulted to 1 for most occupations in this iteration of this process, but over time, we expect additional business input to increase the value of this factor. Source: OED Research staff review of information provided DCCWD and Business Oregon (Economic Development); analyst judgment.
  • Links to key industries – start each occupation with a default value of 1, meaning that the occupation's link to one of Oregon's key industries was not known. Change the factor to as high was 1.1 if an occupation were determined to be of critical value to one of the Governor's (or other policy-makers') highest priority areas of focus (e.g. manufacturing, health care, green jobs). This factor allows for previously agreed on economic and workforce development priorities to be included in this prioritization model. This factor is flexible and we expect this column to change as priorities and policy interests change. Source: OED Research staff review of public policy priority statements from the Governor's office and others, review of staffing patterns in priority industries, and health care occupations determined in partnership with industry in 2006.

At the end of this step, Oregon's 214 high-wage, high-demand occupations each have a composite (additive and multiplicative) score ranging from about 50 (for an occupation that was in the top quintile for all of the step two measures and at the highest level for each of the step three measures) to about 4 (for an occupation in the lowest quintile for all step two measures and at the lowest level for each of the step three measures). Similarly, scores on regional prioritizations have the potential to range from 41 to 3.

Step Four:

Policy-makers and other leaders use the prioritization identified in steps one to three to identify occupations for further consideration, then invite subject matter experts to "dig deeper" into those occupations.

Subject matter experts may include, but are not limited to, business owners, hiring managers, labor representatives, policy-makers, workforce system leadership, and education planners.

 "Digging deeper" implies a thorough discussion of a variety of background information beyond the data provided in steps one through three:

  • Why is the occupation expected to need additional training?
  • Are certain skills particularly in need?
  • Is the occupational shortage caused by factors other than the supply of trained workers?
  • Is the occupation impacted by specific training, education, certifications, or skills requirements?
  • Are there barriers to additional training such as lack of equipment, faculty, or internships?
  • Are there opportunities for positive steps that would attract additional students toward particular occupations, such as scholarships or other incentives?

Important Note: Exceptions

As with any other public policy-related effort, it should be expected that some occupations or related training programs may be selected for priority resource allocation, even if they do not come out on top based on the process outlined above. The Governor, the Legislature, Oregon's Workforce Investment Board, and others must retain the right to make policy choices based on exceptions to this process. However, it is to be hoped that a process similar to the one above would become the driver for many, even most, workforce training-related resource prioritization decisions.