In July 2012, we signed an interagency agreement (IAA) with the Bureau of Labor Statistics (BLS) to develop and test occupational data collection methods that could lead to the development of a new Occupational Information System (OIS). The OIS will replace the Dictionary of Occupational Titles (DOT) that we currently use in our disability adjudication process. The new OIS will include many occupational descriptors similar to those adjudicators currently use in the DOT. However, the OIS will expand on DOT information by describing basic mental and cognitive work requirements and providing more details about occupations’ exertional and skill ratings.
(See OIS Project History for more background information.)
BLS Occupational Requirements Survey
Under the IAA, BLS is using the Occupational Requirements Survey (ORS) to collect occupational characteristics data that is critical to our disability adjudication process. ORS data, such as the physical demands, skill requirements, and environmental conditions of work, mirror the occupational descriptors found in the DOT.
BLS Fiscal Year 2013 Testing
In fiscal year (FY) 2013, the BLS ORS team conducted three phases of data collection field-testing to evaluate survey design options; to develop, test, and refine collection protocols and aids; and to assess the feasibility of using the National Compensation Survey (NCS) as a platform to collect the occupational data. In the first phase, BLS collected data from 27 establishments. By Phase 3, BLS was collecting data from 690 establishments for about 3,259 occupations.
BLS Fiscal Year 2014 Testing
In FY 2014, BLS continued testing the feasibility of using the NCS platform to capture data relevant to the Social Security disability program. The primary goals were improving data quality, minimizing the burden on employers, and resolving outstanding issues from the FY 2013 testing. Specifically, in 2014 BLS conducted the following test activities:
- A joint NCS/ORS test to determine the feasibility and best practices for jointly collecting NCS and ORS data elements;
- A central office collection test to determine how best to collect ORS data elements from large firms and State governments;
- The collection of the new data elements that describe the mental and cognitive demands of work;
- An evaluation to determine the feasibility of using the occupational task lists contained in the Occupational Information Network (O*NET) during data collection;
- An “ORS only” efficiency innovations test to develop more efficient approaches for data collection identified during FY 2013 testing;
- An alternative modes test to determine how best to collect occupational requirements data elements by phone or email when a personal visit is not optimal; and
- An observation test that captured and evaluated changes in data coding that resulted from observing the work environment, the sampled occupation, or both.
BLS Fiscal Year 2015 Activities
The results of FY 2013 and 2014 testing were encouraging and indicated that BLS could move forward to pre-production testing in FY 2015.
In FY 2015, BLS is:
- Conducting a large-scale nationwide pre-production test to complete the evaluation of whether ORS can produce estimates of sufficient quantity and quality to meet our needs and prepare to move the survey into a production environment;
- Continuing to work on issues from the FY 2014 testing, including further refinement and testing of the new mental and cognitive data elements, survey design processes, protocols, aids, and collection procedures; developing best practices to collect data from large employers; and conducting additional research on methods of collection and approaches for testing the validity and reliability of occupational requirements data; and
- Contracting with an outside expert to produce a methodological report that provides advice on achieving high levels of reliability, validity, and accuracy given the ORS’ parameters. Specific concerns include the relative merits of different sources of information and modes of data collection. The central recommendation is that BLS conducts its own systematic tests and validation studies in order to ensure that the final survey design aims for the most reliable, valid, and accurate measures possible given the various alternatives. (See Occupational Requirements Reliability and Validity Literature Research)
The DOT contains discrete and well-established descriptions of the physical demands of jobs, but it does not provide information on their mental and cognitive requirements. The current ORS testing represents a unique opportunity to establish the comparable mental and cognitive requirements of work so that we can add this data to the new OIS.
In FY 2013, we organized an internal workgroup of disability vocational policy experts and contracted psychiatric and psychological consultants to develop data elements that describe the mental and cognitive requirements of work. The workgroup started with the Disability Evaluation Constructs -- a list of all of the possible elements that may be useful to disability adjudication that we gathered from external experts and public comment from the Occupational Information Development Advisory Panel -- and a list of the mental and cognitive data elements from O*NET. The workgroup also reviewed the results of an internal study that gathered information from 5,000 disability claims, which is a representative sample of claims decided at steps 4 and 5 of sequential evaluation at the initial and hearings levels. To ensure that the mental and cognitive elements met the needs of our disability programs, the workgroup used the results from the study showing the most commonly assessed mental limitations in claims with mental impairments to develop a manageable set of mental and cognitive data elements focused on the demands of lower skilled work.
During FY 2014 feasibility testing, BLS tested the mental and cognitive survey questions. OIS project staff and staff from the Office of Disability Policy observed several data collection appointments. We worked with BLS, whose field economists have experience collecting similar mental and cognitive occupational requirements for the NCS, to make refinements and adjustments to the questions.
The test captured the following mental and cognitive data:
- The complexity of the work tasks;
- The frequency and nature of social interactions; and
- The frequency of required changes in work tasks, work schedules, and work location.
Results from testing and feedback from stakeholders have resulted in changes to the mental and cognitive survey questions and responses. In the summer of 2015, BLS will test the revised mental and cognitive questions.
Working with DOL’s Employment and Training Administration (ETA)
The Department of Labor (DOL) last updated the DOT in 1991, and, in its place, launched O*NET in 1998. O*NET does not contain the detailed physical job requirements needed to adjudicate disability claims under current Social Security regulations and policy.
However, there are aspects of O*NET that we could effectively integrate into the OIS to avoid duplicating work already done by DOL. We are working with DOL’s ETA to identify and incorporate into the OIS O*NET task descriptions, lay titles, and information on occupational tools and technology.
OIS Technology Platform
In FY 2015, we began work on an easy-to-use, web-based platform to support the new OIS. We assembled a workgroup of the agency's occupational information users, including representatives from the Office of Disability Adjudication and Review and adjudicators from State disability determination services, to help outline system requirements. When we complete development, the OIS will be available to the public to use online at no cost.
For more information regarding the development of the OIS, please contact us at firstname.lastname@example.org.