VI. Verification and Validation of Performance Measures

General Discussion

SSA is committed to ensuring that those who use our reported performance information to make decisions can do so with the assurance that our data are valid and reliable.

The Section V discussion on "Performance Goals, Means and Strategies" includes a detailed definition of each of the plan's performance measures as well as the data source used to measure performance.

Performance data for the plan's quantifiable measures, including the budgeted output measures, are generated by automated management information and workload measurement systems as a by-product of routine operations.

The performance level for several indicators relating to the accuracy of our processes and public satisfaction comes from surveys and workload samples designed to achieve very high levels (usually 95 percent confidence level) of statistical validity. The Office of Quality Assessment (OQA) reviews a stratified sample of recently completed actions and of ongoing entitlement rolls to determine the accuracy of SSA payments and service transactions.

To help ensure the integrity of management information (MI) data, SSA implemented a Management Information Integrity Monitoring (MIIM) team in February 1998. The MIIM team serves as the clearinghouse for resolving allegations of inappropriate practices affecting MI. Line managers and staff throughout the Agency also review MI data regularly to identify anomalies and correct reporting problems.

For those indicators for which SSA can not define performance goals in an objective, quantifiable form, descriptive statements are provided on how we will consider the goal to have been achieved.

Program Performance Report

SSA's annual Accountability Report has been the vehicle used to report on SSA's key goals and performance measures as well as the Agency's progress in meeting its GPRA goals. SSA is continuing to use the Accountability Report as the program performance report for reporting on performance for FY 1999 and beyond.

The Accountability Report describes, in detail, SSA's comprehensive program of conducting reviews of management and security controls in both SSA's administrative and programmatic processes and in accounting controls in financial management systems. The effectiveness of these reviews was confirmed when the independent accounting firm of PricewaterhouseCoopers performed its FY 1998 financial audit of SSA processes and systems. Such reviews and assessments assure that our systems are secure and not vulnerable to manipulation by intruders, and confirm SSA's confidence in the reliability of its performance data.

Role of the Inspector General

The Office of the Inspector General (OIG) plays a key role in assuring that the data systems used by SSA to measure performance are reliable. Each year since 1998, the OIG's workplan has included reviews of SSA performance measures in which the OIG evaluates the processes and systems being used to measure progress in each of the measured areas, so as to assure that they provide reasonable assessments of performance. So far, the OIG surveyed the existence of sources of information for the 68 performance measures (i.e., indicators) appearing in SSA's FY 1999 APP, and found that SSA has methods to collect data for all indicators.

The OIG is currently auditing several performance measures contained in SSA's FY 1999 and FY 2000 APP's. SSA is taking appropriate action to correct any reported deficiencies. These actions may include disclosure of data gaps, changes in performance measures, improvements to or additions of data collection systems, or some combination thereof.

For example, the OIG identified two deficiencies with regard to the data used to measure the accuracy of social security number card processing and the accuracy and timeliness of earnings posted to individuals' records by September 30. In response, our Revised Final FY 2000 APP and FY 2001 APP correct the definition of the former indicator and clarify that self-employment earnings are not included in the latter measure. The OIG also found a weakness in the system SSA uses to measure SSI debt management. In July 1999, we implemented a new software release that improves control over SSI debts and collections, and results in SSA having more and better information to accurately measure SSI debt management.

The OIG uses a four-point approach to reviewing SSA's performance measures:

• Assess SSA's system capacity to produce performance data;

• Assess whether reported performance measure data is valid;

• Ensure that SSA has the appropriate measures to indicate vitality of its programs; and

• Ensure that the performance measures fully capture the program segments that they are intended to capture.

General Accounting Office Reviews

In its review of SSA's FY 2000 APP, the General Accounting Office (GAO) found that the plan "provides general confidence that agency performance information will be credible."] The GAO noted that SSA's annual Accountability Report provides additional assurances of credibility in its discussion of systems reviews that are routinely performed by independent contractors as well as SSA's actions to respond to their recommendations.

Coordinated Agency Evaluation Plan

Each fiscal year, SSA develops a coordinated Agency evaluation plan. All key components involved in the evaluation function, including the OIG, conduct a joint review of evaluation workplans to assure an appropriate match between planned evaluation activities and Agency priorities, identify and address any gaps in needed information and eliminate any overlap or duplication.

Additional information on the Agency Evaluation Plan is provided in Appendix 4.