CHAPTER 2
Beneficiary Participation in Ticket to Work

Participation rates for beneficiaries who are eligible for TTW have continued to rise each month since early in the program rollout, although they remain low in absolute terms. As of March 2004, the participation rate in Phase 1 states had risen to 1.1 percent, up from the 0.7 percent we reported for August 2003 in the initial evaluation report. Participation rates vary somewhat among subgroups and states, with some small groups participating at rates up to 5 percent.

While these findings are mildly encouraging, they are counterbalanced by the early data from Phase 2 states, which suggest that participation rates were lower for those states than for the Phase 1 states at a similar point in the TTW rollout. These lower rates are entirely due to lower participation at SVRAs; Phase 2 participation at ENs is essentially the same as in the Phase 1 states at a comparable number of months after the beginning of the rollout. SVRA closure statistics indicate that the lower SVRA participation in Phase 2 states is not due to a decline in the number of beneficiaries served. Instead, we attribute the lower Phase 2 rates to three other factors. First, SSA distributed Tickets more gradually in Phase 2 states than in Phase 1 states to ease the burden of the rollout on providers. Second, it appears that the Phase 2 SVRAs are being less aggressive than the Phase 1 SVRAs initially were in obtaining Ticket assignments. Third, the Phase 1 states may differ systematically from those in Phase 2 because of the way SSA selected them. Specifically, SSA selected states for Phase 1 on the basis of their readiness for rollout and then selected Phase 3 states to match the chosen Phase 1 states. Phase 2 states were those that remained after this process and therefore are likely to differ from the matched Phase 1 and 3 states. The evaluation team will continue to track this issue as additional information becomes available, paying particular attention to changes in SVRA services provided to disability beneficiaries before and after the TTW rollout.

The overwhelming majority of Tickets continues to be assigned to SVRAs. As in August 2003, approximately 90 percent of assigned Tickets are with SVRAs. As noted in the initial evaluation report, this pattern suggests that TTW does not yet represent a dramatic break from the past. Instead, its progress so far represents the introduction of new choices and incentives whose ultimate effect will depend on the eventual level of participation and the extent to which ENs and SVRAs offer newer and more effective services.

Only a few beneficiary characteristics predict whether they participate with different types of providers or with providers using different TTW payment systems. Further, even for those having all of the characteristics that have a positive association with participation, the participation is low—under 10 percent. Groups that are more likely to assign their Tickets to ENs rather than SVRAs include older beneficiaries, new beneficiaries, and DI beneficiaries in their extended period of eligibility. Groups that are more likely to assign their Tickets to SVRAs include younger beneficiaries, those who require communication from SSA in some form other than written English or Spanish, and those with hearing impairments. As a rule, SVRAs accept very few assignments under either of the new payments systems, although there are two significant exceptions among Phase 1 states (Oklahoma and Vermont) and another two among Phase 2 states (Connecticut and Louisiana).

This chapter extends the findings presented in our initial evaluation report through March 2004 (the initial report included data through August 2003). The chapter incorporates data elements that were not available to us for the earlier report and presents additional analyses. We therefore summarize the findings from the more recent analysis, focusing on how old findings have changed and on findings that are completely new. The major sections present updated rollout and participation statistics as well as analyses of how participation rates vary among beneficiary subgroups and of how provider and payment type vary with beneficiary characteristics. More extensive statistics appear in Appendix B.

 

 

A. ROLLOUT STATISTICS

1. Ticket Mailings and Eligible Beneficiaries with Tickets

As of March 2004, SSA had mailed over 7.3 million Tickets to beneficiaries. By design, the schedule for the Phase 2 and 3 mailings was slower and more uniform than the schedule for the Phase 1 mailings (Figure II.1). The change was made in response to the difficulties experienced by providers and others in Phase 1 states in handling the large number of beneficiary inquiries generated by the mailings.1

As of March 2004, there were 2.64 million eligible beneficiaries in Phase 1 states and 2.79 million in Phase 2 states. The Phase 3 data through March 2004 imply that approximately 3.5 million beneficiaries were selected to receive tickets, so approximately nine million Tickets were to be mailed by the end of the TTW rollout in September 2004 (by March 2005 more than 10 million Tickets had been mailed).

 

Click for Figure II.1. Ticket Mailings, by Month and Phase (in thousands) (Opens in new window)

 

2. Participation Rate

The TTW participation rate is defined as the number of “in-use” Tickets (i.e., Tickets assigned to providers) as a percentage of current Ticket-eligible beneficiaries. By March 2004, 52,270 Tickets were in use (by March 2005 more than 85,000 Tickets were in use). Over half of these Tickets were held by beneficiaries residing in Phase 1 states, reflecting the rollout schedule. As documented in our initial evaluation report, the participation rate in Phase 1 states was only 0.74 percent as of August 2003. We have revised this August 2003 estimate upward to 0.88 percent, based on more recent data that show that the reporting of assignments is often delayed by several months. By March 2004, the rate was still low in Phase 1 states, but the increase has continued, reaching 1.12 percent (Figure II.2).2

 

Click for Figure II.2. Participation Rate, by Months Since Rollout Start and Phase (Opens in new window)

 

The participation rate in Phase 2 states is low (0.6 percent) but still rising. It appears to be on a track that is well below that of the participation rate in the Phase 1 states. Figure II.2 shows the number of months since the first rollout month (the zero month) on the horizontal axis. Because the Phase 2 rollout began nine months after the beginning of the Phase 1 rollout, the calendar month corresponding to any point in the Phase 2 series is nine months after the calendar month for the corresponding point in the Phase 1 series. When trends in participation are compared in this way, it is clear that the participation rates in Phase 2 states are substantially lower than in Phase 1 states at the same stage of the rollout.

3. Participation by Provider Type

In addition to the overall participation rate, the participation rates at each of the two provider types (ENs and SVRAs) are also an important indicator of program success. As in the initial evaluation report, we again note that the vast majority of in-use Tickets were assigned to SVRAs. There is, however, an important difference between the Phase 1 and 2 states. The participation rate in Phase 2 SVRAs is substantially lower than the participation rate in Phase 1 SVRAs at the comparable point in the rollout (Figure II.3); in fact, the lower Phase 2 SVRA participation rate entirely accounts for the lower overall participation rate. In contrast, beneficiaries participate at ENs in Phase 2 states at essentially the same rate that we observed for Phase 1 states in the comparable month. As a result, the percentage of in-use Tickets assigned to SVRAs in Phase 2 states is 88.7 percent, compared with 91.4 percent in Phase 1 states as of the 16th month.

 

Click for Figure II.3. Participation Rates, by Months Since Rollout Start and Provider Type (Opens in new window)

 

Holding months since the beginning of the rollout constant, we would expect the participation rate to be lower in Phase 2 states than in Phase 1 states because the Phase 2 rollout was more gradual; on average, beneficiaries in Phase 2 states received their Tickets later in their rollout than did beneficiaries in Phase 1 states. If that were the only reason for the difference between the SVRA rates in the two phases, we would expect the difference to eventually diminish after month 11 when the initial mailings for Phase 2 were completed. Instead, the difference appears to have become wider over the last few months we can observe.

Statistics on first assignments (i.e., number of beneficiaries who assign their Ticket for the first time) by provider type make it more apparent that the gap between assignments at the Phase 1 and Phase 2 SVRAs is not likely to narrow soon, because first assignments at Phase 2 SVRAs have already dropped to the same level as those at Phase 1 SVRAs (Figure II.4). First assignments at Phase 2 SVRAs were initially lower than at Phase 1 SVRAs, and they peaked later, reflecting the more gradual Ticket mailing schedule. The decline of first assignments at Phase 2 SVRAs from month 9 to month 16 looks remarkably similar to the decline in Phase 1 states from month 17 to month 25. Because the Phase 2 rollout began 9 months after the Phase 1 rollout, these two periods are essentially contemporaneous (i.e., the same calendar months).

 

Click for Figure II.4. First Assignments at SVRAs, by Months Since Rollout Start and Phase (Opens in new window)

 

Although Ticket assignment data show that fewer Tickets were assigned to Phase 2 SVRAs than to Phase 1 SVRAs at a comparable point in that rollout, other administrative data from SSA and the RSA, along with findings from our interviews of eight Phase 2 SVRAs, indicate that the Phase 2 SVRAs were serving beneficiaries at rates comparable to the rates for the Phase 1 SVRAs. What appears to be different is that the Phase 2 SVRAs were initially much less aggressive than the Phase 1 SVRAs in obtaining Ticket assignments from the beneficiaries they served.

The RSA data are based on individual case records submitted by SVRAs to RSA in each fiscal year for all cases closed (i.e., cases officially completed) in that year—the RSA 911 data. These data (top section of Table II.1) show that the numbers of DI/SSI beneficiaries age 18 to 64 exiting the Phase 1 and 2 SVRAs are quite similar from FY2001 (before the Phase 1 rollout) through FY2003. The first of these years precedes the Phase 1 rollout, and the last year encompasses the Phase 2 rollout (from November 2002 through September 2003).

SSA data on payments to SVRAs under the traditional payment system (from SSA’s Vocational Rehabilitation Reimbursement Management System, VRRMS) also show that the number of cases for which SSA made payments to Phase 1 and Phase 2 SVRAs under this system is quite comparable for the year before the Phase 2 rollout (FY2002) and the rollout year (FY2003) (middle section of Table II.1).

 

Table II.1. SVRA Statistics on Services Provided to SSA Beneficiaries
Period Phase 1 Phase 2 Phase 2 as % of Phase 1
Case Closures per Month (based on RSA 911 data)
FY2001 4,025 3,922 97%
FY2002 4,119 4,035 98%
FY2003 4,357 4,278 98%
Traditional Payment Cases Paid per Month (based on SSA VRRMS data)
FY2001 191 144 75%
FY2002 253 244 96%
FY2003 163 159 98%
New SVRA Assignments per Month (based on SSA January 2005 Disability Control File)
Months 14-16 1,418 997 70%
Months 23-25 1,044 N/A N/A

 

The Ticket data show, however, that Phase 1 SVRAs received an average of over 1,400 assignments per month in months 14 to 16 after the start of their rollout, compared to just under 1,000 for Phase 2 SVRAs in the comparable period (bottom section of Table II.1). The difference is all the more remarkable because the more rapid rollout in the Phase 1 states means that the Phase 1 SVRAs had, on average, more time to obtain assignments from “pipeline” cases (i.e., existing beneficiary clients) before months 14 to 16.3 The Ticket data also show that new assignments per month to the Phase 1 SVRAs fell to almost the same level by months 23 to 25.

Based on these statistics and on interviews of SVRA staff, it appears that the SVRAs are serving many beneficiaries without obtaining Ticket assignments. It appears that either there has been a decline in SVRA’s perceived value of obtaining Tickets assignments and/or an increase in the perceived cost of obtaining assignments. That is, it seems that SVRAs view the cost of processing the paperwork for many beneficiary clients as exceeding the expected benefits.

The SVRAs do not expect to receive payments from SSA for many of their beneficiary clients under any payment system. This is not surprising given that in FY2001, before TTW, the number of cases for which SSA made payments was only about five percent of the number of beneficiary closures.4 In fact, given the number of cases generating payments before Ticket, it is perhaps surprising that the Phase 1 SVRAs obtained so many assignments initially—the number obtained per month in months 14 to 16 was almost nine times as large as the number of cases per month for which SSA made payments in FY2001, the fiscal year preceding the initial TTW rollout.

It is possible that the total number of beneficiaries obtaining services from the Phase 2 SVRAs has declined markedly since the Phase 2 rollout began, but so far, we do not have quantitative or anecdotal indication of such a decline. (Because the RSA statistics are for closures and SVRA service delivery can take many months or even years, it will be some time before we can generate statistics on the post-TTW rollout entry of SSA beneficiaries into SVRA services, including those whose Tickets are not assigned to the SVRA.) As shown in Table II.1, the number of SVRA closures in the Phase 1 states did not fall after the introduction of TTW in FY2002, and we have no reason to expect that they fell in the Phase 2 states either.5 Moreover, if the Ticket rollout were the cause of the drop in the number of cases for which SSA made traditional payments in FY2002, we would expect the drop to be later in Phase 2 states than in Phase 1 states; instead it is contemporaneous. The RSA data suggest a different proximate cause: a decline of similar magnitude in the percentage of beneficiary clients who are competitively employed6 in all three state groups.7 One possible explanation of this decline is the weak economy. Interestingly enough, the percentage of non-beneficiary clients closed as competitively employed did not decline in the same period.8 This might suggest that some factor other than the economy was responsible for the decline for beneficiaries. It also might be that competitive employment of beneficiary clients is more sensitive to the decline in the economy because beneficiaries have a means of support that is not available to nonbeneficiaries, or because beneficiaries have more severe disabilities than nonbeneficiaries.

Federal officials and SVRA representatives we spoke with also said it is unlikely that TTW was responsible for the reduction in payments under SSA’s traditional payment system. Because the average SVRA client receives services for 25 months (Gilmore 2004), it is likely that almost all the claims submitted in FY2002 and even FY2003 came from beneficiaries who began receiving services before TTW.

Staff in four of the eight SVRAs we spoke with did suggest that their agencies were serving fewer beneficiaries than in the past. Some attributed the decline to factors external to SSA programs (e.g., inadequate SVRA resources and changes in state mental health agency policy). None suggested that TTW had a direct effect on the number of beneficiaries served, but two indicated that the elimination of referrals from the Disability Determination Services to their agencies had a negative effect. Any such effects are not likely to be captured in the closure data reported above, but we will continue to track this issue in subsequent evaluation reports.

Other information obtained in these interviews was also consistent with the hypothesis that the Phase 2 SVRAs, having observed the Phase 1 experience, were simply less aggressive about assigning Tickets because of the administrative burden and low payment probabilities. Respondents at the eight SVRAs all said that they received Ticket assignments from only 20 to 50 percent of beneficiaries they served. They also uniformly said that TTW had added administrative requirements to serving individuals under the traditional payment system that were burdensome. For instance, SVRA central office staff and local rehabilitation counselors spent a substantial amount of time explaining the program to beneficiaries, encouraging them to assign their Tickets to the SVRA, and ensuring that beneficiaries made an informed choice in making a Ticket assignment. As one respondent explained:

“The only difference [introduced by TTW] is that reimbursement issues are more visible to the local counselors now than previously. Reimbursement issues used to be handled largely without their knowledge; they focused just on services. Now counselors are more aware of the reimbursement system. For example, they think about the point at which a certain outcome will lead to payment. Counselors are generally questioning of their role of encouraging Ticket assignments and processing Ticket related paperwork. They view it as an extra step that really does not give anything of benefit to consumers.”

Given these comments and the closure statistics above, it seems plausible that the Phase 1 SVRAs accepted many Ticket assignments from beneficiaries who were very unlikely to generate payments, that they now are not bothering to take Ticket assignments from similar clients, and that the Phase 2 SVRAs have learned from the Phase 1 experience.

The Phase 2 SVRAs may also respond to TTW differently from the Phase 1 agencies because of underlying differences in their employment support infrastructure. SSA selected states for Phase 1 that were judged to be most ready for rollout (Stapleton and Livermore 2002). Readiness was judged using factors such as whether the state already had employment promotion programs for people with disabilities (such as a Medicaid Buy-In or a project in the State Partnership Initiative), the strength of the disability advocacy community, and the strength of the provider community including the SVRA and Department of Labor one-stop sites. SSA also considered administrative factors such as whether a state was a Disability Redesign prototype state or had been recommended by the SSA regional commissioner. Once the Phase 1 states were selected, a matching set of states was selected for Phase 3. This matching process was used to enable the evaluation to compare beneficiary outcomes for states that were similar except for the availability of TTW. A result of this process was that the Phase 2 states tend to differ from the Phase 1 and 3 states with respect to the intensity and duration of their programs to support employment for disability beneficiaries. Those differences, in turn, may at least partially account for the apparent different levels of interest in TTW. The evaluation will be able to assess the effect of these differences better once the data on Phase 3 assignment rates is available.

4. In-Use Tickets by Payment Type

In the initial evaluation report, we noted that most in-use Tickets were assigned under the traditional payment system, necessarily to SVRAs. Based on updated data, 87.7 percent of Tickets in the Phase 1 states were assigned under the traditional payment system as of August 2003.9 That figure rose slightly by March 2004, to 88.6 percent.

A larger percentage of in-use Tickets have been assigned under the two new systems in the Phase 2 states than were assigned under those systems in Phase 1 states, but this difference is only a reflection of the lower number assigned to SVRAs. The participation rates associated with each of the two new payment types in Phase 2 states are somewhat higher than the corresponding rates in Phase 1 states at comparable points in their rollouts (Figure II.5).10 In March 2004 (month 16 for the Phase 2 states), the participation rate for the two new payment systems combined was .115 percent in the Phase 2 states compared with .101 percent in the Phase 1 states in the comparable month.

5. Deactivations and Reassignments

We also examined administrative data on deactivations and reassignments to determine whether substantial numbers of beneficiaries who have assigned their Tickets are changing providers or formally withdrawing, or being withdrawn, from participation. What we found in the initial evaluation report continues to be true: reassignments rarely occur, and deactivations are also very low—typically 0.5 percent of assigned Tickets in each month, with no substantive difference between the Phase 1 and 2 states.

We do not know, however, how many in-use Tickets are effectively inactive. Ticket users who stop their return-to-work effort entirely have little motivation to withdraw their Tickets. Providers are to deactivate Tickets if beneficiaries are not making timely progress at the 24-month review, but almost no Tickets had been assigned for that long during the period considered. While some providers have unassigned Tickets of inactive clients prior to the 24-month mark, others have not done so, either because they did not know they could or because they wanted to avoid the administrative costs of processing deactivations even if warranted.

 

Click for Figure II.5. Participation Rates for the New Payment Systems, by Months Since Rollout and Phase (Opens in new window)

 

 

B. PREDICTORS OF PARTICIPATION

This section updates the analysis of how the participation rate varies with beneficiary characteristics.11 The statistics presented here show the bivariate relationship between the participation rate and each characteristic. These statistics are useful for determining differences in participation rates among groups that could be targeted using data available to SSA. However, some of these relationships reflect relationships between participation and other characteristics that co-vary with the specific characteristic examined in the bivariate analysis (e.g., age and Title). Findings from a multivariate analysis (which are presented later in this section) net out these various interrelationships and indicate how participation is related to specific characteristics when other characteristics are kept constant. We report primarily statistics for Phase 1 states. Bivariate statistics for the Phase 2 states are very similar.

All of the statistics in this section pertain to March 2004 and are based on SSI and DI program data matched to an April 2004 extract from SSA’s Ticket MI Universe File. These data differ from those used for the earlier analysis in this chapter, which were extracted in January 2005 from SSA’s new Disability Control File, mostly because of delayed reporting of Ticket assignments.12 The more recent data show that about 10 percent of in-use Tickets in Phase 1 states in March 2004 had not been reported as assigned as of April 2004. Due to time and resource constraints, we did not update the analysis below. But we have no reason to think that the relationships reported here would be qualitatively different from those in updated data; most of the participation rates reported for specific groups would presumably be on the order of 10 percent higher, however.

1. Months Since the Ticket Mail Date

One of the most obvious predictors of assignment is the number of months since the Ticket mail date, which confirms findings in the initial evaluation report. As of March 2004, 25 months had elapsed since SSA mailed the first Tickets. The beneficiaries included in that first mailing had the highest participation rate, 1.33 percent. Those beneficiaries who were sent Tickets in the previous month had the lowest rate, 0.21 percent. The participation rate rises steadily throughout the observed range, showing no evidence of abating in the near future.

2. Concurrent Beneficiaries

We previously reported that participation rates for concurrent beneficiaries are somewhat higher than for DI-only or SSI-only beneficiaries, with the latter two groups participating at about the same rates. The same is true in the current findings: the 13 percent of Phase 1 eligible beneficiaries who were concurrent beneficiaries had a participation rate of 1.53 percent in March 2004, compared with 0.93 percent for DI-only beneficiaries and 0.98 percent for SSI-only beneficiaries.

3. Age

As with the initial analysis, the current analysis shows that the participation rate is highest for the youngest eligible beneficiaries and declines with age. This too continues to be true. In March 2004, the Phase 1 participation rate for the relatively small share of eligible beneficiaries in the youngest age group, 18 to 24, was 2.79 percent, compared with just 0.16 percent for the oldest age group, 60 to 64. The participation rate for all those 50 to 64 years old is only 0.4 percent.

4. Primary Impairment Category

The initial evaluation report indicated that participation rates vary substantially by primary impairment category, and the updated findings are very similar. Beneficiary groups in Phase 1 states with March 2004 participation rates that were at least half a percentage point above or below the overall value of 1.0 percent are shown in Table II.2. As indicated by the statistics in the second column, some of these groups are very large, but others are quite small. Also, while some participation rates are high relative to the overall average, for example, the rate for those with hearing impairments (5.9 percent) is almost six times the overall participation rate, they are not high in an absolute sense. Impairment categories containing relatively large numbers of beneficiaries and participation rates above the 1.0 percent average rate include schizophrenia/psychoses/neuroses (1.5 percent), major affective disorders (1.3 percent), nervous system disorders (1.3 percent), and mental retardation (1.2 percent). The impairment category with the largest percentage of beneficiaries, musculoskeletal (16 percent of beneficiaries) has the third lowest participation rate (0.5 percent).

5. Race and Ethnicity

Hispanic beneficiaries participate at a lower rate than other racial/ethnic groups (0.7 percent in Phase 1 states), and the rate for those requesting that SSA send them all written information in Spanish is even lower, 0.4 percent. African Americans participate at a higher rate (1.3 percent in Phase 1 states). Participation rates for whites, Asians/Pacific Islanders, and Native Americans are very close to the overall rate.

6. Time Since Disability Program Entry

As documented in the initial evaluation report, the participation rate in Phase 1 states increased with time from the point of entry into a disability program up through month 24 and then is stable through 120 months (10 years) before declining somewhat. A different finding emerged from our analysis of more recent data. Participation rates are lower than average for those who have been on the rolls for six or fewer months (.77 percent), the rate rises to 1.00 percent for those on the rolls from seven to 12 months, and there is no substantial variation with additional months on the rolls.

7. State of Residence

Like our initial analysis, the recent analysis shows that participation rates vary substantially across the Phase 1 states, although even the highest rate as of March 2004 rate (2.3 percent for Vermont) was low (top of Figure II.6). The lowest rate, 0.4 percent, was in Oregon.

Participation rates also vary substantially across Phase 2 states (bottom of Figure II.6). As of March 2004, South Dakota had the highest rate (1.6 percent,) and New Hampshire had the lowest (0.1 percent).

 

Table II.2. Beneficiary Groups with High or Low Participation Rates in March 2004, Phase 1 States
Participation Rate at Least 1.5 Percent
Participation Rate No More than 0.5 Percent
Beneficiary Groups
Participation Rate (%)
Percent in Group (%)
Beneficiary Groups
Participation Rate (%)
Percent in Group (%)
Age
18-24
2.8
5.9
  50-64
0.3
31.7
25-29
2.1
4.5
 
30-39
1.7
15.1
 
Primary Impairment
Hearing
5.9
0.9
  Respiratory impairment
0.4
2.5
Speech
2.3
0.1
Circulatory impairment
0.5
6.9
Congenital anomaly
2.0
0.4
Musculoskeletal
0.5
16.4
Visual
1.7
2.0
 
Schizophrenia/ psychoses/ neuroses
1.5
7.6
 
State of Residence
Vermont
2.3
0.7
  Arizona
0.5
5.0
Delaware
2.2
0.8
Colorado
0.4
3.1
 
Massachusetts
0.4
7.4
 
Oregon
0.4
3.3
Other
13-16 years of schoolinga
1.5
17.4
  1-8 years of schoolinga
0.3
16.7
Concurrent
1.5
13.0
  SSA mailed Ticket 0 to 6 months ago
0.3
3.4
Work incentive participation
  Spanish preferred
0.4
4.3
Section 1619(b) (SSI)
3.1
0.3
   
Section 1619(a) (SSI)
2.4
0.4
   
Extended period of eligibility (SSDI)
2.3
0.4
   

Source: Ticket Research File.
Note: "Beneficiary groups" are groups of Ticket-eligible beneficiaries defined by the characteristic indicated; "participation rate" is the percentage of the group in Phase 1 states with in-use Tickets in March 2004; and "percent in group" is the percentage of eligible beneficiaries having the indicated characteristic.
aThe education statistics are based on the relatively small share of Ticket-eligible beneficiaries for which education is reported in the administrative data (38.6 percent).

 

Click for Figure II.6. Ticket Participation Rates in Phase 1 and Phase 2 States, by State, Provider Type, and Payment Type, March 2004 (Opens in new window)

 

In all Phase 1 and Phase 2 states, a large majority of Tickets were assigned to SVRAs (left side of Figure II.6). Indeed, in Vermont and South Dakota, all in-use Tickets were assigned to SVRAs, and both of these states had the highest participation rates in their respective phase. Assignment rates for ENs were relatively high in Arizona, Tennessee, and Nevada. Similarly, a large majority of Tickets were assigned under the traditional payment system in all Phase 1 and 2 states. Participation under the milestone-outcome system was exceptionally high in Oklahoma, and participation under the outcome-only system was exceptionally high in Vermont, where the SVRA accepts a relatively large number of beneficiary clients under that system. We also found that the number of Ticket assignments increased at least to some extent in Phase 1 SVRAs from August 2003 through March 2004. No Phase 1 state experienced a very substantial change in the number assigned to ENs.

8. Use of Work Incentive Programs

We also found that TTW participation was higher than average for beneficiaries using two SSA work incentive programs. In particular, the Phase 1 participation rates were considerably higher for the 0.4 percent of Ticket eligible beneficiaries who were DI beneficiaries in their extended period of eligibility (EPE) on their mail date (Figure II.7).13 Phase 1 rates were also higher for the 0.7 percent of Ticket-eligible beneficiaries who were SSI recipients and in the Section 1619b programs on their mail date than for other beneficiaries—over 3 percent of those in Section 1619b participants.

These beneficiaries were all working when the Ticket was rolled out, and some may have been close to leaving the rolls. In fact, the 1619b participants were not receiving an SSI benefit on their Ticket mail date, although they presumably were getting benefits on the earlier Ticket selection date, and the DI beneficiaries in EPE status had only a limited number of months remaining in which they would be able to collect benefits if their earnings fell below the substantial gainful activity level. Given the circumstances of the 1619b participants, it is perhaps surprising that their participation rate was not even higher. That is not true, however, for those in 1619a status who continue to receive cash payments.

 

Click for Figure II.7. Ticket Participation Rates in Phase 1 States, by Use of Other Work Incentives, March 2004 (Opens in new window)

 

9. Education

The analysis of the relationship between education and participation rates is new as of this report. We had correctly anticipated that participation rates would be positively related to education. However, as a beneficiary’s education often plays no direct role in the administration of the disability programs,14 the administrative data on education are incomplete; we found that education information was missing for 61 percent of eligible beneficiaries in Phase 1 states. But even so, we found a substantial positive relationship between education and the participation rate for those with education data (Figure II.8). The statistics show that those with the highest participation rate have 13 to 15 years of schooling (i.e., completed some post high school education but less than four years). The data also show, however, that only 12 percent of eligible beneficiaries with reported education are in this category.

 

Click for Figure II.8. Ticket Participation Rates in Phase 1 States by Education, March 2004 (Opens in new window)

 

10. Allowance Level

The Beneficiaries who were awarded disability benefits only after appealing an initial denial to an administrative judge participate at a lower rate than those awarded benefits on the basis of their initial application.15 In Phase 1 states, the participation rate for the former group is 0.63 percent, compared with 1.20 percent for the latter.

11. Other Predictors of Participation

We also examined the relationship between participation rates and several other characteristics not reported on above. We were particularly interested in determining whether “newly eligible” beneficiaries—those who became eligible for Ticket after the rollout began in their state—would participate at different rates than “existing eligible” beneficiaries—those eligible when the rollout began in their state. We found that newly eligible beneficiaries were participating at rates that were just slightly lower than existing eligible beneficiaries—a difference that reflects the lower participation rate for those whose Tickets had been mailed up to six months earlier. We also found no substantive difference by sex.

12. Multivariate Analysis of Participation

We extended the bivariate analysis of factors correlated with beneficiary participation by considering how all factors together are related to beneficiaries’ participation decisions. This multivariate approach lets us assess how well we can predict participation on the basis of measured characteristics. It also lets us assess the extent to which any specific factor is related to participation once we control for the effects of other beneficiary characteristics that may be correlated with that factor. For example, the multivariate analysis lets us assess whether the bivariate finding that concurrent beneficiaries participated at a higher rate than DI-only or SSI-only beneficiaries is really due to the fact that concurrent beneficiaries tend to be younger than other beneficiaries (younger beneficiaries also participate at higher rates).

Overall, we found that each characteristic that has a substantial bivariate relationship with the participation rate, as described above, also has a substantial relationship with the participation rate after accounting for co-variation with other characteristics. In fact, the bivariate and multivariate relationship between participation and each characteristic is almost the same for many characteristics. A few exceptions are noteworthy, as discussed below. The characteristics included in this analysis are title, sex, age, attainment of Ticket eligibility after the initial selection date, race, ethnicity, education, monthly benefit, months since initial DI or SSI award, language preference, level of allowance, participation in the EPE, Section 1619a or Section 1619b during the ticket mail month, primary impairment, state of residence, and a series of county characteristics (enumerated later). Details of the specification appear in Appendix B. Variables were defined as they were in the bivariate analysis.

One important finding from this analysis is that it is very difficult to predict which beneficiaries participate from characteristics that are observed in administrative data. Only about one percent of the variation in participation can be accounted for by all of the characteristics used in this analysis.16 Further, even when we use all of the characteristics as predictors, the highest estimated probability of participation is less than 10 percent, and only one percent of beneficiaries has a predicted probability of 4.4 percent or higher. Because many beneficiaries do participate, we would have expected that some beneficiaries would have had a high predicted participation probability if our model had been accurate. The low predicted probabilities suggest that other unmeasured factors, such as the nature/severity of their impairment, social supports, and personal motivation, play an important role in beneficiaries’ decisions.

After other characteristics are controlled for, the relationship between Title and participation changes. DI-only and concurrent beneficiaries participate at almost identical rates, and the participation rate for all DI beneficiaries is 0.2 percentage points higher than for SSI-only beneficiaries.

Adjustments for all other characteristics increased the participation rate for New York relative to that for other Phase 1 states by about 0.5 percentage points. The apparent reason for the difference is the delay in the rollout in New York combined with the strong positive relationship between participation and the number of months since the Ticket mail date; compared with their counterparts in other Phase 1 states, New York beneficiaries have had less time, on average, to assign their Tickets.

Controlling for other factors strongly affects the differences between both racial/ethnic groups and language preference groups. African American beneficiaries still participate at higher rates than white beneficiaries, and Hispanic beneficiaries participate at lower rates, but the differences between the groups are smaller. The difference between participation rates of those who prefer to communicate in Spanish and those who prefer to communicate in English is also smaller after controlling for other factors, but it remains substantial (0.3 percentage points versus the unadjusted difference of 0.8 percentage points). Hispanic ethnicity and a preference for the Spanish language are obviously connected. Together, the model estimates imply that Hispanics who prefer that SSA communications be in Spanish participate at a rate that is 0.4 percentage points lower than that of whites who prefer English, holding other factors constant.

After other factors are controlled for, the participation rate for those with 16 or more years of schooling is slightly higher than the rate for the 13- to 15-year group. Thus, one or more other factors that are correlated with education appear to explain why the participation rate for the 16+ group is lower than for the 13-to-15 group (see Figure II.8).

Variation in participation rates among impairment categories changes substantially after controlling for other beneficiary characteristics, particularly for categories that have relatively low participation rates. We conjecture that the changes are mostly due to co-variation between impairment category and age, and the strong relationship between age and participation. Before adjustment, the two categories with the lowest participation rates are respiratory system and circulatory system, both categories that are relatively more common for older beneficiaries. After adjustment the lowest two are blood/blood-forming diseases and mental retardation. Both of the latter categories have above-average participation rates before controlling for other variables—about 1.2 percent for each. The change for mental retardation is particularly striking, because this category is large (11.6 percent of beneficiaries in this analysis). Thus, although beneficiaries with mental retardation are participating at an above average rate, after controlling for their age and other factors they are participating at a rate that is well-below average.

After controlling for other factors, we still found that those who received their benefit allowance only after appealing an initial denial beyond the reconsideration level were less likely to participate than those who received their allowance at the initial determination level, but the difference is smaller than the simple bivariate difference (0.25 percentage points versus 0.48). We also found that some of the difference in the rates for those participating in the other work incentive programs (EPE for DI and Section 1619 for SSI) and those not participating in the other programs are considerably smaller, albeit still substantial, after other factors are controlled for.

As mentioned, the bivariate analysis indicates that beneficiaries who received their benefit allowance in the six months before March 2004 are less likely than others to participate, and that participation rates vary little with duration of time on the rolls for those who received their allowance earlier. In the multivariate analysis, we experimented with a simple model that had only one category for those on the rolls for 24 months or more and accounted for a possible interaction between those on the rolls for less than 24 months and those enrolled in DI. The purpose of the experiment was to capture the possibility that some DI-only beneficiaries might refrain from participating in TTW until they were entitled to Medicare (that is, until they had completed the two-year waiting period). Using the model, we found that those on the rolls for 7 to 23 months as of March 2004 participated at a rate that was only slightly higher than the rate for those who received their award at any point up through six months. We also found that DI-only beneficiaries were just slightly less likely than SSI beneficiaries to participate. We plan to examine this issue further in the future with more detailed statistical models.

We also included a variety of county characteristics17 in the multivariate analysis to examine whether these characteristics are predictive of participation after controlling for beneficiary characteristics and state of residence (see Appendix B). The results indicate that, most county characteristics individually have very little predictive power after individual characteristics are controlled for. We also found that the results are difficult to interpret because of correlations between the numerous county characteristics. In the future, we will develop models in which county variables are treated in a way that makes the results easier to interpret.18

 

C. PREDICTORS OF PROVIDER AND PAYMENT TYPE

For our earlier report, we found that provider and payment type for in-use Tickets vary with age. We continue to find, in both Phase 1 and 2 states, that older beneficiaries are more likely than younger beneficiaries to assign their Tickets to ENs and to use one of the new payment systems. For example, in March 2004, 13.0 percent of Phase 1 beneficiaries age 60 to 64 with in-use Tickets had assigned their Tickets to ENs, 21.0 percent of which were assigned under one of the new payment systems; the corresponding percentages for those 18 to 24 years old were 3.4 and 7.9 percent, respectively.

In general, provider and payment type do not vary much with other beneficiary characteristics, or characteristics of their county. A few exceptions are noteworthy. In Phase 1 and Phase 2 states, Ticket participants who were new eligibles (i.e., first became eligible for Ticket only after the start of the rollout) are somewhat more likely than existing eligibles to use ENs (15.0 percent versus 11.6 percent); relatively few participants requiring communication from SSA in some form other than written English or Spanish assigned their Tickets to ENs (5.6 percent); participants with hearing impairments are much less likely than others to use ENs (2.1 percent); and DI participants in their EPE are more likely than other DI participants to use ENs (18.7 percent versus 7.6 percent).


1 See Appendix A for the rollout schedule and a list of states by phase. Return to Text.

2 This March 2004 estimate is based on reporting through mid January of 2005. Return to Text.

3The more rapid rollout is likely an important cause of the large number of assignments to Phase 1 SVRAs in months 4 through 7. Return to Text.

4Compare the FY2001 SSA cases paid per month in Table II.1 to the number of beneficiary closures per month in the same year. Return to Text.

5Because statistics for total closures might mask a decline in closures for beneficiaries who entered SVRA programs recently, we also tabulated the number of closures in each year for beneficiaries found eligible for SVRA services in the same year. We did not find evidence of a decline in such closures for either Phase 1 or Phase 2 states. We also checked the RSA 911 data to see if the number of closures for beneficiaries might have declined relative to those for non-beneficiaries from FY2001 to FY2003, and found virtually no relative change in the Phase 1 states and a relative, but small increase for beneficiaries in the Phase 2 states. Return to Text.

6Competitive employment excludes employment at sub-minimum wages and employment in sheltered workshops. Return to Text.

7As discussed in Chapter VI, in Phase 1 states, the percentage of beneficiary clients aged 18 to 64 who SVRAs closed to competitive employment dropped from 29.6 percent in FY2001 to 26.1 percent in FY2003. For Phase 2 states, the comparable drop is from 29.8 percent to 24.4 percent, and for Phase 3 states it is from 30.9 percent to 26.1 percent. Return to Text.

8For all non-beneficiary clients, the percentage competitively employed at closure increased from 33.3 percent in FY2001 to 34.0 percent if FY2003, compared with a decline from 29.8 percent to 25.6 percent for all beneficiary clients. Return to Text.

9Our earlier estimate was 75.9 percent. Most assignments reported after the extraction of data for the initial evaluation report were to SVRAs. Return to Text.

10The participation rate for a specific payment system is defined as the number of Tickets in-use under that payment system divided by total eligible beneficiaries. Return to Text.

11Back-up tables for the figures in this section appear in Appendix B. Return to Text.

12The more recent data also incorporate minor changes in (1) the definition of a beneficiary’s state for purposes of assigning Phase and (2) the classification of beneficiaries by program (i.e., Title). Return to Text.

13Information on participation in another component of the DI work incentives, the Trial Work Period, status was not available for this report. Return to Text.

14While education is a consideration at step five of the medical eligibility determination process (ability to do any other work), it is not relevant to eligibility for those allowed at a lower step, and does not play any other role in the administration of the program. Return to Text.

15For purposes of this analysis we include people who are awarded benefits at the reconsideration level along with those awarded benefits based on their initial application. Return to Text.

16This statement is based on the value of the adjusted R2 from the regression of the participation indicator on all of the characteristics (1.2 percent). Return to Text.

17The variables are: population density, population loss between 1990 and 2000, percent African American, percent non-white, percent Hispanic, percent of populations living in households with income below the poverty line, percent of employment in manufacturing, the unemployment rate, another low employment index, an urban/rural index, the percentage of workers using public transit, a housing stress index, and a low education index. Return to Text.

18There are several possible reasons for the low predictive power of individual county characteristics. One is that they are highly correlated with individual characteristics already captured by other variables—especially the state indicators, as well as race, education, and possibly others. Another is that they are highly correlated with each other across counties, making it difficult to separate the predictive power of any one from the others. Agodini et al. (2002) found similar results in an evaluation of the State Partnership Initiative. We will explore these explanations further in future analyses. Return to Text.