|DHS Contract Mgr
|Annual Grant $
|Project Manager Name and Contact Information
|Contact for copy of the Evaluation Data Analysis Report
Richard Sherman, Ph.D.
|Annual Eval $
Interim data analyses are described in the Interim Summary of Project Data Collection and Reporting Activities, July 2009.
In September 2007, the Illinois Office of the Governor received a Notice of Award (NoA) in response to the Access to Recovery II (ATR-II) cooperative agreement application that was submitted by Illinois to the Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment (CSAT). ATR-II funding awards were made to 18 states, 5 Native American tribal organizations, and the District of Columbia. The original ATR-II application submitted by Illinois proposed serving 9,000 clients over a 3-year period, at a requested CSAT funding level of $7 million for each of 3 project years.
On August 29, 2007 a facsimile transmission was received from CSAT stating that the Illinois application was being considered for funding but that the Year 1 award would be reduced to $4,636,800. There was a stipulation that at least $556,633 of Year 1 funds be spent in support of treatment and recovery support services for clients with methamphetamine abuse issues. It was further indicated in this transmission that there would be an expectation to serve at least 6,210 clients over the three years of federal funding, with the following annual client intake targets: Year One - 1,164; Year Two - 2,523; and, Year Three - 2,523. As required in this transmission, a response was returned to CSAT indicating agreement with these conditions. Following receipt of the initial Notice of Award, a request was made to CSAT that the State of Illinois, Office of the Governor would transfer programmatic and fiscal management responsibilities for the Illinois ATR-II project to the Illinois Department of Human Services, Division of Alcoholism and Substance Abuse (IDHS/DASA). This request was approved by CSAT.
Through this CSAT award, Illinois is continuing and expanding its Pathways to Recovery Access to Recovery (ATR) initiative that was implemented in 2004 through a grant from CSAT. Authority for administrative management of this ATR-II project was delegated to the Illinois Department of Human Services, Division of Alcoholism and Substance Abuse (IDHS/DASA). A revised project scope was submitted to CSAT and subsequently approved that described programmatic and fiscal changes which were necessitated on the basis of a reduced award of $4,636,800 for each of Years 1 and 2 and $4,404,960 for Year 3, as compared to an originally-requested funding level of $7,000,000 per year for each of three years.
As proposed in the original application to CSAT, Illinois ATR-II supported services are being continued in Cook County and the 5th and 6th Illinois Judicial Districts, and expanded to the 10th District in west central Illinois. The total number of unduplicated clients that will be enrolled in the Illinois ATR-II supported services is projected to be at least 6,210 individuals.
As originally proposed, the continued Illinois ATR Program includes a methamphetamine treatment component that is targeted to residents of the 11 counties in east central Illinois that comprise the 5th and 6th Judicial Districts. Early in Year 2, a change in scope request was made to CSAT to expand Illinois ATR-II activities to additional judicial districts in the southernmost areas of the state. This request was approved by CSAT and is designed to assist in the identification of methamphetamine-involved clients.
The Illinois ATR-II screening and enrollment processes are structured in a manner that ensures objective and genuine client choice in selection of the providers to whom they will be referred for ATR-II supported treatment and recovery support services. As was the case with the Illinois ATR-I project, a primary emphasis is placed on clients involved with the criminal justice system. Clients of Illinois TASC are a major source of participating individuals within Cook County, as are women involved with the Department of Women's Justice Services within the Cook County Jail. Furthermore, it is anticipated that the majority of clients served through the methamphetamine treatment expansion component will be involved with the criminal justice system. The Illinois System for Tracking Addiction and Recovery Services (I-STARS), a customization of the CSAT WITS data infrastructure, supports a screening, enrollment, referral, vouchering, billing, and services utilization management system. Key partners in this initiative are TASC, Far East Industries (FEI) Inc., the Lighthouse Institute (LI), Family Guidance Centers (FGC), Inc., and IOTA, Inc. TASC, FGC and IOTA function as client enrollment and referral entities. TASC and LI provide recovery management services, to include the conducting of six-month follow-up interviews. TASC and FGC provide client toxicology services. FEI assists in the further customization of I-STARS required for the continued program and on-going maintenance support of the system.
Summary of Key Findings
Status of Progress toward Client Intake Target
This project has an intake goal of enrolling 6,210 clients to the expanded treatment and recovery support services during the 3 years of CSAT funding support. The experience gained by Illinois in ATR-I was a substantial advantage in rapid start-up of ATR-II. ATR-II grantees were expected to start serving clients at the start of January 2008. Illinois ATR-II enrolled its first client on January 3, 2008. Beginning with February 2008, a monthly intake target of 129 clients was established for the Illinois ATR-II project on the CSAT SAIS web site. For each subsequent month in Year 1, the client enrollment target to-date for Illinois ATR-II increases by either 129 or 130. These incremental increases were obtained by dividing the project's Year 1 client intake target (1,164) by 9 (months). The Illinois ATR-II project has an intake target of enrolling 2,523 clients in each of Years 2 and 3. Therefore, in each of Years 2 and 3, the Illinois client target to-date increases by 210 for 9 of the 12 months, and 211 for three of the 12 months. The data analyzed in this interim report was downloaded from the CSAT SAIS web site on July 8, 2009. The intake coverage report table that appeared on the CSAT web site for the Illinois ATR-II project on that date is provided below. As can be seen from the below table, our project had a 106.5% interim client enrollment rate as of this point in time.
Intake Coverage Report GFA Program: Access to Recovery 2
|Intake Coverage Rate
|Avg. Rate of
in this GFA
|State of Illinois -
09/30/07 - 09/29/10
Summary of Illinois ATR-II Client Baseline Data
This interim summary of Illinois ATR data collection and reporting activities is based on GPRA data for our project that was downloaded from the CSAT SAIS web site on July 08, 2009. As was the case in ATR-I, CSAT is allowing ATR-II grantees to count both negative and positive screen clients to their intake targets. Illinois took this approach in ATR-I. In ATR-II however, Illinois has decided to count only positive screen clients to the project's intake target. A positive screen client is an individual who screens positive for a substance use problem (either currently using or in recovery), is indicated to be in need of treatment and/or recovery support services that are funded through Illinois ATR-II, chooses among alternative providers of the indicated service(s), and accepts a referral to the selected service provider(s). Baseline GPRA item responses are entered into I-STARS, and subsequently uploaded to the SAIS web site only for clients who satisfy these criteria.
Provided below is a summary of the baseline GPRA item responses for the 3,258 positive screen clients enrolled by Illinois ATR-II who were represented in the July 8, 2009 download. This count includes two duplicated clients that are included in the below analyses:
* Nearly 74% of persons thus far enrolled in Illinois ATR II are male, nearly 80% are African American, 16.5% are White, and about 5% report being of Hispanic/Latino ethnicity. Mexican is the most commonly indicated ethnic group among clients who report that they are Hispanic/Latino, followed by Puerto Rican. The clients thus far enrolled are on average 37.4 years of age.
* Less than 4.0% of clients thus far enrolled met the CSAT criteria of being methamphetamine clients.
* About one-third of the clients reported that they had been living in their own home or apartment during the past 30 days prior to enrollment, 31.7% reported living in someone else's home or apartment, about 12% reported living in a shelter, and 18% reported being in a halfway house or residential treatment.
* Over 70% of the clients report that they have children and a little less than 5% of the clients reported that they have children living elsewhere due to a court order.
* About 5% of the clients reported at the time of project enrollment that they were currently enrolled in training or school, and nearly 90% stated that they were not currently employed.
* A little over one-third of the enrolled adults reported that they did not have a high school diploma, and an additional nearly 40% of the clients stated that they had no further educational experience or training beyond completion of high school.
* The majority of enrolled clients reported little or no feelings of stress, reduction in important activities, or emotional problems that were attributable to their current or past substance use.
* A little less than 45% of the clients reported having attended self-help groups through a non-Faith-Based Organization (FBO) during the 30 days prior to enrollment, a little over 20% reported attending groups through an FBO organization during this time period, and about 12% reported participation in other activities that support recovery during this time.
* Over 90% of clients reported having had contact with family and/or friends who support recovery during the 30 days prior to enrollment in Illinois ATR.
* Less than 8% of the clients reported that there is no one who they turn to when they are in trouble, and about 60% of the clients report that they primarily turn to family members.
* About 3% of the clients reported having been arrested during the 30 days prior to enrollment, and 6% reported having been incarcerated during this time period.
* Over 45% of the persons enrolled in Illinois ATR II reported being currently on probation or parole, and 7% stated that they were awaiting trial or sentencing.
* The clients enrolled in Illinois ATR II reported generally low average levels of alcohol and other drug use during the 30 days prior to enrollment, and as a group have relatively high rates of recent abstinence from alcohol and other drugs. This reflects a pattern that was observed in Illinois ATR I. This can be attributed to the primary focus of Illinois ATR II on persons who are in recovery.
Summary of Client Outcome Analyses
Illinois ATR II clients first came due for six-month follow-up on June 3, 2008. A total of 1,806 six-month follow-up interviews were entered on the CSAT SAIS web site as of July 9, 2009, for the 1,977 clients who had come due for follow-up. This converted to an interim six-month follow-up rate of 91.4%. The cumulative six-month follow-up rate of all ATR II grantees on July 09, 2009 was 62.9%.
Results from an interim analysis of Illinois ATR II client six-month follow-up GPRA item responses are provided in the below table for data from the 1,825 interviewed clients whose completed surveys were downloaded from the CSAT SAIS web site on July 09, 2009. The results of an interim analysis of baseline (project enrollment) and six-month follow-up GPRA item comparisons are summarized below.
* The comparative pattern of living arrangement status from baseline to six-month follow-up was statistically significant. A slightly higher percentage of interviewed participants reported being housed at follow-up than was the case at project enrollment.
* There was a statistically significant increase from enrollment to six-month follow-up in the percentage of interviewed clients who reported being enrolled in training either part-time (Enrollment - 3.5%; Follow-up - 8.6%) or full-time (Enrollment - 2.0%; Follow-up - 6.7%). It should be noted that this assessment is being made at six-month follow-up. If interviews were conducted somewhat closer to the time of project enrollment the increase may have been larger.
* There was a statistically significant increase from enrollment to six-month follow-up in the percentage of interviewed clients who reported being employed either part-time (Enrollment - 5.3%; Follow-up - 12.3%) or full-time (Enrollment - 3.9%; Follow-up - 15.6%).
* There was a statistically significant increase in reported average income from wages among the interviewed clients from enrollment ($143.62) to six-month follow-up ($296.34).
* There were statistically significant increases from enrollment to follow-up in the percentages of clients who reported an absence of feelings of stress, reduction in important activities, and emotional problems that were attributable to their substance use.
* There was a statistically significant increase from enrollment to six-month follow-up in the percentage of clients who self-reported their health status as very good or excellent.
* There were statistically significant decreases from enrollment to six- month follow-up among the interviewed clients in average days of alcohol, cocaine, marijuana, heroin, methamphetamine, and any illegal drug use during the previous 30 days.
* There were statistically significant increases from enrollment to six-month follow-up among the interviewed clients, in abstinence rates from alcohol, any illegal drugs, cocaine, marijuana, methamphetamine, and heroin.
* There were statistically significant decreases from enrollment to six-month follow-up among the interviewed clients in average days of reported depression, serious anxiety, hallucinations, trouble concentrating or remembering, and trouble controlling violent behavior.
Voucher Information and Transaction Data
ATR II grantees are also expected to enter voucher information and transaction data onto the CSAT SAIS web site within 7 days business days of being collected. Most client vouchers are created at time of enrollment into Illinois ATR II. Each voucher is specific to a particular service provider that is chosen by the individual client. The voucher can include multiple service types if these services are to be provided by the same organization. The voucher is assigned a funding level based upon the service rates and time period caps. The organization can bill against the established voucher amount based upon services provided to the client. It is also indicated whether the voucher is currently active. For vouchers that have been closed, date of closure is indicated and the amount of the voucher reflects the total amount billed against it rather than the amount assigned at time of creation.
Provided below is the Voucher Count Summary Report for the Illinois ATR-II project that was available from the CSAT SAIS web site on July 9, 2009. The following comments summarize the data in this table, in addition to results from additional analyses conducted on voucher information and transaction data downloaded from the SAIS web site on July 9, 2009.
* As reported in the bottom line of the below table, 10,594 vouchers had been uploaded as of this date to the SAIS web site for the clients enrolled in Illinois ATR-II. The total redeemable amount of these issued vouchers was $6,090,218.
* Based on analyses conducted by IDHS/DASA, the average duration between voucher issuance and entry on the SAIS web site was 19.2 days. The median duration for this interval was 7.0 days, which can be interpreted to mean that about 50% of the created vouchers were entered on the web site within 7 days or less time. It should be noted that these measures are in calendar days rather than business days. It can also be assumed that these measures were also influenced by the project's start-up period.
* As of this date, 98.8% (10,464) of the vouchers created had been redeemed in that at least one transaction was billed against the voucher. The total amount redeemed against the created vouchers was $5,443,979.
* As of this date, 88.7% (9,399) of the vouchers created had been cancelled or closed. These decisions are made primarily as a result to clients reaching their service billing caps, the service being otherwise completed, or clients having ceased participation in service delivery at the organization that was issued the voucher.
* Based on analyses conducted by IDHS/DASA, a total of $4,682,574 was billed against the 9,399 vouchers that were cancelled as of July 9, 2009. This accounts for 86.0% of the $5,443,979 that had been redeemed against all vouchers created through Illinois ATR-II. The average duration between issuance of these vouchers and their cancellation was 51.1 days and the median was 40 days. The total amount available to transaction activities for the 1,195 vouchers that remained open at this time was $646,239.
Voucher transactions represent billings submitted by providers against created vouchers. The voucher transaction data includes the amounts billed against individual vouchers by service type with an indication of numbers of service sessions or units represented in the billing. There is indication of the provider organization that submitted the voucher transaction, and whether the organization is "flagged" as a faith-based organization. Provided below is the Service Average Cost Report that was available for the Illinois ATR II project on July 9, 2009. This report provides information regarding the number of voucher transactions processed, the amount of transactions by service type, and the average transaction amount in total and by service type.
* A total of 80,699 voucher transactions were downloaded from the SAIS web site on July 9, 2009. Based on analyses conducted by IDHS/DASA, the average duration between the transaction date and its entry on the web site was 12.5 days. The median duration between the voucher transaction date and entry on the SAIS web site was 1.0 day. Over 75% of the voucher transactions were entered on the website within 7 calendar days of their submission.
* Nearly 55% (44,247) of these voucher transactions were accounted for by faith-based organizations. The total amount redeemed by faith-based organizations was 35.8% ($1,875,128) of the total amount thus far redeemed against vouchers issued by Illinois ATR-II.
Interim Summary of Consumer Satisfaction Survey Responses
Methods were implemented to obtain participating client perceptions of, and satisfaction with, the services they receive through the Illinois ATR II Project. Participants are asked to complete a satisfaction tool as part of their initial enrollment process. This tool asks about their perceptions of various aspects of the enrollment process. Participants are asked to complete a second satisfaction tool when they are contacted for six-month post-enrollment follow-up. This second tool asks about their satisfaction with aspects of the services that they later received through Illinois ATR II. Each satisfaction tool consists of forced choice items that ask for their degree of agreement with the service-related statements being asked, followed by an opportunity for them to provide written comments about their experiences. Below are summary findings from the client satisfaction with enrollment tool responses.
* Over 97% of the responding clients strongly agreed that they were treated with respect by enrollment organization staff, and that these staff spoke to them in a way they could understand.
* Over 90% of the clients strongly agreed that enrollment organization staff was sensitive to gender, racial and ethnic issues, and nearly 90% responded in the same manner to the statement that they were satisfied with the time it took to complete the screening and enrollment process.
* Nearly 90% of the responding clients strongly agreed that the enrollment process was helpful in identifying their service needs, and nearly 90% strongly agreed with the statement that they were able to participate in identifying the kinds of services that they need.
* Over 85% of the clients strongly agreed that they were given the information they needed to choose among alternative treatment and/or recovery support providers, and over 80% strongly agreed that they were given a sufficient number of providers from which to choose.
* Nearly 90% of the responding clients strongly agreed that the statement that they were satisfied with their choice of treatment and/or recovery support service providers.
* Over 85% of the clients strongly agreed with the statement that they know how the issued service voucher works, and a similar percentage of clients strongly agreed with the statement that they were clear about what they needed to do next.
* Satisfaction with Services. The first group of enrolled Illinois ATR II clients came due for follow-up in early June 2008. As of the preparation of this interim report, 1,590 clients who completed six-month follow-up interviews also completed the Illinois ATR II consumer satisfaction survey. This represents over 85% of the clients who have completed six-month follow-up interviews. Nearly 95% (1,497) of these interviews were completed by Lighthouse Institute staff. Responses to the 15 items on this tool are on a four point scale, with a value of 4 indicating the most positive response. Below are summary findings from the average responses to the 15 items on the consumer satisfaction survey completed at six-month post-enrollment follow-up.
* Over or slightly less than 80% of the responding clients rated the services they received as excellent or good, and responded completely or mostly to statements that they were the kind of services they wanted and were sufficient or effective in helping them deal with their problems and/or needs.
* Approximately 85% of the clients responded definitely or probably to statements that they would recommend the services they received to others who needed similar help, and that they would seek out these services if they needed assistance again in the future.
* Approximately 75% of the responding clients responded excellent or good to statements regarding the location of services, professionalism of service provider staff, perceived understanding of their problems, length and frequency of service sessions, and the extent that the service sessions were helpful in resolving their problems or concerns.
Access to Recovery II (ATR II) Evaluation/Performance Assessment Design
Richard E. Sherman, Ph.D. and Maria Bruni, Ph.D.
|Indicators / Measures
||Tools / Instruments / Data Sources
||Other Deliverables / Comments
- # of Clients Enrolled
- # of Vouchers Issued
- # of Service Billings
- Follow-up GPRA Completion Rate
- # of Treatment Providers Enrolled
- # of Recovery Support Providers Enrolled
- # of Faith-based Providers Enrolled
Abstinence Rates among Participating Clients
Various Changes in Client Functioning
CSAT ATR-II GPRA Tool
Service Voucher Forms
Service Encounter/Billing Forms
Treatment and Recovery Support Provider MOUs and Service Profiles
Clientt Satisfaction Surveys
|Data collection and analysis are focused on ensuring compliance with federal reporting requirements. Data is required to be collected from clients at project enrollment,, discharge from services, and six-month follow-up . Voucher information and transaction data must also be reported to CSAT.