DHS Project Director Theodora Binion-Taylor
DHS Contract Mgr Deborah Levi, Ph.D.
Annual Grant $ $3,500,000
Evaluator Name & Contact Info

Richard E. Sherman, Ph.D.

IOTA, Inc.

&

Christy K. Scott, Ph.D.

Lighthouse Institute

Annual Eval $

$35,000 (IOTA)

$315,000 (Lighthouse)

Funding Source SAMHSA/CSAT
Eval Period 9/30/2003 through 6/30/2009

Evaluation findings are described in Final Summary of Project Data Collection and Reporting Activities, July 2009, by Richard E. Sherman, Ph.D., IOTA, Inc.

Project Summary

In August 2003, the State of Illinois, Office of the Governor received notice of approval of the application submitted by the State of Illinois, Cooperative Agreements for Screening, Brief Intervention, Referral and Treatment (SBIRT). The primary goal of the Illinois SBIRT Initiative (ISI) is to expand the state's continuum of care by implementing substance use disorder (SUD) screening, brief intervention, and referral to treatment (SBIRT) services within hospitals and community health clinics operated by the Cook County Health and Hospitals System (CCHHS). The scheduled end of this five-year CSAT-funded project was September 29, 2008. CSAT approved a nine-month no-cost extension of project activities through June 30, 2009.

The Illinois Department of Human Service, Division of Alcoholism and Substance Abuse (IDHS/DASA) managed the Illinois SBIRT Initiative (ISI). The following three general hospitals were represented in the Illinois SBIRT Project: John H. Stroger, Jr. Hospital of Cook County (formerly Cook County Hospital), Provident Hospital, and Oak Forest Hospital. Four community health clinics located within Chicago also participated in the project. SBIRT health counselors provided screening and brief intervention services within Cook County Health and Hospital System (CCHHS) settings. TASC community care coordinators assist in linking hospitalized substance dependent patients to substance abuse treatment in one of the 11 IDHS/DASA-licensed treatment providers involved in the project.

Patients indicated to be in need of treatment were referred to IDHS/DASA-licensed treatment providers for further assessment and admission. Through this project, Brief Treatment (BT) was being implemented as a new treatment modality for patients determined to be appropriate for this level of care.

Patient Service Categories

All patients who presented to one of the participating hospitals (John H. Stroger Jr. Hospital of Cook County, Provident Hospital, or Oak Forest Hospital) or the community health clinics were eligible to be screened by a Health Counselor. The Health Counselors were grant-funded staff who were added to the clinical team at the health care site and reported to a central SBIRT management team. Patients were approached by a health counselor in the course of receiving the usual medical services and told that screening was a routine part of care. Only about 3% of the patients refused screening. All patients were asked if they used beer, wine, or liquor in the past twelve months. Patients who answered "Yes" to alcohol use were asked more questions about the quantity and frequency of alcohol use. All patients were asked if they used marijuana, cocaine, heroin, sedatives or pain medicines not as prescribed or other drugs within the past 3 months. Patients who were screened as not being at risk for a substance use disorder were categorized as Screening and Feedback Only (SF).

Using a structured brief assessment tool, the health counselor asked screen positive patients additional assessment questions. Patients who completed this Brief Assessment and answered, "Yes" to 0-2 of the alcohol/drug use items were given the diagnostic impression of Substance Abuse or Use with Consequences. Patients in this group who received no further SBIRT services were categorized as Brief Intervention (BI).

Patients who completed the Brief Assessment and answered, "Yes" to more than 2 of the alcohol/drug items were given the diagnostic impression of Substance Dependence. The Health Counselor asked these patients to participate in a Referral Assessment. The Referral Assessment included additional information necessary to make a level of care recommendation and develop a treatment referral. Substance dependent patients could be referred to Brief Treatment (BT) or Referral to Treatment (RT) depending on the patient's needs and the availability of desired services.

Evaluation Activities

A primary purpose of the data collection and reporting activities conducted through SBIRT was to assess the project's responsiveness to the Center for Substance Abuse Treatment initiative goals to: 1) expand the state's continuum of care to include screening, brief intervention and referral to treatment services in general medical and other community settings; 2) support clinically appropriate treatment services for non-dependent substance abusers; 3) improve linkages between community agencies performing SBIRT and specialist substance abuse treatment agencies; and, 4) identify systems and policy changes to increase access to treatment in generalist and specialist settings. Data collection activities were put into place to monitor progress toward achievement of project patient service category targets, and assessment of outcomes among a sample of patients receiving Brief Intervention (BI), Brief Treatment (BT) or Referral to Treatment (RT) services.

Capacity

Provided in the table below is the screening only, brief intervention, brief treatment and referral to other treatment service data that had been uploaded to the CSAT GPRA website for the ISI as of 04/01/2009. This represents the final patient intake data summary for the project. This data reflects an unduplicated count of patients served. Upon the conclusion of CSAT-funded ISI services, 97,533 unduplicated patients had received SBIRT services through ISI. This represents a final intake coverage rate of 97.0% of the five-year target of 100,500 unduplicated patients. Also at the close of CSAT-funded patient services, the Illinois SBIRT project had provided services to a duplicated count of 120,510 patients.

Illinois SBIRT Intake Coverage Rate by Service Category

Grantee Information

Client

Target

(Final)

Intakes

Received

(Final)

Intake Coverage Rate (Final) Average Intake Coverage Rate of All Grantees in this Portfolio
Screening and Feedback (SF) 85,440 74,457 1% 6%
Brief Intervention (BI) 9,750 15,562 6% 5%
Brief Treatment (BT) 2,620 3,233 4% 0%
Referral to Treatment (RT) 2,690 4,281 1% 4%
Total 100,500 97,533 0% 1%

SBIRT Patient Demographic Characteristics

This Final Summary of Illinois SBIRT Project Data Collection and Reporting Activities provides results of analyses of the 97,538 patient intake interviews downloaded from the CSAT GPRA web site on April 1, 2009. As discussed above, this total number of interviews reflects an unduplicated count of patients and therefore this report's results vary somewhat from interim reports submitted prior to May 2007. As noted above, the GPRA items administered to SBIRT patients varied according to the service categories into which patients were classified. CSAT required that only the GPRA items pertaining to gender, race/ethnicity, and date of birth (section H) be collected for Screening Only/Feedback (SF) patients. Therefore, these are the only data elements reported for the entire group of Illinois SBIRT patients. Following are summary observations from this analysis of patient demographics for all Illinois SBIRT patients, and separately by the four service categories.

  • Slightly less than 50% of Illinois SBIRT patients were female, these patients were on average 47.6 years of age, 65.1% were African American, 10.4% were white, and 19.0% were Hispanic/Latino.
  • The relationship between patient gender and SBIRT service category was statistically significant (?2 = 5206.00, p<.001, df = 9). The percentage of SF patients who were female was significantly greater than that of any of the three other SBIRT service categories.
  • There was a statistically significant relationship between patients reporting themselves to be of Hispanic/Latino ethnicity and SBIRT service category (?2 = 822.40, p<.001, df = 3). The percentage of SF patients who reported themselves to be of Hispanic/Latino ethnicity was significantly greater than that of any of the three other SBIRT service categories.
  • There was a statistically significant difference among the SBIRT groups on the basis of average age (F = 805.00, p<.001, df = 3). The screening only patients had a significantly higher average age than that of any of the other service groups. The BI patients were on average two years (BI vs. BT) to over one year (BI vs. RT) younger than patients in the other non-screening only service groups. The group differences in average age between the BI patient group and the BT and RT patient groups were statistically significant on the basis of both the Scheffe and Least Significant Difference (LSD) post-hoc tests.

Effectiveness

The Lighthouse Institute, using the legal alias of the Illinois Health Survey Laboratory (IHSL) and under the direction of Christy K Scott, Ph.D., was responsible for contacting and conducting six-month follow-up interviews with consenting patients. During the six-month follow-up interview, patients were asked to complete the follow-up version of the CSAT GPRA tool. SBIRT patients first began to be enrolled in the follow-up study in May 2004, and interviews began in October 2004. As of the conclusion of follow-up activities, 1,865 completed Illinois SBIRT patient follow-up GPRA tools were entered onto the CSAT SAIS web site. These 1,865 contacted patients were broken down by the three SBIRT service categories as follow: BI - 1,042 (55.9%); BT - 292 (15.7%); and RT - 531 (28.5%). A total of 2,009 patients came due for follow-up over the five years of the project, plus the nine-month no-cost extension. The 1,865 follow-up interviews completed by IHSL converts to an interim six-month follow-up interview rate of 92.8%. The average six-month follow-up rate of all CSAT SBIRT grantees was 43.8%. The six-month follow-up rates across the three SBIRT patient categories were as follows: BI - 93.4%; BT - 92.7%; and, RT - 91.9%. During their follow-up tracking activities, Lighthouse determined that 123 (5.8%) of the total 2,132 patients enrolled in follow-up were deceased at time of their interviews. These individuals were not included in calculation of the follow-up rate. Some of the partial results for the 1,865 interviews include:

  • There was a significant change in the pattern of living arrangements from baseline to six-month follow-up for the entire follow-up sample, and separately for the BT and RT patient groups. Overall and for each of the patient groups there was a decrease from baseline to six-month follow-up in the percentage of patients who reported being housed.
  • There was a significant change from baseline to follow-up for the entire follow-up sample in the extent to which they report feelings of stress, reduction in important activities, and emotional problems that were considered due to their substance use. In each case, there were substantial increases from baseline to follow-up in the percentage of patients who reported no difficulties that were related to their substance use. This pattern of significant change from baseline to follow-up was evident for both the BT and RT groups for each of the three GPRA items.
  • There was a significant change in employment status from baseline to follow-up for the entire sample and separately for the RT group. For the entire sample and separately for the BT and RT groups there was an increase from baseline to follow-up in the percentage of patients who reported full-time employment. Although these changes were statistically significant, it is questionable whether the changes reflect any practical or clinical significance.
  • There was a significant change from baseline to follow-up in patient self-report of overall health status. The percentage of patients who reported their overall health to be either fair or poor decreased from baseline to follow-up for the entire sample and separately for each of the BT and RT patient groups.
  • At time of six-month follow-up patients reported significant reductions during the past 30 days in average days of use of alcohol, any illegal substance, cocaine, marijuana, and heroin as compared to the 30 days prior to admission. The statistically significant reductions from baseline to follow-up were evident for each of the BI, BT and RT patient groups, with but one exception. There was little change from baseline to six-month follow-up in the average days of marijuana use among the BT patients.
  • There were corresponding increases from baseline to six-month follow-up in the percentages of patients in each group who reported abstinence from alcohol, any illegal substance, cocaine, marijuana, and heroin. The sole exception was again the reported marijuana abstinence rate among BT patients.
  • There was a significant reduction from baseline to follow-up in the average number of crimes committed during the past 30 days for the entire sample. A significant reduction in average crimes committed was observed separately for the BT and RT patient groups.
  • Despite the reduction from baseline to follow-up in average crimes committed, there was a statistically significant increase over this time interval in average days incarcerated during the past 30 days for the entire sample, and separately for both the BT the RT groups. It cannot be determined from available data to what extent the days of incarceration can be attributed to crimes committed prior to participation in SBIRT.
  • There were statistically significant increases from baseline to six-month follow-up in the average number of days during the past 30 that BT patients experienced serious depression, serious anxiety and tension, and trouble concentrating, understanding and remembering. There were statistically significant increases from baseline to six-month follow-up in the average number of days during the past 30 that RT patients experienced serious anxiety and tension. Further study would be needed to explore a possible explanation for these changes.
  • There were statistically significant increases from baseline to six-month follow-up in the average number of days during the past 30 that both BT and RT patients reported use of psychotropic medications. This increase could be attributable to medications prescribed in conjunction with their hospitalization.

Final Summary of Consumer Satisfaction Survey Responses

Patients contacted for six-month follow-up were asked to complete a satisfaction with services tool as part of the interview process. This tool asked about their perceptions of various aspects of the SBIRT service experience. The satisfaction with services tool consisted of eight (8) forced choice items that asked for their degree of agreement with the service-related statements being asked. A total of 1,867 Illinois SBIRT patients completed this satisfaction with services tool as part of their six-month follow-up interviews. This represents 99.2% of the Illinois SBIRT patients who completed six-month follow-up interviews. Below are final summary findings from an analysis of patient responses on the satisfaction with SBIRT services tool.

  • Over 95% of the responding patients rated the services provided by CCHHS health counselors as Excellent or Good.
  • Nearly 95% of the patients stated that they definitely or generally received the kind of services they wanted.
  • Nearly 85% of the responding patients indicated that the SBIRT services provided by the CCHHS health counselors met almost all or most of their needs. This was the item that received the lowest average response score, which could be a reflection of the broad nature of the question's content.
  • Nearly 97% of the responding patients stated that they would recommend the SBIRT services to a friend who needed similar help.
  • Nearly 95% of the patients indicated that they were very or mostly satisfied with the amount of help they received from the SBIRT health counselor,
  • Over 95% of the responding patients stated that the SBIRT services they received helped a great deal or somewhat in dealing with their health issues.
  • Over 95% of the patients stated that they were very or mostly satisfied with the SBIRT services they received from the CCHHS health counselors.
  • Over 97% of the patients indicated that they would definitely or generally want to make use of these SBIRT services if they looked for similar help again in the future.

Screening, Brief Treatment and Referral to Treatment (SBIRT) Evaluation/Performance Assessment Design

Christy K Scott, Ph.D. and Richard E. Sherman, Ph.D., Evaluators

Indicators / Measures Tools / Instruments / Data Sources Other Deliverables / Comments

Process

  • # of Screening Only Patients.
  • # of Brief Intervention Patients.
  • # of Brief Treatment Referral Patients.
  • # of Other Treatment Referral Patients.
  • Characteristics of Patients by Service Categories.

-Enrollment to follow-up changes in patient behavior and functioning.

- Various Forms of Project Documentation.

CSAT GPRA Tool

Project Screening and Assessment Tools

Service Referral Documentation Tools

Patient Satisfaction Tool

The process evaluation includes monitoring progress toward project implementation, achievement of patient enrollment targets, patterns of referral to service providers, fulfillment of project federal reporting requiremennts, and analysis of patient demographics.
Outcome
  • % of Brief Intervention Patients Interviewed
  • % of Brief Treatment Patients Interviewed
  • % of Patients Referred to Other Treatment interviewed
  • Alcohol and Other Drug Abstinence Rates by Service Category and Substance of Use

CSAT GPRA Tool

Patient Satisfaction Tool

Patient interviews

The outcome evaluation consists of conducting six-month post-enrollment folllow-up interviews with a random sample of patients who are provided brief intervention or referral to treatment services. It is an expectation of the federal funder that at least 10% of patients in these service categories are actually interviewed for six-month follow-up.