2009 IDHS Summary of Program Evaluation Finding - Update

November 2009

Dear Colleagues:

We are pleased to distribute the Illinois Department of Human Services (IDHS), Summary of Program Evaluation Findings 2009. As of October 2009, fifteen (15) programs representing $33,554,483 in federal grants funded through the Illinois Department of Human Services (IDHS) for FY2009 had independent evaluations that were ongoing or recently completed. A total of $2,788,830 of the $33,554,483 was used to fund these evaluations.

This booklet represents the fourth report to combine a mixture of detailed reports and summaries of all of the IDHS evaluations into a single document. The fundamental purpose of evaluation is to examine services and/or systems to determine what works most efficiently and effectively. When the findings are fully integrated into the operations of a system or the delivery of services, it results in an overall improvement of services and outcomes for Illinois residents. Some findings presented in this year's report include:

  • In September 2003, the Illinois Department of Human Services was awarded a cooperative agreement by the U.S. Department of Health and Human Services for its Screening, Brief Intervention, Referral, and Treatment (SBIRT) application. All patients who presented at one of the participating Cook County Health and Hospitals or community health clinics were eligible to be screened by a health counselor for substance use and/or abuse. Upon conclusion of this project, a total of 97,533 Illinois residents had received services. Largely due to the evidence of demonstrating the cost-effectiveness of providing screening and interventions for substance use in the context of general healthcare that was gained during this project, the Cook County Health and Hospitals Board of Directors approved integration of the SBIRT activities and program personnel into the permanent staff of the health system. The Cook County Commissioners also recommended that SBIRT become a permanent component of their health care delivery system. In February 2009, the Cook County Commissioners and the Cook County President approved the overall budget that included the establishment of ten positions in their 2009 budget.

    IDHS, DASA Contract Manager: Deborah Levi, Ph.D.

    Evaluator: Richard Sherman, Ph.D. and Christy K. Scott, Ph.D

  • During the final phase of the evaluation of the Illinois Adolescent Substance Abuse Treatment Coordination Initiative, one of the criticisms heard most often was parents' disappointment with the lack of information in schools about adolescent substance abuse treatment. In response, the Division of Substance Abuse partnered with the Illinois Federation of Families to write and produce "The Parent Handbook." This publication guides parents through the adolescent treatment process. "The Parent Handbook" which was published at the end of 2008 (in English and Spanish), has been distributed to parents and schools and is posted on the Illinois Treatment Works website.

    IDHS, DASA Contract Manager: Danielle Kirby

    Evaluator: Maria Bruni, Ph.D.

  • In September 2007, the Division of Alcoholism and Substance Abuse was awarded a federally funded Targeted Capacity Expansion/HIV grants. The purpose of this project is to expand and enhance the evidence-based outpatient methadone treatment services that are available to African-American and Hispanic/Latino injecting males, including men who have sex with other men (MSN), who are residents of the near north and west side of the City of Chicago community areas that have high rates of HIV infection and AIDS. A total of 98 follow-up interviews were completed as of July 9, 2009. A significant interim finding reports only four of the clients were employed at admission, but at six-month follow-up, 17 reported being employed part-time.

    IDHS, DASA Contract Manager: Carolyn Hartfield

    Evaluator: Richard Sherman, Ph.D.

  • The Division of Mental Health's McHenry County Family CARE emphasizes the use of the System of Care values and principles to empower youth and families as well as to enhance system-wide collaboration. A key component of the system of care is the formation of Child and Family Teams using traditional and non-traditional supports in care planning. Families participate in the development of their plans of care, learn to manage their care, resources, and establish desired outcomes for their families. Parents of youth with serious emotional disturbances, are employed throughout the community as Family Resource Developers, Child and Family Team Facilitators, data collectors, and peer supports to family members in need.

    During the past year, Family CARE partners and stakeholders participated in an assessment of the cultural and linguistic competence of the system of care. Respondents to the surveys included senior management of five (5) agencies participating in the system of care, consumers, volunteers, and agency staff. The assessment highlighted key strengths and challenges in the ways in which organizations have to date embraced the concept of cultural and linguistic competence.

    Greatest strengths:

    • Organizations have instituted policies against discrimination and harassment.
    • Organizations actively enforce policies against discrimination and harassment.
    • Board members and volunteers are interested in and supportive of cultural diversity.

    Greatest challenges:

    • Need for multi-cultural training across the system of care.
    • Need for orientation to the concept of cultural diversity as well as ongoing educational opportunities that reinforce basic learning about issues of cultural diversity and competence.
    • Need to recognize and embrace the role of natural healers within various cultures.

    Key recommendations:

    • Develop, implement, and evaluate long term comprehensive professional development training for all staff and board members in issues related to cultural and linguistic competency.
    • Offer conversational Spanish to all staff and board members as soon as possible.
    • The recruitment, hiring, and retention of qualified minority staff should become an agency wide priority.
    • Increase minority participation on the Board of Directors to provide balance at the governance level given the diversity of the clientele being served.
    • Extend Professional Development curriculum beyond African American and/or Hispanic cultural awareness to include knowledge of other ethnic groups, social classes, sexual orientations, and disabilities.
    • Determine if there are benefits to greater involvement of clients, staff, and volunteers into the agency decision-making process.
  • Several system of care committees are working to ensure that strengths identified in the cultural competence needs assessment are reinforced and challenges are addressed. In particular, efforts are being made to ensure that through training and other forms of experiential learning, system of care providers can deliver more culturally and linguistically competent services that meet the needs of the culturally diverse population that they serve.

    State Project Director: Todd P.Schroll, MSW

    Evaluator: Mary Spooner Ph.D.
    Mental Health Services and Policy Program of Northwestern University

  • The State of Illinois was one of six states awarded the Substance Abuse and Mental Health Services Administration Jail Diversion - Trauma Recovery grant. This grant for approximately $2 million over five years has enabled the Division of Mental Health to establish the Illinois Veterans Reintegration Initiative to increase diversion for criminal justice-involved veterans with trauma histories in Cook and Rock Island counties.

    Veterans in the criminal justice system with mental illnesses and combat-related trauma disorders represent a growing population with unique service needs. The significant number of returning veterans to Illinois also underscores the importance of adapting current training and treatment strategies to meet the needs of these soldiers and their families.

    Without these services, veterans with mental health disorders or co-morbid substance abuse may lack the supports necessary to achieve successful reintegration, and usually find themselves caught in a cycle of homelessness, hospitalization, and incarceration. In accordance with the federal requirement, services under this grant could not be provided until the Strategic Plan was approved by the U.S. Department of Health and Human Services, Substance Abuse Mental Health Services Administration. Illinois was the first state to submit its Strategic Plan for review and to receive approval for the plan.

    IDHS, DMH Contract Manager: Debra Ferguson, Ph.D.

    Evaluator: Sue Pickett, Ph.D, Associate Professor
    University of Illinois-Chicago Department of Psychiatry
    Center for Mental Health Services Research and Policy

  • The fundamental purpose of the Division of Community Health and Prevention's Illinois Steps AHEAD (ISA ) program is to increase the number of low-income students who attend and succeed in college. The Illinois Steps AHEAD program works with students in middle and high school to improve their achievement, build study skills, assist in course selection, and increase their knowledge of admission requirements for college education. Illinois Steps AHEAD provided services to 1,914 youth in FY08 with 1,270 participating in activities on an ongoing and regular basis throughout the year. The core components of tutoring, mentoring, and college advisement were delivered by participating providers. Students received an average of 32.1 hours of tutoring, an average of 10.6 hours of mentoring, and an average of 13.6 hours of counseling/advising throughout the year. Key outcome findings to date include:
    1. When youth were asked in the 2009 survey about their academic performance and study habits since participating in the program, 50% said their grades in school this year were better than last year.
    2. More than 85% of all youth reported numerous positive outcomes from their participation in the program, including: ability to set goals, complete homework more often, improved academic skills, and stated they feel more prepared for college.
    3. Ninety one (91%) percent of youth in 2009 'strongly agree' or 'agree' that since attending the program, they are more interested in going to college.
    4. Additionally, 85% of the youth 'strongly agree' or 'agree' that since attending the program they are more prepared to attend college.

    IDHS, CHP Contract Manager: Karrie Reuter

    Evaluator: Peter Mulhall, PhD.
    Center for Prevention Research and Development (CPRD), and University of Illinois at Urbana-Champaign;

  • The Division of Community Health and Prevention's Teen Parent Family Services (TPFS) project is a five year federally-funded demonstration project. The project is designed to help teen mothers who are enrolled in Teen Parent Services with the necessary supports to develop and strengthen family unity. These supports include connecting each family member to community resources and involving them in a variety of programs, customized to their own interests and needs typically focusing on education, employment and personal development. A statistically significant finding related to financial resources and self-sufficiency was the teen mothers in the intervention group use of public benefits decreased over time, and was lower than the comparison group's use of public benefits, which increased. This may be related to the greater number of teen mothers in the comparison group who had a subsequent birth (48%) compared to the teen mothers in the project group that did not report any subsequent births at all (0%).

    IDHS, CHP Contract Manager: Denise Simon

    Evaluators: Peter Mulhall, Ph.D. and Erin Ficker M.P.A.

    Center for Prevention Research and Development
    University of Illinois at Urbana-Champaign

  • Between January 2008 and July 2009, The Division of Alcoholism and Substance Abuse's Access to Recovery II (ATR-2) project provided services to 3,258 Illinois residents. Six month follow-up interviews were completed on 1,825 clients. The six month follow-up rate for this project was 91.4%. This is significant because the current follow up rate for ATR II grantees is 62.9%. Results from an interim analysis of these clients include:
    1. Statistically significant increases from enrollment to six month follow up of clients who reported being enrolled in training part time (from 3.5% at enrollment to 8.6% at follow-up; and of clients who reported being enrolled in training full time (from 3.9% at enrollment to 15.6% at follow-up).
    2. Statistically significant increase in reported average income from wages among the interviewed clients from enrollment ($143.62) to six-month follow-up ($296.34).
    3. Statistically significant increase from enrollment to six month follow-up of clients who self-reported their health status as good or excellent.
    4. Statistically significant increase from enrollment to six-month follow-up in abstinence rates from alcohol, and illegal drugs including cocaine, marijuana, methamphetamine, and heroin. There were statistically significant decreases from enrollment to six month follow-up among the interviewed clients in average days of reported depression, serious anxiety, hallucinations, trouble concentrating or remembering, and trouble controlling violent behavior.

    IDHS, DASA Contract Manager: Rex Alexander

    Evaluator: Richard Sherman, Ph.D.

This report produced by the IDHS Office of Grants Administration presents each evaluation in place or completed by the end of June 2009. Sources for the information in this report range from continuation applications to full evaluation reports provided by IDHS staff and their evaluators.

The Illinois Department of Human Services envisions that this booklet, Summary of Program Evaluation Findings 2009 will:

  • Assist state agencies and the human services field by providing state of the art information necessary to improve human services and systems;
  • Fulfill a key requirement for federal grants. This report is a mechanism to disseminate evaluation findings;
  • Assist IDHS staff in the development of evaluations, a key requirement of all federal grant applications;
  • Demonstrate to funding entities the extent of IDHS's experience with program evaluations; and,
  • Provide a vehicle for Illinois evaluators to share information.

The following individuals are to be acknowledged for their diligence and hard work in developing, editing, and providing the information that is presented in this report.

  • IDHS staff: Rex Alexander; Danielle Kirby; Richard Sherman, Ph.D.; Deborah Levi Ph.D.; Peggy Alexander; Kim Fornero; Denise Simon; Carolyn Hartfield; Karrie Reuter; Debra Ferguson, Ph.D.; Gloster Mahon; Tanya R. Anderson, M.D.; Philip Matute; Denise Eligan; and Sharon Zahorodnyj.
  • Evaluators: Maria Bruni, Ph.D.; Beth-Anne Jacob, Ph.D.; Richard E. Sherman Ph.D.; Peter Mulhall, Ph.D.; Nancy Flowers; Beth Welbes M.S.P.H.; John Lyons, Ph.D.; Mary Spooner, Ph.D.; Christy K. Scott, Ph.D; Erin Ficker M.P.A.; Julie Spielberger, Ph.D., and Sue Pickett, Ph.D.

Sincerely,

Michelle R.B. Saddler
Secretary
Illinois Department of Human Services

FY09 Grants Evaluation Report (pdf)

Table of Contents