DHS Contract Mgr Tanya R. Anderson, MD
Annual Grant $ (varies each grant year) $1,989,977 in Federal FY 09
Evaluator Name & Contact Info Mary Spooner Ph.D.
Mental Health Services and Policy Program
Northwestern University
Annual Eval $ $350,000 in Federal FY 09
Funding Source SAMHSA/CMHS
Eval Period 12/01/05 through 9/29/11

The IDHS/Division of Mental Health is currently operating one System of Care grant funded by the United States Department of Health and Human Services, Substance Abuse and Mental Health Services Administration/Center for Mental Health Services. The McHenry County Family CARE cooperative agreement, a Children's Mental Health Initiative, was awarded to McHenry County in December 2005.

Project Summary for the System of Care McHenry County

The Illinois Department of Human Services, Division of Mental Health, in collaboration with service providers, youth and families in McHenry County, are developing the Child/Adolescent Recovery Experience (CARE) system of care for the county's youth with serious emotional disturbances. The overarching goal of this project is to transform McHenry County's system of care for serious emotionally disturbed youth and their families through system-wide strategic collaboration and implementation of family-driven and youth-guided services. IDHS/DMH has partnered with the McHenry County Mental Health Board to implement system of care transformation, on a local level. The mission of this project "is to meet the social and/or emotional needs of families, children, and youth by providing leadership to develop and sustain a community of care that provides continuous support and easy access at every level of care."

McHenry County Family CARE emphasizes the use of System of Care values and principles to empower youth and families as well as to enhance system-wide collaboration. Youth and families are empowered and engaged through the formation of Child and Family Teams using traditional and non-traditional supports in care planning. Families participate in the development of their plans of care; learn to manage their care, resources, and to establish desired outcomes. Parents of youth with serious emotional disturbances, are employed throughout the community as Family Resource Developers, Child and Family Team Facilitators, data collectors, and peer supports to family members in need.

Parents are empowered, participating in the development of the system of care in many different capacities. Parents serve as Family Resource Developers and School Sector Coordinators. NAMI-McHenry County, Illinois Federation of Families and Families ETC. have teamed up to develop a McHenry County Family Center, increasing the number of education, training and support activities available to youth. Using the Targeted Parenting Assistance framework (adapted from KEYS for Networking, Kansas), parents are engaged in system development activities consistent with their personal development goals. Parents make up 51% of the Governance Council and are active members of the Local Evaluation Team, Transition Age Youth Workgroup and Early Childhood Workgroup. McHenry County Family CARE has also developed an active Family Council and Youth Council representing youth voice in the system of care. Both councils engage in social marketing, evaluation, governance and service planning activities on behalf of youth in McHenry County.

The Governance Council of Family CARE meets on a monthly and quarterly basis with workgroups focused on collaboration, budget, evaluation and sustainability planning. Over the last year, the Governance Council has promoted additional system of care activities through the use of partnership agreements. As Family CARE moves forward, significant effort is being placed on enhancing school/mental health partnerships, developing a trauma-informed system of care, collaborative cross-system training and professional development, the sustainability of local family organizations, and cultural competency in organizational planning and implementation. Based on the year four SAMHSA site visit, priority activities include:

  • Strengthening the role of the Governance Council;
  • Broadening opportunities for stakeholder input;
  • Updating the strategic/sustainability and cultural competency plans with increased stakeholder participation;
  • Strengthening local family organizations;
  • Increasing enrollment into the National Evaluation;
  • Developing local research questions and using findings to guide service response;
  • Developing a youth engagement plan and dedicated budget.

Evaluation Activities for System of Care McHenry

The evaluation contract was transferred to Northwestern University (NU) as of July 15, 2008. As a result of the new agreement, project management re-defined the system of care more broadly, to include youth receiving outpatient and community services and not only the youth enrolled in the Wraparound and SASS programs. This strategy made it possible to extend the national study to more youth and families. The Northwestern Evaluation team also implemented a strategy to collect data that remained outstanding from the beginning of the study.

To date, the evaluation team has successfully collected enrollment demographic data for 544 youth and assessment data for 117 youth enrolled into the national longitudinal study. The evaluator worked with program managers and community partners to develop logic models and the played a key role in providing guidance in use of the logic models for outcomes management purposes. Data are now being used routinely as an integral part of the decision making process. Additional effort is now being placed on engaging youth and families in the evaluation of the system of care. Youth are being engaged to participate in data collection and together with caregivers are being engaged in quality assurance processes. In particular, parents participate in monthly program meetings where program managers share data from various program serving youth and families. The evaluation team is presently in the process of meeting federal requests for development of research questions, guidelines, and methodologies for the remaining two years of the cooperative agreement.

The evaluation team presently comprises the lead evaluator, a data coordinator, and five part-time parent interviewers. The evaluator leads the local evaluation team meeting in conjunction with the project director and support from Families ETC. As part of the quality improvement effort, arrangements have been made for ongoing analysis of CANS data by Northwestern University. This will enable programs to monitor youth outcomes more consistently and incorporate the findings into the decision making and continuous quality improvement processes.

The evaluation team and project management completed a cultural competency self-assessment process with six local mental health organizations in May 2009. Additionally, the McHenry County Wraparound program has initiated a sub-study utilizing the Wraparound Fidelity Index, WFI, to assess fidelity with the National Wraparound Initiative model. Furthermore, the findings of the study will help to assess the extent to which youth and families who receive Wraparound services are effectively engaged in and guide the process and are able to achieve the outcomes established as part of Wraparound planning goals.

Preliminary Evaluation Findings for the System of Care McHenry

The National Evaluation provides CQI data on a quarterly basis. Data are extracted from interviews collected at baseline, 6 months, 12 months, 18 months, and 24 months after the initial baseline interviews. These data are used to gain an understanding of the experiences of youth and families.

Preliminary analyses of data related to families indicate that families are experiencing significant stress in caring for children with serious emotional disorders. Data from a sample of caregivers enrolled in the national evaluation indicate that caregivers experience significant subjective internalized strain, i.e., stress related to personal sadness, worry, and fatigue. The findings suggest an association between family strain and socio-structural factors such as socio-economic status, gender, family composition, and living arrangements, and issues such as the presence of family histories of depression, substance abuse, and mental illnesses. Findings from the continuous quality improvement data show positive improvement in service accessibility and service quality. Noticeable positive change was also evident in school attendance, stability of living arrangements, hospitalization rates, and emotional and behavioral problems. The evaluation team continues to mine the national longitudinal study data and to use program data, where necessary, to contribute to the continuous quality improvement process.

During the past year, Family CARE partners and stakeholders participated in an assessment of the cultural and linguistic competence of the system of care. Respondents to the surveys included senior management of five (5) agencies participating in the system of care, consumers, volunteers, and agency staff. The assessment highlighted key strengths and challenges in the ways in which organizations have to date embraced the concept of cultural and linguistic competence.

  • Greatest strengths:
    • Organizations have instituted policies against discrimination and harassment.
    • Organizations actively enforce policies against discrimination and harassment
    • Board members and volunteers are interested in and supportive of cultural diversity
  • Greatest challenges:
    • Need for multi-cultural training across the system of care.
    • Need for orientation to the concept of cultural diversity as well as ongoing educational opportunities that reinforce basics of cultural diversity and competence.
    • Need to recognize and embrace the role of natural healers within various cultures.
  • Key recommendations:
    • Develop, implement, and evaluate long term comprehensive professional development training for all staff and board members in issues related to cultural and linguistic competency.
    • Offer conversational Spanish to all staff and board members as soon as possible.
    • The recruitment, hiring, and retention of qualified minority staff should become an agency wide priority.
    • Increase minority participation on the Board of Directors to provide balance at the governance level given the diversity of the clientele being served.
    • Extend professional Development curriculum beyond African American and/or Hispanic cultural awareness to include knowledge of other ethnic groups, social classes, sexual orientations, and abilities.
    • Determine if there are benefits to greater involvement of clients, staff, and volunteers into the agency decision-making process.

Several system of care committees are working to ensure that strengths identified in the cultural competence needs assessment are reinforced and challenges are addressed. In particular, efforts are being made to ensure that through training and other forms of experiential learning, system of care providers can deliver more culturally and linguistically competent services that meet the needs of the culturally diverse population that they serve.

System of Care - McHenry County

Evaluation Design Mary Spooner Ph.D.

Indicators / Measures Tools / Instruments / Data Sources Other Deliverables / Comments

Process:

  • Level of incorporation of System of Care principles in service planning, delivery, and governance
  • System of care compliance with required grant components, such as reports, audits and match funding
  • Collaboration between system partners in service delivery, planning, finance, and governance
  • Utilization of evaluation findings to refine and improve the quality, timeliness, accessibility, and efficiency of services
  • Youth and family engagement in all aspects of system of care planning, service delivery, outcomes evaluation
  • Cultural and linguistic competency of services delivered at all levels of the system of care

Outcome:

  • Improved functioning of youth in academic, social, family, and economic domains
  • Improved family status
  • Decrease in numbers institutionalized
  • Decrease in service costs
  • Decrease in child welfare and juvenile justice involvement of system of care youth
  • Increase in transitional youth employment
  • Inreased access to mental health care
  • Increased access to parent and professional training and supports
  • Increase in cross-system collaboration to facilitate planning, service delivery, financing and evaluation 
  • National evaluation (i.e., ORC Macro) protocol
  • Child and Adolescent Needs and Strengths (CANS) assessment tool
  • Local evaluation team meetings
  • Youth and family satisfaction with services
  • Local evaluation surveys to collaborating partners
  • Service data from partner agencies
  • Focus groups
  • CLC needs assessment
  • Wraparound fidelity assessment
  • System of care strategic plan  
Participation in national system of care conferences and trainings Publication of research findings and sharing of system of care outcomes at the local and national levels