Early Intervention
Illinois Annual Performance Report

Part C State Annual Performance Report for (FFY07)
(OMB NO: 1820-0578 / Expiration Date: 12/31/2009)

Family Outcomes


Part C State Annual Performance Report (APR) for FFY 07/SFY 08

Overview of the Annual Performance Report Development:

The Illinois APR documents performance data on State targets for each Child and Family Connections (CFC) office and documents CFC and state progress or slippage toward measurable and rigorous targets. The Illinois Early Intervention (EI) Program makes the Illinois APR and State Performance Plan (SPP) available on its web site and through links from the other EI web sites (the Illinois Early Intervention Training Program; Provider Connections, the Early Intervention credentialing office; and the Early Childhood Intervention Clearinghouse). The APR and SPP documents are also available to the public at each of the 25 CFC offices.

The APR is part an ongoing process of performance measurement and strategic planning for the Illinois Early Intervention Program. For a number of years, Illinois has been reporting performance data to key stakeholders including the Illinois Interagency Council on Early Intervention (IICEI), the Child and Family Connections (CFC) offices, and the general public through various reporting mechanisms. Work groups of the IICEI have been formed to address specific indicators (Service Delay Work Group and Transition Work Group) and overall operation issues (Finance Work Group). These work groups have helped analyze data and suggest improvement strategies.

The Illinois and Texas Early Intervention Programs received funding through an IDEA General Supervision Enhancement Grant (GSEG) for a joint project to develop and pilot a family outcomes survey and to complete analysis of the results. The EI Bureau has provided updates on the project to the Illinois Interagency Council on Early Intervention (IICEI) and other interested bodies since before the grant was received. Illinois and Texas were coordinating their efforts on a family outcomes survey even before requesting grant funding.

As the only GSEG project focused exclusively on the measurement of family outcomes, Illinois and Texas volunteered to be one of the projects the Early Childhood Outcomes Center (ECO) worked with closely. This allowed Illinois to utilize their expertise, most notably that of Dr. Don Bailey, then of the University of North Carolina Frank Porter Graham Center. The survey Illinois has developed with their help is now more commonly known as the ECO tool. Dr. Bailey is now working at RTI. RTI is one of two major contractors on the project, along with the University of Illinois.

In addition to a formal project advisory group process, focus group reviews of the English version of the tool were carried out, with the help of the Early Childhood Outcomes (ECO) Center in FFY 05/SFY 06. That provided feedback on the tool itself and aspects of survey administration. ECO has also incorporated input from stakeholders nationally. The Illinois-Texas-ECO tool is being used in whole or in part by about one-third of states. During FFY 06/SFY 07 RTI conducted focus groups on the Spanish version of the tool with Illinois and Texas families who predominately speak Spanish.

During FFY 07/SFY 08, Illinois distributed a total of 3,135 surveys. As in the past, the goal was to distribute surveys primarily by hand at six-month reviews and annual IFSP renewal meetings. Some were sent by mail from CFC offices when meetings had already been held or otherwise could not be distributed by hand. However, in the first two years of survey administration there were major discrepancies in return rates between CFCs. Texas return rate have consistently been more representative than Illinois' in both years.

In an attempt to overcome that problem 1,000 of the surveys sent to the Chicago area CFCs were mailed from the central project office. This did have a positive impact on return rates in several areas and, as will be discussed in more detail later, may be a contributing factor to lower average ratings. Given the extra work necessary to do hand-delivery and the poor results outside of downstate, all FFY08/SFY09 surveys will be mailed.

In our CFC determination scorecard the two agencies with the lowest return rates receive a mark against them, unless their return rate is at least 50% of the state average. That will still be in place for FFY07/SFY08 performance, based just on hand-delivered surveys, but that will no longer be a factor once all surveys are mailed.

Monitoring Priority: Early Intervention Services In Natural Environments

Indicator 4: Percent of families participating in Part C who report that early intervention services have helped the family:

  1. Know their rights;
  2. Effectively communicate their children's needs; and
  3. Help their children develop and learn.

(20 USC 1416(a)(3)(A) and 1442)

Measurement:

  1. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family know their rights divided by the # of respondent families participating in Part C times 100.
  2. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family effectively communicate their children's needs divided by the # of respondent families participating in Part C times 100.
  3. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family help their children develop and learn divided by the # of respondent families participating in Part C times 100.

FFY: 2007(7/1/07-6/30/08)

Measurable and Rigorous Target:

  1. 76% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
  2. 86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
  3. 90% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

Actual Target Data for FFY 07/SFY 08

  1. 75.0% of respondent families participating in Part C will report that early intervention services have helped the family know their rights [(631/842) x 100]=75.0%
  2. 87.1% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs [(735/843) x 100]=87.1%
  3. 90.8% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn [(766/843) x 100]=90.8%

Actual Target Data for FFY 07/SFY 08:

Survey results have been weighted to correct for geographic disparities in returns. Although, the resulting changes in the results were minor for all three measures.

Weighted Survey Results:  Formula: [(Responses 5 or Higher/Returns) x 100] Percent 5 or Higher

To what extent has early intervention helped your family know and understand your rights? [(631/842) x 100] 75.0%

To what extent has early intervention helped your family effectively communicate your child's needs? [(735/843) x 100] 87.1%

To what extent has early intervention helped your family be able to help your child develop and learn? [(766/843) x 100] 90.8%

Weighted Survey Results: Formula: [(Responses 5 or Higher/Returns) x 100] Percent 5 or Higher
To what extent has early intervention helped your family know and understand your rights? [(631/842) x 100] 75.0%
To what extent has early intervention helped your family effectively communicate your child's needs? [(735/843) x 100] 87.1%
To what extent has early intervention helped your family be able to help your child develop and learn? [(766/843) x 100] 90.8%

Mainly as a result of the decision to incorporate some mail survey sent directly from the central project office to Chicago and suburban areas, returns were far more representative than they were in the first two years. The overall response rates were actually about the same, at just under 27% both last year and for this report year. A more representative sample meant a significantly smaller adjustment to the results. In fact, the adjustments to the percentage of families responding 5, 6, or 7 were just 0.05%, 0.07% and 0.09% respectively.

The relative differences seen in previous years remained the same. The highest percentage of families responding with a five or better was still in regards to helping their child develop and learn, 90.82%, and the lowest was in knowing and understanding their rights, 74.94%. Chicago families continued to report

the lowest levels of satisfaction and downstate the highest. However, with just one exception, the average ratings were lower on every measure for every region. The exception was for the proportion of families downstate that reported positively on their ability to help their children develop and learn.

The fall was relatively small downstate compared to the three Chicago area CFC regions. It is probably not a coincidence that there was no central mailing to downstate CFC offices but there was to the three Chicago CFC area geographic areas. The reasons for this will be discussed in more detail under the explanations for progress or slippage. We had already decided to move to an all-mail survey because it was less administratively burdensome and because the assumed return rate benefits of direct hand-off had not materialize, at least for Chicago area CFC offices. However, we must conclude that a mail sample is actually more representative of the caseload, even if there was no notable difference in the demographics of those who returned the surveys in the past and this reporting year.

This was an unexpected finding and it raises a number of questions the program will explore in the coming year. However, it also probably means that moving to an all-mail survey is the right thing to do but it also renders the assumed baseline data as questionable at best.

To what extent has early intervention helped your family know and understand your rights?


Chicago
Suburban
Cook
Collar
Counties

Downstate
Illinois
Total
Raw Totals
Distributed 1,012 663 830 630 3,135
Returns 232 147 223 240 842
Return Rate 22.92% 22.17% 26.87% 38.10% 26.86%
Responses 5 or Higher 160 104 163 204 631
% 5 or Higher 68.97% 70.75% 73.09% 85.00% 74.94%
Average Response 4.90 5.22 5.16 5.74 5.26
Statewide Return % 25.36% 16.48% 25.83% 32.33% 100%
Totals Adjusted for Geography
Avg. IFSP in Period 4,713 3,764 4,793 5,348 18,618
Caseload % 25.31% 20.22% 25.74% 28.72% 100%
Adjusted Returns 213.15 170.23 216.76 241.86 842
Responses 5 or Higher 147.00 120.43 158.44 205.58 631
% 5 or Higher 68.97% 70.75% 73.09% 85.00% 74.99%
Average Response 4.90 5.22 5.16 5.74 5.27

To what extent has early intervention helped your family effectively communicate your child's needs?


Chicago
Suburban
Cook
Collar
Counties

Downstate
Illinois
Total
Raw Totals
Distributed 1,012 663 830 630 3,135
Returns 231 149 223 240 843
Return Rate 22.83% 22.47% 26.87% 38.10% 26.89%
Responses 5 or Higher 196 129 187 222 734
% 5 or Higher 84.85% 86.58% 83.86% 92.50% 87.07%
Average Response 5.45 5.65 5.64 6.05 5.71
Statewide Return % 27.40% 17.67% 26.45% 28.47% 100%
Totals Adjusted for Geography
Avg. IFSP in Period 4,713 3,764 4,793 5,348 18,618
Caseload % 25.31% 20.22% 25.74% 28.72% 100%
Adjusted Returns 213.40 170.43 217.02 242.15 843
Responses 5 or Higher 181.07 147.55 181.99 223.99 735
% 5 or Higher 84.85% 86.58% 83.86% 92.50% 87.14%
Average Response 5.45 5.65 5.64 6.05 5.71

The one specific area of concern that has come out of both responses and focus groups is the difference between Spanish and English speakers, and to a lesser extent Hispanic families who indicate their primary language is English. The proportion of Hispanic families returning surveys for FFY07/SFY08 was again below the state average and the responses given on almost all the survey questions, not just the specific three OSEP questions, were notably lower on the Spanish version than on the English version.

On the positive side, the proportion of Blacks and Asians responding more closely mirrored their proportions in the caseload for the first time. Although they still were underrepresented in the sample. Since the vast majority of both of those groups lives in and around Chicago, it is logical to assume mailing helped with those groups but not with Hispanics.

FFY 07/SFY 08 Family Outcome Survey ResultsReturn Rates & Unweighted Results by CFC

CFC Know Rights Communicate Child Needs Help Child Develop & Learn
Surveys Returns Return Rate Scores 5 or > Average Score Scores 5 or > Average Score Scores 5 or > Average Score
1 - ROCKFORD 53 17 32.1% 87.5% 5.63 75.0% 5.56 94.1% 6.18
2 - LAKE COUNTY 141 40 28.4% 67.5% 4.83 77.5% 5.35 85.0% 5.53
3 - FREEPORT 23 8 34.8% 50.0% 4.63 75.0% 5.63 100.0% 6.38
4 - KANE-KENDALL 137 36 26.3% 72.2% 4.81 77.8% 5.44 91.4% 5.60
5 - DUPAGE CO. 244 73 29.9% 77.8% 5.53 87.5% 5.81 91.8% 6.05
6 - NORTH COOK 310 68 21.9% 69.1% 5.04 86.8% 5.74 91.2% 6.09
7 - WEST COOK 147 28 19.0% 81.5% 5.74 85.7% 5.68 84.6% 6.08
8 - SW CHICAGO 131 31 23.7% 73.3% 5.30 80.0% 5.33 80.6% 5.42
9 - CENTRAL CHICAGO 177 47 26.6% 67.4% 4.74 77.8% 5.18 82.6% 5.43
10 - SE CHICAGO 171 51 29.8% 72.9% 5.10 85.4% 5.46 85.4% 5.58
11 - NORTH CHICAGO 533 109 20.5% 66.7% 4.77 88.9% 5.60 91.7% 5.97
12 - SOUTH COOK 206 54 26.2% 67.3% 5.17 86.8% 5.53 90.4% 6.08
13 - MACOMB 63 23 36.5% 73.9% 5.39 87.0% 5.83 95.7% 6.17
14 - PEORIA 92 41 44.6% 95.0% 6.13 95.0% 6.18 97.4% 6.33
15 - JOLIET 215 64 29.8% 71.4% 5.03 84.1% 5.63 87.3% 5.83
16 - E. CENTRAL ILLINOIS 84 26 31.0% 88.0% 5.56 100.0% 6.28 100.0% 6.40
17 - QUINCY 22 5 22.7% 40.0% 5.00 100.0% 6.20 100.0% 6.25
18 - SPRINGFIELD 35 13 37.1% 92.3% 6.38 92.3% 6.46 92.3% 6.54
19 - DECATUR 44 20 45.5% 90.0% 5.75 95.0% 5.85 95.0% 6.05
20 - EFFINGHAM 62 24 38.7% 75.0% 5.79 95.8% 6.50 100.0% 6.54
21 - METRO E. ST. LOUIS 54 33 61.1% 87.9% 5.67 90.9% 5.94 93.9% 6.09
22 - CENTRALIA 56 24 42.9% 87.0% 5.70 95.7% 5.87 95.7% 6.35
23 - NORRIS CITY 23 7 30.4% 100.0% 6.43 100.0% 6.29 100.0% 6.71
24 - CARBONDALE 19 3 15.8% 100.0% 6.00 100.0% 6.00 100.0% 6.33
25 - MCHENRY COUNTY 93 12 12.9% 75.0% 5.75 100.0% 6.17 83.3% 6.08
STATEWIDE 3,135 857 27.3% 74.9% 5.26 87.1% 5.71 90.7% 5.97

Discussion of Improvement Activities Completed and Explanation of Progress or Slippage that occurred for FFY 06/SFY 07:

For the report year, families were selected in the same way as they were in the past, based on their child having a six-month review or annual IFSP coming due during a given span of time. The majority of surveys distributed during FFY07/SFY08 were distributed in the same way but 1,000 of those in the Chicago area, both urban and suburban, were distributed by mail in an attempt to overcome return rate problems. We believe that change produced a significant and unexpected change in survey results.

The percentage of respondents that indicated a positive response (a score of 5 or better) exceeded Illinois' target percentages for being able to communicate child needs and for being able to help their child develop and learn but fell short for knowing and understanding their rights.

Of greater concern is that the number of families giving positive responses, 5 or greater on the 7-point scale, fell for all three questions. Downstate responses remained high and changed little from the first two years but Chicago area responses fell sharply. Most factors would affect the entire state but one change only affected the Chicago area, the move to mail out some of the surveys to address return rate problems. We did not anticipate that eliminating Service Coordinators hand-off would change the nature of who responded or the responses we received but that is the logical conclusion, particularly since downstate acts as a kind of control group and changed little.

Naturally, the program did not knowingly take steps that resulted in lesser outcomes for families. However, there were several changes in the program environment that help explain the declines:

  • The nation's financial situation had not worsened at the time of the survey but the mood was not good and the state's financial situation was weaker. This could explain some of the uncertainty and lack of satisfaction reported by families.
  • Service delay problems, explained in more detail under Indicator 1, were growing through FFY07/SFY08. The areas with the lowest average responses also happen to be the areas with the greatest service delay problems, north and west of Chicago.
  • Given the greater decline in average ratings in the areas where the program tested mailing surveys from the program centrally, it is logical to assume that mailing produces different responders, more comfort in giving lower responses, or both. Focus groups preferred that families receive surveys directly from Service Coordinators as a means to increase return rates but that assumption has not proven to be correct and it is administratively burdensome. So, the program had already decided to move to an all-mail survey for FFY08/SFY09. However, it now appears that the hand-off method may also skew results positively. There are several possible explanations for this but if a mail survey produces lower ratings than a hand-off version we have to anticipate our FFY08/SFY09 results will be lower again, due to the complete conversion to central mail distribution. This also may mean that mail results will not be comparable to the direct hand-off results from the first two years of surveying.

Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 08/SFY 09 - FFY 10/SFY 11 [If applicable]

The proportion of families reporting that the EI program had a positive impact on them fell in all three areas for FFY 07/SFY 08. We believe economic factors, including increased service delays, contributed to some of this decline but we believe the move to mail some surveys, instead of hand delivering them, was a bigger factor. Assuming mailing surveys will produce less positive results, we must anticipate another decline for FFY08/SFY09 before we can affect positive change. As a result, targets are being changed as follows:

2008 (2008-2009) 

  • 74% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
  • 86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
  • 89% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

2009 (2009-2010)

  • 75% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
  • 87% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
  • 90% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

2010 (2010-2011)

  • 75% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
  • 88% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
  • 91% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

The following new improvement activity will be undertaken:

  • The program is creating a new Program Integrity Project to supplements its existing records review based monitoring system. The project supports conformity with the spirit of Early Intervention rules, laws and philosophy. The Program Integrity Project will include monitoring and recommendations, when needed, on local practice as it relates to the three family outcomes.