Early Intervention
Illinois Annual Performance Report
Part C State Annual Performance Report for (FFY07)
(OMB NO: 1820-0578 / Expiration Date: 12/31/2009)
Family Outcomes
Part C State Annual Performance Report (APR) for FFY 07/SFY 08
Overview of the Annual Performance Report Development:
The Illinois APR documents performance data on State targets for each Child and Family Connections (CFC) office and documents CFC and state progress or slippage toward measurable and rigorous targets. The Illinois Early Intervention (EI) Program makes
the Illinois APR and State Performance Plan (SPP) available on its web site and through links from the other EI web sites (the Illinois Early Intervention Training Program; Provider Connections, the Early Intervention credentialing office; and the Early
Childhood Intervention Clearinghouse). The APR and SPP documents are also available to the public at each of the 25 CFC offices.
The APR is part an ongoing process of performance measurement and strategic planning for the Illinois Early Intervention Program. For a number of years, Illinois has been reporting performance data to key stakeholders including the Illinois
Interagency Council on Early Intervention (IICEI), the Child and Family Connections (CFC) offices, and the general public through various reporting mechanisms. Work groups of the IICEI have been formed to address specific indicators (Service Delay Work
Group and Transition Work Group) and overall operation issues (Finance Work Group). These work groups have helped analyze data and suggest improvement strategies.
The Illinois and Texas Early Intervention Programs received funding through an IDEA General Supervision Enhancement Grant (GSEG) for a joint project to develop and pilot a family outcomes survey and to complete analysis of the results. The EI Bureau
has provided updates on the project to the Illinois Interagency Council on Early Intervention (IICEI) and other interested bodies since before the grant was received. Illinois and Texas were coordinating their efforts on a family outcomes survey even
before requesting grant funding.
As the only GSEG project focused exclusively on the measurement of family outcomes, Illinois and Texas volunteered to be one of the projects the Early Childhood Outcomes Center (ECO) worked with closely. This allowed Illinois to utilize their
expertise, most notably that of Dr. Don Bailey, then of the University of North Carolina Frank Porter Graham Center. The survey Illinois has developed with their help is now more commonly known as the ECO tool. Dr. Bailey is now working at RTI. RTI is
one of two major contractors on the project, along with the University of Illinois.
In addition to a formal project advisory group process, focus group reviews of the English version of the tool were carried out, with the help of the Early Childhood Outcomes (ECO) Center in FFY 05/SFY 06. That provided feedback on the tool itself and
aspects of survey administration. ECO has also incorporated input from stakeholders nationally. The Illinois-Texas-ECO tool is being used in whole or in part by about one-third of states. During FFY 06/SFY 07 RTI conducted focus groups on the Spanish
version of the tool with Illinois and Texas families who predominately speak Spanish.
During FFY 07/SFY 08, Illinois distributed a total of 3,135 surveys. As in the past, the goal was to distribute surveys primarily by hand at six-month reviews and annual IFSP renewal meetings. Some were sent by mail from CFC offices when meetings had
already been held or otherwise could not be distributed by hand. However, in the first two years of survey administration there were major discrepancies in return rates between CFCs. Texas return rate have consistently been more representative than
Illinois' in both years.
In an attempt to overcome that problem 1,000 of the surveys sent to the Chicago area CFCs were mailed from the central project office. This did have a positive impact on return rates in several areas and, as will be discussed in more detail later, may
be a contributing factor to lower average ratings. Given the extra work necessary to do hand-delivery and the poor results outside of downstate, all FFY08/SFY09 surveys will be mailed.
In our CFC determination scorecard the two agencies with the lowest return rates receive a mark against them, unless their return rate is at least 50% of the state average. That will still be in place for FFY07/SFY08 performance, based just on
hand-delivered surveys, but that will no longer be a factor once all surveys are mailed.
Monitoring Priority: Early Intervention Services In Natural Environments
Indicator 4: Percent of families participating in Part C who report that early intervention services have helped the family:
- Know their rights;
- Effectively communicate their children's needs; and
- Help their children develop and learn.
(20 USC 1416(a)(3)(A) and 1442)
Measurement:
- Percent = # of respondent families participating in Part C who report that early intervention services have helped the family know their rights divided by the # of respondent families participating in Part C times 100.
- Percent = # of respondent families participating in Part C who report that early intervention services have helped the family effectively communicate their children's needs divided by the # of respondent families participating in Part C times
100.
- Percent = # of respondent families participating in Part C who report that early intervention services have helped the family help their children develop and learn divided by the # of respondent families participating in Part C times 100.
FFY: 2007(7/1/07-6/30/08)
Measurable and Rigorous Target:
- 76% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
- 86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
- 90% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn
Actual Target Data for FFY 07/SFY 08
- 75.0% of respondent families participating in Part C will report that early intervention services have helped the family know their rights [(631/842) x 100]=75.0%
- 87.1% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs [(735/843) x 100]=87.1%
- 90.8% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn [(766/843) x 100]=90.8%
Actual Target Data for FFY 07/SFY 08:
Survey results have been weighted to correct for geographic disparities in returns. Although, the resulting changes in the results were minor for all three measures.
Weighted Survey Results: Formula: [(Responses 5 or Higher/Returns) x 100] Percent 5 or Higher
To what extent has early intervention helped your family know and understand your rights? [(631/842) x 100] 75.0%
To what extent has early intervention helped your family effectively communicate your child's needs? [(735/843) x 100] 87.1%
To what extent has early intervention helped your family be able to help your child develop and learn? [(766/843) x 100] 90.8%
| Weighted Survey Results: |
Formula: [(Responses 5 or Higher/Returns) x 100] |
Percent 5 or Higher |
| To what extent has early intervention helped your family know and understand your rights? |
[(631/842) x 100] |
75.0% |
| To what extent has early intervention helped your family effectively communicate your child's needs? |
[(735/843) x 100] |
87.1% |
| To what extent has early intervention helped your family be able to help your child develop and learn? |
[(766/843) x 100] |
90.8% |
Mainly as a result of the decision to incorporate some mail survey sent directly from the central project office to Chicago and suburban areas, returns were far more representative than they were in the first two years. The overall response rates were
actually about the same, at just under 27% both last year and for this report year. A more representative sample meant a significantly smaller adjustment to the results. In fact, the adjustments to the percentage of families responding 5, 6, or 7 were
just 0.05%, 0.07% and 0.09% respectively.
The relative differences seen in previous years remained the same. The highest percentage of families responding with a five or better was still in regards to helping their child develop and learn, 90.82%, and the lowest was in knowing and
understanding their rights, 74.94%. Chicago families continued to report
the lowest levels of satisfaction and downstate the highest. However, with just one exception, the average ratings were lower on every measure for every region. The exception was for the proportion of families downstate that reported positively on
their ability to help their children develop and learn.
The fall was relatively small downstate compared to the three Chicago area CFC regions. It is probably not a coincidence that there was no central mailing to downstate CFC offices but there was to the three Chicago CFC area geographic areas. The
reasons for this will be discussed in more detail under the explanations for progress or slippage. We had already decided to move to an all-mail survey because it was less administratively burdensome and because the assumed return rate benefits of direct
hand-off had not materialize, at least for Chicago area CFC offices. However, we must conclude that a mail sample is actually more representative of the caseload, even if there was no notable difference in the demographics of those who returned the
surveys in the past and this reporting year.
This was an unexpected finding and it raises a number of questions the program will explore in the coming year. However, it also probably means that moving to an all-mail survey is the right thing to do but it also renders the assumed baseline data as
questionable at best.
To what extent has early intervention helped your family know and understand your rights?
|
Chicago |
Suburban
Cook |
Collar
Counties |
Downstate |
Illinois
Total |
| Raw Totals |
| Distributed |
1,012 |
663 |
830 |
630 |
3,135 |
| Returns |
232 |
147 |
223 |
240 |
842 |
| Return Rate |
22.92% |
22.17% |
26.87% |
38.10% |
26.86% |
| Responses 5 or Higher |
160 |
104 |
163 |
204 |
631 |
| % 5 or Higher |
68.97% |
70.75% |
73.09% |
85.00% |
74.94% |
| Average Response |
4.90 |
5.22 |
5.16 |
5.74 |
5.26 |
| Statewide Return % |
25.36% |
16.48% |
25.83% |
32.33% |
100% |
| Totals Adjusted for Geography |
| Avg. IFSP in Period |
4,713 |
3,764 |
4,793 |
5,348 |
18,618 |
| Caseload % |
25.31% |
20.22% |
25.74% |
28.72% |
100% |
| Adjusted Returns |
213.15 |
170.23 |
216.76 |
241.86 |
842 |
| Responses 5 or Higher |
147.00 |
120.43 |
158.44 |
205.58 |
631 |
| % 5 or Higher |
68.97% |
70.75% |
73.09% |
85.00% |
74.99% |
| Average Response |
4.90 |
5.22 |
5.16 |
5.74 |
5.27 |
To what extent has early intervention helped your family effectively communicate your child's needs?
|
Chicago |
Suburban
Cook |
Collar
Counties |
Downstate |
Illinois
Total |
| Raw Totals |
| Distributed |
1,012 |
663 |
830 |
630 |
3,135 |
| Returns |
231 |
149 |
223 |
240 |
843 |
| Return Rate |
22.83% |
22.47% |
26.87% |
38.10% |
26.89% |
| Responses 5 or Higher |
196 |
129 |
187 |
222 |
734 |
| % 5 or Higher |
84.85% |
86.58% |
83.86% |
92.50% |
87.07% |
| Average Response |
5.45 |
5.65 |
5.64 |
6.05 |
5.71 |
| Statewide Return % |
27.40% |
17.67% |
26.45% |
28.47% |
100% |
| Totals Adjusted for Geography |
| Avg. IFSP in Period |
4,713 |
3,764 |
4,793 |
5,348 |
18,618 |
| Caseload % |
25.31% |
20.22% |
25.74% |
28.72% |
100% |
| Adjusted Returns |
213.40 |
170.43 |
217.02 |
242.15 |
843 |
| Responses 5 or Higher |
181.07 |
147.55 |
181.99 |
223.99 |
735 |
| % 5 or Higher |
84.85% |
86.58% |
83.86% |
92.50% |
87.14% |
| Average Response |
5.45 |
5.65 |
5.64 |
6.05 |
5.71 |
The one specific area of concern that has come out of both responses and focus groups is the difference between Spanish and English speakers, and to a lesser extent Hispanic families who indicate their primary language is English. The proportion of
Hispanic families returning surveys for FFY07/SFY08 was again below the state average and the responses given on almost all the survey questions, not just the specific three OSEP questions, were notably lower on the Spanish version than on the English
version.
On the positive side, the proportion of Blacks and Asians responding more closely mirrored their proportions in the caseload for the first time. Although they still were underrepresented in the sample. Since the vast majority of both of those groups
lives in and around Chicago, it is logical to assume mailing helped with those groups but not with Hispanics.
FFY 07/SFY 08 Family Outcome Survey ResultsReturn Rates & Unweighted Results by CFC
| CFC |
|
|
|
Know Rights |
Communicate Child Needs |
Help Child Develop & Learn |
|
Surveys |
Returns |
Return Rate |
Scores 5 or > |
Average Score |
Scores 5 or > |
Average Score |
Scores 5 or > |
Average Score |
| 1 - ROCKFORD |
53 |
17 |
32.1% |
87.5% |
5.63 |
75.0% |
5.56 |
94.1% |
6.18 |
| 2 - LAKE COUNTY |
141 |
40 |
28.4% |
67.5% |
4.83 |
77.5% |
5.35 |
85.0% |
5.53 |
| 3 - FREEPORT |
23 |
8 |
34.8% |
50.0% |
4.63 |
75.0% |
5.63 |
100.0% |
6.38 |
| 4 - KANE-KENDALL |
137 |
36 |
26.3% |
72.2% |
4.81 |
77.8% |
5.44 |
91.4% |
5.60 |
| 5 - DUPAGE CO. |
244 |
73 |
29.9% |
77.8% |
5.53 |
87.5% |
5.81 |
91.8% |
6.05 |
| 6 - NORTH COOK |
310 |
68 |
21.9% |
69.1% |
5.04 |
86.8% |
5.74 |
91.2% |
6.09 |
| 7 - WEST COOK |
147 |
28 |
19.0% |
81.5% |
5.74 |
85.7% |
5.68 |
84.6% |
6.08 |
| 8 - SW CHICAGO |
131 |
31 |
23.7% |
73.3% |
5.30 |
80.0% |
5.33 |
80.6% |
5.42 |
| 9 - CENTRAL CHICAGO |
177 |
47 |
26.6% |
67.4% |
4.74 |
77.8% |
5.18 |
82.6% |
5.43 |
| 10 - SE CHICAGO |
171 |
51 |
29.8% |
72.9% |
5.10 |
85.4% |
5.46 |
85.4% |
5.58 |
| 11 - NORTH CHICAGO |
533 |
109 |
20.5% |
66.7% |
4.77 |
88.9% |
5.60 |
91.7% |
5.97 |
| 12 - SOUTH COOK |
206 |
54 |
26.2% |
67.3% |
5.17 |
86.8% |
5.53 |
90.4% |
6.08 |
| 13 - MACOMB |
63 |
23 |
36.5% |
73.9% |
5.39 |
87.0% |
5.83 |
95.7% |
6.17 |
| 14 - PEORIA |
92 |
41 |
44.6% |
95.0% |
6.13 |
95.0% |
6.18 |
97.4% |
6.33 |
| 15 - JOLIET |
215 |
64 |
29.8% |
71.4% |
5.03 |
84.1% |
5.63 |
87.3% |
5.83 |
| 16 - E. CENTRAL ILLINOIS |
84 |
26 |
31.0% |
88.0% |
5.56 |
100.0% |
6.28 |
100.0% |
6.40 |
| 17 - QUINCY |
22 |
5 |
22.7% |
40.0% |
5.00 |
100.0% |
6.20 |
100.0% |
6.25 |
| 18 - SPRINGFIELD |
35 |
13 |
37.1% |
92.3% |
6.38 |
92.3% |
6.46 |
92.3% |
6.54 |
| 19 - DECATUR |
44 |
20 |
45.5% |
90.0% |
5.75 |
95.0% |
5.85 |
95.0% |
6.05 |
| 20 - EFFINGHAM |
62 |
24 |
38.7% |
75.0% |
5.79 |
95.8% |
6.50 |
100.0% |
6.54 |
| 21 - METRO E. ST. LOUIS |
54 |
33 |
61.1% |
87.9% |
5.67 |
90.9% |
5.94 |
93.9% |
6.09 |
| 22 - CENTRALIA |
56 |
24 |
42.9% |
87.0% |
5.70 |
95.7% |
5.87 |
95.7% |
6.35 |
| 23 - NORRIS CITY |
23 |
7 |
30.4% |
100.0% |
6.43 |
100.0% |
6.29 |
100.0% |
6.71 |
| 24 - CARBONDALE |
19 |
3 |
15.8% |
100.0% |
6.00 |
100.0% |
6.00 |
100.0% |
6.33 |
| 25 - MCHENRY COUNTY |
93 |
12 |
12.9% |
75.0% |
5.75 |
100.0% |
6.17 |
83.3% |
6.08 |
| STATEWIDE |
3,135 |
857 |
27.3% |
74.9% |
5.26 |
87.1% |
5.71 |
90.7% |
5.97 |
Discussion of Improvement Activities Completed and Explanation of Progress or Slippage that occurred for FFY 06/SFY 07:
For the report year, families were selected in the same way as they were in the past, based on their child having a six-month review or annual IFSP coming due during a given span of time. The majority of surveys distributed during FFY07/SFY08 were
distributed in the same way but 1,000 of those in the Chicago area, both urban and suburban, were distributed by mail in an attempt to overcome return rate problems. We believe that change produced a significant and unexpected change in survey
results.
The percentage of respondents that indicated a positive response (a score of 5 or better) exceeded Illinois' target percentages for being able to communicate child needs and for being able to help their child develop and learn but fell short for
knowing and understanding their rights.
Of greater concern is that the number of families giving positive responses, 5 or greater on the 7-point scale, fell for all three questions. Downstate responses remained high and changed little from the first two years but Chicago area responses fell
sharply. Most factors would affect the entire state but one change only affected the Chicago area, the move to mail out some of the surveys to address return rate problems. We did not anticipate that eliminating Service Coordinators hand-off would change
the nature of who responded or the responses we received but that is the logical conclusion, particularly since downstate acts as a kind of control group and changed little.
Naturally, the program did not knowingly take steps that resulted in lesser outcomes for families. However, there were several changes in the program environment that help explain the declines:
- The nation's financial situation had not worsened at the time of the survey but the mood was not good and the state's financial situation was weaker. This could explain some of the uncertainty and lack of satisfaction reported by families.
- Service delay problems, explained in more detail under Indicator 1, were growing through FFY07/SFY08. The areas with the lowest average responses also happen to be the areas with the greatest service delay problems, north and west of Chicago.
- Given the greater decline in average ratings in the areas where the program tested mailing surveys from the program centrally, it is logical to assume that mailing produces different responders, more comfort in giving lower responses, or both. Focus
groups preferred that families receive surveys directly from Service Coordinators as a means to increase return rates but that assumption has not proven to be correct and it is administratively burdensome. So, the program had already decided to move to
an all-mail survey for FFY08/SFY09. However, it now appears that the hand-off method may also skew results positively. There are several possible explanations for this but if a mail survey produces lower ratings than a hand-off version we have to
anticipate our FFY08/SFY09 results will be lower again, due to the complete conversion to central mail distribution. This also may mean that mail results will not be comparable to the direct hand-off results from the first two years of
surveying.
Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 08/SFY 09 - FFY 10/SFY 11 [If applicable]
The proportion of families reporting that the EI program had a positive impact on them fell in all three areas for FFY 07/SFY 08. We believe economic factors, including increased service delays, contributed to some of this decline but we believe the
move to mail some surveys, instead of hand delivering them, was a bigger factor. Assuming mailing surveys will produce less positive results, we must anticipate another decline for FFY08/SFY09 before we can affect positive change. As a result, targets
are being changed as follows:
2008 (2008-2009)
- 74% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
- 86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
- 89% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn
2009 (2009-2010)
- 75% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
- 87% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
- 90% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn
2010 (2010-2011)
- 75% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
- 88% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
- 91% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn
The following new improvement activity will be undertaken:
- The program is creating a new Program Integrity Project to supplements its existing records review based monitoring system. The project supports conformity with the spirit of Early Intervention rules, laws and philosophy. The Program Integrity
Project will include monitoring and recommendations, when needed, on local practice as it relates to the three family outcomes.
