Part C State Annual Performance Report (APR) for FFY 06/SFY 07

Overview of the Annual Performance Report Development:

The Illinois and Texas Early Intervention Programs received funding through an IDEA General Supervision Enhancement Grant (GSEG) for a joint project to develop and pilot a family outcomes survey and to complete analysis of the results. The EI Bureau has provided updates on the project to the Illinois Interagency Council on Early Intervention (IICEI) and other interested bodies since before the grant was received. Illinois and Texas were coordinating their efforts on a family outcomes survey even before requesting grant funding.

Each state has formed an advisory council to provide input to the project. The Illinois advisory council includes representation from parents, service providers, and CFC offices, along with a researcher and a developmental pediatrician. Each state meets periodically with their advisory group and occasionally the groups meet together via conference call. With the release of the instructions for the SPP, responsibilities and membership of the Illinois advisory council for the family outcomes project was expanded to include the discussion of a process for measuring child outcomes.

As the only GSEG project focused exclusively on the measurement of family outcomes, Illinois and Texas volunteered to be one of the states the Early Childhood Outcomes Center (ECO) would work more closely with. This allowed us to utilize their expertise, most notably that of Dr. Don Bailey, then of the University of North Carolina Frank Porter Graham Center. The survey we have developed with their help is now more commonly known as the ECO tool. Dr. Bailey is now working at RTI. RTI is one of two major contractors on the project, along with the University of Illinois.

In addition to the formal grant advisory group process, focus group reviews of the English version of the tool were carried out, with the help of the Early Childhood Outcomes (ECO) Center in FFY 05/SFY 06. That provided feedback on the tool itself and aspects of survey administration. ECO has also incorporated input from stakeholders nationally. The Illinois-Texas-ECO tool is being used in whole or in part by about one-third of states. During FFY 06/SFY 07 RTI conducted focus groups on the Spanish version of the tool with Illinois and Texas families who predominately speak Spanish.

During FFY 07/SFY 08, the project will consider how to incorporate this feedback into the tool and into survey administration. We also will consider if it is advisable to make changes to the survey based on what we have learned from the project so far, possibly including shortening its current 21questions for the Illinois-Texas. Other than the collection of new surveys for FFY 07/SFY 08, the primary focus of the project for the year will be assessing it reliability and validity.

During FFY 06/SFY 07, Illinois distributed a total of 6,288 surveys, primarily by hand at six-month reviews and annual IFSP renewal meetings. Some were distributed by mail when meetings had already been held or otherwise could not be distributed by hand. Approximately 2,500 were distributed during January and February. The remaining surveys were distributed in May and June. The second wave included an additional one-page questionnaire of 25 questions drawn from the National Center for Special Education Accountability Monitoring (NCSEAM) list. Responses to the NCSEAM questions will be used compare with ECO responses on like questions.

During FFY 05/SFY 06, the Texas return rate was more representative than Illinois'. Illinois had employed a relatively hands-off approach to avoid skewing results. Texas took several steps to improve return rates. For our FFY 06/SFY 07 surveys, Illinois took an approach more like Texas had used the year before. For both waves Illinois tracked the cumulative return rates and communicated that to service coordination agencies (CFC). In our CFC determination scorecard the two agencies with the lowest return rates receive a mark against them, unless their return rate is at least 50% of the state average. The areas with the lowest return rates also tend to be areas with larger black and Hispanic populations. So, return rates also were not racially and ethnically representative.

To help overcome these problems Illinois allowed CFCs to mail out surveys that were not hand delivered. Some CFCs made special trips to families to deliver surveys. It was emphasized that they should not help a family complete the survey or wait for them to complete it. That would create too much chance that responses would be skewed. Finally, UCP of Greater Chicago, which coordinates the survey process, called minority households to try to improve the response rate among underrepresented groups. With these efforts, our return rates and representative-ness did improve. Additional efforts will be undertaken to get a more representative sample in FFY 07/SFY 08.

Monitoring Priority: Early Intervention Services In Natural Environments

Indicator 4: Percent of families participating in Part C who report that early intervention services have helped the family

  1. Know their rights;
  2. Effectively communicate their children's needs; and
  3. Help their children develop and learn.

(20 USC 1416(a)(3)(A) and 1442)

Measurement

  1. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family know their rights divided by the # of respondent families participating in Part C times 100.
  2. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family effectively communicate their children's needs divided by the # of respondent families participating in Part C times 100.
  3. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family help their children develop and learn divided by the # of respondent families participating in Part C times 100.

FFY: 2006 (2006-2007) (SFY 07)

Measurable and Rigorous Target:

  • Surveying will take place between February and June 2006
  • 76% of respondent families participating in Part C will report that early intervention services have helped the family know their rights
  • 86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs
  • 90% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn
Actual Target Data for FFY 06/SFY 07:
Survey: Formula: [(Responses 5 or Higher/Returns) x 100] Percent 5 or Higher
To what extent has early intervention helped your family know and understand your rights? [(1,356/1,653) x 100] 82.0%
To what extent has early intervention helped your family effectively communicate your child's needs? [(1,495/1,656) x 100] 90.3%
To what extent has early intervention helped your family be able to help your child develop and learn? [(1,547/1,658) x 100] 93.3%

The proportion of families who responded that EI had helped them increased in all three areas and so did average scores. Responses improved in most of the geographic areas as well. The return rate for suburban Cook County fell but the proportion of those who returned their surveys giving ratings of 5 or higher and their average scores increased more than for any of the other geographic areas.

As was the case with the baseline data, weighting the raw responses for geography lowered the ratings for all three questions but the differences were only about half of what they were for the baseline data, reflecting the improved representativeness of the sample.

The greatest difference between the baseline and the report period was for knowing and understanding their rights and the lowest was for being able to help their children develop and learn. However, the scores for the rights question was still the lowest at 80.5% positive (weighted) and the highest was for "the develop and learn" question at 92.8% (weighted).

To what extent has early intervention helped your family know and understand your rights?
Raw Totals
Raw Totals Chicago Suburban
Cook
Collar
Counties
Downstate Illinois
Total
Distributed 1,743 1,305 1,377 1,803 6,228
Returns 370 252 333 698 1,653
Return Rate 21.2% 19.3% 24.2% 38.7% 26.5%
Responses 5 or Higher 283 195 271 607 1,356
% 5 or Higher 76.5% 77.4% 81.4% 87.0% 82.0%
Average Response 5.29 5.34 5.51 5.90 5.60
Statewide Return % 22.4% 15.2% 20.1% 42.2% 100.0%
To what extent has early intervention helped your family know and understand your rights?
Totals Adjusted for Geography
Totals Adjusted for Geography Chicago Suburban
Cook
Collar
Counties
Downstate Illinois
Total
Avg. IFSP in Period 4,539 3,704 4,539 4,782 17,564
Caseload % 25.8% 21.1% 25.8% 27.2% 100.0%
Adjusted Returns 427.17 321.55 427.19 477.09 1,653
Responses 5 or Higher 327 249 344 411 1,331
% 5 or Higher 76.5% 77.4% 80.5% 86.2% 80.5%
Average Response 5.28 5.34 5.48 5.87 5.51
To what extent has early intervention helped your family effectively communicate your child's needs?
Totals Adjusted for Geography
Raw Totals Chicago Suburban
Cook
Collar
Counties
Downstate Illinois
Total
Distributed 1,743 1,305 1,377 1,803 6,228
Returns 371 253 335 697 1,656
Return Rate 21.3% 19.4% 24.3% 38.7% 26.6%
Responses 5 or Higher 318 223 305 649 1,495
% 5 or Higher 85.7% 88.1% 91.0% 93.1% 90.3%
Average Response 5.70 5.77 5.86 6.10 5.91
Statewide Return % 22.4% 15.3% 20.2% 42.1% 100.0%
To what extent has early intervention helped your family effectively communicate your child's needs?
Totals Adjusted for Geography
Raw Totals Chicago Suburban
Cook
Collar
Counties
Downstate Illinois
Total
Avg. IFSP in Period 4,539 3,704 4,539 4,782 17,564
Caseload % 25.8% 21.1% 25.8% 27.2% 100.0%
Adjusted Returns 427.94 322.14 427.96 477.95 1,656
Responses 5 or Higher 367 282 387 444 1,480
% 5 or Higher 85.7% 87.7% 90.4% 92.9% 89.4%
Average Response 5.70 5.76 5.84 6.09 5.86
To what extent has early intervention helped your family? Be able to help your child develop and learn?
Raw Totals
Raw Totals Chicago Suburban
Cook
Collar
Counties
Downstate Illinois
Total
Distributed 1,743 1,305 1,377 1,803 6,228
Returns 374 252 334 698 1,658
Return Rate 21.5% 19.3% 24.3% 38.7% 26.6%
Responses 5 or Higher 336 239 315 657 1,547
% 5 or Higher 89.8% 94.8% 94.3% 94.1% 93.3%
Average Response 5.95 6.13 6.11 6.26 6.14
Statewide Return % 22.6% 15.2% 20.1% 42.1% 100.0%
To what extent has early intervention helped your family? Be able to help your child develop and learn?
Totals Adjusted for Geography
Totals Adjusted for Geography Chicago Suburban
Cook
Collar
Counties
Downstate Illinois
Total
Avg. IFSP in Period 4,539 3,704 4,539 4,782 17,564
Caseload % 25.8% 21.1% 25.8% 27.2% 100.0%
Adjusted Returns 428.46 322.53 428.48 478.53 1,658
Responses 5 or Higher 384 304 402 448 1,538
% 5 or Higher 89.6% 94.4% 93.8% 93.6% 92.8%
Average Response 5.94 6.09 6.08 6.25 6.10

The one specific area of concern that has come out of both responses and focus groups is the difference between Spanish and English speakers, and to a lesser extent Hispanic families who indicate their primary language is English. In spite of extra efforts to reach Hispanics and Blacks, the proportion of Hispanic families returning surveys was still below the state average and the responses given on almost all the survey questions, not just the specific three OSEP questions, were notably lower on the Spanish version than on the English version. The proportion of Spanish speaking surveys returned was 19.3%, compared to 18.9% for all Hispanics and 21.2% for African-Americans.

FFY 06/SFY 07 Family Outcome Survey ResultsReturn Rates & Unweighted Results by CFC
CFC Surveys Returns Return
Rate
Know Rights
Scores
5 or >
Know Rights
Average
Score
Communicating
Child Needs
Scores
5 or >
Communicating
Child Needs
Average
Score
Help Development & Learn
Scores
5 or >
Help Development & Learn
Average
Score
#1 - ROCKFORD 143 52 36.4% 86.54% 5.98 96.15% 6.27 94.23% 6.29
#2 - LAKE COUNTY 221 48 21.7% 79.17% 5.48 87.50% 5.60 93.75% 5.90
#3 - FREEPORT 136 45 33.1% 73.33% 5.62 86.67% 5.98 91.11% 6.07
#4 - KANE-KENDALL 228 48 21.1% 65.96% 4.83 79.17% 5.27 85.42% 5.69
#5 - DUPAGE COUNTY 396 88 22.2% 85.06% 5.77 96.55% 6.18 95.35% 6.20
#6 - N. SUBURBS 602 119 19.8% 77.31% 5.28 92.44% 5.87 98.32% 6.36
#7 - W. SUBURBS 261 56 21.5% 77.78% 5.37 85.45% 5.62 90.91% 5.82
#8 - S.W. CHICAGO 209 44 21.1% 78.57% 5.17 85.71% 5.76 86.36% 5.77
#9 - CENTRAL CHICAGO 296 89 30.1% 73.86% 5.02 83.15% 5.27 85.39% 5.49
#10 - S.E. CHICAGO 361 81 22.4% 82.50% 5.44 83.75% 5.59 87.50% 5.79
#11 - N. CHICAGO 877 161 18.4% 74.38% 5.40 88.13% 5.97 94.41% 6.34
#12 - S. SUBURBS 307 79 25.7% 77.22% 5.41 83.54% 5.73 92.31% 5.99
#13 - MACOMB 176 33 18.8% 90.63% 5.84 93.75% 6.22 93.94% 6.33
#14 - PEORIA 227 95 41.9% 85.26% 5.84 95.79% 6.05 95.79% 6.15
#15 - JOLIET 415 135 32.5% 84.21% 5.54 92.54% 5.91 96.27% 6.24
#16 - BLOOMINGTON 240 62 25.8% 79.03% 5.47 90.16% 5.82 93.55% 6.05
#17 - QUINCY 72 22 30.6% 86.36% 5.82 95.45% 6.05 81.82% 6.14
#18 - SPRINGFIELD 123 59 48.0% 89.83% 6.07 93.22% 6.27 94.92% 6.51
#19 - DECATUR 166 81 48.8% 96.30% 6.20 95.06% 6.33 96.25% 6.41
#20 - EFFINGHAM 221 78 35.3% 93.51% 6.14 93.51% 6.19 97.40% 6.27
#21 - METRO E. ST. LOUIS 186 69 37.1% 85.51% 5.88 89.86% 6.01 92.75% 6.42
#22 - CENTRALIA 136 57 41.9% 85.96% 5.77 92.98% 5.88 96.49% 6.21
#23 - NORRIS CITY 66 27 40.9% 88.89% 6.15 92.59% 6.26 88.89% 6.30
#24 - CARBONDALE 46 20 43.5% 80.00% 5.55 95.00% 5.90 90.00% 5.85
#25 - MCHENRY 117 18 15.4% 88.89% 5.83 94.44% 6.22 100.00% 6.50
STATEWIDE 6,228 1,666 26.8% 82.03% 5.60 90.28% 5.91 93.31% 6.14

Discussion of Improvement Activities Completed and Explanation of Progress or Slippage that occurred for FFY 06/SFY 07:

The percentage of respondents that indicated a positive response (a score of 5 or better) exceeded Illinois' target percentages in all three survey items. There were several differences between the baseline data collection in FFY 05/SFY 06 and collection for the report period. The most important difference was that far more surveys were sent out and far more were returned. In FFY 05/SFY 06, the highest number of responses for any of the questions was 401 in the baseline. The lowest of the three in FY 06/SFY 07 was 1,653, more than four times as many. The program made more efforts to monitor and promote returns and a special follow-up effort was undertaken to reach underrepresented minorities. This did result in higher return rates and produced a more representative sample.

The overall return rate improved from 20.8% to 26.6%. The special efforts resulted in the greatest improvement in Chicago, where the return rate increased to 21.2%, from 11.8% in the pilot. All Chicago CFCs are predominately black or Hispanic. It also improved some in the collar counties and more substantially downstate but the return rate was lower in suburban Cook County. Weighting for geography (CFC) brings the racial distribution much more in balance. It decreases the influence of the mainly white downstate areas and increases the more heavily minority Chicago areas.

Another important difference between the baseline and report period surveys is more likely to have influenced the results. The baseline survey did not include children within three-months of turning three. This was because pre-tests found that rather than hold meetings at that point a significant portion of cases were simply extended, as allowed by rule. However, this situation changed for the report period. The program now requires exit/transition conferences of the team to include evaluation of child outcomes. So, for the report period families of children near age three were surveyed. It is logical to assume that longer exposure to the program will increase the chances that it will result in positive outcomes for families.

Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 07/SFY 08 - FFY 10/SFY 11 [If applicable]

The proportion of families reporting that the EI program had a positive impact on them in all three areas increased and exceeded the goals for FFY 06/SFY 07 and for FFY 07/SFY 08 for all three areas. However, the program does not choose to change its goals at this time. There is not yet enough evidence to support making changes. However, the following improvement activity will be added:

  • The program will work with the Illinois Interagency Council on Early Intervention, the Minority Outreach Strategies group and CFCs to identify and implement ways to be more responsive to the needs of both Spanish and English Speaking Hispanic families.