Indicator 3: Percent of infants and toddlers with IFSPs who demonstrate improved

Overview of the State Performance Plan Development:

See Indicator 1 for a description of this process.

On April 1, 2006, the Illinois Early Intervention program began utilizing the Early Childhood Outcomes (ECO) Center child outcomes summary form (COSF) statewide for all initial and renewal IFSPs as well as for exit conferences, to be held within 90 days of a child turning three. A few entries were made prior to that time in a pilot phase and are included in this analysis. The Illinois Child Outcomes Rating Scale and Summary form is attached. This form includes instructions for its use. Illinois also follows the ECO Center guidance that scores of six or seven on its seven-point scale be considered demonstrating development equivalent to same age peers.

Monitoring Priority Early Intervention Services In Natural Environments
Indicator 3 Percent of infants and toddlers with IFSPs who demonstrate improved:
  1. A. Positive social-emotional skills (including social relationships);
  2. Acquisition and use of knowledge and skills (including early language/communication); and 
  3. Use of appropriate behaviors to meet their needs. (20 U.S.C. 1416(a)(3)(A) and 1442)

Measurement

Outcomes:

  1. Positive social-emotional skills (including social relationships);
  2. Acquisition and use of knowledge and skills (including early language/communication); and 
  3. Use of appropriate behaviors to meet their needs.
 Progress categories for A, B and C:
  1. Percent of infants and toddlers who did not improve functioning = [(# of infants and toddlers who did not improve functioning) divided by (# of infants and toddlers with IFSPs assessed)] times 100.
  2. Percent of infants and toddlers who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers = [(# of infants and toddlers who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers) divided by (# of infants and toddlers with IFSPs assessed)] times 100.
  3. Percent of infants and toddlers who improved functioning to a level nearer to same-aged peers but did not reach it = [(# of infants and toddlers who improved functioning to a level nearer to same-aged peers but did not reach it) divided by (# of infants and toddlers with IFSPs assessed)] times 100.
  4. Percent of infants and toddlers who improved functioning to reach a level comparable to same-aged peers = [(# of infants and toddlers who improved functioning to reach a level comparable to same-aged peers) divided by (# of infants and toddlers with IFSPs assessed)] times 100.
  5. Percent of infants and toddlers who maintained functioning at a level comparable to same-aged peers = [(# of infants and toddlers who maintained functioning at a level comparable to same-aged peers) divided by (# of infants and toddlers with IFSPs assessed)] times 100.
Summary Statement #1

Summary Statement 1: Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turn 3 years of age or exited the program.

Measurement for Summary Statement 1: Percent = #of infants and toddlers reported in progress category (c) plus # of infants and toddlers reported in category (d) divided by [# of infants and toddlers reported in progress category (a) plus # of infants and toddlers reported in progress category (b) plus # of infants and toddlers reported in progress category (c) plus # of infants and toddlers reported in progress category (d) times 100.

Summary Statement #2

Summary Statement 2: The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turn 3 years of age or exited the program.

Measurement for Statement 2: Percent = # of infants and toddlers reported in progress category (d) plus [# of infants and toddlers reported in progress category (e) divided by the total # of infants and toddlers reported in progress categories (a) + (b) + (c) + (e)] times 100.

Overview of Issue/Description of System or Progress:

An infant or toddler's developmental status upon entry to the Part C Early Intervention Services System is determined via a comprehensive assessment using multiple sources of information, including one or more approved assessment instruments, a review of pertinent records, clinical observation, and parent interview. The assessment process for eligible children yields valuable information that is used in the development of child outcomes. Progress toward the outcomes identified on the Individualized Family Service Plan (IFSP) is measured annually or more frequently if necessary, by the IFSP team through the use of approved assessment instruments, clinical observation and/or parent interview. Strategies, services and supports are created and/or modified as necessary to best meet the child and family's changing needs.

Measurement of progress toward the three child outcomes identified by OSEP has been incorporated into Illinois' system of progress measurement. IFSP teams continue to have a variety of formal assessment instruments available to them, including both curriculum based and norm-referenced instruments. In order to summarize the outcome data in a format consistent with OSEP's measurement criteria, IFSP teams complete a Child Outcomes Summary Form (COSF) as outlined by the ECO Center. The COSF has been integrated into Illinois' current IFSP document, which is - for the most part - an electronic document and has been completed online.

In order to integrate the Outcome Summary Form into the IFSP document, modifications to the Cornerstone data management system were completed. A workgroup made up of stakeholders representing service providers, service coordinators and families assisted in identifying needed system changes as well as policy and procedure changes related to assessment and IFSP development. The Illinois Interagency Council on Early Intervention (IICEI) discussed the process at several of its meetings and reviewed and approved final procedures prior to implementation. All data system changes were complete by March 2006. Testing was completed during March and statewide implementation went into effect on April 1, 2006. All measures are entered into the Cornerstone data system so that it can be queried and analyzed.

The Early Intervention Training Program developed and implemented a training module that adequately addressed the new reporting requirements, new policies and procedures, methods for translating assessment information into progress measurement, achieving consensus and completing the new Outcome Summary Form. The training module was reviewed by the IICEI in January 2006 and was disseminated statewide in February 2006. Early Intervention Bureau staff provides technical assistance as needed. Monitoring of appropriate assessment procedures and accurate data input will be conducted by the EI Monitoring Program.

Progress/Baseline Data for FFY09/SFY10:

Measurable Rigorous Targets for FFY09/SFY10 Percent
Positive Relationships Summary Statement 1 64.5%
Positive Relationships Summary Statement 2 64.5%
Acquire Knowledge & Skills Summary Statement 1 78.5%
Acquire Knowledge & Skills Summary Statement 2 52.5%
Able to Meet Needs Summary Statement 1 75.5%
Able to Meet Needs Summary Statement 2 57.0%

SUMMARY STATEMENT SCORED BY CFC

CFC City

Outcome 1

Summary Statement 1

Outcome 1

Summary Statement 2

Outcome 2

Summary Statement 1

Outcome 2

Summary Statement 2

Outcome

Summary Statement 1

Outcome 3

Summary Statement 1

1 Loves Park 68.20% 67.90% 81.00% 48.70% 80.50% 65.10%
2 Waukegan* 62.60% 67.10% 76.90% 54.50% 78.60% 59%
3 Freeport 58.50% 66.70% 72.10% 59.00% 70.20% 63%
4 Geneva* 52.10% 71.70% 71.40% 54.60% 67.70% 63%
5 Lisle 57.60% 72.90% 79.90% 52.50% 73.30% 64%
6 Arlington Heights*** 67.80% 72.70% 83.90% 48.10% 80.50% 56%
7 Hillside*** 68.20% 59.70% 86.40% 47.20% 81.80% 53%
8 Chicago - Hoyne ** 63.30% 56.30% 75.80% 46.30% 74.80% 48%
9 Chicago - Harrison ** 67.00% 57.30% 76.40% 41.60% 77.00% 52%
10 Chicago - East 61st St ** 62.00% 40.00% 68.50% 28.10% 68.30% 27%
11 Chicago - George St ** 74.30% 63.40% 79.10% 53.20% 76.40% 57%
12 Tinley Park*** 67.50% 72.50% 75.80% 57.30% 72.90% 60%
13 Monmouth 52.60% 67.70% 66.70% 54.80% 64.30% 63%
14 Peoria 53.40% 73.20% 67.90% 61.00% 62.00% 71%
15 Joliet* 54.30% 63.70% 71.60% 52.20% 72.50% 63%
16 Danville 66.90% 60.20% 78.30% 49.90% 78.60% 53%
17 Quincy 69.70% 45.30% 77.40% 23.30% 79.10% 27%
18 Springfield 55.90% 49.40% 58.70% 19.50% 63.50% 34%
19 Decatur 83.50% 45.20% 80.50% 42.30% 81.30% 41%
20 Effingham 52.30% 56.60% 68.30% 47.30% 70.50% 51%
21 Belleville 67.10% 52.70% 82.20% 46.00% 84.10% 44%
22 Centralia 77.50% 47.80% 85.50% 43.30% 82.30% 47%
23 Norris City 69.50% 44.00% 83.20% 41.00% 73.30% 49%
24 Carbondale 63.60% 46.40% 82.40% 36.20% 77.00% 42%
25 Crystal Lake* 47.60% 76.00% 67.10% 58.90% 68.30% 69%
  • *Collar Counties
  • **City of Chicago
  • ***Suburban Cook County 

Balance of the State

Complaince Improvement

The following two tables represent the distribution of Child Outcomes matched entry-exit pairs compared by fiscal year. This data illustrates the improvement in compliance with program rules by fiscal year.

FFY09/SFY10 Improvement

FFY09/SFY10 Eligible State % Matched Pairs State % Compliance
State Total 13,959 100.00% 8,358 100.00% 59.88%
Collar 3,575 25.61% 2,367 28.32% 66.21%
Cook 2,694 19.30% 1,310 15.67% 58.25%
Chicago 3,614 25.89% 1,924 23.02% 35.83%
Downstate 4,076 29.20% 2,757 32.99% 67.64%

FFY08/SFY09 Improvement

FFY08/SFY09 Eligible State % Matched Pairs State % Compliance
State Total 14,116 100.00% 5,924 100.00% 42.00%
Collar 3,616 25.62% 1,954 32.98% 54.00%
Cook 2,872 20.03% 1,114 18.80% 39.40%
Chicago 2,759 19.55% 584 9.86% 21.20%
Downstate 4,914 34.81% 2,272 38.35% 46.20%

Discussion of Progress/Baseline Data

In the FFY09/SFY10 APR, Illinois proposed to replace its FFY08/SFY09 baseline data with the FFY09/SFY10 data as it is more fully reflective of Illinois' EI population for the following reasons:

  • There was a 41 percent increase from FFY08/SFY09 to FFY09/SFY10 in the number of matched pairs data available for analysis.
  • Increased use of the decision tree and participation in training sessions have increased the understanding of the child outcome measurement process, resulting in data that more accurately reflects child performance.
  • With an additional year of data collection, a full cadre of EI participants is reflected in the data, including children with more significant developmental concerns.

    In terms of progress or slippage, there are two areas of focus: (1) quality of data, and (2) quality of services. Illinois' data quality has improved over the last fiscal year as demonstrated by the significant increase in the number of matched entry-exit pairs. This is also evident to the EI Program based on feedback from CFC offices and providers regarding increased use of the decision tree and increased overall understanding of the Child Outcome measurement process, including use of the Child Outcome Summary Form (COS Form).

    Illinois continues to work to understand the relationship between the Child Outcomes ratings and the quality of EI services and supports being provided. FFY09/SFY10 Progress Data for Part C Children reveals an almost across the board decrease in the percent of children either reaching a level of development comparable to same-aged peers or maintaining functioning at a level comparable to same-aged peers (progress categories D and E). Summary Statement data, which focuses on children who have made significant progress and/or have exited the program functioning at a level comparable to same-aged peers experienced a similar lag in progress or slippage - failing to meet a majority of the rigorous targets set for FFY09. The EI program, in collaboration with the IICEI, the Outcomes Work Group and other key stakeholders has considered this lag in progress or slippage and an explanation follows:

    • Current data is more fully reflective of our EI population. With this being the first year in which the state is reporting a full cadre of EI participants, the data pool is reflective of children who entered the program at or near birth and participated in the program until age 3. Children who meet these criteria are typically children with medical diagnoses which make them eligible for services early in life or are children with such significant developmental concerns that those concerns are identified very early in the child's life. By comparison, due to reporting requirements, earlier reported data would have only captured children who entered the program later in life (i.e. 18 - 24 months) and exited at or near age 3 or children who entered and exited the program early in life. Given that the earlier reported data did not include children who entered the program at or near birth and participated in the program until age 3, it is reasonable to assume that progress would decline from the earlier reporting period to this reporting period as children with more significant developmental concerns are added to the data pool.
    • Accuracy and compliance have improved. While the natural response to improved accuracy and compliance may be an expectation to see higher numbers, in this case, the opposite may very well be true. Nationally, state Part C programs using the ECO Child Outcome Summary Form have reported concerns with early intervention service coordinators and/or providers rating children higher than they should be. This was a concern in Illinois as well. However, as training has become more prolific and focused and with more widespread use of the decision tree, early intervention service coordinators and providers in Illinois report a better understanding of the rating process and a feeling that ratings are more accurate now than they may have been early in the COS Form implementation process. This means that while early intervention service coordinators and providers may have rated children slightly higher than they should have at entry, exit scores are less likely to have received that artificial bump and therefore progress does not appear to be as great as it may otherwise have been. However, over time, entry scores will reflect the same level of accuracy as current exit scores and the playing field will level.

FFY Measurable and Rigorous Target

FFY  Measurable and Rigorous Target

2009

(2009-2010)

Summary Statement 1: Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 64.5% for positive social-emotional skills (including social relationships);
  2. 78.5% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 75.5% for use of appropriate behaviors to meet their needs.

Summary Statement 2: The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 64.5% for positive social-emotional skills (including social relationships);
  2. 52.5% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 57.0% for use of appropriate behaviors to meet their needs

2010

(2010 - 2011)

 Summary Statement 1: Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 65.6% for positive social-emotional skills (including social relationships);
  2. 77.0% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 74.5% for use of appropriate behaviors to meet their needs.

Summary Statement 2: The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 63.3% for positive social-emotional skills (including social relationships);
  2. 48.0% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 55.0% for use of appropriate behaviors to meet their needs.

2011

(2011 - 2012) 

Summary Statement 1: Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 66.0% for positive social-emotional skills (including social relationships);
  2. 77.5% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 75.0% for use of appropriate behaviors to meet their needs.

Summary Statement 2: The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 63.3% for positive social-emotional skills (including social relationships);
  2. 49.0% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 55.5% for use of appropriate behaviors to meet their needs.

2012

(2012 - 2013) 

Summary Statement 1: Of those infants and toddlers who entered or exited early intervention below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 66.5% for positive social-emotional skills (including social relationships);
  2. 78.0% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 75.5% for use of appropriate behaviors to meet their needs.

Summary Statement 2: The percent of infants and toddlers who were functioning within age expectations in each Outcome by the time they turn 3 years of age or exited the program.

Outcomes:

  1. 63.3% for positive social-emotional skills (including social relationships);
  2. 49.6% for acquisition and use of knowledge and skills (including early language/communication); and
  3. 56.0% for use of appropriate behaviors to meet their needs

With input from the Outcomes Work Group, the proposed performance targets for FFY10, FFY11 and FFY12 have been set using the new baseline data. The targets indicate an increase in two summary statements, while the remaining targets decrease and then improve to target value levels. Based on the information shared under the Progress or Slippage for Indicator 3 discussed earlier, the EI Program is not confident that improvement in terms of increases in stated percentages should be expected.

Improvement Activities/Timelines/Resources:

This is the initial reporting of paired entry and exit data. So, no progress or slippage can be reported. No targets will be established until 2/1/2010.

  • Continued training on the importance of completing the Child Outcomes Summary Form as a normal part of the IFSP and exit process.
  • Reassessment by the EI Bureau and the EI Monitoring agency of the uniformity of the administration of the summary form by the end of FFY 08/SFY 09.
  • Semi-annual evaluation to assure there are no patterns in the instances where assessments are not being completed at entry and at exit as required.
  • During FFY 07/SFY 08 the EI program increased training that emphasized the importance of completing child outcomes assessments at every IFSP meeting and the particular importance at the initial IFSP and at exit.
  • To improve uniformity of administration, having one of the two lowest percentage of compliant child outcomes entry-exit pairs was made a negative factor in the CFC determination scorecard, if the percentage is below 50% of the state average effective with CY 2007 and each year thereafter.
  • During FFY 07/SFY 08 the EI program emphasized the importance of correctly filling in the child outcome ratings at each IFSP, with particular emphasis on indicating progress was made.
  • During FFY 07/SFY 08 and FFY 08/SFY 09 the EI program reviewed the rates of compliance with rules regarding child outcomes measurement with CFCs, the IICEI and other interested parties and develop strategies to assure uniformity of administration.
  • During FFY 07/SFY 08 and FFY 08/SFY 09 the EI program reviewed the aggregate results from child outcome measurement and discuss what the results say about the program and make initial plans for activities to improve results.
  • During FFY 08/SFY 09 and FFY 09/SFY 10 the EI program worked with stakeholders, including the IICEI and CFCs to develop goals to improve child outcomes.
  • During FFY 08/SFY 09 and FFY 09/SFY 10 the EI program worked with contractors and stakeholders to educate the public on the early results on child outcome measurement and why it is important, with the help of the Outcomes Workgroup which will meet at least quarterly starting in December 2009.
  • By the end of FFY 10/SFY 11 the EI program will implement specific goals to improve child outcomes.
  • The System Ombudsman position will begin work with the field in February 2010 on improving compliance with program rules and principles. Better compliance with principles will result in better outcomes for children.
  • By June 30, 2010, the IICEI will create a workgroup to study issues that prevent good outcomes for Hispanic children and families. This group will issue an initial report no later than December 31, 2010.
  • Special training will be undertaken with CFC's and providers in Chicago to improve the amount of useable data, to be completed no later than June 30, 2010.
  • The improvement activities described in the SPP are ongoing efforts. The following are new improvement activities to be implemented through FFY12/SFY13.

New Improvement Activities Timelines & Resources

New Improvement Activities  Timelines & Resources
The EI program will add two Child Outcomes measurement modules to the Systems Overview training which is currently required of all new service providers. This improvement activity will target both the quality of Illinois' Child Outcomes data as well as the quality of services designed to improve children's outcomes. This activity will be completed no later than February 1, 2011. Resources include, but are not limited to the Early Intervention Training Program.
Utilizing a multifaceted approach to training and support related to the Child Outcomes measurement process, the EI program will develop an online training module in order to improve access to information about the Child Outcomes measurement process. Additionally, the EI program will offer post-training team discussions, mentoring and support to better ensure generalization of skills and consistency of practice. This improvement activity will target both the quality of Illinois' Child Outcomes data as well as the quality of services designed to improve children's outcomes. This activity will be completed no later than February 1, 2011. Resources include, but are not limited to the Early Intervention Training Program.
The EI program will offer post-training team discussions, mentoring and support to better ensure generalization of skills and consistency of practice. This improvement activity will target both the quality of Illinois' Child Outcomes data as well as the quality of services designed to improve children's outcomes. This activity will begin no later than February 1, 2011 and will continue through June 30, 2013.

This activity will begin no later than February 1, 2011 and will continue through June 30, 2013.

Resources include, but are not limited to the Early Intervention Training Program.

The EI program will begin analyzing Child Outcome data by race/ethnicity and comparing this data to Family Outcome data. This improvement activity will primarily target the quality of services designed to improve children's outcomes. This activity will be completed no later than June 30, 2011.

This activity will be completed no later than June 30, 2011.

Resources include, but are not limited to the workgroup convened to address issues related to the Hispanic and African American communities in Illinois and the EI Data Manager.

The IICEI will create a workgroup to investigate the correlation, if any, between poor family outcomes reported in a prior year by Spanish-speaking families and the child outcome ratings for children in Spanish-speaking households. The focus of the work group will be expanded to include African American families.

This work group will be created by December, 31, 2011 and will issue a report no later than June 30, 2012.

Resources include the IICEI, the EI Training Program, and the Bureau of Early Intervention.