Overview of the Annual Performance Report Development:

Monitoring Priority:

Indicator 4: Percent of families participating in Part C who report that early intervention services have helped the family:

  1. Know their rights;
  2. Effectively communicate their children's needs; and
  3. Help their children develop and learn.

(20 USC 1416(a)(3)(A) and 1442)

Measurement:

  1. Percent =# of respondent families participating in Part C who report that early intervention services have helped the family know their rights divided by the# of respondent families participating in Part C times 100.
  2. Percent =# of respondent families participating in Part C who report that early intervention services have helped the family effectively communicate their children's needs divided by the# of respondent families participating in Part C times 100.
  3. Percent = # of respondent families participating in Part C who report that early intervention services have helped the family help their children develop and learn divided by the# of respondent families participating in Part C times 100.

FYI 2008 (2008-2009)

Measurable and Rigorous Target:

74% of respondent families participating in Part C will report that early intervention services have helped the family know their rights

86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs

89% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

Actual Target Data for FFY 08/SFY 09:

Survey results have been weighted to correct for geographic disparities in returns. Although, the resulting changes in the results were minor for all three measures.

Weighted Survey Results: Formula: (Responses
5 or Higher/
Returns) x 100]
Percent 5
or Higher
To what extent has early intervention helped your family know and understand your rights? [(444/815) x 100] 71.85%
To what extent has early intervention helped your family effectively communicate your child's needs? [(516/616) x 100] 83.81%
To what extent has early intervention helped your family be able to help your child develop and learn? [(548/621) x 100] 88.27%

Moving to an all mail survey resulted in more representative sample overall. Less was done to followup and increase returns than in past years. The net return rate of about 24% was only modestly lower than the 27% of past year. Weighting to make the results better reflect the state caseload decreased the proportion of scores of 5 or over by 0.35% for knowing rights and by 0.12% for the effective communications question. The "helping your child develop and learn" question actually increased by a scant 0.03% through weighting.

It is generally agreed that the all mail approach is best for Illinois but the program will work with the IICEI Outcomes Workgroup on ways to improve representativeness and return rates.

To what extent has early intervention helped your family know and understand your rights?

TABLE

To what extent has early intervention helped your family effectively communicate your child's needs?

TABLE

To what extent has early intervention helped your family? Be able to help your child develop and learn?

TABLE

FFY 08/SFY 09 Family Outcome Survey Results Return Rates & Unweighted Results by CFC

TABLE

Discussion of Improvement Activities Completed and Explanation of Progress or Slippage that occurred for FFY 08/SFY 09:

Progress or Slippage for Indicator 4: The percentage of families that indicated a positive response for being able to communicate child needs, for being able to help their child develop and learn, and for knowing and understanding their rights fell just short of targets and demonstrated a decrease from FFY07/SFY08 data (A. 75.0%, B. 87.1%, and C. 90.8%). Moving to mailed surveys has made families feel freer to send back lower scores than when they were handed the survey directly by a representative of the program.

The geographic patterns seen in past years changed in this report. Chicago has always recorded the lowest proportion of families responding with scores of 5 or more. Downstate has always had the highest scores. This year, the proportion of Chicago respondents who gave a score of 5 or higher, Increased, for knowing rights and for helping children develop and learn while those scores declined for all the other regions. The only other improvement from last year was for the collar counties in the communications question. They had the highest scores on that question.

Downstate again had the highest scores on knowing rights. Chicago actually had the highest scores for helping their children develop and learn. Scores for respondents from Suburban Cook County fell sharply in all three measures. As a result, their scores were the lowest on all three questions. The scores on all three measures also fell sharply for downstate respondents but in the past their responses have been so high this drop mainly resulted in their being more in line with the other regions.

Concerns continue about the responses of Hispanics relative to other groups. This year the return rates for both Spanish and English speaking Hispanics did not differ as much as they have in past years. However, Spanish language survey results continued to come in much lower than the average, both in average scores and in the percentage responding with scores of 5 or higher. Interestingly, the responses from Hispanic families who say their primary language is English were much more like those for English language surveys overall than they were for Spanish speaking Hispanics. One of the primary improvement activities for the coming year will be to study the situation of Hispanic families in general, with a special emphasis on Spanish speaking families.

Local/CFC level data continues to illuminate the differential return rate problem. Weighting the four large regions helps adjust for this but there are also disparities within those larger regions. For instance, CFC 10, which serves Southeast Chicago, had the lowest rating scores and the lowest return rate. Scores for the other three Chicago CFC's were near or even above the state average. In theory, if more families from CFC 10 responded and also reported low scores it would pull down the Cook County average significantly and the statewide scores would be somewhat lower as well. CFC 10 is an economically poor area. At the other end of that spectrum CFC 5 - DuPage County is a wealthy area and they had the state's highest return rate. Their responses were higher than the state average. Their responses did not differ as much as those for the collar counties as a whole but their results do pull up the averages somewhat.

Local/CFC level data continues to illuminate the differential return rate problem. Weighting the four large regions helps adjust for this but there are also disparities within those larger regions. For instance, CFC 10, which serves Southeast Chicago, had the lowest rating scores and the lowest return rate. Scores for the other three Chicago CFC's were near or even above the state average. In theory, if more families from CFC 10 responded and also reported low scores it would pull down the Cook County average significantly and the statewide scores would be somewhat lower as well. CFC 10 is an economically poor area. At the other end of that spectrum CFC 5 - DuPage County is a wealthy area and they had the state's highest return rate. Their responses were higher than the state average. Their responses did not differ as much as those for the collar counties as a whole but their results do pull up the averages somewhat.

It was concluded that unanticipated negatives countered the assumed benefits of direct hand-off. The pool of responders is more representative, although that is still a significant concern. In addition, the proportion of responders who indicated a positive response (a score of 5 or better) fell again. We believe that one reason is that there is less selection bias with a mail survey and families feel freer to give low answers when the receipt of the survey is anonymous. Other problems identified in last year's Annual Performance Report also became even more pronounced in FFY08/SFY09 and contributed to lower scores in all three measures, resulting in the program missing all three targets.

As was the case last year, the program focused on taking steps that would improve outcomes for families. However, there were several changes in the program environment that help explain the declines:

  • Last year we noted that the nation's financial situation had not worsened at the time of the survey but the mood was not good. At the time of the survey this year financial conditions were substantial worse. The proportion of families with health insurance has fallen steadily, as has the proportion of families required to pay fees because they have income over 185% of poverty. The proportion covered by various forms of the state's Medicaid program increased significantly. The uncertainty many families were feeling last year was significantly worse this year. It is understandable that more challenging personal and financial situations translate into a lack of satisfaction and confidence in the program and in their own abilities to cope.
  • Service delay problems, explained in more detail under Indicator 1, continued to grow through FFY08/SFY09. As the proportion of families waiting to receive services for their child grows their dissatisfaction with the program grows as well.
  • Provider payment delays are a new problem that may contribute to unease for families. A year ago in November providers were being paid on a regular basis. From that point the program started to experience payment delays because the state was unable to transfer enough money into the EI Revolving fund to allow for the payment of bills on time.
  • As noted before, we believe moving to mailing surveys has made families feel freer to send back low scores than they were when they were handed the survey directly from a representative of the program, even if it was in a sealed envelope.

Improvement Activities

  • The Illinois and Texas Early Intervention Programs received funding through an IDEA General Supervision Enhancement Grant (GSEG) for a joint project to develop and pilot a family outcomes survey and to complete analysis of the results. The EI Bureau has provided updates on the project to the Illinois Interagency Council on Early Intervention (IICEI) and other interested bodies since before the grant was received. Illinois and Texas were coordinating their efforts on a family outcomes survey even before requesting grant funding.
  • As the only GSEG project focused exclusively on the measurement of family outcomes, Illinois and Texas volunteered to be one of the projects the Early Childhood Outcomes Center (ECO) worked with closely. This allowed us to utilize their expertise, most notably that of Dr. Don Bailey, then of the University of North Carolina Frank Porter Graham Center. The survey we have developed with their help is now more commonly known as the ECO tool. Dr. Bailey is now working at RTI.
  • In addition to a formal project advisory group process, focus group reviews of the English version of the tool were carried out, with the help of the Early Childhood Outcomes (ECO) Center in FFY 05/SFY 06. That provided feedback on the tool itself and aspects of survey administration. ECO has also incorporated input from stakeholders nationally. The Illinois-Texas-ECO tool is being used in whole or in part by about one-third of states. During FFY 06/SFY 07 RTI conducted focus groups on the Spanish version of the tool with Illinois and Texas families who predominately speak Spanish.
  • During FFY 08/SFY 09, Illinois distributed a total of 2,568 surveys. As in the past, the goal was to distribute surveys primarily by hand at six-month reviews and annual IFSP renewal meetings. Some were sent by mail from CFCs when meetings had already been held or otherwise could not be distributed by hand. However, in the first two year of survey administration there were major discrepancies in return rates between CFCs. In an attempt to overcome that problem, for FFY 07/SFY 08 1,000 surveys were sent to families in the Chicago area directly from the central project office. This did have a positive impact on return rates in several areas. We also believe it was a factor in the lower average ratings. Given the extra work necessary to do hand-delivery and the poor results outside of downstate, all FFY08/SFY09 surveys were mailed. The resulting return rates were somewhat more representative, although the resulting sample still under represented the Chicago area. Average scores were again lower. Given our experience with a partial mail process in FFY 07/SFY 08, we speculate that the move to an all mail process was one of the reasons for lower scores.
  • The creation of an IICEI workgroup to study issues related to Hispanics is slated to begin working prior to the end of FFY09/SFY10.
  • The Early Intervention Program has worked with its Training Program and CFC offices to fully implement new bilingual translator training and proficiency testing qualifications. Currently, systems overview and interpreter training are being combined to improve training outcomes. The combined training will incorporate the Code of Conduct requirements from the Department's Provision of Interpretive services for Limited English Proficient Persons administrative directive.
  • As part of minority outreach strategies, the Early Intervention Clearinghouse is developing bilingual information for families and will collect and retain translated program documents for use by CFC offices.
  • The Program Integrity Project, described in Indicator 1, includes the addition of a system ombudsman position. Delays in the contracting processes resulted in delays in filling of this position, but an ombudsperson should be in place in February 2010. Other Program Integrity Project strategies have been piloted, with some system impact realized.

Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY09/SFY10:

The proportion of families reporting that the EI program had a positive impact on them fell in all three areas for FFY 08/SFY 09. We believe the economic downturn, the negative affect on short-term state finances, and the somewhat related increase in service delays were the primary reason for the decline. We also assume the conversion to an all mail survey produced less positive results. We do not anticipate that the state's economy or the state's finances will be appreciably better when the survey is mailed in the spring of FFY09/SFY 10. Therefore, we do not anticipate much change from the report data. As a result, targets are being changed as follows:

2009 (2009-2010)

73.5% of respondent families participating in Part C will report that early intervention services have helped the family know their rights

85% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs

89% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

2010 (2010-2011)

74% of respondent families participating in Part C will report that early intervention services have helped the family know their rights

86% of respondent families participating in Part C will report that early intervention services have helped the family effectively communicate their children's needs

89.5% of respondent families participating in Part C will report that early intervention services have helped the family help their children develop and learn

The following new improvement activity will be undertaken:

  • The System Ombudsman will work to enhance high-level conformity with the spirit of Early Intervention rules, laws and philosophy. The System Ombudsman also will observe and make recommendations on local practice as it relates to the three family outcomes.
  • The IICEI will create a workgroup to study issues related to Hispanics. This workgroup will recommend program changes that will have a positive impact on the way Hispanics experience the program and thus their outcomes.
  • The program will do whatever it can to limit provider payment delays. Effective January 1, 2010 the state has included EI payments under its state Prompt Payment Act, which says the state must make an additional payment if bills are not paid within 60 days.
  • The ECO/Illinois/Texas survey tool is being revised to make it appear shorter and less intimidating to families. It is anticipated this will improve the response rate.
  • The IICEI, through its Outcomes Workgroup, will recommend steps to be taken that will help increase survey return rates and help make returns more representative of the caseload for FFY09/SFY10 and following years.
  • The ECO/Illinois/Texas survey tool is being revised to make it appear shorter and less intimidating to families. It is anticipated this will improve the response rate starting with the FFY09/SFY10 survey.