Overview of the State Performance Plan Development:
See Indicator 1 for a description of this process. The EI Program will make the Illinois APR and SPP available on its web site and through links from the other EI web sites (the Illinois Early Intervention Training Program; Provider Connections, the Early Intervention credentialing office; and the Early Childhood Intervention Clearinghouse). The APR and SPP documents will also be available to the public at each of the 25 CFC offices.
The Illinois and Texas Early Intervention Programs received funding through an IDEA General Supervision Enhancement Grant for a joint project to develop and pilot a family outcomes survey and to complete analysis of the results. The EI Bureau has provided updates on the project to the Illinois Interagency Council on Early Intervention (IICEI) and other interested bodies since before the grant was received. Illinois and Texas were coordinating their efforts on a family outcomes survey even before requesting grant funding.
Each state has formed an advisory council to provide input to the project. The Illinois advisory council includes representation from parents, service providers, and CFC offices, along with a researcher and a developmental pediatrician. The first meeting of Illinois' advisory council to the project was held in August 2005, followed by a joint meeting with the Texas advisory council. With the release of the instructions for the SPP, responsibilities and membership of the Illinois advisory council for the family outcomes project have been expanded to include the discussion of a process for measuring child outcomes. An expanded group meeting was held prior to the November 3, 2005 IICEI meeting. Subsequent meetings have been held since that time at important points in the project. The group will continue to meet and provide input and will assist in developing implementation strategies for measurement of both family and child outcomes.
In addition to the formal advisory group process, Illinois and Texas carried out focus groups to review the tool, which has been developed with the help of the Early Childhood Outcomes (ECO) Center. They have provided feedback on the tool itself and aspects of survey administration. ECO has also incorporated input from stakeholders nationally. The tool is now the ECO tool and is used by a number of other states in addition to Illinois and Texas.
(The following items are to be completed for each monitoring priority/indicator.)
||Early Intervention Services In Natural Environments
Percent of families participating in Part C who report that early intervention services have helped the family
- Know their rights;
- Effectively communicate their children's needs; and
- Help their children develop and learn.
(20 USC 1416(a)(3)(A) and 1442)
- Percent = # of respondent families participating in Part C who report that early intervention services have helped the family know their rights divided by the # of respondent families participating in Part C times 100.
- Percent = # of respondent families participating in Part C who report that early intervention services have helped the family effectively communicate their children's needs divided by the # of respondent families participating in Part C times 100.
- Percent = # of respondent families participating in Part C who report that early intervention services have helped the family help their children develop and learn divided by the # of respondent families participating in Part C times 100.
Overview of Issue/Description of System or Process:
Most of Illinois' Child and Family Connections (CFC) offices have surveyed parents on the program for a number of years. However, those surveys tended to focus on satisfaction with the program rather than outcomes achieved for the family. Also, those surveys are not uniform and they were not developed in anticipation of needing to answer the specific questions required by the Office of Special Education Programs (OSEP).
In SFY 03/FFY 02, the EI program completed a uniform statewide survey of parents. This allowed a view of all programs statewide on the same platform. However, that was a one-time study and the data would be too old to provide a baseline. That tool also focused mainly on program satisfaction. It has not been duplicated and was not developed with the specific OSEP questions in mind. However, Illinois recognized the need to measure program outcomes. We felt there was a greater opportunity to get meaningful feedback more quickly by focusing on family outcomes first. Given the critical importance of families in the development of children and their life-long learning opportunities, we felt assessing family outcomes was just as important for the Part C program as assessing child outcomes.
The program also felt that the National Early Intervention Longitudinal Study (NEILS) offered a good platform from which we could build a statewide outcomes survey to measure family outcomes. We found that the Texas EI program had a similar idea and it was agreed that we would work jointly to develop a single tool, as a means of sharing costs and limited staff resources. When the General Supervision Enhancement Grants (GSEG) became available to help states build outcome measurement systems, Illinois applied for funding to complete the joint project. Illinois was awarded funding and the joint project is the only one working exclusively on family outcomes. Since that time, Texas applied for and received a grant to continue the joint project.
Since ours was the only GSEG project focused on family outcomes, the Early Childhood Outcomes (ECO) Center provided us with extensive assistance, mainly through Dr. Don Bailey. The survey tool that was developed measures the five ECO outcomes as well as the three OSEP family outcomes questions. The Illinois-Texas Tool is more widely known as the ECO tool, as they took the lead in its final development and in promoting it nationally. The Illinois-Texas version of the tool also addresses family feelings about their future. Illinois and Texas feel this is an important consideration, based on reviews of the research on family outcomes.
The survey is essentially an extension of ECOs work on family outcomes with additional questions added for areas Illinois and Texas want to test. The tool has not only been reviewed and commented on by Texas and Illinois program staff, family focus groups, and advisory groups but also by ECO staff and ECO advisory groups. Based on additional feedback and other considerations after the completion of the pilot study, the survey tool in use for FFY 06/SFY 07 has been modified slightly. Although, the basic structure and root questions remain the same.
Illinois/Texas and ECO made a presentation on the tool and related issues to all states on September 29, 2005 in a national conference call sponsored by the National Early Childhood Technical Assistance Center (NECTAC). The current tool is attached and is also is available on the following websites, (NOTE: Only the Early Intervention Statewide Outcomes Survey website includes the Illinois/Texas only questions.)
Surveys are handed to families by Service Coordinators at six-month reviews and IFSP renewal meetings. Focus groups indicate direct hand-off will increase returns but allowing Service Coordinators to directly assist families would compromise anonymity and skew results. It is important that families feel comfortable providing negative responses. Each survey has a code distinct to the child, so that responses can be analyzed on demographic and service factors. However, each survey is in a sealed envelope so that nobody at the CFC knows what the numbers are. English and Spanish language surveys are put in different colored envelopes to further ensure proper distribution. For the pilot, Coordinators were specifically directed that they should not assist the family with completion of the survey. Families complete the surveys and mail them back to the project office. The project office notes how many have been returned from each CFC and forwards completed surveys to the University of Illinois, which completes initial statistical analysis of responses received. CFCs also inform the project office of surveys that were not delivered for various reasons.
The survey being used in FFY 06/SFY 07 differs slightly from the one used in the FFY 05/SFY 06 pilot based on additional feedback. The new version does not change the basic structure of the survey or the intent of any of the questions.
In addition, the program was disappointed in the return rates in the Chicago area, particular in Chicago, during the pilot study. It was decided that one reason for this was that Service Coordinators were directed to be too "hands-off", to avoid skewing responses. In the future Coordinators will be encouraged to be more helpful and encourage families to respond, as long as they do not assist families in completing the survey. We also are exploring ways to increase return rates of minorities, those with fewer resources, and low literacy clients.
The original plan was to include survey responses in an insolated section of the Cornerstone database but data is actually kept completely outside of Cornerstone. This means it is available to an even smaller number of individuals and further improves confidentiality. This process still allows the assessment of results on a wide range of factors without wasting the time of families providing demographic detail the system already has. The program will share general survey statistics with CFCs and others, as long as the number of surveys being analyzed does not compromise client confidentiality. As required, summary results will be published regularly at both statewide and CFC levels. The tool also provides families with a place to write additional comments and those comments will also be reviewed and analyzed. The GSEG project is testing both English and Spanish language versions of the survey tool. Versions in other languages may be added as well. During FFY 06/SFY 07, a total of 6,000 surveys will be distributed in two waves. The first will last about six weeks starting in February. The second wave will take place in May and June. Decisions on when to distribute surveys and how many to distribute in coming years will depend on discussions with Illinois and Texas stake holders and the continued needs of the GSEG project and ECO.
As suggested by the ECO Center, a score of five or above on the seven-point scale is considered positive.
In FFY09/SFY10, Illinois utilized the revised version of the Family Outcomes Survey (FOS-R) for the first time to collect the data for this indicator. The FOS-R uses a 5-point rating scale, versus a 7-point scale used in previous versions, to assess the helpfulness of early intervention.
The scale includes the following responses: 1 = Not at all helpful, 2 = A little helpful, 3 = Somewhat helpful, 4 = Very helpful, and 5 = Extremely helpful. Also new with the FOS-R are 17 new helpfulness indicators, including five for "know their rights," six for "effectively communicate their children's needs," and six for "help their children develop and learn." These additional indicators have been added with the belief that the data collected would be more informative and valid than data collected from the previous version of the FOS. For the second consecutive year Illinois used an all mail survey, with the result of a more representative sample overall. Families were selected in the same way as they have been in the past, based upon a representative sample of children having a six-month review or annual IFSP coming due during a given span of time.
Beginning in FFY10/SFY11, Illinois will discontinue the use of mailing surveys to a sampling of families participating in the program. All families enrolled in the program during a designated month will be sent a Family Outcomes Survey.
Baseline Data for FFY 2005 (2004-2005)/SFY 06:
To what extent has early intervention helped your family know and understand your Rights?
||Responses 5 or Higher
||Total Number of Responses
||% Responses > or = 5
To what extent has early intervention helped your family effectively communicate your child's needs?
||Responses 5 or Higher
||Total Number of Responses
||% Responses > or = 5
To what extent has early intervention helped your family be able to help your child develop and learn?
||Responses 5 or Higher
||Total Number of Responses
||% Responses > or = 5
Discussion of Baseline Data:
The percentages of responders who answered positively with a score of five or better on the seven-point scale to the three specific OSEP questions was high:
- Know and understand your rights - 79.1%
- Effectively communicate your child's needs - 87.8%
- Able to help your child develop and learn - 91.0%
However, both response rates and ratings differed in a number of ways. For instance, an identical number of surveys (608) were prepared, although not necessarily distributed, in Chicago and downstate but 184 were returned from downstate but just 71 from Chicago. The respective return rates were 30.3% downstate and 11.8% for Chicago. In addition, there were return rate differences based on economic and social factors. The return rate for whites was 25.8%, compared to 12.3% for blacks and 12.1% for Hispanics. The Spanish speaking return rate was 11.5%. Families required to cost share through family fees, indicating income in excess of 185% of poverty had a 26.4% return rate. Families who are not assessed fees constitute more than two-thirds of the caseload but had a return rate of just 18.4%. Families that had never been Medicaid eligible had a 25.0% return rate, compared to 18.6% for those who were on Medicaid or who had been in the past.
If we weight returns by region to account for return rates we find that each of the positive response rates are lower, particularly the question regarding knowing and understanding rights. The following are the weighted response rates:
- Know and understand your rights - 76.8%
- Effectively communicate your child's needs - 86.5%
- Able to help your child develop and learn - 90.2%
The program is only now receiving initial data on survey responses. It will take most of the rest of the year to analyze the data, discuss possible responses, and initiate improvement plans. Therefore, during FFY 06/SFY 07 we do not anticipate being able to affect any positive change. However, we are implementing a series of changes in how the survey is administered geared towards improving return rates, particularly in CFCs that had low return rates during the pilot. We also will make extra efforts to improve response rates among minorities, the economically disadvantaged and low literacy families. Assuming those efforts are successful, we anticipate positive response rates will be closer the weighted rates for FFY 06/SFY 07. It is possible that positive responses will be even lower because we will be reaching families who are less engaged in the process. Therefore, although we will begin implementing improvement activities in FFY 07/SFY 08, we do not anticipate being able to see meaningful progress until FFY 08/SFY 09.
The overall responses on all three questions showed that the client population was very satisfied with the services provided by the state. Approximately 90% of the respondents rated the programs 5 or above (on a scale from 1 to 7) in helping them to effectively communicate their children's needs (Question 17) and in helping their children to learn (Question 18). In terms of helping their families to understand their rights (Question 16), approximately 80% rated the programs 5 or over; while this is still an extremely good score, it indicates that the state needs to put more effort in this area.
The response patterns for all three questions were similar. For each question, the ratings in the Collar Counties and Downstate were very similar and were the consistently the highest scores reported. On the other hand, the ratings in suburban cook county were consistently the lowest, and were approximate 5% below the ratings reported for Cook County, the second lowest reported.
For social science survey a response rate of around 30% is typical; this rate was obtained in the Downstate portion of the state. Both suburban Cook County and the Collar Counties showed a response rate of around 20%, an interesting finding considering that the actual reported responses were much more favorable in the Collar Counties that in suburban Cook County. The response rate in the City of Chicago was only around 12%, and obviously much work needs to be done in future surveys to improve this.
The proposed target values maintain the baseline target for two indicators (effectively communicate your child's needs and able to help your child develop and learn) in FFY2012/SFY2013, as the baseline values are well over current performance. Interim years demonstrate a gradual increase to baseline values. Improvement over baseline for the third indicator (know and understand your rights) has been demonstrated and should continue as improvement activities are implemented. Survey data are still stabilizing as strategies to increase both the number of surveys distributed and return rates will have an impact on the ability of the data to be representative across CFC office areas and populations served.
- Development of survey tool in conjunction with the Texas EI program and the Early Childhood Outcomes Center by the end of November 2005
- Testing of finalized survey tool to be completed by February 2006
- Development of preliminary results of testing by March 2006
- Additional improvement activities, timelines and resources will depend on the development and analysis of baseline data
- Complete development of revised survey tool revision by December 2006
- Share results of pilot survey with advisory groups and the public by May 2007
- Complete a initial improvement plan based on survey results by July 2007
- Complete FFY 06/SFY 07 survey between February and June 2007
- Complete analysis of FFY 06/SFY 07 responses by October 2007
- Complete discussions of FFY 06/SFY 07 survey responses with stakeholders and approve formal, long-term improvement plan by December 2008
- The program will work with the Illinois Interagency Council on Early Intervention, the Minority Outreach Strategies group and CFCs to identify and implement ways to be more responsive to the needs of both Spanish and English Speaking Hispanic families.
- The program is creating a new Program Integrity Project to supplements its existing records review based monitoring system. The project supports conformity with the spirit of Early Intervention rules, laws and philosophy. The Program Integrity Project will include monitoring and recommendations, when needed, on local practice as it relates to the three family outcomes.
- The System Ombudsman will work to enhance high-level conformity with the spirit of Early Intervention rules, laws and philosophy. The System Ombudsman also will observe and make recommendations on local practice as it relates to the three family outcomes.
- The IICEI will create a workgroup to study issues related to Hispanics. This workgroup will recommend program changes that will have a positive impact on the way Hispanics experience the program and thus their outcomes.
- The program will do whatever it can to limit provider payment delays. Effective January 1, 2010 the state has included EI payments under its state Prompt Payment Act, which says the state must make an additional payment if bills are not paid within 60 days.
- The ECO/Illinois/Texas survey tool is being revised to make it appear shorter and less intimidating to families. It is anticipated this will improve the response rate.
- The IICEI, through its Outcomes Workgroup, will recommend steps to be taken that will help increase survey return rates and help make returns more representative of the caseload. The ECO/Illinois/Texas survey tool is being revised to make it appear shorter and less intimidating to families. It is anticipated this will improve the response rate.
- The IICEI, through its Outcomes Workgroup, will recommend steps to be taken that will help increase survey return rates and help make returns more representative of the caseload.
Revisions to proposed targets and improvement activities were discussed with the Outcomes Work Group. The work group includes parent representatives from the IICEI, CFC office managers, Early Intervention providers, and research and training staff. The improvement activities described in the SPP are ongoing efforts. The following are new improvement activities to be implemented through FFY12/SFY13.
New Improvement Activity
|New Improvement Activity
||Timelines & Resources
|The Outcomes Work Group will develop a guidance document to help CFCs report the local results of the FOS to the community. Based on what is learned from the survey data statewide and locally, providers and programs will have an opportunity to reflect on the data and may choose to make changes or adjustments in their practice with families to see an improvement in family outcomes and/or individual indicators. This will be completed in FFY10/SFY11.
This will be completed in FFY10/SFY11.
Resources include the Outcomes Work Group, the EI Training Program and the Bureau of Early Intervention.
|The Illinois EI Training Program will imbed training on the FOS indicators in both their online training modules and as a part of face-to face training opportunities for providers. The intent of this training will be to highlight the importance of what is asked of families as a part of the FOS, and to highlight how data from the FOS can help states see how their families are doing, identify any areas in need of improvement, and then, after program adjustments, assess the impact of those changes-with the goal of moving to ever higher percentages of families reporting outcomes attained.
This will be completed in FFY10/SFY11 and will continue as an ongoing activity.
Resources include EI Training Program and the Bureau of Early Intervention.
|The IICEI will create a work group to study issues related to Hispanics. This work group will recommend program changes that will have a positive impact on the way Hispanics experience the program and thus their outcomes. The focus of this group will be expanded to include African American families. This work group will be created by December, 31, 2011 and will issue a report no later than June 30, 2012.
This work group will be created by December, 31, 2011 and will issue a report no later than June 30, 2012.
Resources include the IICEI, the EI Training Program, and the Bureau of Early Intervention.
|Illinois will discontinue the use of mailing to a sampling of families participating in the program. All families enrolled in the program during a designated month will be sent a Family Outcomes Survey.
This will be initiated in FFY10/SFY11 and will continue as an ongoing activity.
Resources include the EI Training Program and the Bureau of Early Intervention.