Purpose of Program Monitoring

State Service Commissions and National Programs face many challenges in administering AmeriCorps programs. Limited staff capacity, as well as unusual program models, can make monitoring and management particularly difficult. As stewards of public funding, the SERVE ILLINOIS COMMISSION needs to ensure that the programs and sites they fund are aware of their contractual requirements and are in compliance with all of the rules, regulations, and provisions governing AmeriCorps funds. To accomplish this, your Serve Illinois Commission Program Officer must create adequate systems for monitoring program/sites that are fully implemented and available in writing.

Components of a Monitoring Strategy

An adequate monitoring strategy features an array of oversight of program/sites, including:

  • Document Review
  • Compliance Reviews
  • Program Quality and Performance Reviews
  • Special Audits or Surveys
  • One-on-one technical assistance to resolve any issues

General

The Serve Illinois Commission conducts site visits to programs for two purposes:

  1. To provide assistance on program development and implementation issues; and
  2. To ensure compliance with AmeriCorps rules and regulations.

Programs in their first three years of funding typically require more assistance than programs that have been operating for four or more years. The majority of staff time and energy will be spent visiting and providing assistance to new programs.

Programs not scheduled for a visit may request assistance or a site visit to focus on specific program issues at any time. Staff will accommodate these requests as efficiently as possible.

In addition, all programs with one or more high-risk factors, determined by the Risk Assessment Tool, will receive additional visits from Serve Illinois Commission Program Officers.

This policy will help to ensure high quality AmeriCorps programs are implemented while maximizing staff time and meeting the recommendations of the Inspector General in regards to program monitoring.

Preparations

To prepare properly for a site visit, program staff should:

  • Review the Monitoring Tool provided by your Program Officer and be prepared to present any requested documentation.
  • Schedule the visit with your Program Officer at a time when the appropriate people will be available for meetings.
  • Verify that member time sheets are entered and up to date two weeks prior to the meeting.

Procedure

All visits will be scheduled in advance. Your Program Officer will contact you to schedule your monitoring visit(s).

Written feedback will be provided to programs after each visit. Copies of all completed forms and correspondence will be kept in program monitoring files.

Monitoring Visits

Based on a state or organization's risk-based monitoring strategy, a programmatic site visit may be made to every grantee or site one time per grant cycle, one time per year, or more often as may be appropriate.

Typical site visit activities include the following:

Activity Individuals Involved
Interview AmeriCorps Program/Site Director
Member File Checks (other types of files?) AmeriCorps Program/Site Director
Multi-Site Observation of Member Activities AmeriCorps Members
Field Interviews Site Supervisor, AmeriCorps Members
Mid Year/Close-Out Interview AmeriCorps Program/Site Director and Staff

All AmeriCorps programs will receive a monitoring site visit during each program year. The purpose of the visit is to assess the program's compliance with federal regulations and to help the program improve systems to pass a formula OIG audit. Member files, program policies and fiscal records will be reviewed during the visit. The AmeriCorps Program Review Instrument will be used to assess program compliance. (Please see AmeriCorps Program Review Instrument at the end of this chapter).

The results of the visit will be reviewed with program staff at the end of the day. The site visit feedback letter sent to the program should include only those issues discussed during the visit.

The modules of the AmeriCorps Program Review Instrument to be completed are as follows:

This Module will be completed before the visit. Results will be reviewed with the program and important issues or concerns will be discussed. A program may only receive a "Yes - Appears OK" rating if it has been in compliance with the item being reviewed 100% of the time. Programs that are not performing in this area must show improvement and demonstrate a commitment to improving. Any concerns in this area will be addressed in the site visit feedback letter and a corrective action plan to improve future performance will be required.


Module A: Reporting and Communication Compliance and Early Issue Detection

This Module will be completed before the visit. Results will be reviewed with the program and important issues or concerns will be discussed. A program may only receive a "Yes - Appears OK" rating if it has been in compliance with the item being reviewed 100% of the time. Programs that are not performing in this area must show improvement and demonstrate a commitment to improving. Any concerns in this area will be addressed in the site visit feedback letter and a corrective action plan to improve future performance will be required.


Module B: Financial Compliance

The majority of this module will be completed with the assistance of the program's fiscal staff, but some questions will most likely be answered by program staff. Programs should have copies of their expenditure forms and the Federal Financial Reports submitted via OnCorps available for review. A copy of the program's original budget should also be available. In general, a program must have all required documentation for each item and every cost sampled to receive a "Yes - Appears OK" rating.


Module C: Policies and Procedures Compliance

All programs should be in compliance with all of these issues. Written documentation and/or proof of compliance is necessary in case of an audit and should be provided here. Programs should have written documentation readily accessible, such as training agendas or memos. Programs must have written documentation to receive a "Yes - Appears OK" rating.


Module D: Member Documentation Compliance
Member files will be reviewed to determine if required documentation is being kept by the program. Program Officers will randomly select the member files to be reviewed. All member files reviewed must contain required documentation for a program to receive a "Yes - Appears OK" rating. A program may receive a "Needs Improvement" rating if one file is missing required documentation. If more then one file is missing documentation, the program will receive a "Does Not Meet Requirement" rating. Programs that do not meet a requirement must submit documentation showing full compliance and improve future performance to ensure continued funding.

* See Chapter 8 - Attachments: 27. Member File Checklist (pdf)

Review Outcome

After the monitoring visit is complete, the Program Officer will provide written feedback relating to the review. It is expected that all programs will receive "acceptable" in all categories. The monitoring review will be used to determine if additional site visits are necessary and may impact a program's renewal application and ability to access additional training funds.

* See Chapter 8 - Attachments: 28. Program Review Instrument (pdf)

High-Risk Site Visits

High-risk site visits are conducted to provide targeted assistance on specific issues or concerns.

The agenda for program visits will vary based on the reason for the visit. In general, Program Officers (and Commission members or other staff, as appropriate) will meet with members, site supervisors or other stakeholders to discuss the program. When possible, members will be observed in service. The Commission acknowledges that issues such as confidentiality may prevent this. Program Officers will try to obtain as much information as possible about program activities and services in order to identify best practices or areas for improvement.

For program-related visits being conducted because of high-risk factors, the agenda will be determined by the reason for the visit. For example, the agenda for a visit that is being held because of a change in program director will focus on expectations of programs, AmeriCorps regulations and relationship building. The agenda for a visit that is being conducted because of concerns regarding prohibited activities will focus on meeting with members, site supervisors, and program staff to assess whether prohibited activities were taking place and to ensure that prohibited activities do not occur in the future.

The site feedback letter will serve as written documentation of what occurred during the visit.

High-Risk Programs

Programs designated as high-risk or who request a site visit to focus on specific issues will receive additional visits.

High-risk factors may include, but are not limited to:

  • A change in program director or other key staff
  • Legitimate member complaints to the Commission
  • Poor past performance by the program (based on progress reports and/or previous site visits)
  • Concerns regarding prohibited activities
  • Compliance with reporting deadlines

These visits will be conducted on an on-going basis throughout the program year.

Risk-based Criteria Form

This tool will be used to identify which of the currently funded programs will require formal monitoring in the next program year.  If a programneeds improvement or are out of comliance in any of the first 9 criteria, they will receive both formal monitoring visits for the next program year.  If the program is compliant in all areas, the program will receive only the necessary visits.

Risk-based Criteria includes:

  • Prior monitoring visits identify insufficient documentaion and/or systems
  • Program has newlyhired staff member(s)
  • Program demonstrates a need for additional support
  • Program maintained less than 100% recruitment and/or 90% retention
  • Program has significantly changed the performance measures
  • Program has signficantly increased their grant
  • Program fell signficantly short in reaching their performance measures

Link for the 2010-11 Risk-based Criteria Form (pdf).  Please contact AmeriCorp with questions; 800-592-9896.

Monitoring Checklist

In general, it is expected that program staff will monitor grantees and sites to examine the following:

Member Documentation

  • AmeriCorps Application
  • Determination of Eligibility
  • Background Checks
  • Enrollment Forms
  • Retention Statistics (end of term/exit, change of status/term, terms of release)
  • Hours Worked and Timesheets
  • Training Received
  • Benefits (child care, health care, etc.)
  • Member Service Agreements
  • Position Descriptions
  • Performance Evaluations (at least mid-year and end of year)

* See Chapter 8 - Attachments: 27. Member File Checklist (pdf)

Program Documentation

  • Grievance Procedures
  • Prohibited Activities
  • Staff Timesheets
  • Progress Toward Meeting Performance Measure Targets
  • Measurement Tools
  • Use of OnCorps
  • National Identity (signage, uniforms, published materials)
  • Mechanism for Community Input and Collaboration
  • Subcontracts or Host Site Agreements (if applicable)

* See Chapter 8 - Attachments: 28. Program Review Instrument, Module C for a more detailed list of documents (pdf)

Desk-Based Monitoring

While an on-site monitoring visit can provide a valuable, direct view of a grantee's/site's operations, it is a resource-intense form of monitoring. Desk-based monitoring is more sparing of scarce resources of staff time and money and, properly used, can actually forestall the need for more intensive intervention later on to resolve grantee problems.

Throughout the year, your Program Officer manages Onorps accounts for each program/site in their portfolios. Areas that require your Program Officers attention include:

  • Approving performance measures
  • Reviewing reports

Your Program Officer will also utilize My AmeriCorps as an effective management and monitoring tool. For instance, through My AmeriCorps, it is possible to:

  • Review each program/site's progress on member recruitment and retention
  • Track member progress toward completing service hours
  • Check timeliness of reporting, including member forms, progress reports, and financial reports.
  • Conduct member enrollment queries and monitor member hours tables

Your Program Officer will also utilize OnCorps as an effective management and monitoring tool. For instance, through OnCorps, it is possible to:

  • Monitor expenditures
  • Review progress towards performance measures
  • Check timeliness of reporting, including member forms, progress reports, and financial reports.

Member Service Agreement

The standard member service agreement in particular should be reviewed carefully to ensure that it contains the following items:

  • Minimum number of hours and other requirements necessary for a post-service education award
  • Start and end dates of the member's term of service
  • Amount of the living allowance
  • How the member will be paid
  • Description of the other benefits available to the member
  • Standards of conduct and sanctions for improper conduct
  • Prohibited activities
  • Requirements under the Drug-Free Workplace Act
  • Termination and suspension rules (including the specific circumstances under which a member can be released for cause)
  • Position description for the member
  • Grievance procedures
  • Any other program-specific requirements
  • Signatures of both the member and the program director

Member Orientation

Orientation materials should also be reviewed carefully to ensure that the grantee/site has provided members with appropriate training for beginning their term of service. This should include:

  • An overview of national service
  • Program rules, regulations, and expectations
  • Review of the member service agreement
  • Training appropriate to activities
  • Grievance Procedures

* See the Orientation Checklist (pdf) for a detail description of mandated topics.

Site Monitoring

Site visits by program staff can have many purposes, some of the most important of which are summarized below:

  • LEARNING

Finding out about AmeriCorps grantees'/sites' service activities and community partnerships.

  • PUBLICITY

Hearing success stories.

  • COMPLIANCE

Assuring that all federal, state, local, contractual, and organization-specific regulations, policies, and provisions are being followed.

  • MONITORING PROGRESS

Comparing actual outputs and outcomes to the performance measures outlined in a grantee/site contract.

  • TECHNICAL ASSISTANCE

To improve overall quality of the program.

* See Chapter 8 - Attachments: 29. Host Site Supervisor Evaluation Form (pdf) for more details.

Financial Monitoring

Typically, program staff is assisted by fiscal staff in assuring that program and sites are in compliance with the fiscal requirements of their grants. Though program staff do not have primary responsibility in this area, they should be aware of grantee obligations to submit periodic expense reports and financial status reports, meet match requirements, and (in most cases) conduct annual audits.

Financial Documentation

  • Match (where applicable)
  • Receipts for Grant Expenditures
  • Internal Controls Systems
  • Accounting Systems
  • Payroll System
  • A-133 Audit (if applicable)

* See Chapter 8 - Attachments: 28. Program Review Instrument, Module B (pdf) for a more detailed list of items.

Recruitment/Retention Monitoring Tool

In light of the CNCS standards for member enrollment and retention, it is important for your Program Officer to monitor their grantees/sites closely to keep track of how they are doing in these areas. The standards are:

ENROLLMENT RATE = Total slots filled/total slots awarded.

Enrollment for programs should be 100%.

RETENTION RATE = Total members exiting with an Education Award/total slots filled.

A target of 90% for retention or completion is desired.

CNCS realizes that many programs not yet meeting these rates are nevertheless performing well in their individual circumstances. Also, retention may vary among effective programs depending on the target populations for those programs. However, in order to leverage limited program dollars, CNCS expects programs to strive toward improving their rates of enrollment and retention. During the application process, programs that are not meeting these standards need to offer a rationale for why not, as appropriate.

Recruitment/Retention Rate Calculation

To calculate enrollment and retention using the data in My AmeriCorps:

  1. Log into your account via eGrants
  2. Select S/N Reports
  3. Select Report via pull down menu
  4. Select Program year
  5. Select format (PDF, Excel etc.)
  6. Select Submit
  7. You may open or save the report created

Program Close-Out

Each year, near the end of your program year, your Program Officer will conduct a program close-out visit. This visit will be conducted by using the AmeriCorps Program Review Instrument. Programs will be required to present documentation relating to fiscal policy, program policy and member files.  Please see the AmeriCorps Program Review Instrument for details on specific documentation that will need to be available.

Final Close-Out

The Corporation grants are usually awarded for a three year Project Period, consisting of three yearly budget periods. The Serve Illinois Commission is required to submit documentation of program close-outs to the Corporation. Close-out documents are due to your Program Officer no later than 60 days after the close of the grant and should include the following:

Final Project Report

The narrative should include a three page, double-spaced summary of the qualitative accomplishments that the AmeriCorps grant has made for the duration of the grant period. Include a discussion of the impacts of any special initiatives that fall within this grant. Discuss what exists now in the communities that are served that did not exist prior to this grant. Quantitative data may be used to support the impact statements.

Please attach the appropriate cover page, that can be found via the link below: http://www.americorps.gov/for_organizations/tta/forms.asp and submit to the Commission. This report is not submitted via OnCorps.

Final Federal Financial Report

A program completing the final year of its three-year grant cycle, must submit a final Federal Financial Report (FFR) that is cumulative over the entire project period. This FFR is due within 60 days after the end of the project period. This Final FFR should be completed on the required form and not in OnCorps.

Equipment and Supply Inventory Form

A program completing the final year of its three-year grant cycle must submit two (2) inventory forms. One for any equipment inventory with a current fair market value of $5,000 or more purchased with federal grant funds. The second form for unused/residual supplies inventory purchased with federal funds exceeding $5000. Reports should be completed on the required forms. If no equipment or supplies were purchased, forms should reflect such. For more information please refer to the AmeriCorps Provisions.

IDHS Grant Close-Out

At the end of each IDHS grant period, providers will be required to complete a "Grant Reconciliation Form". This form will be provided to the Grantee upon completion of the IDHS grant period.

* See Chapter 8 - Attachments:

 30. Federal Financial Report (pdf)

 31. Equipment Inventory (pdf)

 32. Inventory of Residual Supplies (pdf)

 33. Certification of Subgrant Closeout (pdf)

Monitoring Tools

The Pre-Site Visit Checklist should be completed by the Program Officer prior to each Monitoring Visit.

* See Chapter 8 - Attachments: 34. Pre-Site Visit Review Tool (pdf)

The Risk-Based Criteria Form will be completed the beginning of each program year. This form will be used to determine the number of visits a Program will receive in the upcoming year.

* See Risk-Based Criteria Form (pdf), in Chapter 7, page 80.

The AmeriCorps Program Review Instrument is the primary tool used to assess program quality and compliance with regulations.

* See Chapter 8 - Attachments: 28. Program Review Instrument (pdf)

The Member File Checklist Service will be used to evaluate member files.

* See attached file: 27. Member File Checklist (pdf)

The Illinois AmeriCorps*State Program Evaluation survey tool, based on Module E of the Program Review Instrument, will be used during program-related or high-risks site visits to gather additional input from members.

* See Chapter 8 - Attachments: 35.AmeriCorps member evaluation (pdf)

PROGRAM EVALUATION

As articulated in the AmeriCorps regulations 45 CFR §§ 2522.700-740, all AmeriCorps State and National grantees that receive an average annual Corporation grant of $500,000 or more must conduct an independent evaluation. An independent evaluation uses an external evaluator who has no formal or personal relationship with, or stake in the administration, management, or finances of the grantee or of the program to be evaluated.

All other AmeriCorps State and National grantees must conduct an internal evaluation. An internal evaluation is designed and conducted by qualified program staff or other stakeholders, such as board members, partners, or volunteer affiliates.

If you are a . . . You will submit an . . . 
State formula grantee Evaluation as specified by your state commission
State competitive grantee with an average annual
Corporation grant under $500,000
Internal Evaluation
State completive grantee with an average annual
Corporation grant of $500,000 or more
Independent Evaluation
National grantee with an average annual
Corporation grant under $500,000
Internal Evaluation
National grantee with an average annual
Corporation grant of $500,000 or more
Independent Evaluation
State and National Education Award Program (EAP) grantee,
regardless of funding
Internal Evaluation

If you are recompeting you are required to submit "a summary of your evaluation efforts or plan to date, and a copy of any evaluation that has been completed, as part of your application for funding" (45 CFR § 2522.730). If you recompete again in 2011 or beyond, you are required to submit a completed evaluation with your application. The Corporation will consider the results of your evaluation "in assessing the quality and outcomes of your program" (45 CFR § 2522.470).

If you receive an average of $500,000 or more per year from the Corporation, averaged over the last three years of funding you have received before you recompete, we expect you to conduct an independent evaluation by contracting with an external evaluator. The AmeriCorps regulations describe how this evaluation should provide evidence of a causal relationship between program activities and outcomes (45 CFR § 2522.700). You may consider using an experimental or quasi-experimental design, or compare your results with national/state/local data. Your external evaluation method should match the size, scale, and purpose of your program.

In our ongoing effort to reduce burden on grantees, especially those with smaller grants, grantees an average annual grant under $500,000 may submit an internal evaluation. The primary difference between the independent evaluations that grantees that receive $500,000 or over are required to submit and the internal evaluation is who conducts the evaluation study. Your own staff and other stakeholders can serve as internal evaluators.

We encourage you to design your internal evaluation so that it will yield data most useful to you. You may opt for an impact evaluation, or you may conduct a process or management evaluation. You are not required to conduct an experimental or quasi-experimental evaluation that proves causality, which is required of grantees that receive $500,000 or over, although you are allowed to conduct this type of study. We expect the same high quality that we expect of a larger grantee, regardless of the type of evaluation you decide to conduct.