OCC: Legislative Process for Performance Based Contracting

The Office of Community Corrections (OCC) has been executing plans for performance-based contracting (PBC) within the community corrections system for several years. PBC represents an opportunity to maximize the performance, efficacy, and quality of community corrections services through the use of incentives. Under PBC, the community corrections system becomes one of continuous improvement. An improved community corrections system in turn leads to greater public safety, successful rehabilitation for individuals, and long-term fiscal savings in relationship to reduced recidivism. The move to PBC embodies the mission of the OCC: Through regulation, innovation and collaboration, the Office of Community Corrections strives to support our community partners in delivering superior services to clients, building a stronger, safer Colorado. 

The original plan submitted in 2015 by the Governor’s Community Corrections Advisory Council detailed a methodical and progressive approach to the implementation of PBC. Since then, OCC has undertaken large, system-wide improvements to pave the way for the implementation of PBC. Those changes include:

• Comprehensive revisions of the Colorado Community Corrections Standards;
• New audit and evaluation tools; 
• A baseline measurement of community corrections programs.

In the 2021 legislative session, the Colorado legislature made a formal Request for Information (RFI) for the Department to provide an update on PBC:

“As part of its FY 2022-23 budget request, the Department is requested to submit a proposal for the implementation of performance based contracting. This proposal should include payment models, outcomes to evaluate the performance of community providers and local community corrections boards, baseline targets for the Program Assessment for Correctional Excellence (PACE) and core security audits, the frequency of PACE and core security audits, and a warning system for underperforming providers. The Department is requested to submit this proposal no later than January 3, 2022.”

The Department contracted with Government Performance Solutions, Inc., in April 2021 to help develop and facilitate a transparent and inclusive stakeholder engagement process to inform OCC’s response to the RFI. Stakeholder input is critical to developing and implementing a successful model for PBC in community corrections. Stakeholder views are acknowledged throughout the RFI response.

In order to keep you informed of the proposed plan as submitted to the legislature, we are sharing with you below the full OCC response to the RFI.

RFI Highlights

Key aspects of the department’s plan as submitted to the legislature include:

Performance Metrics and Targets

The performance metrics and targets included in the proposed plan represent items both in the direct and indirect control of system providers, with a focus on program quality, program compliance, and individual client outcomes. 

Risk-Informed Outcomes

Metrics: Successful Completion and Recidivism

Targets: Will be based on data analysis and formulas to set appropriate targets based on multiple years worth of Colorado system performance. 

Program Compliance

Metrics: Core Security Audit

Target: Composite score of 2 or higher

Program Quality

Metrics: PACE Evaluation

Target: Composite score one deviation above the mean of statewide baseline measurement

Key Performance Indicators

Metrics: Staff training and retention indicators

Target: Specific metrics and targets to be individualized with input from local community corrections board in consideration of local needs, barriers, etc.

Payment Model

The payment model proposed includes several key factors:

• Slow progression into increased incentives and decrease of base per diem to allow for the system to become accustomed to the new contracting and reimbursement method. 
• A model that allows for future flexibility in metrics, weights, and payments that can evolve with the system over time.
• Solutions to address concerns about funding and initial investment to ensure the success of PBC. 
• Increased accountability and financial impacts for poor performance.

Timeline

The timeline submitted aims to get the first incentive payments out as soon as Fiscal Year 2022-23 for risk-informed outcomes, while proposing a slower timeline for other metric areas in regards to a reasonable audit and evaluation cycle. The audit and evaluation cycle takes into account resources needed and adequate time between audits and evaluations for improvements and corrections to be made. 

In August of 2015, the Governor-appointed Community Corrections Advisory Council submitted the original plan for performance-based contracting (PBC) in the community corrections system. Since that time, the department has been working towards the goal of implementing PBC by establishing the foundational groundwork. Accomplishments to date include:

• Revised the standards for community corrections to better align with best practices and the principles of effective intervention and published the 2017 Colorado Community Corrections Standards (Standards).
• Completed the development of the Program Evaluation for Correctional Excellence (PACE).
• Developed the Core Security audit covering core public safety functions related to the Standards.
• Completed a baseline PACE and Core Security audit at all active community corrections programs prior to the COVID-19 pandemic. 
• Contracted with the Urban Institute to make recommendations for PBC in regards to risk-informed outcomes, payment models, and the implementation. 
• Released baseline reports on the completed PACE and Core Security audits. 

Metrics to Evaluate Performance

The original PBC plan defined performance as program compliance, program quality and program efficacy. Defining performance as such allows for metrics both in the direct and indirect control of the provider. The plan continues to be recommended to align PBC with this definition of performance. 

Risk-Informed Outcomes 

Risk-informed outcomes represent metrics that are not in the direct control of the provider, but are important indicators of program performance and measure program efficacy. All risk-informed outcomes are individual outcomes such as success, escape, and recidivism that take into consideration the risk level of the individual being supervised by the community correction program. Analyzing outcomes in relationship to risk mitigates concerns that jurisdictions and providers will be incentivized to serve low risk populations. The ability to successfully measure and analyze outcomes is rooted in the data that is both available and reliable. Given this, the Urban Institute recommended the measures of successful completion and recidivism. After evaluating the report, available data, and considering stakeholder input, the Department agrees with these recommendations for the start of PBC. 

Successful completion is a longstanding and reliable data set contained in the Community Corrections Information and Billing (CCIB) system and allows for the use of a positive outcome as a measurement of program efficacy. While utilizing recidivism as an outcome measure in criminal justice comes with some imperfections, recidivism is still a widely accepted, utilized, and expected measurement of success in the field. In addition to representing an individual's success, improved recidivism outcomes represent cost-savings for the state. The Urban Institute recommended a definition of recidivism as a new felony conviction starting from entry into the community corrections program. For the purposes of PBC, the Department will adopt this definition and evaluate at two (2) years from program admission. Two years should represent both time in program and time in the community for the vast majority of individuals. While the stakeholder group identified some questions and concerns about this definition, it was supported over other recidivism definitions.

While successful completion and recidivism will be the initial outcome measures for PBC, the model proposed has the flexibility to adjust and change these measures in future years.  In the stakeholder process, the largest supported outcome measure was the reduction of risk. Currently, risk is measured by the Level of Supervision Inventory (LSI) in the community corrections system. The LSI is given every six (6) months, meaning any client discharged before 6 months will most likely not have an updated LSI. Due to this, in Fiscal Year 2021 only 45.5% of individuals received an updated LSI that could be utilized to demonstrate risk reduction. In Figure 1 it is demonstrated that the majority of the 45.5% are successful program completions and a review of risk reduction with the currently available data would be skewed by this. 

Figure 1 

Currently the department is updating CCIB and will be collecting additional data points related to risk reduction and the LSI. The department will work towards the goal of including risk reduction and other areas identified by stakeholders as performance measures in the future. Other areas could include employment retention, community engagement, and treatment matching.

Core Security Audit

One of the areas of performance in the direct control of the provider identified in the original PBC plan was program compliance with core security functions.  Including core security functions as a measurement of performance is critical to the mission of community corrections. Community corrections is tasked with ensuring both the safety of the local communities where they operate, as well as the individuals participating in the programs. Safety is an essential component to an individual’s ability to grow and progress.  After the Standards were revised in 2017, standards related to core security functions were identified and an audit of those specific standards was developed. The specific audit is referred to as the Core Security audit. The importance of maintaining safety was reinforced by the stakeholder working group and the Department will include the Core Security audit as a performance metric. 

PACE Evaluation

The final area defined as performance in the original PBC plan was program quality as defined by adherence to the Principles of Effective Intervention. The Principles of Effective Intervention are those that are most likely to impact outcomes including risk reduction, program success, and post-release recidivism. The Program Assessment for Correctional Excellence (PACE) was developed with national experts on the Principles of Effective Intervention as well as evaluation practices. The evaluation is in congruence with the Standards revised in 2017. The evaluation of program quality defined in this way represents another area of performance in the direct control of providers and the stakeholder working group reinforced the value of including it as part of the definition of performance. The department will continue with the original plan to include the PACE evaluation as a performance metric.

Specific Measures/Key Performance Indicators

During the workshops focused on the Core Security and PACE audits, there was discussion on the use of specific measures within these processes versus a total composite score for performance metrics. Stakeholder feedback varied across workshops and discussions of these different processes. The department believes it is best to have a holistic measurement for both the Core Security and PACE audits, and feels that some of the benefits of having specific measures can be achieved by including an additional performance metric of key performance indicators (KPIs). While this was not a part of the original plan, it adds the opportunity to capture areas of performance that were also valued by the stakeholder working group. Throughout the workshops the topics of staff training and retention were evident. The department proposes the addition of KPIs in the PBC model. The department will collaborate with the local boards to set the KPIs for their respective programs.  This will allow for flexibility for each program to focus on the performance indicators based on independent factors for a specific program, taking into account their geographical location, needs and values.

Metric Targets

Essential to developing a successful PBC model is determining the targets at which performance will be incentivized and the magnitude of those incentives. Metric targets will be based on statewide data and level setting. The department is currently in the process of hiring a Statistical Analyst III to ensure the internal expertise exists for the required data analysis. 

Weights

The original PBC plan recommended that each area of performance (quality, compliance, efficacy) have a different weight or importance associated with incentive payments. Quality was considered the highest area of weight followed by compliance and efficacy. The stakeholder workgroup participants agreed with this recommendation and wanted to see the PACE metric weighted as the highest indicator of performance. In order to ensure equity and fairness within the model, initially the department recommends equal weights for PACE, Core Security, and KPI performance metrics and a lower weight for risk-informed outcomes. After every program has had its first PACE and Core Security audit completed and equity becomes possible, the department will adjust the weights in line with the original recommendation and the recommendation of the stakeholder workgroup. This is highlighted with more details in the payment model and timeline sections of the RFI.

Risk Informed Outcomes

The department will utilize the Urban Institute’s formulas and analysis methods for risk adjustment and determining performance targets for successful completion and recidivism. The Urban Institute developed two categories of risk: low/medium and high/very high as defined by client LSI scores. This category of risk is used to determine whether a program served high/very high risk clients, or low/medium risk clients is defined by a program having more than 50 percent of their clients assessed by the LSI at that high/very high risk level in a year. Programs fall into the low/medium risk category when more than 50 percent of their clients are assessed by the LSI as low/medium risk.

Core Security

Security functions are critical to ensuring both individual and community safety. During the stakeholder working groups, input was received about the importance of these fundamental standards and the expectation of compliance. Stakeholders expressed the importance of adherence to the Standards as being mandatory or expected in this area. The performance target will therefore be set at a composite score of 2. A score of 2 is defined as satisfactory performance in the Core Security audit and setting the target at 2 requires programs to perform at this level. Programs that receive a composite score of 2 or higher will receive incentive funding. Of the 24 programs that received a baseline Core Security audit, 8 programs received a score of 2 or higher. 

PACE 

The PACE is an innovative evaluation tool that utilizes a variety of methods to determine congruence with the Principles of Effective Intervention. The scoring of the PACE is intricate and follows a different scale than utilized by the Core Security audit. This type of evaluation is newer to the community corrections system and was carefully designed with leading experts. Stakeholders believed those practices being evaluated in the PACE were the most important to achieving better outcomes and required more practice to master. Setting the target based on the results of the baseline evaluations was preferred. To set targets for this performance metric that are fair and rooted in data, the target will be set by using standard deviation. The PACE composite metric target will be set at one standard deviation above the statewide baseline mean. Setting the target in relation to baseline data, allows the model to adjust as the statewide baseline improves. As performance is improved across the state and the baseline score raises, the standard deviation will also be updated. Of the 29 programs that received a baseline PACE, 8 programs received a composite score one standard deviation above the mean. 

Specific Measure/Key Performance Indicators

The specific KPIs for each program related to staff training and retention will be set in partnership with the local community corrections board. KPIs and their targets should be based on available data. One geographical area in Colorado may have a different job market and staffing variables than other locations. Using available data allows both the KPIs and their targets to be based on the individual program. The state will provide guidelines and oversight for setting the KPIs and targets.

Payment Model

Model

In consideration of the feedback from stakeholders since the beginning conception of PBC and the recommendations from the Urban Institute, it is important that the payment model be flexible and consider both gradual and timely adjustments. Adjustments in payment are recommended to be gradual in the beginning to allow time for programs to adjust to the new funding model before potentially seeing a decrease in funding and getting acclimated to the level of performance required to receive additional funding. At the same time it is important that the model allow for adjustments to incentives to be frequent enough to keep programs engaged with PBC. 

In regards to flexibility, the department proposes a model that allows for flexibility in both schedule and performance metrics. While the proposed payment model and the timeline introduced later rely on a three year performance cycle for the performance metrics of PACE, Core Security, and KPIs, the model also has the flexibility to change this schedule if needed. In addition to schedule flexibility, the payment model proposed allows for flexibility in updating or changing the performance metrics utilized in PBC over time. This flexibility allows the model to grow and change as the system grows and changes. Within this model it is possible to update the risk-informed outcomes, KPIs, targets, and incentive percentages.

In addition to these considerations, a model for payment must align with realistic timelines and provide for equitable opportunities for incentive payment. To honor the ability to have equitable opportunities for payment and take into consideration the weighting of performance metrics, the department proposes an initial payment schedule reflected in Table 1. Once all programs have had an equitable opportunity for incentive payments in the performance metrics of PACE, Core Security, and KPIs, PBC will transition to the payment model reflected in Table 2. 

Table 1

Table 2

Funding

There were two main concerns expressed by stakeholders regarding funding. The first concern was the state’s ability to ensure that funds are available to provide incentive funding for all providers that have earned them. The second concern was that the proposed model reduces the percentage of base per diem funding and that investing in improving the quality of services requires upfront investment. 

In regards to the first concern, during the Fiscal Year 2021-22 budget setting process, the Joint Budget Committee analyst presented a staff-initiated request to sponsor legislation creating a cash fund with community corrections-reverted general fund dollars. The time-limited cash fund would be utilized to ensure the needed funding is available for the first years of PBC. The department supports the staff-initiated request. A cash fund of this nature would aid in ensuring needed funding is available in the first years of PBC while any barriers and issues are being resolved, most importantly in regards to timelines. As will be discussed further in the timeline section of the RFI, once the evaluation timeline is successfully offset from the funding cycle, it will allow the department to be able to submit accurate budget requests eliminating the need for the cash fund. 

As mentioned, concerns were also raised regarding the investment needed to improve quality services and earn incentive funding. More specifically, concerns were raised that lower performing programs would not have the funding needed to improve and earn incentive funding before the base per diem starts to be lowered. The department recommends allocating an additional facility payment in Fiscal Years 2022-23 and 2023-24. The availability of funding for this would be made easier by the creation of a cash fund with reversional dollars. This additional funding would allow programs to make upfront investments in training, coaching, curriculums, and other areas that will improve their performance on the Standards. 

Warning System 

It is the statutory responsibility of the department to promulgate standards for community corrections and audit compliance to those standards. The utilization of PBC in the community corrections system does not replace or supersede this obligation. Stakeholders and the department believe it is imperative regulatory functions remain intact with the ability to issue corrective actions or take more serious sanctions upon a program up to and including program closure. This allows for immediate action and consequences to be applied if it is warranted. It is important that PBC work in conjunction with this role. To ensure that programs in poor standing with Standards compliance are not also receiving financial incentives, the department will create a probationary status for non-compliance. Probation will be utilized when attempts at corrective action are unsuccessful.  Placement on probation will be in consideration of the scope and severity of the compliance issues. Once a program fulfills the requirements to be removed from probationary status, they will then again become eligible for PBC incentive payments. 

In addition to regulatory compliance considerations, the PBC model itself includes inherent consequences for poor performance. As demonstrated in the payment modeling, a program will be eligible for less per diem if the program does not meet the metrics for incentive payments. In addition to this, the three year performance cycle means a program that does not meet the target for incentive will have to wait until their next evaluation on the three year cycle to try for the incentive again. This acts as a natural warning system and consequence for poor performance. For example, in Fiscal Year 2027-28 a program earns incentives in the areas of program completion (1%) and KPI(1%) only. That program will receive just 99% of the per diem and will have to wait for their re-valuation of PACE and Core to become eligible. At the local level, PBC disincentivizes local units of government for awarding contracts to lower performing programs given that board administrative funds are a percentage of the total allocation. The higher performing the program or programs are in the jurisdiction, the larger the overall allocation amount for the jurisdiction, which directly translates to more board administrative funds. At the state level, lower performing programs will be less competitive in the procurement process for specialized community corrections contracts.

Timeline

The timeline for PBC must integrate with the payment model and align with available resources. The incentive payments can begin as soon as FY 2022-23. To be prepared for the first incentive payments on the other metrics, there are tasks that need to be completed. The tasks needed to be completed in the current and upcoming fiscal year are noted in Table 3. 

Table 3

Table 3 shows the RFI, Standards Review and Corre Security and PAC updates were completed in FY22 and in FY23 KPIs were developed, the Core Security PACE updates happened, and evaluations and audits begin.

In determining an appropriate timeline for PBC in community corrections, the department carefully considered the current staff and available resources, as well as the amount of time needed between audits for a provider to make improvements positively affecting their performance. The timeline represented below in Figure 2 demonstrates a three (3) year performance cycle for the metrics of PACE, Core Security, and KPIs, while the risk-informed outcomes metrics represent an opportunity to improve each year. A three-year cycle ensures the department has the sufficient staffing and resources to meet the demands of the timeline while also staying committed to regulation, technical assistance, and contract management. In addition, it leaves time between audits and evaluations for the program to implement changes and receive technical assistance in pursuit of increased performance. The stakeholder working groups indicated a preference for a program to receive a PACE or Core Security audit each year. The department cannot meet this demand with the current staffing levels and also wants to ensure that the staffing and resources exist to meet all of the demands of the statutory requirements for community corrections. In addition, the department believes it is critical to ensure there is proper time between audits and evaluations for improvements to be made. 

The evaluation cycle will be offset from the payment cycle to ensure the data needed to inform the Governor and General Assembly of the funds needed for each fiscal year is available. Programs will be assigned to 3 cohorts, with each cohort receiving an update every year in either the PACE, Core Security audit, or KPI performance metrics.

Figure 2: PBC Timeline by Cohorts

As was demonstrated in the payment model (Table 1), for the first 3 year cycle, each of these metrics will be weighted the same and eligible for one percentage on the metric most recently updated. Once equity has been achieved by all programs having received the PACE, Core Security audit, and KPI metrics evaluation, the model changes to each of these performance metrics being eligible for a weighted incentive (Table 2). At this point, once an incentive has been earned in one of these metric areas, the program will retain this incentive until the next evaluation of the performance metric. Figure 3 provides an example of the payment possibilities and evaluation cycle for a program in Cohort 1, and reflects the update of the payment model in Fiscal Year 2027-28 (Table 2). The example assumes the program meets the performance targets for all of the PBC measures. It also demonstrates the evaluation cycle, however it is important to note that this cycle will be offset from the fiscal year and will run January - November.

Figure 3

Other Considerations

New Programs

During the stakeholder working groups, a concern was raised on how to integrate new programs into the PBC model. The department recommends that new programs be given three (3) years of the full per diem rate before joining the PBC model. This allows the provider time to establish the program, receive technical assistance, and prepare for PBC. This also allows the department time to provide technical support and begin the program evaluation process. 

Updates to the Standards and/or Evaluation Methods

The department maintains the raw data for all audits and evaluations. As Standards and Methods are updated that will have a significant impact on overall composite scores, raw data will be reanalyzed to make adjustments as needed both on baseline numbers and metric targets. For example, if the Standard and evaluation methods related to headcounts as part of the Core Security audit changes significantly, the data from that standard will be removed for the analysis of the composite score.