Skip to content

Establishing A Public-Private Partnership Program: A Primer

November 2012

« PreviousNext »

Chapter 7 - Oversight and Monitoring

P3 agreements can create efficiencies through establishing long-term design-build-finance-operate-maintain (DBFOM) contracts that include outcome-based performance specifications. Outcome-based performance specifications focus on what a facility is intended to achieve rather than prescribing methods and materials for achieving facility goals. The goal of using outcome-based performance specifications is to make service delivery more efficient by allowing the concessionaire to decide how best to achieve the intended results.

This shifts the public agency's primary role in the project from oversight of design and construction to management of a performance-based contract. In this role, the challenge for the public agency is to find ways to monitor and manage contract performance without reclaiming transferred risks or impinging on the efficiencies gained from allowing the concessionaire to choose the best way to meet performance specifications.

Effective performance-based P3 contracts align the concessionaire's interest with those of the public sector throughout the duration of the agreement. Performance management is a way to maximize project efficiency while at the same time ensuring that the contractor not only meets performance standards at the time of construction, but manages the dynamic risks to performance over the period of the agreement.

This chapter describes the elements of effective P3 performance management. The first section discusses public sector performance management responsibilities and challenges. The second section identifies factors that contribute to effective performance management.

Performance Management Responsibilities

Public sector responsibilities for managing the performance of P3 agreements begin prior to the close of the agreement and last for the duration of the agreement. These responsibilities include:

  • Defining performance measures and setting performance standards;
  • Monitoring performance;
  • Assessing payments and penalties for performance;
  • Designing and managing dispute resolution processes; and
  • Managing handback of the facility.
Setting Performance Standards

In setting performance standards for a P3, agencies need to consider:

  • The types of performance standards that should be used. Are these standards critical to the performance of the project? Does the agency have the staff and resources to monitor the performance?
  • The level at which the performance standards should be set. High standards are desirable, but standards that are set too high will raise the cost of a project and will result in a project that is at a higher standard than others in the State, or possibly a project with higher standards than are needed for user benefit. For example, requiring that roads be litter-free may lead to a better driving experience for road users, but requiring that litter be removed hourly may not produce enough benefit to offset the additional cost.

There are tradeoffs associated with committing to certain standards and consequent required levels of funding. In this regard, P3 agreements are less flexible than traditional methods of publicly maintaining and operating infrastructure, where the public agency retains year-to-year flexibility in the allowable performance standards. Public sector agencies sometimes relax these standards by delaying or reducing investments, or by lowering maintenance standards, in order to conform to financial realities.

Private sector input on performance standards may be sought at the onset of a procurement process by soliciting alternative technical concepts (ATCs) or changes to project scope, design, or construction criteria. However, in considering ATCs, the public agency must balance the benefits of private sector innovations with the benefits of maintaining a fair and competitive procurement process.

In setting performance standards, public agencies may look to benchmarks set in other P3 agreements or equivalent facilities. Public agencies that are already applying performance management to State-operated transportation facilities may set goals and measures for P3 projects that are consistent with or contribute to the goals and measures the agency has set for the rest of the system. Public agencies may also set policy goals for specific facilities and set performance standards based on those explicit policy goals, such as mobility, safety, environmental stewardship, or economic development.

Public agencies must also consider that desired performance standards are likely to change over time. As a public agency's own standards change due to changing conditions or policy goals, they will likely expect the concessionaire to conform to those changes. For example, future land development may necessitate changes in environmental standards.

To a certain degree, the concessionaire is typically willing to take on the risks associated with changes in law that do not specifically discriminate against the project or the concessionaire, and such an agreement can be written into the contract. However, the concessionaire will typically ask for some assurance that the standards won't be changed so quickly or completely that it becomes financially onerous to meet new standards. As a result, some P3 contracts specify a limit to the number or percentage of changes to standards that can be made on an annual basis or include procedures for the private partner to be compensated for unexpected costs or lost revenues resulting from changes.

There is a natural tension between flexibility and accountability in performance management. If a standard is too flexible, the public sector risks not obtaining the highest possible level of performance from a concession. If a standard is inflexible, it may not adapt to changing technology needs. For example, in one agreement, the concessionaire's performance was based on the operations of its call center for its toll payment accounts. However, most users preferred to use a web interface to communicate with the concessionaire. The contract performance standard failed to anticipate technology changes or to use a more flexible measure of success, such as customer satisfaction.

Monitoring Performance

The public agency is responsible for monitoring the performance of the concessionaire. P3 contracts will typically establish roles and responsibilities (see Table 7-1) and monitoring procedures. Performance monitoring procedures can include self-reporting procedures, independent audits, regular meetings and reports, and the use of intelligent transportation systems that automate data collection and reporting processes.

Table 7-1. Potential Performance Monitoring Responsibilities
Party Responsibility
Concessionaire
  • Develop management plans and procedures
  • Collect monitoring data
  • Develop status reports
  • Self-report violations
Public Agency
  • Set performance standards
  • Review plans, procedures, and status reports
  • Perform audits and inspections
  • Assess penalties and awards
3rd Party
  • Perform independent audits and inspections
  • Data collection
  • Resolve disputes
Shared
  • Daily communication and problem solving
  • Conduct regular face to face meetings
  • Complete annual performance reviews
Assessing Payments and Penalties for Performance

Most P3 agreements prescribe processes for penalizing noncompliance, but rewards for superior performance are rarely used. The public agency is responsible for tracking concessionaire performance and penalizing the concessionaire when contractual obligations are not met. Before penalties are assessed, P3 agreements typically prescribe a series of actions that must be taken to notify the concessionaire of the issue and a period of time to correct the noncompliance issue after it is detected.

Penalties typically consist of payment reductions or retentions and non-compliance or default points. Once noncompliance or default points reach a specified level, they can result in increased oversight, work by the owner at the contractor's expense, suspension of work, or termination of the contract.

Contractors may prefer default points to financial penalties because they may fear the public agency will abuse financial penalties to meet short-term financial objectives. Furthermore, if the cause of underperformance is lack of finances, fines may inhibit the concessionaire's ability to correct the problem. On the other hand, if financial penalties are set too low, the concessionaire may lack sufficient incentive to take corrective action or may perceive fines are simply part of the cost of doing business. Default points incentivize performance without money changing hands by raising the risk of default. This in turn may raise the concerns to private lenders, who may then pressure the concessionaire to correct the issue.

Resolving Disputes

P3 contracts typically specify dispute resolution processes to reduce the risk of legal conflict over technical issues or differences in contract interpretation. Alternative dispute resolution processes may include mediation and third party arbitration following a period of time allowed for both parties to make good faith efforts to resolve the dispute themselves. Arbitration may be conducted by an agreed-upon expert or by a designated board with members selected by both the public agency and the concessionaire. In particularly large projects, a permanent, independent dispute resolution office may be established to quickly resolve any contract dispute.

P3 contracts typically specify alternative dispute resolution processes for various reasons including the time sensitivity of many P3 projects and the speed advantage of these extrajudicial processes. Professional arbitrators or mediators can be selected for their industry knowledge and will seek resolution through a collaborative non-adversarial process. Another consideration favoring alternative dispute resolution procedures on P3 contracts is that the public agency may not be sued even when in breach of the contract. This "sovereign immunity" can become an obstacle for the private sector to financing a project unless the agency waives this immunity in favor of contractually-defined alternative dispute resolution mechanisms.

Prior to mediation or arbitration, dispute resolution processes often define tiered systems of problem identification and resolution through negotiation to encourage problems to be resolved at the lowest levels. For example, the contract may specify a process whereby the parties to a dispute are given a set time period to seek ways to resolve their dispute before it is elevated to their respective managers. In elevating the dispute, the parties must write a memo to their supervisors, summarizing the nature of the dispute and the steps they attempted to take to resolve the issue. This can serve as an incentive for parties to seek a speedy resolution to disputes.

In the worst case scenario, underperformance can lead to contract failure. Contract failure occurs when one party is unable or refuses to comply with the contract or the parties to an agreement are unable to resolve disputes concerning the meaning of contract specifications. Contract failure can result in the need to amend or renegotiate a contract, resolve disputes in courts, replace parties to the agreement or terminate the agreement. These events may ultimately lead to higher costs for the public sector.

Managing Handback

P3 contracts generally specify the required condition of the facility at the end of the contract term. The condition of a facility at handback depends on the maintenance and operation procedures employed throughout the lifecycle of the facility, so the concessionaire is typically required to develop a capital replacement or asset management plan for equipment, systems and assets. In addition, the concessionaire may be required to develop a plan that specifies the processes for turning over operation of the facility to another party at the conclusion of the contract.

Review of handback conditions may involve the use of a third party to assess remaining design life or the residual value of assets through inspections, materials testing, and a review of the history of maintenance and capital investments. If the facility is not in acceptable condition, the concessionaire may be required to make additional capital investments.

To manage the financial risks associated with handback, some P3 agreements require the concessionaire to establish a handback reserve account that begins to accrue toward the end of an agreement and may be used for unplanned repairs required prior to or shortly after handback of a facility to the public owner. This handback reserve (or replacement letter of credit) typically serves to alleviate uncertainties and unforeseen costs at the end of the concession.

Success Factors

Performance management can address the risks of underperformance by:

  • Designing a contract that aligns private sector incentives with public sector goals and clearly defines performance standards and performance management systems;
  • Assigning a competent, long-term team to govern the contract; and
  • Establishing communication processes that facilitate an engaged and adaptive relationship between the public and private parties.

These elements allow parties to a P3 agreement to effectively manage the risks that occur throughout the term of the contract, allowing the private party to find the best way to meet its contractual obligations while the public agency effectively safeguards the public interest (see Figure 7-1).

Figure 7-1. Elements of Effective Performance Management

Figure 7-1. Elements of Effective Performance Management.

Defined Management Systems and Incentives

A critical factor in successful performance management is an agreement that aligns the interests of the public agency and concessionaire over time by defining effective performance management systems and compensation structures. Flexible, outcome-based performance management systems are essential for P3 agreements because they allow the public agency to ensure that a facility continues to meet policy goals over time.

In designing and managing P3 contracts, however, policymakers must consider the tradeoffs associated with designing a performance-based contract. An effective performance-based contract is one that is sufficiently detailed to ensure that potential bidders understand what their responsibilities will be over the contract term and compete on their capability to meet those responsibilities efficiently, yet flexible enough to allow for changing conditions and needs over time.

Contracts can allow for flexibility by accounting for contingencies that can be anticipated, such as the need for expanded capacity when usage reaches a projected level. However, some changes may be more difficult to predict, such as the emergence of new technologies. To account for such changes, P3 contracts typically define processes to adjust performance specifications, amend contracts and resolve disputes.

Compensating the concessionaire with revenues from highway user fees provides an incentive to concessionaires to retain and attract customers to the facility by maintaining customer satisfaction with levels of service, particularly when many actual and potential customers have viable travel alternatives. When contractor payments are not tied directly to facility usage, as in availability payments, the incentive to provide quality service may instead be tied to provisions in the agreement that allow the public agency to withhold payments or apply default points if performance standards are not met.

Effective Contract Governance

The duration, size, and complexity of P3 agreements make them unlike most contracts public agencies must manage. Public agencies often establish contract management teams to manage P3 contracts. Contract management teams need to have the appropriate skills, experience and authority.

Public agencies can promote effective contract governance by facilitating knowledge sharing between the procurement team and the contract management team, planning for skill and knowledge retention over the period of the contract, and balancing the use of internal capacity and external advisors to ensure retention of that knowledge and skill.

Some public agencies have found that the best way for the contract management team to understand and manage contract provisions is for team members to have played a role in the development and negotiation of the contract. Public agencies can also improve the sustainability of effective contract governance practices by ensuring that decisions and processes are documented and that succession planning takes place.

Engaged Parties

An adaptive contractual arrangement requires active cooperation between the government and the concessionaire throughout the agreement. The hallmarks of active cooperation are a mutual recognition of shared goals, clear lines of communication at both the strategic and the tactical level, and open information sharing. The relationship between the two parties can be expected to evolve over time and the learning curve may be steep for both parties. Mechanisms such as regularly scheduled face-to-face meetings can facilitate the development of an effective relationship. To maintain this relationship, enforcement mechanisms should be used consistently and proportionally.

« PreviousNext »

back to top