P3 agreements can create efficiencies through establishing long-term design-build-finance-operate-maintain (DBFOM) contracts that include outcome-based performance specifications. Outcome-based performance specifications focus on what a facility is intended to achieve rather than prescribing methods and materials for achieving facility goals. The goal of using outcome-based performance specifications is to make service delivery more efficient by allowing the concessionaire to decide how best to achieve the intended results. Defining, measuring, and monitoring outcome-based performance specifications can be challenging and costly, so outcome-based performance measures may be more appropriate for long-term contracts that span multiple phases of a facility’s lifecycle (e.g., design, construction, operations and maintenance) or for large, complex projects where there are potential efficiencies to be gained from innovation. P3 projects typically meet both of these conditions. As a result, public agencies using P3 agreements normally employ performance-based contracts. This shifts the public agency’s primary role in the project from oversight of design and construction to management of a performance-based contract. In this role, the challenge for the public agency is to find ways to monitor and manage contract performance without reclaiming transferred risks or impinging on the efficiencies gained from allowing the concessionaire to choose the best way to meet performance specifications.
Effective performance-based P3 contracts align the concessionaire’s interest with those of the public sector throughout the duration of the agreement. Over the period of a P3 agreement, economic conditions will fluctuate, technology will evolve, policy needs will shift, and the contracted parties are likely to change. Changing economic conditions may lead to unexpected changes in facility demand or financial terms. New technologies may require increased capital investment in a facility. Changes to the contracted parties, through elections on the public side or sale on the private side, may bring new understandings and capabilities to an agreement, but may also lead to financial and technical underperformance. Performance management is a way to maximize project efficiency while at the same time ensuring that the contractor not only meets performance standards at the time of construction, but manages the dynamic risks to performance over the period of the agreement.
This chapter describes the elements of effective P3 performance management. The first section discusses public sector performance management responsibilities and challenges. The second section identifies factors that contribute to effective performance management.
Public sector responsibilities for managing the performance of P3 agreements begin prior to the close of the agreement and last for the duration of the agreement. These responsibilities include:
Outcome-based performance measures can be used to specify standards across three phases of a P3 project: design and construction; operation and maintenance; and handback. Outcome-based performance specifications define indicators of quality and functionality that can be measured over time, such as ride smoothness, material durability, lane availability, incident response times, and work zone safety. These measures are typically specified in the contract, but they are rarely set in stone once the contract is completed. Many P3 agreements include provisions for reviewing and updating performance requirements to meet evolving industry standards.
During the design and construction phase, specifications in P3 contracts are generally drawn from agreed-upon designs, and typically include some prescriptive design standards. These standards should be made clear to proposers in the procurement process. Similar to traditional design-bid-build contracts, the public agency may choose to include procedural specifications in a P3 contract to ensure, for example, that a concessionaire conforms to safety and environmental standards. Additional performance specifications, either procedural or outcome-based, may be drawn from project management and quality assurance plans that the concessionaire may be required to develop. Performance measures may also be used to monitor aspects of construction performance such as work zone safety, minimization of service disruption, and the provision of timely and accurate communication with the public. Finally, public agency may set outcome-based goals, such as congestion-relief and allow private partners to propose alternative designs to help achieve those goals.
During the operations phase, performance measures are typically used for managing capital assets, as well as daily operations and maintenance (these are summarized in Table 4-1). Performance measures related to facility maintenance and operations can be specified as processes in detailed maintenance plans (e.g. daily graffiti patrols) or as outcomes to be achieved (e.g. graffiti cleaned within one hour). For some concessions, the concessionaire may be asked to manage a facility so as to meet mobility goals, which may be specified with measures such as average vehicle speed. Often an agreement will require that the concessionaire develop management plans and systems for meeting outcome-based specifications. The role of the public agency or independent auditor in such cases is to verify that the concessionaire is complying with the specified performance requirements.
Performance measures must also be defined for measuring the condition of the facility at the end of an agreement when it reverts to public control (handback). To assess asset conditions prior to handback, the government can require the concessionaire to develop and follow an asset management plan that describes planned capital investments and systems for monitoring asset condition. Relevant performance measures for asset conditions at handback include residual asset value and remaining design life. In an availability payment P3 project, the agreement may allow the public agency to hold back payments in the latter years of the concession if the facility is judged to be in poor condition. In a toll-based P3 project, the public agency can protect against handback risk by requiring that the concessionaire set aside a portion of facility revenue in a special account that can be used once the contract ends for unanticipated capital expenditures resulting from the conditions of the facility.
|Measure Type||Elements Measured|
|Asset Management||Pavement conditions
Drainage and ventilation systems
Intelligent transportation systems
Public agencies may set performance standards at different phases of project development. The parameters of a project’s design will be set during the environmental review process. Functional specifications will be further set during preliminary design and the development of a procurement package. Specific construction or operations standards should be clearly stated in the procurement documents for bidders to accurately price their bids.
Setting performance standards that are representative of a public agency’s desired levels of service requires careful deliberation during the development of the project. In procuring a P3 agreement, a public agency can set high standards for a facility, but it may have to pay more to the concessionaire to achieve higher standards. If the standard is too high, the project may become financially infeasible. Furthermore, by committing to higher standards for P3 facilities, the public agency may have less funding available in the future to invest in other infrastructure. In setting performance standards, public agencies may want to carefully consider the tradeoffs associated with committing to certain standards and levels of funding. In this regard, P3 agreements are less flexible than traditional methods of publicly maintaining and operating infrastructure, where the public agency retains year-to-year flexibility in the allowable performance standards. Public sector agencies sometimes relax these standards by delaying or reducing investments, or by lowering maintenance standards, in order to conform to financial realities. By specifying performance standards contractually, a P3 agreement lessens the flexibility of public agencies to make such compromises, including those that save money in the short term but are more costly from a life-cycle perspective. On the other hand, during periods when agency budgets are strained, the loss of flexibility to relax performance standards on a P3 facility will increase the pressure on public agencies to reduce spending on non-P3 facilities.
Early private sector involvement in the development of P3 projects, through the use of unsolicited proposals or pre-development agreements, can help to ensure that design elements that may determine the financial feasibility from a private sector perspective are considered. For example, early private sector involvement in the development of the I-35W north of Fort Worth (NTE 3A3B project) allowed the private sector to suggest connectivity improvements that, while they required additional up-front private investment, resulted in much higher revenues over the term of the contract, helping the project achieve greater financial feasibility. Private sector input on performance standards may also be sought at the onset of a procurement process by soliciting alternative technical concepts (ATCs) or changes to project scope, design, or construction criteria. However, in considering ATCs, the public agency must balance the benefits of private sector innovations, with the benefits of maintaining a fair and competitive procurement process.
In setting performance standards, public agencies may look to benchmarks set in other P3 agreements or equivalent facilities. Public agencies that are already applying performance management to State-operated transportation facilities may set goals and measures for P3 projects that are consistent with, or contribute to, the goals and measures the agency has set for the rest of the system. Public agencies may also set policy goals for specific facilities and set performance standards based on those explicit policy goals, such as mobility, safety, environmental stewardship, or economic development. In the case of Florida’s I-595 project, for example, Florida chose to retain control over tolling policy and demand risk by using an availability payment compensation model. This control allows Florida to modify tolling levels to help achieve a policy goal of corridor mobility optimization. Mobility improvements may also be achieved in toll-based compensation agreements by benchmarking toll escalation to traffic flows, or through contract mechanisms that allow for the public agency to modify toll policies as long as the private partner is financially compensated for lost revenues.
Public agencies must also consider that desired performance standards are likely to change over time. As a public agency’s own standards change due to changing conditions or policy goals, they will likely expect the concessionaire to conform to those changes. For example, future land development may necessitate changes in environmental standards. The concessionaire is typically willing to take on the risks associated with non-discriminatory changes in law to a certain degree, and such an agreement can be written into the contract. However, the concessionaire will typically ask for some assurance that the standards won’t be changed so quickly or completely that it becomes financially onerous to meet new standards. As a result, some P3 contracts specify a limit to the number or percentage of changes to standards that can be made on an annual basis or include procedures for the private partner to be compensated for unexpected costs or lost revenues resulting from changes.
There is a natural tension between flexibility and accountability in performance management. If a standard is too flexible, the public sector risks not obtaining the highest possible level of performance from a concession. If a standard is inflexible, it may not adapt to changing technology needs. For example, in one agreement, the concessionaire’s performance was based on the operations of its call center for its toll payment accounts. However, most users preferred to use a web interface to communicate with the concessionaire. The contract performance standard failed to anticipate technology changes or to use a more flexible measure of success, such as customer satisfaction.
The government is responsible for monitoring the performance of the concessionaire. P3 contracts will typically establish roles and responsibilities (see Table 4-2) and monitoring procedures. Performance monitoring procedures can include self-reporting procedures, independent audits, regular meetings and reports, and the use of intelligent transportation systems that automate data collection and reporting processes.
|Concessionaire||Develop management plans and procedures
Collect monitoring data
Develop status reports
|Government||Set performance standards
Review plans, procedures, and status reports
Perform audits and inspections
Assess penalties and awards
|3rd Party||Perform independent audits and inspections
|Shared||Perform daily communication and problem solving
Conduct regular face to face meetings
Complete annual performance reviews
Many P3 contracts require the concessionaire to develop project plans that explain how the concessionaire will monitor and report the project’s performance. Project plans may include: asset management plans, operations and maintenance manuals, quality and performance management plans, and communications and customer care plans. The concessionaire then assumes responsibility for quality management and performance reporting. The government approves the concessionaire’s project plans and validates performance reports. While this model can conserve public resources, there must be significant consequences if the concessionaire falsifies or fails to provide
P3 contracts may also establish processes whereby an independent engineer or certified auditor is responsible for spot checks and audits of the facility. The independent engineer is generally used to assure the design and construction complies with the concession agreement technical requirements, but they can be used through all phases of project. As a rule of thumb, independent engineers are typically employed to monitor high risk areas, whereas self-reporting is used for areas of lower risk. The cost of the independent engineer’s services may be borne by the concessionaire or it may be split between the public agency and the concessionaire. By sharing the cost of the independent engineer’s services, the public agency may reduce the risk that conflicts of interest arise. In Australia, some P3 agreements have followed a reimbursable payment structure for independent engineers where the costs are shared up to an established threshold; beyond the threshold, the costs are borne by the concessionaire. The logic of this payment model is that costs beyond a certain threshold are likely the result of the need for increased oversight due to poor compliance by concessionaire.
Additional oversight or monitoring of the facility conducted by the government can be coordinated with the independent engineer. An independent engineer may also be hired by the lenders as a technical adviser, to ensure that the concessionaire will be able to meet performance targets and comply with contract specifications, or in the case of availability payments, to receive payments accordingly. While this engineer will represent the lenders’ interests, in many cases the public agency’s interest and the lenders’ interest are aligned in terms of ensuring that the concessionaire meets performance targets.
Most P3 monitoring regimes include regular monthly or quarterly meetings between the government and the concessionaire for which performance reports are prepared and reviewed. Such monthly performance reports typically are also required for the monthly payment from the government to the concessionaire.
P3 projects are increasingly using technology such as closed-circuit television and electronic tolling systems to monitor and report operations performance. Intelligent transportation systems (ITS) can be used to monitor toll operations, incident response and reporting, and traffic flows. Florida DOT (FDOT), for example, uses video monitoring systems and other ITS features on I-595 to monitor traffic conditions, incidents, and toll operations in real-time and to generate monthly performance reports.
Setting Penalty Provisions
Penalty provisions must be carefully crafted to achieve desired performance. For the London transit operations and maintenance concession, a provision penalized the concessionaire for the first 48 hours of a service interruption. The provision was intended to encourage the concessionaire to fix all service problems within 48 hours, but it had the opposite of the intended effect. Once a problem had gone unfixed for more than 48 hours, resolving it was no longer as high a priority, because the fines would not increase over time.
Most P3 agreements prescribe processes for penalizing noncompliance, but rewards for superior performance are rarely used. The government is responsible for tracking concessionaire performance and penalizing the concessionaire when contractual obligations are not met. Before penalties are assessed, P3 agreements typically prescribe a series of actions that must be taken to notify the concessionaire of the issue and a period of time to correct the noncompliance issue after it is detected. Penalties typically consist of payment reductions or retentions or noncompliance or default points. Once noncompliance or default points reach a specified level, they can result in increased oversight, work by the owner at the contractor’s expense, suspension of work, or termination of the contract. For I-595, for example, if the concessionaire compiles 100 non-compliance points in a 3-year period, FDOT may increase levels of oversight. If noncompliance issues are not rectified in a timely manner, FDOT may reduce payments or even step in to fix the problem itself at the concessionaire’s expense. Capital Beltway (I-495) employs a similar system of performance points (see Table 4-3).
Contractors may prefer default points to financial penalties because they may fear the public agency will abuse financial penalties to meet short-term financial objectives. Furthermore, if the cause of underperformance is lack of finances, fines may inhibit the concessionaire’s ability to correct the problem. On the other hand, if financial penalties are set too low, the concessionaire may lack sufficient incentive to take corrective action or may perceive fines are simply part of the cost of doing business. Default points incentivize performance without money changing hands by raising the risk of default. This in turn may raise the concerns to private lenders, who may then pressure the concessionaire to correct the issue.
|Heading||Subheading||Breach or Failure||Cure Period||Max Default Points|
|Communication||Public information||Issues factually incorrect information to the public||None||5|
|Operation||Work zone management||Fails to meet work zone safety requirements||60 minutes||5|
|Inspection||Quality of inspection||Fails to identify material defects in inspection reports, maintenance plans, or current work||None||5|
From: Key Performance Indicators in Public-Private Partnerships, 2011, FHWA
Renegotiation and Default
In some cases, contract terms have to be renegotiated to ensure that incentives remain aligned, performance standards remain achievable, or contract disputes are resolved. In the United States, several P3 agreements have been renegotiated, including: Dulles Greenway, Orange County SR91 Express Lanes, South Bay Expressway, and Pocahontas Parkway. Many of these renegotiations took place under the threat of default and resulted in refinancing with losses to equity partners and private lenders and bondholders. In several cases, ownership eventually reverted to the public sector. Renegotiations occurred because of lower than expected demand or because of disagreements over specific contract provisions. The Dulles Greenway project, for example, went into default in 1999, four years after it opened, after traffic levels did not meet expectations. The concession was refinanced, resulting in losses to the original equity lenders and bondholders, and the duration of the agreement was extended to 60 years to allow the sale of the concession to new owners. In the case of SR91, disagreements arose between the concessionaire and Caltrans over a clause in the agreement that prohibited expansion improvements of competing facilities within 1.5 miles of the SR91 right of way without the concessionaire’s consent. To resolve the dispute and address congestion issues in the area, the Orange County Transportation Authority eventually purchased the concession from the concessionaire.
P3 contracts typically specify dispute resolution processes to reduce the risk of legal conflict over technical issues or differences in contract interpretation. Alternative dispute resolution processes may include mediation and third party arbitration following a period of time allowed for both parties to make good faith efforts to resolve the dispute themselves. Arbitration may be conducted by an agreed-upon expert or by a designated board with members selected by both the government and the concessionaire. In particularly large projects, a permanent, independent dispute resolution office may be established to quickly resolve any contract dispute.
P3 contracts typically specify alternative dispute resolution processes for various reasons including the speed advantage of these extrajudicial processes combined with the time sensitivity of many P3 projects. Professional arbitrators or mediators can be selected for their industry knowledge and will seek resolution through a collaborative non-adversarial process. Another consideration favoring alternative dispute resolution procedures on P3 contracts is that the public agency may not be sued, even when in breach of the contract. This "sovereign immunity" can become an obstacle for the private sector to financing a project unless the agency waives this immunity in favor of contractually-defined alternative dispute resolution mechanisms.
Prior to mediation or arbitration, dispute resolution processes often define tiered systems of problem identification and resolution through negotiation to encourage problems to be resolved at the lowest levels. For example, on the Capital Beltway project, the contract specifies a process whereby the parties to the agreement are given a set time period to seek ways to resolve their dispute before it is elevated to their respective managers. In elevating the dispute, the parties must write a memo to their supervisor, summarizing the nature of the dispute and the steps they attempted to take to resolve the issue. This can serve as an incentive for parties to seek a speedy resolution to disputes.
In the worst case scenario, underperformance can lead to contract failure. Contract failure occurs when one party is unable or refuses to comply with a contract or the parties to an agreement are unable to resolve disputes concerning the meaning of contract specifications. Contract failure can result in the need to amend or renegotiate a contract, resolve disputes in courts, replace parties to an agreement, or terminate an agreement. These events may ultimately lead to higher costs for the public sector.
Some P3 agreements set conditional rights or obligations to expand the facility. Capacity expansion can be an option for the concessionaire or a requirement of the contract that is set to a trigger mechanism such as revenue, usage levels, or operating speeds on the existing facility. Capacity triggers can pose a significant financial risk to a concessionaire. If a capacity expansion is triggered towards the end of an agreement, it can result in a significant cost to the concessionaire for which the concessionaire cannot hope to make up from gained revenues. To mitigate this risk, the contract may stipulate adequate compensation to the concessionaire if capacity improvements are required by the last years of the concession or may allow for other means besides capacity expansion to retain levels of service on the facility by improving operations or managing demand through pricing or other means. In some cases, scheduled toll increases agreed to by the public agency may forestall capacity demands by dampening facility demand. In other cases, capacity expansion can be seen as a desirable way for the concessionaire to increase facility revenues, and the concessionaire may bid for the right of first refusal to expand the facility.
P3 contracts generally specify the condition in which the facility should be at the end of the contract term. The condition of a facility at handback depends on the maintenance and operation procedures employed throughout the lifecycle of the facility, so the concessionaire is typically required to develop a capital replacement or asset management plan for equipment, systems, and assets. In addition, the concessionaire may be required to develop a plan that specifies the processes for turning operation of the facility to another party at the conclusion of the contract. Review of handback conditions may involve the use of a third party to assess remaining design life or the residual value of assets through inspections, materials testing, and a review of the history of maintenance and capital investments. If the facility is not in acceptable condition, the concessionaire may be required to make additional capital investments. To manage the financial risks associated with handback, some P3 agreements require the concessionaire to establish a handback reserve account that begins to accrue toward the end of an agreement and may be used for unplanned repairs required prior to or shortly after handback of a facility to the public owner. This handback reserve or replacement letters of credit typically serve to alleviate uncertainties and unforeseen costs at the end of the concession, covering those repairs that may be required prior to reversion of the project.
Performance management can address the risks of underperformance by:
These elements allow parties to a P3 agreement to effectively manage the risks that occur throughout the term of the contract, allowing the private party to find the best way to meet its contractual obligations while the public agency effectively safeguards the public interest (see Figure 4-1).
Figure 4-1. Elements of Effective Performance Management
A critical factor in successful performance management is an agreement that aligns the interests of the public agency and concessionaire over time by defining effective performance management systems and compensation structures. Flexible, outcome-based performance management systems are essential for P3 agreements because they allow the public agency to ensure that a facility continues to meet policy goals over time. In designing and managing P3 contracts, however, policymakers must consider the tradeoffs associated with designing a performance-based contract. An effective performance-based contract is one that is sufficiently detailed to ensure that potential bidders understand what their responsibilities will be over the term and compete on their capability to meet those responsibilities efficiently, yet flexible enough to allow for changing conditions and needs over time. Contracts can allow for flexibility by accounting for contingencies that can be anticipated, such as the need for expanded capacity when usage reaches a projected level. However, some changes may be more difficult to predict, such as the emergence of new technologies. To account for such changes, P3 contracts typically define processes to adjust performance specifications, amend contracts and resolve disputes.
The compensation structure of an agreement typically provides the primary incentive for the private partner to meet performance standards. Incentives may vary depending on whether a concessionaire’s compensation is based on user fees or availability payments. Each arrangement has pros and cons. Compensating the concessionaire with revenues from highway user fees reinforces this incentive because retaining and attracting customers to the facility depends partly on customer satisfaction with levels of service (particularly when many actual and potential customers have viable travel alternatives). Users will make choices to use the facility based on the convenience of the route and the quality of the ride, although the user may have few alternatives. When contractor payments are not tied directly to facility usage, as in availability payments, the incentive to provide quality service may not be tied directly to the choices of potential facility users. Concessionaire incentives in an availability payment structure are instead tied to provisions in the agreement that allow the public agency to withhold payments or apply default points if performance standards are not met.
Longer contract terms can also be used to strengthen the incentives of the private partner to perform, at least early on in the contract, because a failure to perform could lead to a loss in long-term revenue. However, with longer terms, it is more likely that conditions and needs will change and the contract will require amendment or renegotiation. Spanish P3s use a "rebalancing" model that tries to reframe the economic balance of the concession over time. This model has not been used in the United States or the United Kingdom, partially because such a rebalancing would likely become a legal dispute.
The duration, size, and complexity of P3 agreements make them unlike most contracts public agencies must manage. Public agencies often establish contract management teams to manage P3 contracts. Contract management teams need to have the skills, experience, and authority to understand contract provisions, monitor performance, and manage changes and disputes. Public agencies can promote effective contract governance by facilitating knowledge sharing between the procurement teams and the contract management team, planning for skill and knowledge retention over the period of the contract, and balancing the use of internal capacity and external advisors in developing and retaining that knowledge and skill. Some public agencies have found that the best way for the contract management team to understand and manage contract provisions is for team members to have played a role in the development and negotiation of the contract. Public agencies can also improve the sustainability of effective contract governance practices by ensuring that decisions and processes are documented and that succession planning takes place. These skills are discussed in Chapter 5, Organizational Capacity.
An adaptive contractual arrangement requires active cooperation between the government and the concessionaire throughout the agreement. The hallmarks of active cooperation are a mutual recognition of shared goals, clear lines of communication at both the strategic and the tactical level, and open information sharing. The relationship between the two parties can be expected to evolve over time and the learning curve may be steep for both parties. Mechanisms such as regularly scheduled face-to-face meetings can facilitate the development of an effective relationship. To maintain this relationship, enforcement mechanisms should be used consistently and proportionally.
In a P3 agreement, the public agency’s role shifts from that of facility operator and overseer to that of performance-based contract manager. Public agencies must be deliberative and judicious in negotiating and managing this new role. The establishment of a well-defined performance management regime, a strong contract management team, and an open and engaged relationship with the concessionaire can be key to the long-term success of a P3 project. While external risks, such as economic downturns, can always threaten the performance of a P3, performance management systems help agencies and concessionaires manage the risks that they can control and understand and adapt quickly when conditions change. This can help to ensure contract performance while avoiding the potentially costly consequences of contract refinancing, renegotiation, or default.