September 11, 2012
For review by P3 Evaluation Toolkit Beta-Testers
Under traditional public procurement, the public agency retains most of the risks. Even construction risks are often borne by the public agency and are minimal for the contractor. Yet outside of a VfM analysis, these risks are not usually quantified, nor are their costs always included in the project costs. For example, when project cost is calculated, the assumption usually is that there will be no delays, and cost increases due to potential delays are not taken into account. Delay costs may also include the impact of the delay on the traveling public.
A key component of P3 procurement involves the transfer of certain risks from the public agency procuring the project to the private sector partner. The concept of "transferring risk" requires that the private partner will be responsible for cost overruns or expenses associated with the occurrence of that risk.
Risk transfer can include, among others, construction risk (i.e., risk that the project will not be completed on time or on budget), usage or traffic demand risk (i.e., risk of lower than expected revenues from users of the project) and operation and maintenance (O&M) risk. For example, if the public agency transfers the risk of construction to the private sector partner, then any cost overruns or delays during construction will be borne by the private sector partner.
Although the identification of project risks can be detailed and varied, typical project risks include technical, political and environmental issues, as well as financial variables such as interest rates and inflation.
The application of risk management techniques can make enormous contributions to the cost effectiveness of a project. They also make VfM easier to conduct and a more reliable decision tool. Risk management begins with identifying the risks in a structured way, including looking at similar projects, using standard risk checklists, interviewing the various stakeholders and end users, and brainstorming or workshop sessions. In P3 projects, a risk register is often prepared in advance, with public officials choosing among four options for each risk element:
In choosing among these options, the public agency values each risk, and then evaluates which partner is better able to control, retain or mitigate the risk factors at the lowest cost. This Chapter discusses the processes generally used for risk valuation and allocation.
Risk analysis is used in the development of a P3 project for a number of reasons:
A series of risk workshops is generally conducted to develop a project risk register, also called a "risk matrix," which is used to manage risks throughout all phases of the project. An example of a risk register is presented in Table 5-1. The risk register will usually comprise the following components:
The risk register may also include the results of Risk Valuation – either a qualitative priority ranking or a quantitative estimate of the potential financial cost or "risk premium" based on the consequence and likelihood of a risk occurrence. This Chapter focuses on quantitative risk valuation methods and on allocation of risk between the public and private sectors. Note that risk may have an upside as well as a downside, e.g., in the case of toll revenue (discussed in Chapter 6).
Quantitative risk analysis is conducted to quantify risks in terms of both cost and time impact. Two alternative levels of quantitative risk analysis may be undertaken:
|Risk Category||Right of Way (ROW) / Utilities|
|Risk Topic||ROW Acquisition|
|Risk Description||The project is to be constructed in an area that is developing rapidly so land prices are highly volatile. As a result, the cost of ROW acquisition could be significantly higher than in the current estimate|
|Consequence of Risk||Higher prices in future would result in increase in project cots|
|Ability to Transfer Risk||It may be possible to transfer this risk in a PPTA contract but a high risk premium may be included by Offerors if they feel unable to control or influence the underlying economic drivers. It may be more cost effective for the Agency to accept the risk and try to mitigate it.|
|Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011|
With the risk assessment approach used by Virginia DOT, workshop attendees determine specific values for:
The following formula is then used by Virginia DOT to calculate the risk value of each individual risk:
Risk Value = Probability of occurrence x (Min + Max + 4 x ML) / 6
The formula simulates in a very simple way the type of result that might be obtained through use of more sophisticated analyses using probability distributions (discussed in the next section). A contingency amount may be added to account for unknown risks that have not been identified.
Many risk events are likely to have an impact on both cost and schedule. Schedule impact is quantified in units of time, but delays also have a cost associated with them. The direct cost impact of risk events are accounted for under the analysis of cost risk but indirect costs from delays are not. Indirect costs from delays include the added interest costs for financing, and the cost of running site offices, utilities and the time cost of engineers, inspectors and administration staff. Indirect costs will include agency indirect costs (including independent oversight / construction management) and the contractor's indirect costs. The total cost of delay is the sum of the agency indirect costs and the contractor's indirect costs. Additionally, in the case of tolled facility, there will be a loss of revenue that will also need to be accounted for.
Sensitivity analysis may be used to evaluate financial outcomes when critical assumptions are changed. A number of likely scenarios such as low, middle and high cases may be tested. This will provide a more accurate reflection of the potential spread of the total cost to the public agency.
A "Monte Carlo" simulation (named after the Monte Carlo Casino where the uncle of one of the creators of the technique gambled away his money) produces a deterministic sample set of likely project outcomes and the probabilities of their occurrence. The sample set is then used to develop distributions and ranges for aggregate cost and schedule impacts. The simulation provides a range of aggregate risk values that the agency may choose from, depending on what confidence threshold is required. This is not possible with a formula-based analysis.
However, Monte Carlo methods require knowledge and training for their successful implementation. Input to Monte Carlo methods also requires the user to know and specify exact probability distribution information, including mean, standard deviation, and distribution shape. The process is as follows:
The main output of the simulation is total values for retained, transferred and shared risks. Several types of charts may be generated automatically by the Monte Carlo simulation software. Figures 5-1 and 5-2 show examples of impact distribution graphs. Figure 5-1 displays cumulative risks through the use of a histogram. Figure 5-2 shows an alternative method of displaying cumulative risks with an S-Curve.
The S-Curve allows values to be used based on the confidence level required for the project. In Figure 5-2, the 50th percentile (also known as the P50), mean and 80th percentile (P80) are shown since these are the most commonly reported statistics. The mean represents the average of all generated outputs which is not the same as the P50 unless the distribution is symmetrical. The confidence level selected will depend on the stage of assessment, confidence in cost estimates and complexity of the project. The P80 is widely used by public agencies in risk analysis at earlier stages when project information is less well developed in order to show a confidence level of 80% that risk costs will not exceed the estimated value. It should be noted that the public and private sectors have different preferences with regard to the confidence level. For example, a risk averse public agency may use P90 as its confidence level preference, while private entities may be more comfortable using a P50 confidence level.
Figure 5-1: Risk Distribution Histogram (Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011)
Figure 5-2 Risk Distribution S-Curve Showing Confidence Levels (Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011)
Risks identified in a risk register may be categorized in one of three ways:
Risk allocation is at the core of P3s, which are structured around the sharing of risks (and rewards) between the public agency and private sector entity. It is the transfer of risks that provides incentives to the private entity to innovate in the approach it takes to delivering a project under a P3. One study of 17 P3 projects found that risk transfer valuations accounted for 60% of the total forecast cost savings under a P3 approach1.
Transferring too little risk to the private sector would constrain the value for money that could be achieved. Conversely, transferring too much risk (e.g., risk that the private sector is unable to manage) will result in high risk premiums, making the project more costly and driving down the value for money. For example, in the U.K., public agencies retain archeological risks at construction sites since these risks are high and not something that can be managed by the private sector. On the other hand, allowing the private sector to take on part of the geotechnical risk helped bring in VfM for the Port of Miami tunnel project in Florida.
To determine the optimal allocation of risk, an agency compares the public sector's ability to manage each risk to the ability of a potential private partner to do the same. Risks that the private sector is more capable of managing are transferred; risks that the public agency is more capable of managing are retained. Where possible, the party with responsibility for managing the risk will seek to mitigate or avoid that risk. If a risk is difficult to assess or manage, it may be appropriate that it should be shared between the public and private sectors (e.g., earthquake risk). An effective risk allocation should create incentives for the private sector to supply quality and cost-effective services.
While the concept behind optimal risk allocation is clear, the practice of how agencies allocate risks is more of an art than a science. Typically, the public sector will be expected to take on site risks and regulatory risks. The private sector will be expected to take on risks arising from the building, operation, finance, and management of the project. The concessionaire may choose to transfer risks to other private parties by selling equity stakes, holding subcontractors responsible for performance, and/or insuring against certain risks.
Public agencies strive to ensure that this optimal allocation is achieved at the lowest possible cost for taxpayers. Under an optimal risk allocation scheme, risks are generally allocated as shown in Table 5-2, which shows how risk allocation differs for DBFOM projects relative to traditional procurement (Design-Bid-Build) and Design-Build.
|Risk||Design Bid Build||Design Build||Design Build Finance Operate Maintain|
|Change in Scope||Public||Public||Public|
|Right of Way||Public||Public||Shared|
|QA / QC||Public||Shared||Private|
|Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011|
1 Source: Study by Andersen and LSE Enterprises for the Treasury Task Force, UK. Value for Money Drivers in the Private Finance Initiative. January 17, 2000.