Skip to content

Value for Money Assessment for Public-Private Partnerships: A Primer

December 2012

« PreviousNext »

Chapter 5 - Risk Valuation

Accounting for Risk

Under conventional public procurement, while contractors assume significant risks such as labor supply and weather risks, the public agency retains a significant portion of the risks. Yet outside of a VfM analysis, or FHWA's Cost Estimate Review (CER) process for major projects, these risks are not usually quantified, nor are their costs always included in the project costs. For example, when project cost is calculated, the assumption usually is that there will be no delays, and cost increases due to potential delays are not taken into account.

A key component of P3 procurement involves the transfer of certain risks from the public agency procuring the project to the private sector partner. The concept of "transferring risk" requires that the private partner will be responsible for cost overruns or expenses associated with the occurrence of that risk.

Risk transfer can include, among others, construction risk (i.e., risk that the project will not be completed on time or on budget), usage or traffic demand risk (i.e., risk of lower than expected revenues from users of the project) and operation and maintenance (O&M) risk. For example, if the public agency transfers the risk of construction to the private sector partner, then any cost overruns or delays during construction will be borne by the private sector partner, except for certain "compensation events" and "relief events" that may provide the P3 concessionaire with eligibility for additional compensation during construction, e.g., the discovery of unidentified pre-existing hazardous materials, which is generally an owner risk under P3.

Although the identification of project risks can be detailed and varied, typical project risks include technical, political and environmental issues, as well as financial variables such as interest rates and inflation.

The application of risk management techniques can make enormous contributions to the cost effectiveness of a project. They also make VfM easier to conduct and a more reliable decision tool. Risk management begins with identifying the risks in a structured way, including looking at similar projects, using standard risk checklists, interviewing the various stakeholders and end users, and brainstorming or workshop sessions. In P3 projects, a risk register is often prepared in advance, with public officials choosing among four options for each risk element:

  • Retain certain risks;
  • Insure against them;
  • Transfer risk to the private sector partner; or
  • Attempt to mitigate or share the risks.

In choosing among these options, the public agency values each risk, and then evaluates which partner is better able to control, retain or mitigate the risk factors at the lowest cost. This Chapter discusses the processes generally used for risk valuation and allocation.

The Risk Register

Risk analysis is used in the development of a P3 project for a number of reasons:

  • To develop commercial terms that optimize value for money;
  • To calculate risk adjustments as part of value for money assessments;
  • To help determine project contingency amounts; and
  • To identify and monitor mitigation actions (i.e., risk management).

A series of risk workshops is generally conducted to develop a project risk register, also called a "risk matrix," which is used to manage risks throughout all phases of the project. An example of a risk register is presented in Table 5-1. The risk register will usually comprise the following components:

  • Risk Category - type of risk;
  • Risk Topic - identifying the specific risk;
  • Risk Description - including a summary of the potential loss if the risk event occurs;
  • Risk Probability - the likelihood of a risk occurring (e.g., high, moderate, low);
  • Potential Consequence - impact of the risk, should it occur;
  • Allocation of Risk - whether the risk will be transferred, shared or retained; and
  • Treatment Options - actions that can reduce the likelihood or consequences of a particular risk (i.e., risk mitigation).

The risk register may also include the results of Risk Valuation - either a qualitative priority ranking or a quantitative estimate of the potential financial cost or "risk premium" based on the consequence and likelihood of a risk occurrence. This Chapter focuses on quantitative risk valuation methods and on allocation of risk between the public and private sectors. Note that risk may have an upside as well as a downside, e.g., in the case of toll revenue (discussed in Chapter 6).

Quantitative risk analysis is conducted to quantify risks in terms of both cost and time impact. Two alternative levels of quantitative risk analysis may be undertaken:

  • Formula-based analysis using a simple formula to calculate average risk impact using minimum, maximum and most likely cost and schedule impacts;
  • Monte Carlo simulation using specialized software for simulation of expected cost and schedule impacts of each risk to get a range of aggregate risk values along with their probabilities.
Table 5-1: Example of Risk Register
Example 1
Risk Category Right of Way (ROW) / Utilities
Risk Topic ROW Acquisition
Impact Phase Construction
Risk Description The project is to be constructed in an area that is developing rapidly so land prices are highly volatile. As a result, the cost of ROW acquisition could be significantly higher than in the current estimate
Consequence of Risk Higher prices in future would result in increase in project cots
Ability to Transfer Risk It may be possible to transfer this risk in a PPTA contract but a high risk premium may be included by Offerors if they feel unable to control or influence the underlying economic drivers. It may be more cost effective for the Agency to accept the risk and try to mitigate it.
Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011

Formula-Based Quantitative Risk Analysis

With the risk assessment approach used by Virginia DOT, workshop attendees determine specific values for:

  • The probability of occurrence (between 5% and 90%) of each risk; and
  • A Minimum (Min), Maximum (Max) and Most Likely (ML) cost and/or delay impact of the risk in terms of dollars.

The following formula is then used by Virginia DOT to calculate the risk value of each individual risk:

Risk Value = Probability of occurrence x (Min + Max + 4 x ML) / 6

The formula simulates in a very simple way the type of result that might be obtained through use of more sophisticated analyses using probability distributions (discussed in the next section). A contingency amount may be added to account for unknown risks that have not been identified.

Many risk events are likely to have an impact on both cost and schedule. Schedule impact is quantified in units of time, but delays also have a cost associated with them. The direct cost impact of risk events are accounted for under the analysis of cost risk but indirect costs from delays are not. Indirect costs from delays include the added interest costs for financing, and the cost of running site offices, utilities and the time cost of engineers, inspectors and administration staff. Indirect costs will include agency indirect costs (including independent oversight / construction management) and the contractor's indirect costs. The total cost of delay is the sum of the agency indirect costs and the contractor's indirect costs. Additionally, in the case of tolled facility, there will be a loss of revenue that will also need to be accounted for.

Sensitivity analysis may be used to evaluate financial outcomes when critical assumptions are changed. A number of likely scenarios such as low, middle and high cases may be tested. This will provide a more accurate reflection of the potential spread of the total cost to the public agency.

Quantitative Risk Analysis Using Monte Carlo Simulation

A "Monte Carlo" simulation (named after the Monte Carlo Casino where the uncle of one of the creators of the technique gambled away his money) produces a deterministic sample set of likely project outcomes and the probabilities of their occurrence. The sample set is then used to develop distributions and ranges for aggregate cost and schedule impacts. The simulation provides a range of aggregate risk values that the agency may choose from, depending on what confidence threshold is required. This is not possible with a formula-based analysis.

However, Monte Carlo methods require knowledge and training for their successful implementation. Input to Monte Carlo methods also requires the user to know and specify exact probability distribution information, including mean, standard deviation, and distribution shape. The process is as follows:

  1. Quantify probability, cost and schedule impact as per the formula-based analysis described above.
  2. Select a distribution type (also known as an assumption curve) according to the nature of the risk being analyzed. Risk modeling software allows the selection of many different assumption curves.
  3. Perform a Monte Carlo simulation of cost and schedule risk using specialist software such as @RISK or Crystal Ball.

The main output of the simulation is total values for retained, transferred and shared risks. Several types of charts may be generated automatically by the Monte Carlo simulation software. Figures 5-1 and 5-2 show examples of impact distribution graphs. Figure 5-1 displays cumulative risks through the use of a histogram. Figure 5-2 shows an alternative method of displaying cumulative risks with an S-Curve.

The S-Curve allows values to be used based on the confidence level required for the project. In Figure 5-2, the 50th percentile (also known as the P50), mean and 80th percentile (P80) are shown since these are the most commonly reported statistics. The mean represents the average of all generated outputs which is not the same as the P50 unless the distribution is symmetrical. The confidence level selected will depend on the stage of assessment, confidence in cost estimates and complexity of the project. The P80 is widely used by public agencies in risk analysis at earlier stages when project information is less well developed in order to show a confidence level of 80% that risk costs will not exceed the estimated value. It should be noted that the public and private sectors have different preferences with regard to the confidence level. For example, a risk averse public agency may use P90 as its confidence level preference, while private entities may be more comfortable using a P50 confidence level.

Figure 5-1

Figure 5-1: Risk Distribution Histogram (Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011)

Figure 5-2

Figure 5-2 Risk Distribution S-Curve Showing Confidence Levels (Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011)

Risk Allocation

Risks identified in a risk register may be categorized in one of three ways:

  1. Transferrable risks - risks fully transferrable to the private sector.
  2. Retained risks - risks for which the government bears the costs, e.g., the risk of delay in gaining project approvals.
  3. Shared risks -, risks that are shared based on a combination of the above two allocations due to the nature of the risk, e.g., earthquake risk. (If the facility were to be damaged by an earthquake, the private sector may be only partially responsible for repairing the asset, depending on the extent of damage.)

Risk allocation is at the core of P3s, which are structured around the sharing of risks (and rewards) between the public agency and private sector entity. It is the transfer of risks that provides incentives to the private entity to innovate in the approach it takes to delivering a project under a P3. One study of 17 P3 projects found that risk transfer valuations accounted for 60% of the total forecast cost savings under a P3 approach 1.

Transferring too little risk to the private sector would constrain the value for money that could be achieved. Conversely, transferring too much risk (e.g., risk that the private sector is unable to manage) will result in high risk premiums, making the project more costly and driving down the value for money. For example, in the U.K., public agencies retain archeological risks at construction sites since these risks are high and not something that can be managed by the private sector. On the other hand, allowing the private sector to take on part of the geotechnical risk helped bring in VfM for the Port of Miami tunnel project in Florida.

To determine the optimal allocation of risk, an agency compares the public sector's ability to manage each risk to the ability of a potential private partner to do the same. Risks that the private sector is more capable of managing are transferred; risks that the public agency is more capable of managing are retained. Where possible, the party with responsibility for managing the risk will seek to mitigate or avoid that risk. If a risk is difficult to assess or manage, it may be appropriate that it should be shared between the public and private sectors (e.g., earthquake risk). An effective risk allocation should create incentives for the private sector to supply quality and cost-effective services.

While the concept behind optimal risk allocation is clear, the practice of how agencies allocate risks is more of an art than a science. Typically, the public sector will be expected to take on site risks and regulatory risks. The private sector will be expected to take on risks arising from the building, operation, finance, and management of the project. The concessionaire may choose to transfer risks to other private parties by selling equity stakes, holding subcontractors responsible for performance, and/or insuring against certain risks.

Public agencies strive to ensure that this optimal allocation is achieved at the lowest possible cost for taxpayers. Under an optimal risk allocation scheme, risks are generally allocated as shown in Table 5-2, which shows how risk allocation differs for DBFOM projects relative to conventional procurement (Design-Bid-Build) and Design-Build.

Table 5-2. Common Risk Allocation Under Conventional and P3 Procurement
Risk Design Bid Build Design Build Design Build Finance Operate Maintain
Change in Scope Public Public Public
NEPA Approvals Public Public Public
Permits Public Shared Private
Right of Way Public Public Shared
Utilities Public Shared Shared
Design Public Private Private
Ground Conditions Public Public Private
Hazmat Public Public Shared
Construction Private Private Private
QA / QC Public Shared Private
Security Public Public Shared
Final Acceptance Public Private Private
O&M Public Public Private
Financing Public Public Private
Force Majeure Public Shared Shared
Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011

 

Footnotes:

1. Source: Study by Andersen and LSE Enterprises for the Treasury Task Force, UK. Value for Money Drivers in the Private Finance Initiative. January 17, 2000.

« PreviousNext »

back to top