USDOT Federal Highway Administration USDOT Home | FHWA Home | Feedback
Skip to main content

Feedback Forum

Risk Assessment for Public-Private Partnerships: A Primer

September 10, 2012

DRAFT
For review by P3 Evaluation Toolkit Beta-Testers

« Previous | Table of Contents | Next »

Chapter 4 - Risk Valuation Methods

Risk analysis is used in the development of a P3 project for a number of reasons:

  • to develop agreement provisions that optimize value for money (discussed later in this primer);
  • to calculate risk adjustments as part of value for money assessments;
  • to help determine project contingency amounts; and
  • to identify and monitor mitigation actions (i.e., risk management).

For major projects in the U.S., a series of risk workshops is generally conducted to develop a project "risk register," also known as a "risk matrix," which is used to manage risks throughout all phases of the project.  An example of a risk register is presented in Table 4-1. The risk register will usually comprise the following components:

  • Risk Category – type of risk (as discussed in Chapter 3);
  • Risk Topic – identifying the specific risk;
  • Risk Description – including a summary of the potential loss if the risk event occurs;
  • Risk Probability – the likelihood of a risk occurring (e.g., high, moderate, low);
  • Potential Consequence – impact of the risk, should it occur;
  • Allocation of Risk –whether the risk will be transferred to the private sector, shared or retained; and
  • Treatment Options –actions that can reduce the likelihood or consequences of a particular risk (i.e., risk mitigation).

The risk matrix may also include the results of Risk Valuation – either a qualitative priority ranking or a quantitative estimate of the potential financial cost or "risk premium" based on the consequence and likelihood of a risk occurrence.  This Chapter focuses on risk valuation methods.

Qualitative Risk Analysis
Qualitative risk analysis includes methods for prioritizing the identified risks for further action. It assesses the priority of identified risks using their probability of occurrence, the corresponding impact on project objectives if the risks do occur, as well as other factors, such as the time frame and risk tolerance of the project.

A typical qualitative assessment based on the process used by the Virginia DOT is discussed in this section.  Workshop participants are asked to carry out a qualitative risk valuation for each risk using their professional judgment and experience from previous projects. If available, historic data from similar previous projects and details of specific risk events are used to inform the risk assessment. The valuation is carried out by categorizing risks based on their probability of occurring and cost and schedule impact, as noted below.

Table 4-1: Example of Risk Register
Example 1
Risk Category Right of Way (ROW) / Utilities
Risk Topic ROW Acquisition
Impact Phase Construction
Risk Description The project is to be constructed in an area that is developing rapidly so land prices are highly volatile. As a result, the cost of ROW acquisition could be significantly higher than in the current estimate
Consequence of Risk Higher prices in future would result in increase in project cots
Ability to Transfer Risk It may be possible to transfer this risk in a PPTA contract but a high risk premium may be included by Offerors if they feel unable to control or influence the underlying economic drivers. It may be more cost effective for the Agency to accept the risk and try to mitigate it.

Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011

 Probability range: Any risk event that has a probability of occurring of 90% or above would be included in the cost estimate and not on the risk register. One of the following options is selected to define the probability of the risk occurring:

  • Greater than 70% (and below 90%)
  • 40% to 70%
  • 20% to 40%
  • 5% to 20%
  • 0% to 5%

Cost impact: One of the following options is selected to define the cost impact as a percentage of the baseline project cost estimate:

  • Greater than 25%
  • 10% to 25%
  • 3% to 10%
  • 1% to 3%
  • Less than 1%

Schedule impact: One of the following options is selected to define the schedule impact in terms of the period of time that the project would be delayed (or expedited) if a particular risk event were to occur:

  • Greater than 52 weeks
  • 16 to 52 weeks
  • 4 to 16 weeks
  • 1 week to 4 weeks
  • 0 to 1 week

Expected risk impact for cost and schedule are automatically categorized based on Tables 4-2 and 4-3 below. At this stage of assessment, the impact is classified as Very High, High, Medium, Low or Very Low.

The appropriate impact and the color code associated with the risk impact are automatically populated in the risk register once the probability and consequence are selected.

Table 4-2. Qualitative Assessment of Cost Impact of a Risk
  Cost Consequence
Greater than 25% 10% to 25% 3% to 10% 1% to 3% Less than 1%
  Scale   5 4 3 2 1
Probability Greater than 70% 5 Very High High High Medium Low
40% to 70% 4 High High Medium Medium Low
20% to 40% 3 High Medium Medium Low Low
5% to 20% 2 Medium Medium Low Low Low
0% to 5% 1 Low Low Low Low Very Low

Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011

Table 4-3.  Qualitative Assessment of Schedule Impact of a Risk
  Schedule Consequence
Greater than 52 weeks 16 to 52 weeks 4 to 16 weeks 1 week to 4 weeks 0 to 1 week
  Scale   5 4 3 2 1
Probability Greater than 70% 5 Very High High High Medium Low
40% to 70% 4 High High Medium Medium Low
20% to 40% 3 High Medium Medium Low Low
5% to 20% 2 Medium Medium Low Low Low
0% to 5% 1 Low Low Low Low Very Low

Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011

 

Quantitative Risk Analysis
Quantitative risk analysis is performed on risks that have been prioritized by the qualitative risk analysis process as potentially and substantially impacting the project. Quantitative risk analysis is conducted to quantify risks in terms of both cost and time impact. Two alternative levels of quantitative risk analysis may be undertaken:

  • Formula-based analysis using a simple formula to calculate average risk impact using minimum, maximum and most likely cost and schedule impacts;
  • Monte Carlo simulation using specialized software for Monte Carlo simulation of expected cost and schedule impacts to get a range of aggregate risk values along with their probabilities.

A risk workshop is an effective tool for gaining expert input into the quantification of risk probability and potential impact.  Quantitative risk analysis allows an agency to carry out a Value for Money (VfM) assessment during the pre-procurement phase, as well as after bids is received. A quantitative risk analysis may also be helpful in developing key contract terms. 

The approaches used by the Virginia DOT for quantitative analysis are discussed in the following sections.

Formula-Based Quantitative Risk Analysis
With this approach, workshop attendees determine specific values for:

  • The probability of occurrence (between 5% and 90%).
  • A Minimum (Min), Maximum (Max) and Most Likely (ML) cost impact of the risk in terms of dollars.
  • A Minimum (Min), Maximum (Max) and Most Likely (ML) schedule impact of the risk in terms of months.

The following formula is used to calculate the risk value of each individual risk:
Risk Value = Probability x (Min + Max + 4 x ML) / 6

The formula presumably attempts to replicate very simply the result that might be obtained with more sophisticated analyses using simulation (discussed in the next section).  A contingency amount may be added to account for unknown risks.

Many risk events are likely to have an impact on both cost and schedule. Schedule impact is quantified in units of time, but delays also have a cost associated with them. The direct cost impact of risk events are accounted for under the analysis of cost risk but indirect costs from delays are not. Indirect costs from delays include the added interest costs of financing and the cost of running site offices, utilities and the time cost of engineers, inspectors and administration staff.  Indirect costs will include agency indirect costs (including independent oversight / construction management) and the contractor's indirect costs. The total cost of delay is the sum of the agency indirect costs and the contractor's indirect costs.  Additionally, in the case of tolled facility, there will be a loss of revenue that will also need to be accounted for.

In order to get a complete picture of total potential project cost, the agency can calculate the dollar value of schedule impacts by calculating a "per week" value for indirect costs and multiplying this unit rate by the expected schedule impact / delay associated with the risk event. An average rate may be used for risk events during construction and a second value for risk events during operations. Historic data may be used to verify the amounts.

Sensitivity analysis may be used to evaluate financial outcomes when critical assumptions are changed.  This can help decision makers better understand how assumptions shape the expected outcomes of a project and to anticipate the types of conditions that might trigger remedial actions. Sensitivities on key financial and operating conditions may be undertaken through a number of likely scenarios such as low, middle and high cases.  This will provide a more accurate reflection of the potential spread of the total cost to the public agency.   

Quantitative Risk Analysis Using Monte Carlo Simulation
A Monte Carlo simulation (named after the Monte Carlo casino where the uncle of one of the creators of the technique gambled away his money) produces a deterministic sample set of likely project outcomes and the probabilities of their occurrence. The sample set is then used to develop distributions and ranges for aggregate cost and schedule impacts. The simulation provides a range of aggregate risk values that the agency may choose from, depending on what confidence threshold is required. This is not possible with a formula-based analysis.

However, Monte Carlo methods require knowledge and training for their successful implementation. Input to Monte Carlo methods also requires the user to know and specify exact probability distribution information, including mean, standard deviation, and distribution shape. The process is as follows:

  1. Quantify probability, cost and schedule impact as per the formula-based analysis described above.
  2. Select a distribution type (also known as an assumption curve) according to the nature of the risk being analyzed. Risk modeling software allows the selection of many different assumption curves.
  3. Perform a Monte Carlo simulation of cost risk and schedule risk using specialist software such as @RISK or Crystal Ball.

Examples of assumption curves are shown in Figure 4-1.  The curves are probability distributions with different mean values and different standard deviation values. All four distributions have a single high point (the mode) and all have a mean value that may or may not equal the mode. Some of the distributions are symmetrical about the mean while others are not. Selecting an appropriate probability distribution is a matter of which distribution is most like the distribution of actual data. For transportation projects, this is a difficult choice because historical data on unit prices, activity durations, and quantity variations are often difficult to obtain. In cases where data is insufficient to completely define a probability distribution, one must rely on a subjective assessment of the needed input variables.

The main output of the simulation is total values for retained, transferred and shared risks.  Several types of charts may be generated automatically by the Monte Carlo simulation software.  Figures 4-2 and 4-3 present examples of impact distribution graphs.  Figure 4-2 displays cumulative risks through the use of a histogram. Figure 4-3 shows an alternative method of displaying cumulative risks through the use of an S-Curve.

The S-Curve allows values to be used based on the confidence level required for the project. In Figure 4-3, the 50th percentile (also known as the P50), mean and 80th percentile (P80) are shown since these are the most commonly reported statistics. The mean represents the average of all generated outputs which is not the same as the P50 unless the distribution is symmetrical.  The confidence level selected will depend on the stage of assessment, confidence in cost estimates and complexity of the project. The P80 is widely used by public agencies in risk analysis at earlier stages when project information is less well developed in order to show a confidence level of 80% that risk costs will not exceed the estimated value.  It should be noted that the public and private sectors have different preferences with regard to the confidence level.  For example, a risk averse public agency may use P90 as its confidence level preference, while private entities may be more comfortable using a P50 confidence level.

Figure 4-1: Distributions for Risk Analysis

Figure 4-1.  Distributions for Risk Analysis
(Source: NCHRP Report 658, Guidebook on Risk Analysis Tools and Management Practices to Control Transportation Project Costs, 2010)

 Figure 4-2
Figure 4-2:  Risk Distribution Histogram
(Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011)

Figure 4-3

Figure 4-3 Risk Distribution S-Curve Showing Confidence Levels
(Source: Virginia DOT's PPTA Risk Analysis Guidance, September 2011)

 

Submit your comments

 

« Previous | Table of Contents | Next »