Skip to content

Value for Money Assessment for Public-Private Partnerships: A Primer

December 2012

« PreviousNext »

Chapter 8 - Qualitative Assessment of Value for Money

VfM extends beyond the quantitative assessment of project costs and the final outcome is not based solely on this component. VfM may involve substantial qualitative factors that could greatly influence the final decision. Public agencies set goals and objectives for the project (e.g., safety, capacity, reliability, service quality, etc.), which guide them through the planning and procurement stages. Some of the objectives are more qualitative than quantitative. Thus, for those objectives, the qualitative assessment is given equal or more importance relative to the cost factors. The results of the qualitative assessment may be weighted more heavily than the results of the quantitative assessment when the differences between the quantitative results for the conventional option and the P3 option are marginal, or when there is a high level of uncertainty around input variables used in the quantitative assessment and the outputs are highly sensitive to those input variables.

Multiple Criteria Analysis (MCA) may be used to evaluate options based on numerous criteria, including value for money. The MCA approach provides a framework for evaluating potential investment options by evaluating choices against criteria considered critical for the project's success, e.g., the project's goals and objectives.

The outputs of quantitative value for money analysis discussed in Chapter 7 are presented in an MCA process as one of several elements considered in determining the optimal procurement approach for a project.

Other procurement-specific considerations include the ability to address stakeholder interests, meet environmental obligations and ensure a fair and transparent procurement process. Factors that should be addressed in a pre-procurement VfM assessment include any differences in the specifications and service expectations between the PSC and Shadow Bid; whether there are any regulatory or legal restrictions; whether there are any affordability issues; whether there are sufficient opportunities for the private sector to deliver high service quality through innovation; and the robustness of the information used in developing the PSC.

Procurement options that are demonstrated to be clearly inappropriate can be eliminated at an early stage before significant resources are expended on developing detailed quantitative analyses for them. The main output from MCA is a matrix that summarizes how each procurement option being considered scores against the criteria. The comparison between the procurement options usually requires an explicit judgment or "importance weighting" of each criterion. Typically, the results from the MCA are summarized as shown in Figure 8-1.

It is up to decisionmakers to decide which criteria are the most important. Using the matrix in Figure 8-1 as an example, if allowing for innovation is the most important criterion, the Shadow Bid option would be the preferred option, but if user satisfaction is deemed to be the most important, the PSC option might prevail. The most common quantitative criterion is the Net Present Cost (NPC) of the project cash flows under each procurement option.

Figure 8-1 Multi-Criteria Analysis (MCA) Matrix
CriteriaPSCShadow Bid
Competition Good Best
Innovation Limited Best
Service Delivery Outcomes Good Good
User Satisfaction Best Good
Risk Adjusted NPC $763M $698M
(Source: Methodology for Quantitative Procurement Options Analysis, Discussion Paper. Partnerships British Columbia, January 2010)

The VfM qualitative assessment is revisited at every stage of the project. In general, it will seek to identify factors which will influence the project in terms of:

  • Viability - the ability to formulate a sound contract;
  • Performance - the opportunity to encourage risk sharing and innovation; and
  • Achievability - the capability of the public agency and the private sector to deliver the project.

These factors are discussed further below. Examples of additional qualitative

Viability:  The public agency identifies its objectives and desired outcomes and assesses its ability to translate them into output specifications that will form the basis for a P3 contract agreement, including setting the performance standards and the incentives in the payment mechanism. The procuring agency will also determine if there are any regulatory, social equity, efficiency, or accountability issues that may affect the appropriateness of the P3 option to meet the agency's objectives.

Performance: If not already quantified in the quantitative VfM analysis, the qualitative assessment will determine whether the P3 option can provide an improvement over the conventional procurement with regard to risk management and innovation in delivery. This assessment will gauge whether the benefits from the P3 option (e.g., innovation or faster delivery of service) will outweigh the potential costs and disadvantages of pursuing the project as a P3.

Achievability:  This assessment will determine whether the P3 is achievable given a study of the market, the public agency's available resources and experience, and the competitiveness or attractiveness of the proposed project. The procuring agency must determine if there is enough evidence to show that the preferred private consortium is capable of delivering the desired outcome(s) of the project. It will also determine whether there is a sufficient number of private entities interested in the project and the likely strength of competition in the market between private sector entities that submit proposals. It can also include an evaluation of the capabilities of the procuring agency to determine whether or not they have the resources to manage a P3 procurement. The public agency may also want to consider whether its procurement process allows it to effectively execute an efficient procurement plan without causing a significant adverse impact on delivering VfM. Lastly, the procuring agency will determine whether or not the risk transfer it hopes to achieve is feasible based on the market interest and the agency's own limitations.

« PreviousNext »

back to top