U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
FacebookYouTubeTwitterFlickrLinkedIn

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
REPORT
This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-17-072    Date:  February 2018
Publication Number: FHWA-HRT-17-072
Date: February 2018

 

FHWA Research And Technology Evaluation: Public-Private Partnership Capacity Building Program

PDF Version (1.37 MB)

PDF files can be viewed with the Acrobat® Reader®

 

FOREWORD

The Federal Highway Administration (FHWA) has initiated an effort to evaluate the Research and Technology (R&T) development program and communicate the full range of benefits of their program. The R&T evaluation program helps FHWA assess how effectively it is meeting its goals and objectives and provides useful data to inform future project selections.

This report examines how FHWA’s investment in the Public-Private Partnership (P3) Capacity Building Program helps build knowledge and technical expertise and improve decisionmaking for this innovative project delivery method.

This report should be of interest to researchers, practitioners, and decisionmakers involved in the P3 implementation process.

Monique Evans,
Acting Associate Administrator
Research, Development, and Technology

Notice

This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest of information exchange. The U.S. Government assumes no liability for the use of the information contained in this document.

The U.S. Government does not endorse products or manufacturers. Trademarks or manufacturers' names appear in this report only because they are considered essential to the objective of the document.

Quality Assurance Statement

The Federal Highway Administration (FHWA) provides high-quality information to serve Government, industry, and the public in a manner that promotes public understanding. Standards and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its information. FHWA periodically reviews quality issues and adjusts its programs and processes to ensure continuous quality improvement.

 

Technical Report Documentation Page

1. Report No.

FHWA-HRT-17-072

2. Government Accession No.

 

3 Recipient's Catalog No.

 

4. Title and Subtitle

FHWA Research and Technology Evaluation: Public-Private Partnership Capacity Building Program

5. Report Date

February 2018

6. Performing Organization Code

OST-R V-321

7. Author(s)

Lora Chajka-Cadin and Chris Calley

8. Performing Organization Report No.

 

9. Performing Organization Name and Address

Volpe National Transportation Systems Center
55 Broadway
Cambridge, MA 02142

10. Work Unit No. (TRAIS)

 

11. Contract or Grant No.

 

12. Sponsoring Agency Name and Address

Office of Corporate Research, Technology, and Innovation Management
Federal Highway Administration
6300 Georgetown Pike
McLean, VA 22101-2296

13. Type of Report and Period Covered

Covered
Final Report; January 2013–June 2016

14. Sponsoring Agency Code

 

15. Supplementary Notes

John Moulden is the Research and Technology Program Manager and the Contracting Officer’s Representative for Contract DTFH for this program summary.

16. Abstract

This report details the evaluation of the Federal Highway Administration’s Office of Innovative Program Delivery Public-Private Partnership (P3) Capacity Building Program (P3 Program). The evaluators focused on the P3 Program’s P3 Toolkit as an educational resource for transportation practitioners that aimed to build knowledge and technical expertise to inform decisions related to the consideration, development, and implementation of P3 projects. The study concluded that, while less experienced practitioners found value in P3 Program materials, the P3 Program had limited impact on the implementation of new P3 legislation and adoption of P3s. The P3 Toolkit itself is still recognized by practitioners as a valuable resource and is noted by various sources as one of the most comprehensive resources on P3s.

17. Key Words

Public-Private Partnership, evaluation, capacity building, decisionmaking

18. Distribution Statement

No restrictions. This document is available to the public through the National Technical Information Service, Springfield, VA 22161.
http://www.ntis.gov

19. Security Classification
(of this report)

Unclassified

20. Security Classification
(of this page)

Unclassified

21. No. of Pages

98

22. Price

 

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

SI* (Modern Metric) Conversion Factors

Table of Contents

Executive Summary

1. Introduction

2. Evaluation Design

3. Findings on P3 Program Usage

4. Evaluation Findings

5. Recommendations

6. Conclusions

Appendix A. In-Depth Interviews

Appendix B. Online Survey—FHWA Division Office Staff

Acknowledgments

References

List of Figures

List of Tables

List of Abbreviations

Abbreviation

Term

AASHTO

American Association of State Highway and Transportation Officials

BATIC

The Build America Transportation Investment Center

FAST Act

Fixing America’s Surface Transportation Act

FAQs

frequently asked questions

FHWA

Federal Highway Administration

IPD

Office of Innovative Program Delivery

NCSL

National Conference of State Legislators

P3

public-private partnership

P3 Program

Public-Private Partnership Capacity Building Program

PDF

portable document format

PSC

public sector comparator

PWF

public works financing

R&T

FHWA Research and Technology

RD&T

FHWA Office of Research, Development, and Technology

RFP

request for proposal

TRB

Transportation Research Board of the National Academies

VFM

Value for Money

USDOT

U.S. Department of Transportation

Executive Summary

Purpose of the Evaluation

This report is part of the larger compilation of the Federal Highway Administration’s (FHWA) effort to evaluate their Research and Technology (R&T) Development Program. For each evaluation, the FHWA’s R&T Evaluation Program evaluation team (referred to as the “evaluation team” throughout this report) is made up of non-FHWA, third-party evaluators not involved in the research programs and projects being evaluated. This specific report is the evaluation of the Office of Innovative Program Delivery’s (IPD) Public-Private Partnership (P3) Capacity Building Program (P3 Program). The purpose of the evaluation is to understand how the program helps transportation practitioners build knowledge and technical expertise on P3s to inform decisions related to the consideration, development, and implementation of P3 projects.

Program Description

The P3 Program is part of the newly formed Center for Innovative Finance Support (formerly the Strategic Delivery Team in IPD). The center provides tools, expertise, and support for financing to help the transportation community explore and implement innovative financing strategies to deliver costly and complex infrastructure projects.(2)The main component of the P3 Program is the P3 Toolkit. The P3 Toolkit is an educational resource consisting of analytical tools and guidance documents that assist those exploring, developing, and implementing P3s. The P3 Toolkit addresses Federal requirements related to P3s and covers four key phases in P3 development and implementation: legislation and policy, planning and evaluation, procurement, and monitoring and oversight.(3) The Toolkit forms the base of the P3 Program, which also includes a curriculum of training courses and webinars.

Methodology

The evaluation team developed four hypotheses to assess how the activities and outputs of the P3 Program support short-term outcomes and long-term impacts. For each hypothesis, evaluation questions and key performance measures were developed:

  1. The FHWA P3 Program provides the most complete set of information resources to assist transportation practitioners in all phases of the P3 implementation process.
  2. The FHWA P3 Program helps increase legislative and policy support for transportation P3s in State and local governments.
  3. The FHWA P3 Program has led to more informed decisions on consideration and use (approval) of P3s for appropriate transportation projects.
  4. The FHWA P3 Program improves the decisionmaking capabilities of transportation practitioners in the areas of P3 development, procurement, and oversight.

Both qualitative and quantitative methods were used to inform the evaluation. The evaluation team used a literature search and document reviews to gain an initial understanding of the P3 Program, its users, and its outputs. In-depth interviews with FHWA staff, State and local transportation agencies, legislators and policymakers, and P3 advisors provided a deeper understanding of P3 information needs and provided examples of program use. Information from the qualitative research was used to develop an online survey that captured data on P3 information needs and program usage by FHWA Division Office staff. P3 Program resource usage data, including outreach event attendance, website usage, and document downloads, provided additional quantitative data for the evaluation.[1]

Findings

The findings section presents an overview of P3 Program usage and then highlights findings for the four evaluation hypotheses.

P3 Program Users and Resource Use

An analysis of website usage shows that the P3 Program serves hundreds of diverse stakeholders in the transportation community. The P3 Toolkit website reaches 1,500 to 2,500 users per quarter, spiking when new documents, tools, or webinars are introduced. Analysis of other usage data collected from P3 Program outreach registrations and document download forms indicates that there are multiple user groups accessing the program. The most active users fall into three groups: State and local transportation agencies; FHWA Division Office staff; and P3 advisory firms who provide legal, financial, or engineering support. Smaller groups of academics, interest groups, and international users round out the audience.

Aggregate statistics were available to assess usage of the P3 Toolkit website. Since its inception, the website has had two main landing pages, the P3 Toolkit homepage and the P3-VALUE Analytical Tool homepage.(3,4) Although both pages received more than 1,000 views per quarter in early years, the P3-VALUE Analytical Tool homepage has recently become more popular, while P3 Toolkit homepage views have declined. The homepages for other popular resources have also seen views decline, although the fact sheet and publications’ homepages are still widely used.(3)

P3 Program usage data that could be linked to groups show that each has a distinct resource use as follows:[2]

An online survey along with program usage data provide additional detail on the resource usage of FHWA Division Office staff. These users tend to attend webinars and also view fact sheets, primers, and other documents on the P3 Toolkit website. They are also likely to attend P3 outreach events.

Finding: The P3 Program currently has only a limited impact on P3 legislation and policy.

Despite increases in P3 legislation and activity in recent years, P3 Program usage data and in-depth interviews suggest that there is not a strong connection between the P3 Program, legislators, and policymakers. Few involved in legislation or policy directly accessed P3 Program resources. This does not mean, however, that information from the P3 Program does not reach this audience. In many cases, the link between the P3 Program and those in legislative or policymaking positions is indirect. That is, the program provides knowledge to Federal, State, and local transportation agencies and P3 advisors who bring the information to decisionmakers. The P3 Program also provides content to other organizations that develop P3 materials specifically for legislators and policymakers.

Finding: There is not enough evidence in this evaluation to conclude that the P3 Program influences P3 consideration decisions, but evidence does indicate that program materials positively influence project approval decisions.

P3 Consideration Decisions

Neither P3 Program usage data nor in-depth interviews with P3 teams provided solid evidence that program usage impacts P3 project consideration decisions. States using the P3 Program were shown to be slightly less likely to announce consideration of a P3 project than those who did not, and the few interviews conducted did not connect consideration with use of P3 Program resources. To better understand the relationship between P3 Program resources and P3 project consideration, a more thorough identification and exploration of project-screening decisionmakers is needed.

P3 Approval Decisions

There is more evidence demonstrating that P3 Program resources help State and local agencies make informed P3 project approval decisions. An analysis of the Public Works Financing (PWF) Major Projects Database shows that roughly 30 percent of P3 projects announced in the years 2013–2015 were approved. P3 Program use was associated with the vast majority of States who approved transportation P3 projects in this period. Interviews with State and local agencies indicated that knowledge gained from the P3 Program was often used to support the case in favor of P3s.

Finding: Less experienced practitioners value the combination of P3 Program materials and staff when making decisions throughout the P3 process; more experienced practitioners have only limited use of the resources.

Research on State and local agencies led to mixed conclusions about the value of the P3 Program for project decisionmaking. Less experienced States who took advantage of P3 Toolkit resources and FHWA staff found program resources to be valuable during project decisionmaking. States who had little or no FHWA staff support struggled to find helpful resources using the P3 Toolkit website. In addition, very experienced P3 States, most of whom have their own P3 education programs, had only limited use for the current materials, using them mainly for introductory information.

Interviews with a limited set of P3 advisors led to conclusions similar to those seen in State and local agencies. Experienced practitioners at established firms see value in the P3 Program through its use as an introductory resource but rarely use or recommend advanced analytic tools. Newer employees at P3 advisory firms, however, report attending webinars and using documents from the website. Although the evaluation interviews indicate limited P3 Program use among advisors, usage data on P3 Toolkit website downloads contradicts this, showing that advisors are a dominant user group. Additional research is recommended to understand more about this large user group, their use of the P3 Program, and their information needs.

P3 Program resources are increasingly valued by FHWA Division Office staff as more States are considering P3s. According to a survey of FHWA Division Office staff, more than one-third of those in roles that support P3s have accessed the P3 Toolkit website or attended webinars. They look to the program for materials to support teams at each stage of the P3 implementation process. P3 Program documents and webinars provide information that FHWA staff share with State and local agencies. Fact sheets, primers, and webinars provide general P3 information used during planning and evaluation, while model contract guides, project agreements, and oversight documents provide specific examples used when developing project documents during the procurement and implementation phases.

Finding: The P3 Program was found to be one of several resources that are used and valued when researching P3s and making decisions throughout the P3 process.

To address whether the P3 Program offers a complete set of P3 resources, program satisfaction and the use of other resources were assessed. The results indicate that the P3 Program is a valued resource among some user groups but not used by all. In addition, even among satisfied users, other resources are necessary to supplement the information provided by the P3 Program.

Satisfaction

Depending on their level of experience, satisfaction varied between users:

Completeness (Other Resources Used)

Depending on their background, users accessed a variety of resources for P3 information:

Recommendations

The P3 Program is already recognized as a valuable resource, but the evaluation process has identified the following five actions that could improve the reach and usefulness of the program:

  1. The P3 Program should recognize the distinct user groups that access program resources and identify their information needs. Content development should be focused on a few key target groups and communication plans developed to reach them.
  2. Marketing efforts should focus on target groups who are currently underusing P3 Program resources. The program should monitor resource use and seek to improve awareness among target groups not attending events, accessing webinars, or accessing guidance documents.
  3. The P3 Program should segment its offerings, aligning documents and tools with the needs of practitioners at different experience levels to better serve those in all user groups.
  4. The P3 Toolkit website should be organized in a way that allows distinct user groups to easily identify and access the content needed. The P3 Program team should look to its user groups for guidance on how to organize the website to best meet their information needs.
  5. Future resources should include more “real-world” P3 information, examples, and experiences to help teams find information applicable to their unique P3 projects. Examples include actual P3 legislation, peer exchanges, and a P3 project database.

[1]Internal information was received from P3 Program staff and from Google® Analytics™.(42)

[2]Resource usage tied to individuals/groups came from P3 Program event registrations and a form on the P3 Toolkit website download page (https://www.fhwa.dot.gov/ipd/p3/toolkit/download). Document use/views through the website are not included.

 

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101