CHAPTER 6—BENEFIT-COST ANALYSIS OF PERFORMANCE-BASED CONTRACTOR PREQUALIFICATION

This chapter identifies and analyzes the costs and benefits of performance-based contractor prequalification. It also presents contractor and State transportation department perspectives on various aspects of contractor screening from the five State case studies, plus costs of performance-based contractor prequalification versus performance bonds to State transportation departments.

Performance-based contractor prequalification is more comprehensive than the surety industry’s financial prequalification that occurs when it issues a bond. Performance-based contractor prequalification incorporates a contractor’s performance record with a given agency in the prequalification process through a past project evaluation system that creates a record of actual performance. Numerous agencies-FDOT, MTO, and the U.S. Army Corps of Engineers, to name three examples-use this type of performance information to determine eligibility to bid and to limit the amount of work on which a given contractor can bid. Additionally, the increased use of alternative project delivery methods to accelerate project schedules, as well as the decline in number of State transportation personnel, has led public agencies to depend more on contractor QC programs. This shift has turned performance-based contractor prequalification into a risk mitigation strategy and has increased the use of project-specific prequalification.

The evaluation of contractor performance and the integration of these evaluations into the performance-based contractor prequalification system provide a tangible means by which to reward good contractors and a disincentive for marginal contractors to perform badly. Other countries have been motivated to implement performance-based contracts for reasons that should resonate in the United States.

As demonstrated in NCHRP Web Document 38, these factors also motivate State transportation departments in the United States to look for methods by which to create efficiencies through contractor performance evaluations, as well as methods for mitigating the potential risk created by the trend toward the increased use of contractor QC in the project acceptance process.⁽¹⁷⁾ Agency performance evaluation programs have to pass the tests for both fairness and equity, which are essentially reflected in the types of information the agency collects about past performance.

Agencies currently use numerous approaches to incorporate contractor performance into the prequalification process. While the variation is substantial, the motivation for implementing these systems is generally the same; to correlate contractor performance with a contractor’s ability to competitively bid, which thereby creates an incentive for good performance and encourages marginal performers to improve.

This chapter explores the benefits and costs of performance-based prequalification based on data collected through the literature review, the outreach efforts with the State transportation departments and the contractors, and the five State transportation department case studies. However, as a result of these efforts, it was found that there is no specific quantitative data about the benefits of performance-based prequalification available at this time. There is no available data because the benefits were either qualitative, such as an improved relationship with the contractor; the benefit, such as improved schedule, could not be measured in a specific way to show that performance-based prequalification was directly responsible for the benefit; the State transportation department did not track the data required; or there was no baseline data with which to compare the claimed benefit. As a result, instead of a detailed benefit-cost analysis, the benefits of performance-based contractor prequalification will be described in a qualitative manner, and the costs will be described in financial terms.

PERFORMANCE-BASED CONTRACTOR PREQUALIFICATION BENEFITS

Currently, no entity tracks the benefits of performance-based contractor prequalification in a quantitative manner. Quantitative conclusions cannot be drawn based on measures that are currently tracked because the nature of the benefits is qualitative; there are no baseline measures to compare the tracked data to; and/or it is difficult to attribute the measured improvement directly to performance-based prequalification. Nonetheless, the overall opinion of performance-based prequalification in both industry and academia is that it does improve a project. States that currently have performance-based prequalification programs continue to move forward with their existing programs, and some are enhancing their existing programs to make them more robust.   Discussion of this activity can be found in the case studies in appendix B.

Neither State transportation department personnel nor contractors consider the ability to secure a performance bond a reliable indicator of a contractor’s qualification to perform high-quality work; rather, they view a contractor’s past performance as such an indicator. Based on the literature review, outreach efforts with State transportation departments and contractors, and five State transportation department case studies, the overall benefits of performance-based contractor prequalification occur in the following project areas:

• Overall project performance.
  
  
• Project quality.
  
  
• Project timeliness.
  
  
• Number of claims.
  
  
• Contractor and State transportation department relationship.
  
  

The benefits from performance-based prequalification identified by both the contractor survey responses and the State transportation department case studies are the following:

• Improved quality of work.
  
  
• Improved timeliness of delivery.
  
  
• Improved safety.
  
  
• Improved personnel experience.
  
  
• Improved contractor cooperation with the agency.
  
  

The remainder of this chapter will attempt to provide further insight into the benefits of performance-based prequalification.

In the survey of contractors and State transportation departments conducted during this research, there was support for performance-based prequalification. Overall, the contractors express that when performance-based contractor prequalification is used, both the quality of work and the timeliness of delivery improve. All contractors surveyed believe that workmanship quality, safety, timely project completion, timely punch-list completion, personnel experience, and warranty responsiveness improve when performance-based prequalification is implemented. State transportation department respondents did not agree on how these aspects of project delivery are impacted; some felt that there would be no change to these factors.

State transportation departments appeared to be more focused on the impact of performance-based contractor prequalification on State transportation department-contractor relations. State transportation department respondents most often cited improvements in the number of claims filed (i.e., fewer claims) and contractor cooperation with the agency when performance-based prequalification is implemented. The only negative aspect of the implementation of performance-based contractor prequalification mentioned by a minority of the State transportation department or contractor respondents was the thought that there would be a decrease in the number of contractors that bid on a given contract.⁶ Otherwise, the responses for other aspects of project delivery were all either positive or neutral, where neutral indicated no predicted change.

While contractors express greater across-the-board support of performance-based pre-qualification than State transportation departments, on average, both groups view the addition as a positive benefit. Contractors believe that quality and timeliness improve when performance-based contractor prequalification is used, and State transportation departments believe that State transportation department-contractor relations improve when it is used. Table 30 provides a side-by-side comparison of these responses.

Table 30. Impact of performance-based contractor and State transportation department prequalification.

The case studies were conducted with five different states: Iowa, Oklahoma, Utah, Virginia, and Washington. Each of these states has implemented some form of performance-based contractor prequalification. The different versions of performance-based prequalification varied from checking past project references to evaluating contractor performance during projects and incorporating these scores into the prequalification process. During the case studies, it was found that the State transportation departments had difficulty providing specific details about how the overall project performance improved because of performance-based prequalification. However, some participants stated that the project went more smoothly as a result of the performance-based prequalification and that there seemed to be less chaos on the project. The case study participants did all agree that performance-based prequalification was beneficial and that each State transportation department plans to continue to perform this prequalification. During the case studies, each State transportation department was asked to rate the impact of performance-based contractor prequalification on a number of different project aspects that ranged from procurement through completion of the project. Table 31 shows the impact ratings from each of the five State transportation departments. The possible ratings were worse, no change, better, or no opinion.

Table 31. State transportation department perspective of the impact of performance-based contractor prequalification on different aspects of a project.

Iowa rated the impact of performance-based contractor prequalification as no change for every project aspect because Iowa has been doing its own form of performance-based prequalification for more than 20 years, and the participants had no other experience against which to compare it. The remaining four states all gave “better” ratings for both workmanship quality and timely completion of activities, which indicates that both of these are improvements from performance-based prequalification. Unlike the outreach effort with the State transportation departments, these four State transportation departments are in agreement with the overall results of the contractor survey, which found an improvement in quality of work and the timeliness of delivery as the result of performance-based contractor prequalification.

Three out of the remaining four states rated the impact of performance-based contractor prequalification as “better” on the following project aspects:

• Material quality.
  
  
• Safety.
  
  
• Maintenance of traffic.
  
  
• Level/amount of agency inspection required.
  
  
• Personnel experience.
  
  
• Personnel competence.
  
  
• Contractor cooperation with all parties involved in the project.
  
  

While the above aspects are not unanimously rated as an improvement that results from performance-based prequalification, they are still considered benefits because the agencies that responded have very different experiences with performance-based prequalification. Safety and personnel experience are again in agreement with the contractor improvements from performance-based prequalification. Contractor cooperation with all parties involved in the project is also the primary benefit of performance-based prequalification identified by the State transportation departments that participated in the outreach survey.

The only “worse” rating was from Oklahoma, and it was for the number of bidders. It is notable that two of the states rated the number of bidders as “better.” The outreach efforts also identified that the number of bidders was likely to be reduced, due to performance-based prequalification. The variance in the case study ratings could be explained, based on the definition of the response choices and what indicates an improvement. In some circumstances, it may be considered that an increase in the number of bidders results in a more competitive environment, which could be considered an improvement. However, in other circumstances, it could be that the number of bidders decreases, but among that number, more are truly qualified to perform the work or make up the majority of the bidder pool, which could be considered an improvement. Each case study is described in further detail in appendix B.

PERFORMANCE-BASED CONTRACTOR PREQUALIFICATION COSTS

A State transportation department can experience significant costs at the beginning of a performance-based prequalification system, such as development of a performance evaluation process, infrastructure development costs, and internal and external training costs. This analysis does not consider these costs; rather, it considers the costs associated with the operation of a performance-based prequalification system.

The administrative cost of a performance-based contractor prequalification is the only cost a State transportation department will incur with that prequalification system, whereas the State transportation department will pay both administrative costs and premiums for performance bonds, which costs considerably more. The cost of performance-based contractor prequalification results from the additional staff required to manage the system. Research showed across five different States that the additional staff varied between 0.5 full-time employees and 2 full-time employees. Typically, the additional staff required by the performance-based contractor prequalification system has an average fully burdened cost of approximately $50 per hour. This results in an annual cost between $52,000 and $208,000. This cost range is the result of performance-based prequalification programs that are at various stages of implementation.

While the administrative cost of performance-based contractor prequalification can be higher than the administrative cost of performance bonds, this cost is negligible compared to the annual premium costs of performance bonds.

⁶ Any prequalification method will be restrictive to some extent. In the contractor survey, one contractor respondent noted that his or her company found it difficult to secure performance bonds for large projects with contract values above $50 million, due to the company’s relative youth and subsequent inability to rely on cash reserves for operating capital.