CHAPTER 8—CONCLUSIONS

This chapter summarizes the overall conclusions of this research project. Each area of the research will be addressed: the literature review, industry outreach efforts, case studies, benefit cost ratios, performance bond paradox, and the performance-based prequalification model.

LITERATURE REVIEW

Several conclusions follow from a critical analysis of the literature. First, the literature demonstrates that there are essentially three levels of contractor prequalification: administrative, performance-based, and project specific. Second, it shows that agencies take three approaches to the use of performance bonds: they bond the entire contract amount, bond a portion of the contract amount, or require no performance bonds. Third, it shows that the ability to secure a performance bond is not considered a reliable indicator of a contractor’s qualification to perform high-quality work, whereas contractor past performance is considered such an indicator.

This examination found at least 23 State transportation departments that currently use some form of contractor performance evaluation, though the factors they assess vary. Quality of workmanship and materials were the factors most frequently evaluated (19 out of 23 States evaluate these factors). Thirteen States evaluate contract compliance, 11 States evaluate prosecution of the work, and 10 States evaluate safety and project management.

The Florida and South Carolina State transportation departments use sophisticated performance-based contractor evaluation programs and have successfully applied adjustments to the amount of work a marginal contractor can bid on, as a mechanism by which to encourage contractors with less than satisfactory performance records to improve. MTO’s performance-based prequalification system also relies on this mechanism and has completely eliminated performance bonding from its program. A number of States (e.g., California, Minnesota, and Colorado) require less than full-performance bond coverage on certain projects.

The increased use of alternative project delivery methods to accelerate project schedules, as well as the decline in number of State transportation department personnel, has led public agencies to depend more on contractor QC programs. This shift has turned performance-based contractor prequalification into a risk mitigation strategy and has increased the use of project-specific prequalification. SCDOT has implemented a project-specific performance evaluation program to focus further on selecting contractors whose qualifications, competency, and performance records more closely match specific project requirements. Successful performance-based contractor prequalification programs have been shown to consist of the following five components:

• Administrative prequalification, which includes facts about a contractor’s financial status, equipment, plant, and various other factors.

• A formula to determine if a contractor is financially qualified to undertake the given work, based on the administrative data collected.

• A project performance contractor evaluation program that produces the data necessary to assess a given contractor’s record of performance.

• An algorithm that converts the evaluation output into an index that can be used to determine prequalification.

• An appeals process that permits rated contractors to seek redress for ratings that they believe are not accurate, without having to resort to legal action.

INDUSTRY OUTREACH EFFORT

Input was received via surveys from representative State transportation departments, contractors, the SFAA, and surety company representatives, as part of an effort to evaluate the benefits and costs of performance bonds and performance-based prequalification methods. Also, a better understanding of State transportation department and contractor attitudes toward these tools was desired. Contractors noted their misgivings about the value of performance bonds, given the cost of these bonds. However, State transportation departments expressed hesitation at the idea of abandoning the use of performance bonds. The surety industry outlined the benefits of performance bonds during prequalification and construction and presented data on the costs of performance bonds.

A significant number of State transportation departments believed that a performance-based prequalification process would improve the quality and timeliness of project delivery and enhance State transportation department-contractor relationships. Contractors appeared uniformly open to an equitable, performance-based prequalification process as a means to improve project delivery. Survey responses suggest that performance-based prequalification methods can be implemented and/or refined to better emphasize the performance and financial factors that are most relevant to effective project delivery.

Numerous conclusions from the outreach efforts about the potential benefits and structure of performance-based contractor prequalification have been made. In summary, the conclusions are as follows:

• Both contractors and State transportation departments feel that a performance-based contractor prequalification process has the potential to improve overall project performance.

• Both contractors and State transportation departments rate the evaluation of “corporate qualifications” (i.e., qualifications that relate to the experiences and qualifications of the contractor organizations) more highly than they rate the evaluation of contractors’ “programs.”

• Both contractors and State transportation departments believe past performance, relevant experience, illegal behavior, personnel qualifications, and claims history are likely to be the most critical factors when determining a contractor’s effectiveness.

• Contractors believe that a marginal contractor has an unfair advantage over a well-qualified contractor under the current system, which is reinforced by their belief that implementing performance-based contractor prequalification would disqualify the marginal contractors from bidding.

Numerous State transportation department respondents and all contractor respondents expressed the belief that project performance can be quite valuable as an indicator of a contractor’s ability to deliver projects in an effective and timely manner. This belief suggests that improvements in the area of performance-based prequalification could benefit the project delivery process.

While all respondents considered financial factors important to ensuring effective project delivery, contractor respondents did not appear as confident as State transportation departments in the role that surety companies play. These differing opinions may be due, in part, to misconceptions about the nature of performance bonds and the roles that sureties play in the evaluation of contractors and the completion of a contract. These possible misconceptions are described below.

First, performance bonds are not insurance. They do not guarantee against non-completion of a contract under all conditions, as insurance would (if insurance companies made such a product available). Instead, performance bonds come into play only when the contractor has defaulted on completion of the contract and is in financial default (i.e., is unable to provide the funds to remedy the situation). Performance bonds are more a form of credit than insurance in that they are priced like credit; sureties go through the same steps to evaluate contractors as banks go through to evaluate corporate borrowers; and sureties have the same rights to monitor and intervene in the affairs of their contractors as do other creditors.

Second, sureties’ role as creditors gives them a superior ability to assess the financial and managerial capacities of contractors over long periods of time and to intervene in the affairs of contractors to prevent and avoid defaults. However, the advanced evaluation and intervention capabilities are limited by the nature of performance bonds themselves; performance bonds do not guarantee the quality of work, nor do they guarantee that the full costs to complete a project in default will be covered by the performance bond.

Finally, and most relevant to the objective of improving the quality of contracted construction work through the prequalification of contractors, performance bonds provide no protection against mediocre work. Sureties do not evaluate contractors in terms of the completion of timely, high-quality work that satisfies State transportation department expectations. Sureties are unable to obtain data from State transportation departments about contractor performance, and even if they could and did, the low rates of default and sureties’ limited obligations give them little incentive to raise the costs of performance bonds in order to incorporate contractor performance evaluations.

The surety industry’s responses portray the significant benefits surety companies provide to State transportation departments throughout the construction process, which their unique status as the contractors’ creditors enables them to provide. State transportation departments seem more attuned to the specific advantages of this service than contractors are, and consequently, appear unwilling to abandon the perceived security that performance bonds provide. Contractors have greater reservations about this conclusion and feel more strongly that performance-based prequalification methods can lead to improved project delivery, possibly even in place of performance bonds.

Analysis of the responses obtained from State transportation departments, contractors, the SFAA, and select surety companies suggests that opportunities to standardize and integrate performance-based prequalification methods as part of a more comprehensive prequalification process should be addressed, in order to improve project delivery. The results provide an initial indication of State transportation departments’ and contractors’ appetites for improvements to the prequalification process, as well as an indication of potential areas for improvement, supplementation, and consolidation of the contractor evaluation process. Additionally, the results reinforce the conclusion of NCHRP Synthesis 390; that barriers to performance-based prequalification implementation are low among members of the construction industry, and that while State transportation departments show little willingness to completely abandon performance bonds, they acknowledge the potential benefits of evaluating contractor project performance and using the information in the prequalification process.⁽¹⁷⁾

STATE TRANSPORTATION DEPARTMENT CASE STUDIES

Case studies were conducted with five State transportation departments: IOWADOT, ODOT-OK, UDOT, VDOT, and WSDOT, in order to evaluate the performance-based prequalification model in relation to the current prequalification practices of the State transportation departments; to obtain the State transportation departments perspective on performance bonds; and to gather project data for the benefit-cost analysis of both performance bonds and performance-based contractor prequalification. The following conclusions resulted from the case studies:

• All participant State transportation departments are proponents of a performance-based contractor prequalification system.

• None of the State transportation departments were willing to eliminate the use of performance bonds.

• Defaults on State transportation department projects are rare occurrences.

• A performance bond is beneficial because it enables a State transportation department to threaten to call a contractor’s surety to report bad performance, which can be an effective way to motivate the contractor to improve performance.

• State transportation departments have not had a surety become involved in a project, unless the sureties were called in by the State transportation department. State transportation departments also acknowledge that they would not know if the surety was involved in the project before the State transportation department requested the surety’s involvement.

• There was no evidence found about the surety’s involvement in a project that prevents a default, also called a near miss.

Overall, all five states were proponents of performance-based prequalification. However, they are reluctant to change their performance-based prequalification system to the proposed one in this research project, primarily because each believes that their own system works well, and the amount of time and effort to implement a new system is too great when the current one is fine.

BENEFIT-COST ANALYSIS

The following presents the benefit-cost analysis conclusions for both performance bonds and performance-based contractor prequalification. Overall, a benefit cost ratio of less than one indicates a net cost, a benefit cost ratio of greater than one indicates a net benefit, and a benefit cost of one indicates that the costs and benefits are equal.

Performance Bonds

The benefit-cost ratio for performance bonds is equal to the performance bond benefit divided by the performance bond cost. A value greater than one indicates that performance bonds provided a net benefit to the State transportation department; a value less than one indicates that performance bonds caused a net cost to the State transportation department; and a value of one indicates that performance bonds provide no net cost or net benefit.

The default rate of projects is a key factor when determining the benefits of the performance bond. This research determined that the default rate for the highway industry was less than 1 percent, which indicates that a default is a random occurrence. The benefit calculations used a default rate of 0.69 and 0.47 percent, which are the highest rate found and an average rate, respectively.

The benefits of a performance bond are the following:

• The avoided cost of re-bidding the project after a default occurs.

• The avoided schedule delay as a result of a default.

• The avoided cost of the default.

The costs of a performance bond are the following:

• The performance bond premium, which can range between 0.22 and 2.5 percent of the project value.

• The costs to the State transportation department to administer a performance bond program. This cost was found to be negligible in comparison to the performance bond premium and was not used in the overall cost calculation.

In order to complete the benefit-cost analysis, the following different elements required assumed values: number of days the schedule is delayed as a result of a default; cost per day of a schedule delay; default rate; cost to re-bid a project; and the contract value used in each project size category to calculate the benefit-cost ratio. The overall benefit-cost analysis of performance bonds proved to be very sensitive to these assumptions, and as a result, several different analyses were conducted, using a different value for one of the assumptions in each analysis. More details on these analyses can be found in appendix C. These analyses show that if the default rate is held constant at 0.69 percent, projects over approximately $10 million have a net benefit from performance bonds; projects between $100,000 and $1 million have a net cost for performance bonds; and projects less than $100,000 and between $1 million and $10 million vary between net cost and net benefit. However, when the default rate is lowered to 0.46 percent, the average default rate from table 20, the benefit-cost ratios are less than one for all project categories, which indicates a net cost for performance bonds on all projects.

Performance-Based Prequalification Benefit Cost

Even though the benefits of performance-based prequalification are not quantitatively tracked, it is the overall opinion of both the highway industry and academia that performance-based prequalification improves a project. The case studies further validated this finding, based on the fact that all of the State transportation departments continued to use their performance-based prequalification systems and several continued to enhance these systems through the development of further technology, in order to make these systems more robust. As a result of the lack of quantitative measures, the qualitative benefits of performance-based prequalification were investigated.

Based on the literature review, outreach efforts with State transportation departments and contractors, and five State transportation department case studies, the overall benefits of performance-based contractor prequalification occur in the following project areas:

• Overall project performance.

• Project quality.

• Project timeliness.

• Number of claims.

• Contractor and State transportation department relationship.

The benefits from performance-based prequalification identified by both the contractor survey responses and the State transportation department case studies are the following:

• Improved quality of work.

• Improved timeliness of delivery.

• Improved safety.

• Improved personnel experience.

The benefit from performance-based prequalification is improved contractor cooperation with the agency, a benefit that was identified by both the State transportation department survey responses and the State transportation department case studies.

The major cost associated with implementing performance-based prequalification is the cost of State transportation department staff to administer the program. Start-up costs involved with a performance-based prequalification program were not considered in these studies. Based on the case studies, it is estimated that the cost of the State transportation department staff to administer a performance-based contractor prequalification program is between $52,000 and $208,000 annually. This cost range is the result of performance-based prequalification programs that are at various stages of implementation, from checking references to a program that conducts project evaluations and integrates them into the prequalification process.

PERFORMANCE BOND PARADOX

Because defaults happen so infrequently, it seems counter intuitive that State transportation departments are unwilling to eliminate the use of performance bonds; however, the State transportation departments are, in fact, unwilling to eliminate these bonds. Even the contractors weren’t all in favor of eliminating performance bonds. In response to this paradox, an analysis was conducted to determine if performance bonds could be used more effectively. Currently, the majority of the States require performance bonds for 100 percent of the value of a project for all projects.

Based on the default data, it was not found that any particular size or type of project was more likely to default than any other project; default is a random occurrence.

It is recommended to raise the minimum project size that requires a performance bond to between $1 million and $10 million. The following are the reasons for this price increase:

• The default rate is not based on project size, type of project, or delivery method.

• A default is a random occurrence.

• A performance bond does not in itself prevent a default.

• The repercussions that ensue when a smaller project defaults are not of a magnitude that requires the protection of a performance bond.

• The performance bond premium rate for smaller projects is higher than it is for larger projects.

• The amount of savings a State transportation department can receive from this change can be put towards future projects or used to manage the risk of not having a performance bond.

If the five case study states had raised the minimum project size that requires a performance bond to $1 million, the State transportation departments could have saved between $1.9 million and $7.9 million over 5 years. If the minimum project size had been raised to $10 million, then the savings would have been between $6.5 million and $26 million over 5 years.

PERFORMANCE-BASED CONTRACTOR PREQUALIFICATION MODEL

Based on the project research and analysis, a performance-based prequalification model was developed. The model combines elements of the processes used by FDOT, ODOT-OH, and MTO, and it borrows concepts and terminology from each. The model accounts for a contractor’s financial capacity, rewards good performance, and encourages improvement for marginal performance by prequalifying a contractor, based on a bidding capacity that is determined by rating prior performance. The model consists of a two-tier process that is applicable to design-bid-build projects, as well as an optional third tier for alternative project delivery methods, such as design-build, construction manager/general contractor, and public-private partnerships. It also can be used for design-bid-build projects on which a State transportation department wants to do a performance evaluation. A summary of the tiers is provided as follows:

• Tier one consists of administrative prequalification, which is composed of financial analysis conducted by the agency, a records check, and the assembly of optional external documented information.

• Tier two focuses on contractor performance and encompasses two primary areas: the determination of the contractor’s management ability and a post-project evaluation of the contractor performance on each contract. These factors are used in conjunction with tier one’s financial analysis output to determine the amount of work upon which a contractor can bid.

• Tier three is a project-specific prequalification tier designed to closely evaluate the contractor’s qualifications and experience in terms of the specific needs of a given project. This final tier is an optional portion of the prequalification process and is intended for use on projects that are delivered by methods other than traditional DBB and/or on a project that has requirements beyond the standard boilerplate requirements.