CHAPTER 2 — LITERATURE REVIEW

This chapter presents the results of an in-depth literature review of performance based prequalification and the use of performance bonds in the highway industry. Sources reviewed include prequalification research studies, alternative project delivery studies that include prequalification, State transportation department literature available online, Means construction cost data, information from the SFAA, and academic sources. The literature was broken into four focus areas: previous contractor prequalification research, State contractor prequalification evaluation procedures, State contractor bidding procedures, and bonding costs. This chapter presents a summary of the literature results, as well as the details for each focus area.

LITERATURE REVIEW SUMMARY

The literature review included the following four focus areas: previous contractor prequalification research, State contractor performance evaluation procedure, State contractor bidding procedures, and bonding costs. Table 1 summarizes the topics of interest and types of sources for each of the focus areas reviewed.

Table 1. Literature review summary.

Focus area	Topics of interest	Types of sources
Previous contractor prequalification research	Components of a performance-based prequalification program Federal construction contractor appraisal support system Performance-based prequalification implementation Adjusting bidding and bonding capacity Barriers to implementation	Prequalification research studies Alternative project delivery studies that include prequalification
State contractor prequalification evaluation procedures	Overview of existing industry contractor performance evaluation practices Specific processes used by State transportation departments for contractor performance evaluation procedures	Prequalification research studies Alternative project delivery studies that include prequalification State transportation department literature available online
State contractor bidding procedures	Specific processes used by individual State transportation departments for contractor prequalification	Prequalification research studies Alternative project delivery studies that include prequalification
Bonding costs	Cost of a performance bond Default rates	Means construction cost data Myers graduate thesis(11) Surety and Fidelity Association of America

A more detailed review of the literature for focus areas is provided in the following sections.

PREVIOUS CONTRACTOR PREQUALIFICATION RESEARCH

The literature on this subject is both extensive and thorough, which underscores the interest and potential value of performance-based approaches to the transportation industry. NCHRP Synthesis 190 found that State transportation departments rely on the following four strategies to qualify construction contractors to bid:⁽⁶⁾

• Prequalification: The contractor needs to be qualified before it can submit a bid.
  
  
• Post-qualification: Only the lowest responsive bidder is qualified for the State transportation department’s determination of responsibility.
  
  
• Performance bonding: Reliance on the surety industry to identify qualified contractors.
  
  
• Contractor licensing: State-sponsored program to ensure that only qualified contractors can bid based on licensing requirements.
  
  

The authors of literature on this subject believe that the qualifications of a given contractor can have a marked impact on the success of the projects it executes. An article on design-build projects also notes that selecting the most qualified contractor “correlates to the lowest administrative burden” for the State transportation department, which implies that a well-qualified contractor requires less oversight and can be trusted to comply with contract requirements, such as contractor quality control (CQC) methods.⁽¹²⁾ As noted above, in NCHRP Report 561, Scott et al. justifies prequalification by noting, “because of constrained staffing and budgets, it is not possible for State transportation departments to ‘inspect’ quality into the work.”⁽⁴⁾ The same article provides a succinct definition of a prequalification process and a motivation to establish a thoughtful process:⁽⁴⁾

• Prequalification in its simplest form is an assessment of financial responsibility, which often mirrors what sureties look for in making their underwriting decisions relating to issuance of bonds for public works projects. It also may include other factors such as demonstrated ability to perform a certain type of work. Whether by prequalification or other methods, public owners are increasingly exploring ways to include non-price factors, both qualitative and quantitative, in the procurement process to motivate contractors not only to improve their performance during construction, but equally as important, to build value into the end products of construction.

Once again, the merit of using prequalification to add value to the construction process is underscored. The idea of using performance-based prequalification as a means to motivate contractors to improve their performance during construction is also expressed by the authors. The authors note that this idea ultimately leads to the benefits of enhanced construction quality and reduced administrative burden. The New South Wales (NSW) Australia prequalification manual, which calls the process the “scheme,” describes the benefits of a performance-based prequalification process as follows:⁽¹³⁾

• It “allows the NSW Government as a major buyer of construction related services to more effectively implement continuous improvement initiatives in the construction industry to achieve better project outcomes.
  
  
• It results in significantly reduced tender assessment times and simplified contract administration because prequalified tenderers [bidders] have already demonstrated an understanding of and compliance with NSW Government construction industry benchmarks, with management procedures and systems requirements.
  
  
• It is in line with the NSW Government’s direction to do business with the best of the private sector, the Scheme provides for incentives for good performance and also for the application of restrictions or sanctions in the event of poor performance as measured against the respective scheme requirements.”
  
  

Review of Prequalification Studies

The subject of contractor qualification has generated a significant amount of research in the past decade, including five studies that specifically examine performance-based contractor prequalification. These studies are briefly summarized below.

Kentucky Transportation Center Report, KTC-01-24/SPR 212-00-1F, Quality Based Prequalification of Contractors, reports the results of an analysis of the effectiveness of the Kentucky Department of Highways’ (KDOH) contractor performance rating system and its incorporation into the prequalification process.⁽¹⁴⁾ It recommends the inclusion of a given contractor’s past project quality performance record as an integral part of the prequalification process.

Wisconsin Department of Transportation Synthesis Report, Contractor Prequalification Quality-Based Rating reports the practices of 35 State transportation departments that responded to a survey on the topic.⁽¹⁵⁾ The report found that the majority of both contractors and State transportation department respondents believed that performance-based prequalification can be implemented in a fair and equitable manner.

Manchester School of Management Report, Applying Evidential Reasoning to Prequalifying Construction Contractors was conducted in the United Kingdom and essentially proposes a logic-based mathematical model to optimize the contractor prequalification process.⁽¹⁶⁾

NCHRP Synthesis 390, Performance-Based Construction Contractor Prequalification,benchmarked the state of the practice using a survey of 41 U.S. State transportation departments and 7 Canadian provincial ministries of transportation.⁽¹⁷⁾ The synthesis proposed a three-tiered model for performance-based contractor prequalification that will be discussed in detail in a subsequent section of this report.

Analysis conducted for the Michigan Department of Transportation (MDOT) examination, Study of Most Effective Practices for Determining Construction Contractors’ Eligibility to Bid on Construction Projects, built on the analysis work of NCHRP Synthesis 390, and focused on practices in the upper Midwest, surveying the States that border Michigan.⁽¹⁸⁾ MDOT sponsored this research, which resulted in the finding that the current MDOT performance-based contractor prequalification system is both fair and equitable, and is consistent with similar systems in the region.

All five of the reports cited above concluded that implementing performance-based contractor prequalification has the potential to add value to the highway construction procurement process.

Review of Studies on Alternative Project Delivery that Included Prequalification

As part of the research associated with alternative project delivery, a fair amount of information has been published related directly to prequalification of contractors. Each of the relevant studies most specifically assessed the contribution of contractor qualifications to the project selection and award process. A summary of the major studies is provided below.

• AASHTO Guide for Design-Build Procurementprovides a comprehensive set of the qualifications and experience requirements that have successfully been included in project-specific prequalification for design-build contracts.⁽¹⁹⁾
  
  
• NCHRP Synthesis 376, Quality Assurance in Design-Build Projects, analyzes the impact of design-build contractor prequalification and experience on final project quality.⁽²⁰⁾
  
  
• Transportation Cooperative Research Program (TCRP) Report 131, Guidebook for the Evaluation of Transit Project Delivery Methods, presents a comprehensive list of qualifications and experience requirements that have successfully been used on the project-specific prequalification of contractors from transit projects delivered using alternative project delivery methods, such as DB, CMGC, etc.⁽²¹⁾
  
  
• Airport Cooperative Research Program Report 21, A Guidebook for the Selecting Airport Capital Project Delivery Methods, provides a comprehensive list of qualifications and experience requirements that has successfully been used on project-specific prequalification of contractors for airport projects delivered using the alternative project delivery methods, as covered in TCRP Report 131.⁽²²⁾
  
  
• NCHRP Synthesis 402, Construction Manager-at-Risk Project Delivery for Highway Programs, analyzes the impact of contractor prequalification on final project quality for project delivery using Construction Manager-at-Risk (CMR).⁽³⁾ This report found that a major advantage of CMR project delivery was the ability to utilize project-specific contractor performance-based prequalification that allowed the State transportation department to match the contractor’s qualifications with the technical requirement of the project.
  
  

Components of a Performance-Based Prequalification Program

The literature summarized above suggests that most performance-based contractor prequalification programs consist of the same set of components, which are described as follows:

• A completed questionnaire/application furnished by the contractor that details the following information: financial data, available equipment and plant, construction experience for a specified period, names and backgrounds of key personnel, and classes/types of work for which qualification was requested.
  
  
• A formula/algorithm that converts financial data into a rated capacity, which establishes the maximum amount of work that a given contractor can be awarded in a given period.
  
  
• A contractor project performance evaluation system that focuses on ratings of contractor work quality and timeliness on a specific standard form.
  
  
• A formula/algorithm to adjust the rated financial capacity based on the accumulated record of project performance evaluations.
  
  
• An appeals process for a contractor that believes it has been unfairly or improperly rated. (See references 1, 4, 10, 14, 23, 24, 25, and 26.)
  
  

State transportation departments often use general past performance and experience criteria in their administrative prequalification procedures to admit a contractor to the State bidders list.⁽¹⁷⁾ By using performance-based qualification factors in the selection process, a State transportation department can filter out unqualified contractors, thereby increasing the probability that the project will be completed successfully.⁽²⁷⁾ However, the key to successful public sector application of qualification parameters in a bid is that “their application must be justifiable and defensible.”⁽⁴⁾ This observation speaks to the need expressed by Parvin to reduce the probability of bid protest by making the performance-based prequalification system transparent and easy to understand.⁽²⁸⁾ The Minnesota Department of Transportation defended its prequalification method for a design-build project in an award protest by establishing that its prequalification parameters were both “justifiable and defensible.”⁽⁵⁾

Figure 1 is taken from a paper by Hancher and Lambert that details the KDOH’s “performance-based” contractor prequalification process.⁽¹⁴⁾ The process combines the calculation of total maximum financial capacity with a performance-based evaluation through the assignment of an “annual eligibility rating,” which is used to adjust the contractor’s “maximum eligibility amount.” This value equals the amount of work a contractor may be awarded in a given year. Using the formula, “lower-quality work will reduce the allowable work volume, whereas high-quality work will increase the allowable work volume” for a contractor.⁽¹⁴⁾

Figure 1. Flowchart. KDOH performance-based contractor prequalification process.⁽¹⁴⁾

Several authors have conducted research that evaluates the relative importance of the various components listed above. One early study included a survey of construction professionals from both project-owner and contractor organizations and asked them to rank order 20 prequalification factors by importance. Both rated financial stability, past project performance, and personnel availability and experience as the “key decision variables relevant for a generic contractor prequalification knowledge base.”⁽²³⁾ NCHRP Web Document 38 also found project management/control skills, personnel experience, quality of final project, and experience with project type to be the most important, according to a similar group of survey respondents.⁽¹⁾ When these two studies are looked at together, the results of one’s contractor questionnaire/application and the other’s contractor project performance evaluation match one another, which validates the importance of these program components to a prequalification process based on two independent research efforts.

Evaluating contractor performance, then integrating these evaluations into the performance-based contractor prequalification system, provides a tangible means by which to reward good contractors and a disincentive for marginal contractors to perform badly. Other countries have been motivated to implement performance-based contracts for reasons that should resonate in the United States. The motivation for the evaluation of contractor performance in New Zealand has been explained as follows:⁽²⁹⁾

• The concept of performance-based contracts originated from a consideration of four factors, namely, (a) the increasing lack of personnel within the national road departments...(b) the frequency of claims...(c) the need to focus more on customers’ satisfaction by seeking to identify the outcomes, products, or services that the road users expect to be delivered, and by monitoring and paying for those services on the basis of customer-based performance indicators; and (d) the need to shift greater responsibility to contractors throughout the entire contract period as well as to stimulate and profit from their innovative capabilities.
  
  

As demonstrated in NCHRP Web Document 38, these factors also motivate State transportation departments in the United States to look for methods by which to create efficiencies through contractor performance evaluations, as well as methods to mitigate the potential risk created by the trend toward the increased use of contractor QC in the project acceptance process. State transportation departments’ performance evaluation programs have to pass the tests for both fairness and equity, which are essentially reflected in the types of information the State transportation departments collects in regard to past performance.

An attention to fairness should ensure that the evaluation system is transparent and furnishes a mechanism by which contractors can appeal a negative rating. Transparency can be achieved when the evaluation system and all its components are published in advance of the evaluation and when the State transportation department performs the evaluation in line with what has been published.⁽³⁰⁾ The inclusion of guidelines for the ratings used for individual components also contributes to consistency and minimizes biases. Furnishing an appeals process demonstrates to the contracting industry that the State transportation department is open to the challenges of its evaluation system through the use of due process, before a contractor is penalized by a negative rating.⁽⁴⁾ If implemented, these two elements can greatly ameliorate negative perceptions of potential harmful impacts of a new contractor performance evaluation system.⁽³⁰⁾

To further increase fairness in the process, KDOH allows contractors to rate KDOH’s performance at the same time KDOH rates the contractor. These contractor ratings are used by KDOH to “determine quality improvements needed, personnel training needed, and topics for discussion at the annual meetings with the contractor associations and for evaluations of personnel and other uses, as deemed appropriate.”⁽¹⁴⁾

A contractor’s ability to appeal to have a negative rating changed or removed can also serve as a means by which to further ensure fairness. NCHRP Report 561 notes that fairness demands a contractor have recourse to “due process.”⁽⁴⁾ While a State transportation department may not want to create a separate formal disputes resolution system for performance evaluations, it should furnish within its evaluation framework a mechanism whereby a contractor has the ability to protest what it believes is an unfair assessment of its performance. Such a mechanism can be as simple as allowing the contractor to add rebuttal comments to the evaluation form and then charging the chain of command above the evaluator to investigate to determine if the contractor’s protest has merit before the final evaluation is entered into the system. Some jurisdictions have a formal board or committee that will hear and decide appeals on qualification matters. The inclusion of such elements of independence can significantly contribute to both the fairness and, equally as important, the perceived fairness of the system.

Federal Construction Contractor Appraisal Support System

The FHWA’s Federal Lands Highway Divisions utilize the Department of Defense Construction Contractor Appraisal Support System (CCASS). ^(9,31) An NCHRP study reports of the CCASS:⁽⁴⁾

• The Federal Government and a number of State agencies have for many years maintained a database of contractor evaluations on past projects and often use this resource as a means to measure the contractor’s track record. Despite certain drawbacks, this appears to be the best means of assessing past performance as it allows contractors the opportunity to appeal negative ratings.
  
  

The CCASS evaluation system has been in use for decades and serves not only to record actual contractor performance, but also as a means for Federal agencies to make a decision on the “responsibility” of bidders for a DBB project.^(9,32) In this system, a low bidder with several unsatisfactory ratings can be found “not responsible,” and consequently, not be awarded the contract. CCASS requires that the agency evaluate the contractor’s performance in five areas: quality control, timely performance, compliance with safety standards, the effectiveness of management, and compliance with labor standards.

The FHWA requires that the contractor be notified if the State transportation department believes it is not performing at a satisfactory level. This kind of mandated communication between the contract owner and the contractor provides the contractor with the ability to both correct the defect found by the contract owner and, if applicable, to refute or clarify the perceived defects. The CCASS process requires that the State transportation department forward all its ratings to the evaluated contractor and give that entity 30 days to comment on the rating.⁽³²⁾ The State transportation department then reviews the contractor’s comments and determines whether or not to adjust the final rating.

CCASS evaluations are filed and remain in the contractor’s record for six years. They are used as part of the prequalification process on DB and other types of negotiated contracts, as well as to determine responsibility on DBB contracts. This is accomplished by adding the following requirement—that the contractor have no unsatisfactory performance evaluations on file in CCASS—to the Federal definition of a “responsible bidder.”⁽³²⁾

Based on the Federal Lands Highway Divisions experiences, it appears that using some form of evaluation of a given contractor’s actual CQC performance is perceived as having a positive impact on final project quality. If these perceptions are accurate, this approach to evaluation should be part of a performance-based contractor prequalification program.

Performance-Based Prequalification Implementation

NCHRP Synthesis 390 asked, “Can performance-based construction contractor prequalification be implemented in a way to reward good contractors and encourage poor contractors to improve performance?”⁽³²⁾ The FDOT “ability factor,” shown in figure 2 and table 2, is designed to reward a contractor’s past performance by assigning a higher “ability factor” to those who have exceeded the minimum requirements than to those who have not exceeded them.⁽³³⁾ The interviews with FDOT contractors familiar with the system indicated that not only did it furnish an incentive to maintain satisfactory quality, but that it was also generally deemed a fair and equitable system.⁽¹⁷⁾

Figure 2. Line graph. FDOT “ability factor” conversion from contractor’s past performance record.⁽³³⁾

 

Table 2. Impact of the FDOT “ability factor” on maximum capacity rating.⁽³³⁾

Contractor’s Past Performance Record	Ability Factor	Current Ratio Factor	Adjusted Net Worth (ANW) (in $ Millions)	Maximum Capacity Rating (in $ Millions)
Large Company—ANW > $100 million
98—100	15	1.3	$334.1	$6,515
74—76	4	1.3	$334.1	$1,737
64 or less	1	1.3	$334.1	$434
Medium Company—ANW $20 million to $100 million
98—100	15	1.3	$52.7	$1,028
74—76	4	1.3	$52.7	$274
64 or less	1	1.3	$52.7	$69
Small Company—ANW < $20 million
98—100	15	1.3	$1.5	$29
74—76	4	1.3	$1.5	$8
64 or less	1	1.3	$1.5	$2

 

The Canadian province of Ontario has also successfully implemented and sustained a system that encourages good performance. Contractors are rated in five work classifications: general road, structures, electrical, structural coating, and general maintenance. Both rated and new contractors need to be demonstrate satisfactory experience in a given classification to be awarded their full “basic financial rating,” which is an MTO term for the maximum amount of awarded work a given contractor can have ongoing at any given point in time. It is similar to a surety’s bonding capacity and is determined in much the same way. If constructors do not have MTO experience, their experience with other public road agencies is considered and their “basic financial rating” may be reduced in accordance with a published formula.⁽³⁴⁾ Additionally, a fully qualified contractor’s “basic financial rating” is reduced by the amount of its ongoing work to determine an available financial rating. This subtraction parallels the approach used to determine bonding capacity in the United States.

A contractor’s “available financial rating” is determined at the point in time when a bid is tendered, and it needs to equal or exceed the contract rating. For example, a contractor with a basic financial rating of $100 million that has $80 million worth of awarded, ongoing work at the time of the bid has a $20 million available financial rating. If the estimated value of the contract being bid is less than or equal to $20 million, then this contractor is allowed to submit a bid. If the contract value is greater than $20 million, then the contractor is unqualified to bid on this particular project, due to insufficient financial rating or capacity. Contractors with a record of poor performance may also be required to equal or exceed the maximum workload rating (MWL) for the project. The contractor performance index (CPI) is calculated from the contractor’s previously approved contract performance ratings (CPR) for each project and is used to determine if a contractor needs to satisfy the contract rating and the contract maximum workload rating.

Based on the CPI, a contractor is categorized into one of the three following “zones”:

• Green Zone: When CPI ranges from 70 to 100.
  
  
• Yellow Zone: When CPI is greater than 55 and less than 70.
  
  
• Red Zone: When CPI is greater than 35 and less than 55.
  
  
• A contractor with a CPI of less than 35 would not be deemed qualified to bid.
  
  

Green Zone contractors are allowed to bid on work up to their “financial basic rating” without adjustment for performance. Yellow zone contractors may have to meet the contract “maximum workload rating,” and may have their MWL reduced by a factor of up to 20 percent. Finally, Red Zone contractors will have their MWL reduced by a factor “calculated linearly 20 percent and 100 percent depending on their position in the zone (20 percent at 55 and 100 percent at 35).”⁽³⁴⁾ A contractor’s MWL is defined as the highest annual total dollar value of work awarded to a contractor in one of the five fiscal years preceding the current fiscal year.

The MTO includes an integrated infraction report system in the calculation of an adjusted financial rating that accounts for a contractor’s record of infractions issued against it. An infraction is defined as a serious breach of contract, and includes, but is not limited to, the following specific behaviors:⁽³⁴⁾

• Failure to abide by tendering requirements.
  
  
• Incomplete or inaccurate tender declarations.
  
  
• Failure to abide by General Conditions of Contract.
  
  
• Serious issues that affect safety or the environment.
  
  
• The unsatisfactory timeliness of the completion of the work and services.
  
  
• The issuance of any Notice of Default.
  
  
• The manner of the unsatisfactory resolution of any disputes and whether such disputes were resolved, in accordance with the prescribed provisions of the contract.
  
  
• When an Infraction Report is issued, the Qualification Committee may take no action, issue a warning letter, or reduce the contractor’s available financial rating for a specified period.
  
  

The infraction report becomes a very serious matter for contractors who wish to compete for work in Ontario, and the imposition of sanctions (adverse contract actions, such as termination) may create a severe hardship for an Ontario-based company, which then needs to leave the province to seek work. In the case of contractor exclusion (debarment), provisions in the policy create a distinct disincentive to immediately seek redress in the court system; if a contractor sues MTO, it may not do business with (e.g., award a contract to) that contractor. The infraction report also creates an incentive to complete MTO projects in a timely manner and in a fashion that is satisfactory to the MTO. Table 3 and table 4 show how five contractors distributed across the three zones that have the same “basic financial rating” and “maximum workload rating” would be affected by this system.

Table 3. Example of several MTO qualified contractors under a maximum workload rating adjustment.⁽¹⁷⁾

Contract Financial Rating = C$8 Million; Contract Maximum Workload Rating = C$5 Million
Contractor	Basic Financial Rating (MWL Rating)	Contractor Total Ongoing Work (Total Current MTO Work)	Available Rating	Infraction Sanctions	Contractor Performance Index	Impose MWL Rating?
A	C$12 million (C$6 million)	C$2 million ($1.5 million)	C$10 million	0	85 = Green zone	No
B	C$12 million (C$6 million)	C$2 million C$1.5 million)	C$9.4 million	5% (Environmental infraction)	74 = Green zone	No
C	C$12 million (C$6 million)	C$2 million (C$1.5 million)	C$10 million	0	65 = Yellow zone1	Yes: MWL = C$6 million
D	C$12 million ($6 million)	C$2 million C$1.5 million)	C$10 million	0	50 = Red zone2	Yes: MWL = C$6 million
E	C$12 million (C$6 million)	C$2 million (C$1.5 million)	C$8.2 million	15% (Safety infractions)	72 = Green zone	No

MWL = Maximum workload MTO = Ontario Ministry of Transportation 1 Yellow zone penalty is discretionary; MTO can impose a reduction of the MWL of up to 20 percent. For this example, no penalty is assessed, but MWL is imposed. 2 Red zone = Automatic linear reduction from 20 to 100 percent for the rating between 35 and 55.

 

Table 4. Example of several MTO qualified contractors under a maximum workload rating adjustment, part 2.

Contractor	MWL Reduction Adjustment	Exclusion?	Adjusted Financial Rating	Adjusted MWL Rating	Prequalified?
A	N/A	No	C$10 million	N/A	Yes C$10 million > C$8 million
B	N/A	No	C$9,400,000 (5% infraction = C$600,000 reduction)	N/A	Yes C$9.5 million > C$8 million
C	0%	No	C$10 million	C$4.5 million (MWL C$6 million less MTO work C$1.5 million)	No C$4.5 million < C$5 million
D	40%	No	C$10 million	C$2.7 million (60% of MWL C$6 million less MTO work C$1.5 million)	No C$2.7 million < C$5 million
E	N/A	Yes (ongoing lawsuit)	Excluded	N/A	No Excluded, due to ongoing lawsuit

N/A = Not Applicable MWL = Maximum workload MTO = Ontario Ministry of Transportation

The third and final question addressed by NCHRP Synthesis 390 is the following:“Can performance-based construction contractor prequalification add value to the completed construction project?”⁽¹⁷⁾ Both the synthesis survey and its contractor interviews found that the implementation of performance-based contractor prequalification was perceived as having a positive impact on the quality of various components of a project, thereby enhancing the value of that project. Since Ontario has a rigorous prequalification process and has consequently omitted the performance bond requirement for contractors, it has freed up savings in bonding costs, improved project quality, and potentially reduced contract administration effort that can be applied to other projects, which demonstrates the potential that performance-based construction contractor prequalification has to add value to the public transportation construction program.⁽¹⁷⁾

Literature on Adjusting Bidding and Bonding Capacity

The performance-based contractor prequalification programs detailed in NCHRP Synthesis 390 reduce a marginal contractor’s bidding capacity in various ways, and thereby avoid “subsidizing” poor performance (see the Minchin and Smith interviews and those cited in NCHRP Synthesis 390).^(1,17) MTO’s program eliminated performance bonding altogether over 20 years ago.⁽¹⁷⁾

Since the cost of furnishing a performance bond is passed on to the State transportation department in the contractor’s bid, adjustment of the portion of a project value that requires a performance bond seems a logical approach to bonding modification within a performance-based contractor prequalification system. For example, a top performing contractor might only be required to furnish a bond on 50 percent of the contract amount, whereas less qualified contractors or contractors new to the State transportation department would need to bond 100 percent of the contract amount. A reduction in the percentage of the project value that requires a bond, based on a contractor’s past performance, would create an incentive for superior performers by reducing their bid price. U.S. performance bond rates run between 1 and 3 percent of the total bid amount, so this change in bid prices would be significant.⁽³⁶⁾ These savings would be passed on directly to State transportation departments because they would secure lower construction costs for projects. In fact, Florida, Maine, Virginia, and Washington have all experimented with using bonding for less than the entire contract amount.⁽¹⁷⁾

Many State transportation departments perceive that the purpose of bonding is to protect the State against contractor default.⁽¹⁴⁾ However, the majority of U.S. and Canadian survey respondents to the NCHRP Synthesis 390 survey stated that a performance bond was not sufficient evidence of contractor prequalification. A rigorous performance-based prequalification system in place allows contractors who fail to complete a project to be penalized via a reduction in their performance rating or by their ultimate removal from the bidding list.

A rigorous performance-based prequalification system can carry more financial weight than a purely financial bonding system. As stated previously, MTO has a long history of not requiring bonds from their contractors. MTO’s annual construction program is approximately C$2.1 billion. Based on the U.S. 2007 to 2009 national average, the average performance/payment bond costs would be 1.139 percent of the total contract cost.⁽¹¹⁾ Thus, the estimated savings (not accounting for any possible contract defaults) is approximately C$24 million per year in bond costs. MTO utilizes a three-component system to thoroughly review the stability of its bidders:⁽³⁵⁾

• Administrative prequalification.
  
  
• Performance prequalification.
  
  
• Infraction system.
  
  

MTO stresses that the success of the system lies in the interrelationships between the components. While certain components of the system can function independently, the program works best when all the components are integrated.

The NCHRP Synthesis 390 Three-Tiered Prequalification System

NCHRP Synthesis 390 proposes a three-tiered performance-based contractor prequalification system. This approach was developed, based on the study’s comprehensive literature review, including the survey responses recorded from 41 U.S. State transportation departments and 7 Canadian provincial transportation ministries, a content analysis of solicitation documents from 35 State transportation departments, and interviews with 10 construction contractors from firms ranging in size from a local chip seal contractor to a major national Heavy Civil contractor. The recommended process is summarized in figure 3.

Figure 3. Flowchart. Three-tiered performance-based prequalification process.⁽¹⁷⁾

NCHRP Synthesis 390 uses the following definitions for each “tier” in the process detailed in figure 3:⁽¹⁷⁾

• Administrative prequalification: “A set of procedures and accompanying forms/ documentation that must be followed by a construction contractor to qualify to submit bids construction projects using traditional project delivery. These include evaluation of financial statements, dollar amount of work remaining under contract, available equipment and personnel, and previous work experience. This may be on a project-by-project basis or on a specified periodic basis.”
  
  
• Performance-based prequalification: “A set of procedures and back-up documents that must be followed by a construction contractor to qualify to submit a bid on a construction project based on quality, past performance, safety, specialized technical capability, project-specific work experience, key personnel, and other factors. This may be on a project-by-project basis or on a specified periodic basis and the project could be delivered using traditional design-bid-build or alternative project delivery methods such as design-build, construction manager/general contractor, or any other method.”
  
  
• Project-specific prequalification: “Contractor prequalification requirements that exist only for a single project. These normally address project technical/procurement factors that are considered essential for the success of the given project. They may include criteria that require the contractor to have had past experience building a certain technology (i.e., seismic retrofit, information and technology systems, etc.) or a given project delivery method such as design-build. They may also extend to cover specific experience for key project personnel and specific types of plant and equipment.”
  
  

Figure 4 provides additional details on the components of tier three.

Figure 4. Flowchart. Three-tiered process with tier three-project-specific details.

Barriers to Implementation

The literature on performance-based prequalification devotes a significant amount of attention to evaluating the barriers and challenges to implementing a performance-based contractor prequalification system. (See references 1, 2, 17, 35, and 37.) Ultimately, the various analyses reviewed in this study noted very few significant barriers to the implementation of performance-based contractor prequalification. Many State transportation departments already have some form of contractor evaluation included in their bid process; many have some form of performance-based prequalification included as well. Based on input provided by contractors, as collected in the literature, contractors would seem to welcome implementation of this approach as a tool to reduce or remove the number of marginally qualified contractors, against which they need to compete. Nevertheless, NCHRP Web Document 38 notes that the following implementation issues will need to be addressed when a performance-based prequalification process is introduced:⁽¹⁾

• Integration with existing construction administration systems, such as SiteManager.
  
  
• Consideration of the evaluators’ qualifications.
  
  
• Evaluation process administrative rules.
  
  
• Frequency of evaluations.
  
  
• Appeals process development.
  
  
• Lifespan of evaluations/duration of disqualification.
  
  
• Impact on contractor bonding.
  
  
• Legal implications.
  
  

Of the potential barriers listed above, significant focus should be placed on the implementation of administrative rules for the evaluation process.⁽²⁾ The State transportation department will need to ensure that its evaluators are indeed qualified to evaluate the subject contractors. In most cases, contractors should be evaluated by the State transportation department construction personnel who administer the evaluated contract. Implementation will require that an ongoing training program for the evaluators be developed and implemented to ensure consistency between evaluators and across different types of projects. This component of the program will also be necessary to demonstrate the State transportation department commitment to fairness and to ensure the reduction of as much subjectivity in the process as possible. Agencies that currently use this type of system (such as FHWA and FDOT) have found that a review of all contractor evaluations one level above the “evaluator” is also required to make the program as consistent as possible.^(31,33) This issue was highlighted in NCHRP Synthesis 390, in which 8 out of 10 interviewed contractors indicated that their major concern with performance-based prequalification is the agencies’ ability to consistently rate them from project to project.

The administrative rules of the process also need to be transparent and logically derived.⁽²⁸⁾ It is important to determine the frequency of evaluations. The literature on this topic seems to support that at least one interim evaluation is provided to the contractor before the final evaluation. (See references 31, 32, 33, and 34.) FDOT furnishes evaluations on a monthly basis. The crucial element will be to notify the contractor when it is not performing well and to provide the contractor with the opportunity to correct its deficiencies and shortcomings, before negative evaluations become part of its permanent record. There is a need for an appeals process whereby the contractor can refute an unfavorable rating, which provides the contractor with due process before it is penalized by the evaluator.

The question of the appropriate length of an evaluation’s life span should also be addressed as an integral component of the evaluation process. NCHRP Synthesis 390 found that the majority (73 percent) of its survey respondents maintained evaluations in their active record for at least three years. Survey results also support this time interval, and literature on the subject recommends a “rolling 3-year average.”^(14,34) This selected duration creates an incentive for contractors to perform in a satisfactory manner, since a bad evaluation could impact the work that they can secure for a three-year period. The amount of time a contractor can remain disqualified due to certain behavior may be longer. Those that lose their qualification for criminal acts are usually debarred from participation indefinitely. In contrast, those that are disqualified for marginal performance, usually for defaulting on a contract, are able to regain their qualification after proving to the State department of transportation that they have corrected the problems that caused the default(s).

REVIEW OF STATE PREQUALIFICATION EVALUATION PROCEDURES

A survey conducted for NCHRP Web Document 38 found that 29 States used some type of contractor prequalification process.⁽¹⁾ According to Study of Most Effective Practices for Determining Construction Contractors’ Eligibility to Bid on Construction Projects, the major factors used to arrive at a contractor’s prequalification rating are financial resources, experience, availability of necessary equipment, and past performance.⁽¹⁸⁾ MDOT is one example of a State that uses a prequalification procedure. In MDOT’s current procedure, all prime contractors and subcontractors who intend to bid on projects, as well as those who request prequalification, need to be prequalified before they can submit a bid. The prequalification process follows the Bureau of Finance and Administration’s classification and rating of bidders’ administrative rules. MDOT currently requires a bid guarantee from all prime contractors bidding on a project. A performance bond is required from all prime contractors before they can begin work on MDOT projects, and MDOT requires that contractors renew their qualifications on an annual basis. Due to a recent change to the administrative rules, all contractors with a financial rating of more than $10 million can now renew their qualification every two years.⁽¹⁸⁾

Post-qualification practices are also used by numerous State transportation departments. These typically involve consideration of a contractor’s qualifications after the contractor has been selected on a low-bid basis. These qualifications are submitted in response to a State transportation department questionnaire to verify compliance with requirements of the contract. Post-project performance evaluation practices that impact contractor eligibility are in use at many State transportation departments.⁽¹⁸⁾ Ahn conducted a survey of U.S. States in 2008 and found that 28 States rely on some form of post-project evaluation.⁽⁶⁷⁾ The New York State Department of Transportation and the Rhode Island Department of Transportation both currently use post-qualification for contractor selection. Neither State currently has a specific performance-based rating evaluation scale for contractors. They rely on post-qualification instead of performance evaluations.⁽¹⁸⁾

State transportation departments generally review multiple factors in order to prequalify or post-qualify contractors. They typically assign a maximum amount of work a contractor can perform and the type of work they are allowed to perform, based on qualification determinations. The factors most commonly used by State transportation departments to evaluate contractors are as follows:⁽¹⁸⁾

• Past experience of contractor (87 percent).
  
  
• Financial capability (75 percent).
  
  
• Equipment and plant (72 percent).
  
  
• Past performance evaluations (72 percent).
  
  
• Past illegal behavior (72 percent).
  
  
• Detailed financial analysis (66 percent).
  
  
• Qualifications (resumes) (66 percent).
  
  
• Bonding capacity (60 percent).
  
  
• Calculated capacity factor (57 percent).
  
  
• Level of subcontracting (40 percent).
  
  

Not only may State transportation departments rely on different factors, but they may also require slightly different forms of documentation as proof from contractors. The actual amount of effort required for review by each State transportation department depends on the factors evaluated and the type and amount of proof required and reviewed.

State transportation department prequalification and post-qualification practices share certain similarities and differences across States. A majority (25 out of 33) of the States surveyed in Dye Management Group’s eligibility practices reported that they only use prequalification methods to prequalify prime contractors. In addition, most States prequalify contractors in different work categories to ensure that the prequalification process accurately accounts for the fact that construction disciplines are varied and require different skills. The number of work classifications used varies by State transportation department. Most States (31 of 48 surveyed in NCHRP Synthesis 390, and 7 Canadian ministries of transportation) monitor contractor performance on projects, though the information obtained through monitoring is not used in the prequalification/eligibility determination process in all States. While some State transportation departments change a contractor’s prequalification status based on their project performance, others do not. Policies regarding how to modify State limits are well documented and standardized in some State transportation departments, while others use a more subjective, less standardized approach.⁽¹⁷⁾

Most State transportation departments use a combination of methods to determine contractor eligibility. The results of 40 State transportation department surveys, as presented in Dye Management Group’s report, along with information from NCHRP Synthesis 390, show that performance bonding and bid bonding/guarantee are the most widely used methods to determine the eligibility of contractors, followed by general prequalification.^(17,18) Any contractors that do not submit a bond/guarantee are almost always disqualified.

 

Figure 5. Common eligibility determination methods.^(17,18)

 

MDOT currently determines a contractor’s financial rating (also referred to as financial capability) by requiring that proof of financial capability be provided; this requirement needs to be met in order for a contractor to be prequalified. The financial rating for a contractor is calculated as follows: working capital (either positive or negative) is multiplied by nine; depreciation expenses on construction and transportation equipment allowed on contractor/bidder’s book of accounts is multiplied by nine; net booked construction and transportation equipment values, minus any long-term debt the equipment secures, is multiplied by four. The sum of all three factors is the total finance rating. If the contractor is applying for a financial rating of over $1 million, the contractor is required to provide a certified audit (i.e., an audited financial statement). If the contractor is applying for a financial rating of less than $1 million, the contractor submits a compiled financial statement, a reviewed financial statement, or an accounts receivable and bank statement verification form.

MDOT has used this formula to calculate financial rating for a number of years. The financial rating calculated is truncated to the number of thousands of dollars for ease of use and reporting, and is a general rating that applies to all of the 53 different work areas. The more specific financial rating for a work classification, which is calculated separately, can be lower than the overall financial rating. For specific work classifications (e.g., clearing and grubbing, asphalt, concrete, etc.), a contractor requests approval and submits information regarding their equipment, personnel, and team experience, which are considered components of the specific work class financial rating. A review of these factors is subjective, with no specific weights assigned to each, and a financial rating is determined for each work classification.⁽¹⁸⁾

State transportation departments that perform post-project evaluations use a variety of rating systems for their performance evaluations. The rating scales range from 4 to 11 numbers. Many State transportation departments use similar categories to one another to evaluate performance, but group them in different combinations. The narrowness or breadth of categories used varies from State transportation department to State transportation department. Twenty-five of the 26 State transportation departments surveyed that conduct post-project evaluations cited quality and management as categories used in their rating evaluation process. Scheduling was cited by 21 State transportation departments, while safety, traffic, and U.S. Department of Transportation (USDOT) cooperation were cited by 19 State transportation departments.⁽¹⁸⁾

The frequency of evaluation for post-project evaluation practices varies among State transportation departments. Nearly all evaluations are conducted at specific intervals. The Oregon Department of Transportation (ODOT, presented as ODOT-OR for the purposes of this report) evaluates the firm contractually responsible for a project annually, on the anniversary date of the notice to proceed for a project. Once completed, contractor performance evaluations are combined in a 12-month rolling average. The Virginia Department of Transportation (VDOT) incorporates their contractor evaluations into their monthly interim reports and into an annual or final report. In Connecticut, evaluations are conducted annually, as well as after the completion of projects.⁽¹⁸⁾

The South Dakota, North Carolina, Florida, Utah, and Maine transportation departments have minimum thresholds for prequalification that range from $50,000 to $250,000. By contrast, the city of Clearwater, FL, determines its prequalification threshold on a contractor-by-contractor basis that is dependent upon previous project performance. The Delaware Department of Transportation uses a project-specific prequalification process and the Detroit Metropolitan Wayne County Airport uses prequalification based specifically on the technical complexity of a project. VDOT changed its practice in 2006 and 2007 to allow contractors who have consistently performed well to bid on any amount of work, so long as they can provide a performance bond. Newer or marginal contractors are assigned specific limits and may be put on probationary status or given conditional prequalification. VDOT also reserves the right to review applications on a case-by-case basis. VDOT reported consistent contractor performance after it implemented the change, and is able to apply its savings-achieved through the elimination of internally conducted detailed financial reviews-in order to conduct more safety and quality reviews.

Table 5. Summary of State transportation department contractor performance evaluation categories.

State

Quality-Workmanship and Materials

Safety

Sche- dule

Home Office Support

Compli- ance

Environ- mental

Organiza- tion

Equip- ment

Payment of Accounts

Project Management

Prosecution of Work

Relations/ Cooperation

Traffic Control

Connecticut

x

x

x

x

x

Florida

x

x

x

x

Illinois

x

x

x

x

x

x

x

Indiana

x

x

x

x

x

Iowa

x

x

x

x

Kansas

x

x

x

x

x

x

Kentucky

x

x

x

x

x

x

x

x

Maine

x

x

x

x

x

Maryland

x

x

x

x

x

x

x

Massachusetts

x

x

x

x

x

x

x

x

x

Michigan

x

x

x

x

Missouri

x

x

x

Nebraska

x

x

x

x

New Jersey

x

x

x

x

x

x

x

Ohio

x

x

x

x

Oklahoma

x

x

x

x

x

x

x

x

Oregon

x

x

x

x

x

x

x

x

Pennsylvania

x

x

x

South Carolina

x

x

x

x

x

Utah

x

x

x

x

x

x

x

x

Vermont

x

x

x

x

x

x

x

x

Virginia

x

x

x

x

Washington

x

x

x

x

West Virginia

x

x

x

x

x

x

 

Summary of State Performance Evaluation Documentation and USDOT DBE Outreach Efforts

Connecticut DOT (ConnDOT)

ConnDOT requires a contractor performance evaluation for contractors and subcontractors. Performance evaluations are undertaken annually and at the time of project completion for prime contractors, and only at the time of project completion for subcontractors. The rating evaluation form includes fairly detailed descriptions for ratings in five categories. The categories included in the form are quality of the work; performance of work; adherence to project schedule; implementation of Federal, State, and local policies, procedures, and regulations; and procedural/administrative. Connecticut has established consequences and “trigger” values for contractor performance ratings. These are described in ConnDOT’s construction manual, as follows:⁽³⁸⁾

• These forms from all Districts are compiled and the five-year average is used to determine an overall, representative rating for each Contractor. They are used by the Department when determining the qualification of contractors, as well as responsibility issues. The Office of Construction provides the five-year average to the Office of Contracts and other interested units or agencies as requested.
  
  
• If any annual project or overall project rating is at or below a 2.0, the Office of Construction sends a memorandum to the District requesting that the District meet with the firm to discuss the problems or deficiencies noted on the review.
  
  
• If the five-year average rating is at or below a 2.5, the Office of Construction sends the firm a letter (copy the Office of Contracts), noting that their performance was lacking in certain areas and that they need to improve on future projects.
  
  
• If the five-year average rating is at or below a 2.0, the Office of Construction sends the firm a letter (copy the Office of Contracts), putting the firm on notice that the firm is at risk of being found non-responsible.
  
  
• A finding of non-responsibility may impact a firm’s future ability to be awarded work on Department projects.
  
  

Florida Department of Transportation (FDOT)

FDOT details their procedures and policies for the completion of the Contractor’s Past Performance Report in their Construction Project Administration Manual. They also provide a link to a standard spreadsheet for the Contractor’s Past Performance Report. The Contractor’s Past Performance Report is used to establish the contractor’s “Final Rating Score,” which is used to determine the contractor’s maximum capacity rating.⁽³⁹⁾

Indiana DOT (INDOT)

INDOT uses a standard form to rate contractors. The instructions include specific guidelines on the ratings to be given, and place particular emphasis on communication with the contractor throughout the project. Contractors are rated in the following areas:

• Organization, equipment, and personnel.
  
  
• Prosecution of the works.
  
  
• General relationships/cooperation.
  
  
• Quality of workmanship and materials.
  
  

Interestingly, INDOT also uses the same form to rate subcontractors. The positive and negative consequences associated with each rating are noted in the form in terms of positive and negative numbers, which trigger investigatory meetings with INDOT to remedy the specific issues.⁽³⁹⁾

Illinois Department of Transportation (IDOT)

IDOT rates contractors on a single form and guidelines for ratings are provided on the form. The form consists of two parts: the first part focuses on quality, and assigns a numerical rating to different work type categories (from earthwork to electrical to marine construction); the second part focuses on execution of the work. The contractor is evaluated for project execution across six categories: organization/prosecution of the work, cooperation, traffic control/site preparation, Equal Employment Opportunity (EEO)/labor compliance, erosion control, and quality control/quality assurance.⁽⁴¹⁾

Iowa Department of Transportation (IOWADOT)

IOWADOT conducts performance evaluations for every contract in excess of $20,000 and for each bridge painting contract. The Contractor Evaluation Report needs to be completed for contractors and subcontractors. The contractor is evaluated in four areas: organization management, work performance, safety practices, and equipment.⁽⁴²⁾ The referenced document does not specify how the performance rating can impact a contractor or subcontractor’s ability to continue to bid IOWADOT projects.

Kansas Department of Transportation (KDOT)

KDOT uses a relatively simple evaluation form for contractors. The form consists of eight categories, in which contractors are rated by both the field office and the district office:⁽⁴³⁾

• Work quality.
  
  
• Work timeliness.
  
  
• Payment of accounts.
  
  
• Cooperation with owners.
  
  
• Cooperation with the public.
  
  
• Public safety.
  
  
• Work site safety.
  
  
• Contract compliance.
  
  

Kentucky Transportation Cabinet (KYTC)

KYTC requires a Contractor’s Performance Report on every project. Both prime contractors and subcontractors receive a performance evaluation. Prime contractors receive an annual report and a report at contract completion. Subcontractors receive a Performance Report at project completion or annually if they undertake a significant amount of work in a given year. The Contractor’s Performance rating is reviewed by both a section engineer and a chief district engineer. Contractors have an opportunity to appeal the rating within 10 business days. KYTC’s Instructions and Guidelines for Contractor’s Performance Report notes that the contractor’s average performance rating will be used to determine the Contractor’s Maximum Eligibility Amount: “The Contractor’s average performance rating (weighted by dollar amount of work performed) for the previous year will be used in the calculation of the Contractor’s Maximum Eligibility Amount. Utilizing the scores from the Contractor’s Performance Report will provide incentive for the Contractor to consistently perform at a high level of quality.”⁽⁴⁴⁾

Maine Department of Transportation (MaineDOT)

MaineDOT has a form posted online for contractor performance evaluation. Contractors are rated in seven categories as above standard, standard, or below standard. The seven categories are quality of work; subcontractor’s cooperation; environmental; safety; implementation of Federal, State, and local procedures and regulations; and procedural/administrative. Several of the categories consist of one to six subcategories. No administrative guidelines for completing the contractor performance rating were available on the website.⁽⁴⁴⁾

Maryland State Highway Administration (MDSHA)

MDSHA provided a copy of their form for rating contractors and subcontractors. The form details seven categories and assigns various weights to each as follows:⁽⁴⁶⁾

• Contractor administration, personnel, equipment, partnering and public relations.
  
  
• Minority-owned disadvantaged business/DBE/women’s business enterprise compliance.
  
  
• Quality of work.
  
  
• Safety.
  
  
• Project schedule.
  
  
• Erosion and sediment quality assurance (QA) or environmental stewardship.
  
  
• Maintenance of traffic QA.
  
  

Massachusetts DOT (MassDOT)

MassDOT recently updated its policy through the issuance of an Engineering Directive on

April 20, 2010. This directive introduced new Contractor Performance Reports for both contractors and subcontractors. The Contractor Performance Reports indicate that a deduction in the contractor’s maximum amount of work rating is assessed if the rating is below 80 percent or if the project was completed late, due to the contractor’s fault. The contractor’s performance is evaluated in 9 categories with ratings of between 4 (poor) and 10 (excellent) assigned. Different weights are also applied to the individual category ratings. The categories rated and their respective weights out of a total of 10 are as follows:⁽⁴⁷⁾

• Workmanship (2.0).
  
  
• Safety (2.0).
  
  
• Schedule (1.5).
  
  
• Home office support (1.0).
  
  
• Subcontractor performance (1.0).
  
  
• Field supervision/superintendent (1.0).
  
  
• Contract compliance (1.0).
  
  
• Equipment (0.5).
  
  
• Payment of accounts (0.5).
  
  
• Contractor project management and administration (0.5).
  
  

Michigan Department of Transportation (MDOT)

MDOT revised its Contractor Performance Evaluation (CPE) procedure on January 13, 2011. All contracts let through the MDOT letting process require a CPE. Both contractors and subcontractors are evaluated, and the evaluations are completed in Field Manager. Evaluations are completed at the end of each project or annually for multi-year projects. As described in the memorandum, contractors are evaluated in the following four prime categories:⁽⁴⁸⁾

• Organization and management.
  
  
• Resources.
  
  
• Work performance.
  
  
• Subcontractor management.
  
  

Missouri Department of Transportation (MoDOT)

MoDOT undertakes contractor evaluations in the form of a questionnaire. The questionnaire consists of nine sections, with a considerable number of measures included in each. The sections are as follows:⁽⁴⁹⁾

• General provisions.
  
  
• Earthwork.
  
  
• Base and aggregate surfaces.
  
  
• Flexible pavements.
  
  
• Rigid pavements.
  
  
• Incidental construction.
  
  
• Structures.
  
  
• Roadside development.
  
  
• Traffic control facilities.
  
  

Nebraska Department of Roads (NDOR)

NDOR uses a single form to conduct contractor evaluations. The form consists of nine categories, which are evaluated with a rating scale of unsatisfactory, poor, fair, good, excellent, and superior. The NDOR construction manual indicates that poor contractor performance ratings may be considered when determining the contractor’s bid rating.⁽⁵⁰⁾ The construction manual also indicates that this form is normally completed within a computer program, although paper versions are permitted. The date on this form is November 1978, which indicates that it may be an outdated version; an updated version may be available in Nebraska’s SiteManager.

New Jersey Department of Transportation (NJDOT)

NJDOT revised its Contractor’s Performance Evaluations process in June 2011. One unique feature of the New Jersey process is the central office’s ability to promulgate custom evaluations for non-typical projects. This is the only instance found of a project-specific performance evaluation (not to be confused with project-specific prequalification). This process creates a mechanism whereby the evaluation can be directly correlated with the critical areas of performance. An example furnished in the document discusses how a project for building a large retaining wall requires a much higher degree of CQC to achieve the desired material and workmanship quality than an asphalt overlay project. Hence, NJDOT is able to raise the standard for a given project without impacting all other current projects. The degree of flexibility that the project-specific performance evaluation plans provides gives NJDOT the ability to reward excellent performance in project-critical tasks.⁽⁵¹⁾

Ohio Department of Transportation (ODOT-OH)

ODOT-OH evaluates contractors through a series of questions in four categories. Each question is responded to with a rating scale of 1, 5, 8, and 10. The categories are as follows:⁽⁵²⁾

• Organization and management.
  
  
• Equipment.
  
  
• Work performance.
  
  
• Subcontractor management.
  
  

Oklahoma Department of Transportation (ODOT-OK)

ODOT-OK (the Oklahoma Department of Transportation is known as ODOT but is represented as ODOT-OK for purposes of this report) uses a Contractor Performance Evaluation Form, along with supporting information, and the form is available on their website. The evaluation form provides guidance on how to use five numerical ratings to rate contractors. ODOT-OK evaluates contractors in the following six areas:⁽⁵³⁾

• Quality of work.
  
  
• Organization and prosecution of work.
  
  
• Cooperation.
  
  
• Traffic control and maintenance of traffic.
  
  
• Erosion control.
  
  
• EEO and DBE and labor compliance.
  
  

Oregon Department of Transportation (ODOT-OR)

The process to evaluate contractors is described in detail in the Oregon Administrative Rules. ODOT-OR evaluates prime contractors annually, and then within 60 days of a project’s substantial completion. (This is termed “second notification” in ODOT-OR’s materials, and is defined as “the date on which required construction work, including change order work and extra work, has been satisfactorily completed, except for minor corrective work, and the recording of daily time charges cease.”)⁽⁵⁴⁾ Contractor performance evaluations are collected and combined in a 12 month rolling average. Should a contractor’s performance fall below a score of 80, varying degrees of consequences are imposed, with provisions for multiple occurrences of poor performance. Oregon evaluates contractors in nine categories; several sub-criteria are examined within each category. The evaluation is presented in a questionnaire-type format, and each question is answered with a five-point scale. The evaluated categories are as follows:⁽⁵⁴⁾

• Supervision.
  
  
• Progress schedule.
  
  
• Quality of materials and workmanship.
  
  
• Payment.
  
  
• Affirmative action.
  
  
• Safety.
  
  
• Traffic control.
  
  
• Compliance with regulations.
  
  
• Major breach.
  
  

Pennsylvania Department of Transportation (PennDOT)

PennDOT completes all contractor performance evaluations electronically, in a system called the Engineering and Construction Management System. The most current contractor performance evaluation and the preceding five evaluations are used in a semi-annual determination of the Contractor’s Ability Factor. This ability factor is then used to determine the contractor’s maximum capacity rating. The Contractor’s Past Performance Report has three main rated categories (listed below); each consists of several subcategories. The evaluation is conducted in a questionnaire format, with several questions included within each subcategory. The rating scale uses decimal numbers less than or equal to 1.00. The following three categories are evaluated:⁽⁵⁵⁾

• Managing the project.
  
  
• Managing compliance.
  
  
• Managing resources.
  
  

South Carolina Department of Transportation (SCDOT)

SCDOT has an extensive CPE system. Their system is particularly unusual in that 80 percent of the rated categories are objective measures. SCDOT has provisions in their system to address the entry of new contractors, whereby these firms are prequalified on the basis of their safety records, as expressed by the Experience Modification Rate, which is assigned based on their workers’ compensation insurance premiums.⁽⁵⁶⁾ As performance evaluation data is collected, it is integrated into the new contractor’s ratings. SCDOT can choose to create a project-specific prequalification by designating a minimum CPS (Contractor Performance Score) required to be permitted to bid. Such projects are typically more complex than normal or have some requirement that SCDOT deems a justification to restrict the pool of potential bidders to those with CPSs that are above the minimum required CPS. The contractor performance evaluation has 6 categories, each of which can be assigned up to 100 points. The first five scoring categories are all objectively derived from specific performance measures. The different performance measures and their respective weights are as follows:⁽⁵⁶⁾

• Safety (15 percent): based on workers compensation claims files.
  
  
• On-budget (15 percent): based on the paid amount compared to the bid amount.
  
  
• On-time (20 percent): based on how well the contractor met the “Substantial Work Complete Date.”
  
  
• Quality (20 percent): based on the results of a Quality Management Team audit field scores.
  
  
• Claims denied (10 percent): based on the percentage of the claim amount denied.
  
  
• Assessment by the resident engineer (20 percent): this process consists of a subjective evaluation and follows an 18-question questionnaire format.
  
  

SCDOT statistically analyzes the entire population of CPSs for a given year and establishes the Contractor Performance Threshold (CPT), defined as the “CPS below which performance is judged to be substandard.” The year’s CPT is the point equal to the mean CPS, minus two standard deviations. For example, if the mean CPT is 77.9, and the standard deviation of the population is 4.4, the CPT will be 69.0. Once set, a contractor whose CPT falls below 69.0 will not be allowed to bid on projects that have a minimum required CPS.

Utah Department of Transportation (UDOT)

The UDOT Contractor Performance Rating consists of the rating of 40 questions on a scale of 0 to 10, where 0 indicates non-performance and 10 indicates performance 100 percent of the time. The 40 questions included are grouped into 9 categories that are assigned various weightings. The categories, with their corresponding weightings, are as follows:⁽⁵⁷⁾

• Quality control/workmanship: 10 percent.
  
  
• Safety: 15 percent.
  
  
• Work zone traffic control: 15 percent.
  
  
• EEO labor compliance: 5 percent.
  
  
• Environmental compliance: 10 percent.
  
  
• Administration/organization/supervision: 10 percent.
  
  
• Partnering: 5 percent.
  
  
• Scheduling: 15 percent.
  
  
• Public relations: 15 percent.
  
  

Vermont Agency of Transportation (VTrans)

VTrans evaluates contractor performance in 11 categories and asks a series of questions in each category. The Prime Contractor Project Performance Evaluation is considered in the prequalification process by the prequalification committee and is used as the basis from which to accept, deny, or modify the number of projects a contractor is permitted to bid on at any given time. The evaluated categories are as follows:⁽⁵⁸⁾

• Contract administration.
  
  
• Personnel.
  
  
• Submittals/certifications.
  
  
• Environmental.
  
  
• Structures.
  
  
• Covered bridges.
  
  
• Paving.
  
  
• Earthworks.
  
  
• Traffic control.
  
  
• Incidental construction.
  
  
• Safety.
  
  

Virginia Department of Transportation (VDOT)

VDOT has a contractor evaluation process that consists of monthly interim reports and an annual, or final, report. The CPE Interim Report is completed monthly for 48 possible work categories. Monthly interim reports are completed for both contractors and subcontractors. Several questions are included for each work category and are scored on a scale from 0 to 4. The annual or final report consists of several sections, the first two of which relate most specifically to the contractor evaluation. The first section of the report combines all of the monthly scores from the interim reports. The second section of the report consists of an evaluation of the contractor in four categories by the district/area construction engineer. There are several questions within each category, which are scored from one to five, and five is the highest score possible. VDOT uses these performance evaluations in their prequalification process. The consequences for poor performance are laid out in a VDOT directive/memorandum: “a contractor/Subcontractor will be removed from the list of qualified bidders if it receives one score below 60 or three scores in a 24-month period below 70.”⁽⁵⁹⁾ The four rated categories are as follows:⁽⁶⁰⁾

• Safety.
  
  
• Company management of project.
  
  
• Environmental.
  
  
• Final product.
  
  

Washington State Department of Transportation (WSDOT)

WSDOT’s contractor performance evaluation system has two evaluation components. The first component is the numerical rating of contractor performance in four categories, while the second component of the contractor evaluation is comprised of a narrative. The quantitative component’s categories are as follows:

• Administration/management/supervision.
  
  
• Quality of work.
  
  
• Progress of work.
  
  
• Equipment.
  
  

The second component of the contractor evaluation is comprised of a three-part narrative. The purpose of this narrative is to furnish factual explanations with specific citations from the project record to document both good and poor work. The narrative, qualitative categories are as follows:

• General elements.
  
  
• Below standard elements.
  
  
• Superior elements.
  
  

Contractors with poor performance may be assigned a “conditional qualification” status, which may limit their bidding ability as follows: “[c]onditional qualification of a contractor may be affected when the overall performance of the contractor has become less-than-standard... A contractor placed in conditional status may be restricted in bidding ability for highway projects or other sanctions may be placed in effect.”⁽⁶¹⁾ WSDOT does have an appeals process for both the contractor’s performance rating and for restrictions placed on bidding.⁽⁶¹⁾

West Virginia Department of Transportation (WVDOT)

WVDOT staff complete a form for contractor performance. The contractor is evaluated in nine areas rated as outstanding, satisfactory, fair, or unsatisfactory. The nine categories are as follows:⁽⁶²⁾

• Workmanship.
  
  
• Performance.
  
  
• Supervision.
  
  
• Coordination.
  
  
• Labor.
  
  
• Equipment.
  
  
• Contractor/engineer relationship.
  
  
• Contractor/public relationship.
  
  
• Choice of subs and suppliers.
  
  

Wyoming Department of Transportation (WYDOT)

WYDOT evaluates contractors using a six-level scale and a “not applicable” rating option. Contractors are evaluated in 10 categories; each category consists of several subcategories. The categories are as follows:⁽⁵⁹⁾

• Management and organization of prime work.
  
  
• Management and organization of subcontract work.
  
  
• Project processes and submittals.
  
  
• Working relationship with WYDOT personnel.
  
  
• Prosecution of the work.
  
  
• QC.
  
  
• Traffic control.
  
  
• Compliance with work site requirements.
  
  
• Cooperation with others/public relations.
  
  
• Completion of project.
  
  

Through contact with State transportation departments’ staff, it was determined that 10 States do not rate contractor performance. These States are Alabama, Arizona, Arkansas, Hawaii, Idaho, Minnesota, New Mexico, North Dakota, South Dakota, and Texas.

United States Department of Transportation (USDOT)

The USDOT Office of Small Business Bonding Education Program partners with the surety industry nationally to encourage small businesses become bondable. Historically, bondability has been a challenge for many disadvantaged businesses and this program aims to help businesses grow by obtaining or building bonding capacity.

The USDOT Bonding Education Program proactively encourages small contractors to conduct an assessment of the administrative functions of the business to mitigate risks and using the surety prequalification process to build a profitable and a sustainable business, and ultimately become bondable. This contractor development program is a resource that assists small business chief executive officers to not only achieve capacity building goals; it connects federal technical resources to state transportation projects, and increases the pool of viable DBE contractors available in the market.

REVIEW OF STATE CONTRACTOR BIDDING PROCEDURES

Most States (45 of 50) require that performance bonds be used in conjunction with other contractor eligibility evaluation methods. Most States also require that performance bonds be secured for contracts over a specific dollar amount, typically $25,000, although the minimum contract amount ranges from any dollar value (in California) to much higher values (in Indiana, where the minimum is $200,000). The dollar amount of the required performance bond also varies by State, ranging from a percentage of the contract amount to the full contract amount. Performance bonding requirements may also extend to subcontractors.⁽¹⁸⁾

Several States do not always require performance bonds for the full value of the project. FDOT requires that the secured performance bond value be equal to the contract price, except for contracts greater than $250 million (an amount in excess of which is generally too great for a single performance bond to be issued), or if the State otherwise finds that a bond in the amount of the contract is not reasonably available, in which case the bond amount will be set at the largest amount reasonably available. For contracts greater than $250 million, the State transportation department can use a combination of bonds equal to a portion of the contract amount, along with an alternative means of security applied to the remaining portion, such as letters of credit, U.S. bonds and notes, parent company guarantees, and/or cash collateral to replace bond requirements.⁽¹⁸⁾

For design-build contracts, States need to include the cost of design and other non-construction services in the bond amount in order for the bond to be conditioned on performance of those services and for the persons who perform those services to be protected by the bond. In Illinois, the Public Construction Bond Act requires only one bond for the completion of a contract; this includes performance, payment, subcontractors used, and all labor performed. In Louisiana, the performance and payment bond needs to be issued either by a U.S. Treasury, a listed bonding company, or by a Louisiana insurer with a Best’s credit rating of “A-” or better.⁽¹⁸⁾

According to the survey reported on in the effectiveness report, most States (28 of the 41 surveyed) require a 100 percent performance bond on projects. A review of State procurement laws found that only five States, Arkansas, Connecticut, Oregon, Vermont, and Wisconsin, do not always require some type of performance bond on projects. In each case, a performance bond may be waived or the contractor may provide a substitute type of security, such as a cashier’s check for 100 percent of the contract amount.⁽¹⁸⁾

REVIEW OF BONDING COSTS

Determining a generalized cost of performance bonds is not a particularly straightforward task. The surety industry rates each contractor individually, in the context of a specific contract, and develops a separate premium for each individual project performance bond. Hence, it is nearly impossible to generalize or infer a specific cost for the bonding of a given project. A work written by Peurifoy and Oberlender, provides the following guidance:⁽⁶³⁾

• All government agencies and many private owners require a contractor to furnish a performance bond to last for the period of construction of a project. The bond is furnished by an acceptable surety to ensure the owner that the work will be performed by the contractor in accordance with the contract documents. In the event a contractor fails to complete a project, it is the responsibility of the surety to secure completion. Although the penalty under a performance bond is specified as 25, 50, or 100 percent of the amount of the contract, the cost of the bond is usually based on the amount of the contract and duration of the project.
  
  

The cost of a performance bond varies based on a number of factors, but is primarily based on the capacity of the contractor to perform the work and the financial stability of the contractor. Table 6 lists the average performance bond costs in 2002, as provided by Peurifoy and Oberlender in their analysis of the subject, and shows bond costs as a range in cost in terms of dollars per $1,000 of project value.⁽⁶³⁾ When these costs are translated to percentages of project value, the bond costs range from 0.65 to 1.2 percent for Heavy Civil projects.

Table 6. Representative costs of performance bonds per $1,000.⁽⁶³⁾

Project Size	Building Projects ($/$1,000 of project value)	Heavy Civil Projects ($/$1,000 of project value)
First $500,000	$14.40	$12.00
Next $2 million	$8.70	$7.50
Next $2,500,000	$6.90	$5.75
Next $2,500,000	$6.90	$5.25
> $7,500,000	$5.75	$4.80

Means Construction Cost Data (Means), a well-recognized source of construction costs for project estimation, provides percentage values for performance bond costs. In Mean’s construction data book for Heavy Construction, the cost of bonds for Highways and Bridges is listed as a range from 0.4 to 0.93 percent of total contract value.⁽⁶⁴⁾ A thesis on the cost effectiveness of performance bonds, written by Lorena Myers of the University of Florida in 2009, collected State construction data from September 2007 to September 2009. As part of this study, the SFAA reported that the cost of performance bond premiums on projects typically ranged from 2 percent of total contract cost for small projects (i.e., those valued at less than $100,000) to 0.5 percent for very sizeable projects (i.e., those valued at more than $50 million). Table 7 shows one-time performance bond premiums for different ranges of contract amounts, as reported by the SFAA.

Table 7. State transportation department construction performance bond rates.⁽¹¹⁾

Contract Amount	Performance Bond Premium	Project Size Category	Percent
$100,000	$1,200-$2,500	< $1 million	2.50
$1 million	$7,700-$13,500	$1 million-< $10 million	1.35
$10 million	$56,950-$81,000	$10 million-< $50 million	0.81
$50 million	$206,475-$341,000	> $50 million	0.68

 

Myers’ thesis also provided data for 19,135 construction projects for 30 States, shown in table 8, and found that the 2007 to 2009 U.S. national average of bond premiums was 1.139 percent. Surprisingly, only six States reported contractor defaults between 2007 and 2009: Alabama, Georgia, Idaho, Mississippi, South Carolina, and Texas. For these States, there were a total of 10 defaulted contractors over 34 projects, while the rate of default was 0 for all other States. The benefit-cost analysis concluded that only one State realized a net benefit as a result of using secured performance bonds.

Table 8. State default rates.⁽¹¹⁾

State	Number of Defaults	Total Projects	State	Number of Defaults	Total Projects
Alabama	7	631	Michigan	0	1,303
Alaska	0	187	Minnesota	0	447
Arizona	0	205	Mississippi	2	392
Arkansas	0	408	Montana	0	231
California	0	1,237	New Jersey	0	256
Colorado	0	326	New Mexico	0	126
Connecticut	0	134	New York	0	559
Delaware	0	170	Ohio	0	1,393
Georgia	19	513	South Carolina	6	681
Hawaii	0	129	South Dakota	0	292
Idaho	2	188	Texas	1	1,333
Illinois	0	2,682	Washington	0	650
Iowa	0	1,424	West Virginia	0	945
Kansas	0	643	Wisconsin	0	901
Maine	0	545	Wyoming	0	204