Chapter 8. Implementation

Previous | Table of Contents | Next

This chapter presents Phase III, Implementation, of the specification development process. The purpose of this chapter is to provide prescriptive steps that have been used by agencies and that may be useful as an aid in the implementation process. There are many items that can be used in implementing the specification. Some of these may be unnecessary if the specification has been used previously and only a modification is needed. On the other hand, if it is the first use of a QA specification, all of these items should be considered. The primary theme in these suggested steps is to "ease the pain" and to make the learning curve more easily understood by both the private sector and the agency personnel. The steps that are involved in the implementation process are identified in the flowchart in figure 42. The numbers in boxes before the titles of the following sections refer to the corresponding box in the flowchart.

1. Simulate Specification

Without having to do any fieldwork it is possible to use data from several projects either under construction or recently completed to investigate whether the specification works as intended. This will allow the determination of how the acceptance procedures and payment factors would have performed had the new acceptance plan been used on these projects. However, there are potential drawbacks to putting too much emphasis on these data. Caution is urged against drawing conclusions from this analysis because the contractors did not respond under these projects in the same manner as they would have had the new acceptance provisions been part of the project contracts. The data will probably not have been gathered in the same manner on the ongoing projects as will be required in the new acceptance plan. In other words, sublot and lot sizes and sampling frequency will likely be different. Nonetheless, the analysis can provide inexpensive, early, general insights as to how the new acceptance plan might perform.

Also, be sure that random sampling was used to collect the data. This analysis will only be valid if random sampling was used on the projects. Similarly, assure that the same sampling and testing procedures as decided upon in Phase II were used to collect the data. The analysis will only be valid if the same sampling and testing procedures were used that are contained in the new acceptance plan.

When analyzing the data, apply the individual and composite payment factors and critically examine the outcome. The data must be analyzed and the payment factors applied in the same manner as will apply in the new acceptance plan for the analysis to provide meaningful information concerning the new plan. Examine the payment factors in concert with the AQL and RQL and OC and EP curves to assure that incentives would have been paid for only truly exceptional quality work and that if price reductions would have occurred, that they would have been commensurate with less-than-acceptable quality.

Another approach that may be useful is to do "shadow projects" where the project is governed by the current specifications and acceptance plans and those are used in all project decisions, but, in addition, testing is also conducted to simulate the new or proposed acceptance plans. This can be done to evaluate whether the lot size, sublot size, sample size, and other test requirements are realistic.

Once information is available from the specification simulation, revise or "fine-tune" the acceptance plan as necessary into a draft special provision. Depending on the outcome of the analysis, no changes may be necessary or it may be possible to make small changes to the specification to correct an obvious problem. However, major changes based on small problems should be avoided because the small problems may not exist when the contractor bids the new special provision and works with it as part of the contract.

2. Begin or Continue Technician Qualification Training

Technician training and qualification is an extremely important aspect of QA specification implementation. This item often has not been given the attention it needs and deserves. It is important for the technicians, both those from the contractor and those from the agency, to know how to properly perform the tests, to be completely familiar with the sampling and testing frequencies, and to have at least a general understanding of the statistical procedures that will be used to analyze the data and to perform the payment factor calculations. The training should include assurance that the sampling and testing procedures to be used are completely understood with hands-on demonstrations of competence.

Ideally, technician training and qualification will have been previously begun in preparation for this stage of the specification development process. Agencies have found that technician qualification is a necessary and time-consuming step. Such concepts as sharing responsibility between agency and contractor may be entirely new and may require time for technicians to assimilate.

Include basic statistical procedures in the training. The QA initiative will certainly contain some form of statistical procedure. More than likely this will be new to technicians. These procedures are often perceived as being too complicated for technicians to master. Training is important to dispel this notion by explaining the basic statistical procedures that are required to control and/or accept the product, and to have the technicians perform the calculations. The technicians do not necessarily have to know the statistical derivations of the new QA specification, but they should have an idea why the statistical procedures are necessary and how sampling and testing affect the outcome of the analysis.

The training is also not a onetime occurrence. As new technicians come into the employment of the contractor and the agency, training must be given to them. Also, it is generally recognized that retraining, which may require requalification, is also desirable to assure that "bad habits" do not encroach on the techniques and that technicians are familiar with new procedures as they are adopted.

3. Try Specification on a Limited Number of Pilot Projects

The next step in the implementation process is to let pilot projects using the new specification, probably as a Draft Special Provision. Letting a limited number of pilot projects of medium size allows the contractors to develop bidding strategies and to determine how the special provision works. This also allows the agency to examine the outcome and, if desirable, further "fine-tune" the specification under "real-world" conditions.

As mentioned previously, the contractor is most interested in the bottom line, i.e., what is the final payment factor? To ease the uncertainty that may exist in the contractor's mind, and thus in the bidding, initially applying only a percentage of the disincentives has proven to be effective. There have been several strategies used to "ease into" a payment schedule and reduce price adjustment concerns on the part of the contractor, particularly on the pilot projects. Some agencies have applied only a portion of the incentive and disincentive. Others have found that applying only part of the disincentive but all of the incentive helps implementation. There is no consensus regarding how to best implement this strategy, but some effort to do this is recommended.

It should be realized that bid prices obtained on the pilot projects might not reflect future project prices. As with any new specification, the contractors may increase unit bid prices because of unknown risks that they feel may exist. This concern may be reduced if the task force has done a good job of communicating the specification details and processes to the industry during the development phase. An agency should not be discouraged if the initial prices are higher than normal since experience has shown that QA specifications do not increase bid prices over the long-term. Stating this differently, there are usually several other factors that affect competitive bidding to a greater extent than the use of QA specifications.

4. Analyze Pilot Project Results

All of the pilot projects should be examined in their entirety. An attempt to draw conclusions from one or two positive or negative outcomes should be avoided. The pilot projects are part of the learning process and mistakes may be made. One of the purposes of pilot projects is to restrict the mistakes to a small portion of the production/construction processes.

Some questions that should be addressed in the analysis of the pilot projects are similar to those examined under the previous specification simulation process. They include:

• Do the quality levels achieved and payment determinations appear to be consistent with those anticipated in the development of the specification?
  
  
• Do any projects stick out as being out of line with the others? For instance, did one project have much more severe price adjustments than the others? Did all projects have an incentive, irrespective of perceived quality? If any project does stand out as being exceptionally good or poor, the reason should be sought.
  
  
• Did contractors have trouble meeting the special provisions due to geographical location? As an example, are the available materials conducive to meeting the requirements?
  
  
• Did contractors have trouble meeting the special provisions due to technological reasons?
  
  
• Does the agency view the specifications as being too liberal or too severe in relation to the quality provided?

5. Do the Draft Special Provisions Need Major Revisions?

Do any of the problems that were found in the analysis of the pilot projects indicate that major revisions should be made to the draft special provisions?

If the identified problems are substantial and minor revisions will not correct them, then a new draft special provision should be considered. If the steps along the way have been taken with care and forethought, the problems should not be major, but it can happen. If no major problems are found, the next implementation step is to proceed to phasing-in the specification on projects agency-wide.

6. Prepare New Draft Special Provisions if Necessary

If some unfortunate occurrence requires new draft special provisions, the same steps taken in chapters 3 through 7 must be examined to see how the problem can be solved. At the very least, the timeframe for implementation will be adversely affected. However, not correcting the flaw will most likely negatively influence the specification to a greater extent and for a longer period of time.

7. Phase In Specification on Projects Agency-Wide

There is no single scenario that is applicable to all agencies. One note of caution is not to rush the process. It takes time for contractor and agency personnel to become familiar with and comfortable with QA specifications.

Phase in the specification until all major projects are included. Starting with a limited number of medium-sized projects and proceeding to all projects over 2 to 3 years allows contractor and agency personnel to become comfortable with the specification before having to bid large projects.

Consider phasing in payment factors. Phasing in payment factors over the 2-year to 3-year period discussed above in regard to the pilot projects may aid in the implementation process. A schedule of 50 percent of the intended payment adjustment the first year and 100 percent the second year, or 33 percent, 67 percent and 100 percent over 3 years, are two possible ways of phasing in the payment factors. These phased in payment factors may apply to both incentives and disincentives, or to only the disincentives.

8. Monitoring Program for Specification Performance

An annual review or monitoring of the specification is desirable. Some of the items to be examined in the monitoring program include:

• Evaluate quality levels achieved each year. For each quality characteristic, project data should be analyzed to determine how well the process standard deviations that are being attained compare with the "typical" values used when developing the specification. The project standard deviation values should be computed in a manner consistent with the way in which the typical standard deviation values were determined. Inconsistencies, either too high or too low, between the standard deviations being obtained and those used to develop the specifications, may indicate that revisions to the specifications are necessary.
  
  
• Project data should also be analyzed to determine whether or not the populations being obtained on projects are meeting, failing to meet, or exceeding the AQL definitions on which the specifications were based. Data should also be gathered and analyzed concerning payment factors that are being awarded on projects. The payment factor data and the project population data should be compared to see how they relate to the OC and EP curves that were developed for the specifications, and also to identify if there are inconsistencies between the quality that is being achieved and the payment factors that are being awarded.
  
  
• If the desired quality, i.e., the AQL, is not being achieved this may indicate that the technology is not sufficient to allow the contractors to achieve the AQL. However, it may also indicate that it is more economical for the contractor to operate at a quality level less than the AQL. This could be because full payment can be obtained with less than AQL quality, or it may be that the payment schedule is such that it is cheaper for the contractor to operate below the AQL and accept the corresponding payment disincentive than it is to operate at the AQL. In either case, modifications to the payment schedule might need to be considered.
  
  
• Ideally, the quality levels will gradually improve or remain stable with time. If the quality levels start a downward trend or increase significantly, a thorough analysis of the reasons for this is imperative. While the significance might not be as obvious as for a decrease in quality levels, an increase in quality levels is also an indication that the specification may need to be modified. It may be that technology changes are responsible for the improved quality, and the specifications should be revised to reflect this situation.
  
  
• Look for administrative problems. Is the definition for a lot creating any problems such as having lots open for long periods before the payment factor can be determined? Have the acceptance procedures created excessive paperwork? Is the agency having trouble applying incentives or disincentives? Are there other administrative problems? If so, these should be identified and corrected.
  
  
• Consider contractor concerns. Does the industry think the specification is working as intended? Do they have constructive suggestions as to how it can be improved? Are there problems finding enough qualified technicians? Listen to the industry's concerns. If these concerns are not heard and addressed, the industry may bring pressure to bear that may undo the advances made with the new QA specification.
  
  
• Identify technology changes. As stated previously, the specifications should be dynamic. Have technology changes occurred that can positively impact the specifications?
  
  
• Tie results of the QA specification into the pavement management system. For the specification to be a true measure of quality, closing the loop of relating quality and performance requires the integration of the quality levels and/or payment factors with the agency's pavement management system. This may be a source of performance information for future modifications of the specifications.
  
  
• Compare the quality levels with established criteria for success. Close the loop with the item in Phase I in which goals and expectations were established. Is the specification performing the way it was anticipated? If it is not, why not? Are further revisions needed?