U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
|This report is an archived publication and may contain dated technical, contact, and link information|
Publication Number: FHWA-HRT-04-046
Date: October 2004
The objective of the project was to develop a comprehensive QA manual, supported by scientific evidence and statistical theory that provides step-by-step procedures and instructions for developing effective and efficient QA specifications. This technical report summarizes what steps were taken to accomplish this goal and the analyses that were conducted to support the recommendations made in the QA manual.
While the focus and objectives of the two documents are quite different, they are not totally stand-alone documents. As such, the technical report should be read in conjunction with and as a companion to the QA specifications manual that also resulted from the project.
One of the major accomplishments of the project was the development of the three-phase QA specifications development process that is presented in chapter 3. The process was presented to and approved by the panel of representatives from the pooled fund States that provided the funding for this project. The process is presented in flowchart form for ease of understanding.
After seeing and approving the specifications development process flowcharts, the panel was surveyed to identify those topics that they thought were most important for further detailed analyses. The items selected by the panel also comprise some of the major decisions that are necessary during the specifications development process. These items are presented below, along with the recommendations that were included in the QA manual:(1)
What quality measure should be used for individual quality characteristics?
From the many potential quality measures that were identified, three were chosen for detailed analyses PWL (and its complement PD), AAD, and CI. After initial analyses, CI was eliminated because it offered no benefits over AAD and because it provided a slightly biased estimate for the population CI. Further studies showed that PWL and AAD generally performed comparably to one another. PWL was recommended because it provides a direct measure of variability, whereas AAD provides, at best, an indirect measure of variability. In fact, the same population AAD could apply to very different populations. PWL also works effectively with both one-side and two-sided specifications, while AAD only applies to two-sided specifications that have a definite target value.
What payment relationships should be used for individual quality characteristics?
The selection of the payment equation is, to some extent, a subjective decision for each agency based on the level of risk that it considers reasonable. A general method was developed to determine the equivalent PWL and AAD values for any given value of PWL or AAD. From this it was possible to show that separate PWL and AAD payment equations could be developed to provide equivalent EPs for the same population. While no single payment relationship was recommended for individual quality characteristics, the extensive discussion in this report regarding calculating and evaluating risks provides the agency with the tools and techniques to develop payment equations that they believe to be fair and reasonable.
How should multiple quality characteristics be combined into a single payment factor?
Analyses were conducted on six different methods for combining payment factors for individual quality characteristics into one combined payment factor:
The analyses showed that the variability of the combined payment factors for two characteristics was related to whether they were positively correlated, negatively correlated, or not correlated. For four of the methods, the EP values did not vary with the correlation between the variables. However, the EP did vary with the correlation when using either the maximum or minimum individual payment factor.
Another method-using a single composite quality measure derived from a general performance model to predict expected pavement life-was developed and is the recommended approach. If an agency wishes to use a simpler method for determining the composite payment factor, the first four methods listed above are candidates. One of the two averaging methods might be preferred since the averaging process has the effect of reducing the variability in the estimated payment factors.
What procedures should be used to verify the contractor's test results if they are to be used in the acceptance and payment decision?
A detailed discussion and analysis were presented regarding the risks involved in the various approaches to verifying the contractor's test results. OC or power curves, surfaces, and tables were presented to illustrate the risks associated with the different procedures and sample sizes. The relatively high risks that are associated with typical agency verification testing frequencies were pointed out.
A discussion was presented concerning which verification procedures were appropriate for use with split and independent samples. For split samples, the recommended approach is to use a maximum allowable limit for the difference between the agency's results and the contractor's results for each individual pair of split samples, but also to accumulate the results for use in a t-test for paired measurements. The best approach for independent samples is to use the F-test to compare the variances in the contractor's tests and the agency's tests. The result of the F-test then determines the manner in which the t-test is conducted to compare the means of the contractor's tests and the agency's tests. Each agency can select its own testing frequency based on its assessment of the risks provided in the OC curves that were developed during this project.