8. Quality Measures and Payment

Previous | Table of Contents | Next

CANDIDATE QUALITY MEASURES

In chapter 5, candidate quality measures were investigated with respect to their accuracy and precision when estimating the true population parameters. The candidate quality measures that were analyzed were PWL (or its complement, PD), AAD, and CI.

From the simulation analyses summarized in chapter 5, both the sample PWL and the sample AAD were unbiased estimators of the corresponding population values. They both showed decreasing variability in the estimated values as the sample size increased. For PWL, the variability values peaked at a population PWL of 50 and then decreased as the population PWL approached either 100 or 0. The AAD variability values were lowest when the population mean was centered at the target and they increased as the offset from the target of the population mean increased, leveling off as the offset approached 3 standard deviations.

The simulation analyses for CI showed similar variability relationships as were found for AAD. However, the CI appeared to be a slightly biased estimator of the population CI since all of the simulated average CI sample values were negative.

Since there appears to be no appreciable benefit to using CI rather than AAD (i.e., they have similar variability traits, but AAD is unbiased while the CI is slightly biased), the CI was eliminated from further analyses. This chapter presents the results of the further analyses that were conducted on PWL and AAD.

ANALYSES PERFORMED

The quality measure that is selected needs to be capable of being used to determine the payment factor. Therefore, it must be easy to develop an equation that relates the quality measure to the anticipated performance of the pavement. For the analyses to be conducted at this point, it was sufficient to accept the fact that the quality of the material in the population would increase as either its PWL value increased or its AAD value decreased. The relationship of payment to projected performance is discussed in a later chapter.

For each quality measure, it was necessary to determine whether or not it could provide an unbiased estimate for the payment factor for a given population. The amount of variability in the payment factor estimates for individual lots also required study. These results were both generated by computer simulation studies.

Since it is rare that only a single quality characteristic is used to determine the payment factor for a lot, it was necessary to consider how the quality measures performed when multiple acceptance characteristics were used and to consider methods for combining these individual estimated quality measures into a combined (or composite) payment factor. While it was not within the scope of the original project, it was decided that, if multiple quality characteristics were to be investigated, it would be necessary to consider potential correlations among these characteristics.

Finally, it was necessary to investigate the risks associated with the quality measures and how they would be used for acceptance decisions. The concept of risks for acceptance is similar to that discussed in chapter 7 for verification testing. If pass and fail are the only options, the acceptance decision is simply a form of hypothesis testing and the concept is fairly straightforward. When acceptance at payments other than 100 percent is possible (either incentives or disincentives), the evaluation of the risks becomes much more complicated and not nearly as clear-cut as for pass or fail decisions. The analysis of the payment risks associated with PWL and AAD quality measures was conducted for single quality characteristics and for two quality characteristics that were either independent or correlated.

PAYMENT FOR A SINGLE QUALITY CHARACTERISTIC

In chapter 5, it was determined that both PWL and AAD were unbiased estimators for the true population values. As such, they should also provide unbiased payment factor estimates when used in a payment equation. Computer simulation studies were conducted to verify that this was true.

PWL Payment Factors

Computer simulation was used to generate expected payment (EP) curves for a payment equation that based payment on the estimated PWL value. Since PWL is an unbiased estimator of the actual PWL value, the EP curve should follow the payment equation exactly. This will be true provided that an appropriate incentive provision is included in the specification. This is discussed at length in a later section on evaluating risks.

For the purposes of the simulation, any payment equation could have been used. For the simulation studies, the payment equation from the AASHTO Quality Assurance Guide Specification was used.⁽³⁾ This payment equation is:

Pay = 55 + (0.5 X PWL) (18)

This equation was used with sample sizes = 3, 5, and 10. Figures 54 and 55 show the results of the analyses. The plots in figure 54 show the EP curves and the limits within which 95 percent of the estimated payment values fell. As expected, the EP (i.e., the average payment in the long run) follows the same line as the payment equation (equation 18). However, there is considerable variability in the payment values for individual lots. This variability (or spread about the EP line) decreases as the sample size increases from 3 to 10. This was also expected since it follows exactly what was found when PWL values were simulated and since it is the PWL values that determine the payment factor.

To further illustrate the distribution of individual lot payment factors, figure 55 presents histograms of both the individual PWL and the payment factors for populations with an actual PWL = 90 and 50, for sample sizes = 3, 5, and 10.

Figure 54a. EP curves for PWL payment schedule with sample size = 3.

Figure 54b. EP curves for PWL payment schedule with sample size = 5.

Figure 54c. EP curves for PWL payment schedule with sample size = 10.

Figure 55a. Distribution of actual PWL = 90, sample size = 3, and the resulting payment factors.

Figure 55b. Distribution of actual PWL = 50, sample size = 3, and the resulting payment factors.

Figure 55c. Distribution of actual PWL = 90, sample size = 5, and the resulting payment factors.

Figure 55d. Distribution of actual PWL = 50, sample size = 5, and the resulting payment factors.

Figure 55e. Distribution of actual PWL = 90, sample size = 10, and the resulting payment factors.

Figure 55f. Distribution of actual PWL = 50, sample size = 10, and the resulting payment factors.

AAD Payment Factors

Computer simulation was also used to generate EP curves for a payment equation that based payment on the estimated AAD value. Since AAD is an unbiased estimator of the actual AAD value, the EP curve should follow the payment equation exactly. For the purposes of the simulation, any payment equation could have been used.

Since it is known that the expected value for the distribution of absolute deviations for a normal distribution is 0.798s, a logical equation for relating AAD to payment is:

Pay = 105 - 24.75(AAD - 0.798), and Pay ≤ 105% (19)

This equation provides a maximum payment of 105 percent for an AAD = 0.798s (i.e., the value for a normal distribution centered on the target). This payment equation was used in the comparison analysis. It should be noted that this equation was not intended to provide exactly the same EP curve as the PWL payment equation.

Figure 56 shows an EP curve for a sample size = 5, using equation 19 to determine the payment factor. It is shown in the figure that the EP curve follows a straight line. This is true because the payment equation is a straight line and AAD is an unbiased estimator. Also, the same trends of increasing variability about the EP line with a smaller sample size and decreased variability with a larger sample size would hold true for AAD.

Table 38 shows the EP values from the plot in figure 56 and the payment factor that should have been received for a population with the AAD value shown. The final column shows the difference between the EP and the correct payment. For lower AAD populations (i.e., those with higher quality), the EPs are lower than called for in the payment equation (equation 19). As the AAD value increases, the EP values come in line with the correct values. This issue may be related to the selection of the payment equation rather than the AAD estimation process. This issue is discussed in the section on risks.

Matching PWL and AAD Payment Equations

Although at first PWL and AAD seem very different, they are both based on the assumption that the population being sampled is normal. All normal distributions can be related to the standard normal distribution (i.e., μ = 0.0 and σ = 1.0) by transforming the measurement axis to standard deviation units by the relationship:

(20)

where: Z = number of standard deviation units from the population mean

Figure 56. EP curve with the 90^th and 10^th payment percentiles for the AAD payment schedule.

Table 38. Simulated EP factors and correct payment factors based on AAD.

A similar approach can be taken for AAD. If AAD is measured in standard deviation units, then AAD can be related to the number of standard deviation units that the population mean is offset from the target value. This works for any normal distribution as long as AAD is measured in standard deviation units. In this way, AAD computer simulation results can be applied to any normal distribution in much the same manner that the standard normal curve can be used to calculate areas under any normal distribution. Figure 57 illustrates how this concept works. Z_Targ represents the distance that the population mean (μ) is offset from the target value (T) when measured in units of standard deviation (σ).

Figure 57. Illustration of measuring AAD in standard deviation units (Z_Targ) from the mean.

If we also measure the specification limits to be used in calculating PWL in terms of standard deviation units from the target value, then once the PWL specification limits are set, we can develop a relationship between any given PWL value and its equivalent AAD value for the same normal population.

In the simulation studies that were conducted to compare PWL results with AAD results for the same normal population, the PWL specification limits were set at ±1.645 σ from the target value, which corresponds to 90 PWL for a population centered on the target. Since it is now possible to determine AAD and PWL values for the same offset from the target (Z_Targ), an empirical relationship can be developed that for any normal population can convert any given PWL value to its corresponding AAD value, and vice versa. Equations 21 and 22 show this approximate empirical relationship in which all the terms have already been defined:

     (21)

     (22)

First, it should be noted that it is known that AAD = 0.7979σ if the population mean is centered at the target value. Therefore, since the AAD value for a population is approximately 0.798 times the population standard deviation, the only way that the AAD value could be zero is if the standard deviation was zero. Also, the smallest population AAD value is 0.798 times the population standard deviation. Therefore, if the population AAD value is measured in standard deviation units, then 0.798 is the smallest population AAD that is possible, thus, AAD ≥ 0.798 is a requirement.

Equations 21 and 22 allow the corresponding PWL value to be obtained for a given AAD value, and vice versa. For example, for a population with an AAD of 1.00, the offset of the population mean from the target value (Z_Targ), measured in standard deviation units, can be calculated as 0.7403 by using equation 22. Since the specification limits are assumed to be set at ±1.645 σ, PWL can be determined from tables of areas under the standard normal curve using figure 58.

Tables 39 through 41 show the relationships between Z_Targ, measured in standard deviation units, and the AAD and PWL values for a normal population, provided that the PWL specification limits are two-sided and are set at ±1.645 σ. The two-side specification limits are a requirement since AAD is always two-sided because it is measured as the deviation from a target value. With these tables, it is easy to identify the appropriate PWL and AAD values for a given normal population, provided that the specification limits are set at ±1.645 σ.

Figure 58. Example illustrating the PWL specification limits and the offset in s units between the population mean and the target.

Table 39. Relationship between Z_Targ, AAD, and PWL for a normal population when the PWL specification limits are set at 1.645σ.

Table 40. Relationship between AAD, Z_Targ, and PWL for a normal population when the PWL specification limits are set at 1.645σ.

Table 41. Relationship between PWL, AAD, and Z_Targ for a normal population when the PWL specification limits are set at 1.645σ.

Using the information from tables 39 through 41, a computer simulation program was developed that generated the EP and the standard deviation of the individual lot payment factors for any value of either PWL or AAD and the corresponding payment equation. The program output also identifies both the PWL and AAD values for the population. This program makes it simple to compare PWL and AAD payment plans provided that equivalent payment equations are developed for AAD and PWL. Tables 39 through 41 allow the development of such equivalent AAD and PWL payment schedules.

As an example, suppose an agency was considering the use of PWL with the payment factor determined by equation 18. However, the agency would also like to evaluate a payment equation based on AAD that would provide for a given normal population the same EP as the PWL equation. This is not difficult to do and, indeed, was done in this project to compare PWL and AAD payment procedures for the same population.

First, equation 18 was used to determine the payment factor for a number of actual PWL values ranging from 90 to 10. Table 41 was then used to identify the AAD value that corresponded to each of the PWL values. This then allowed the payment factors to be associated with their respective AAD values. The results of this process are shown in table 42.

Table 42. AAD values equivalent to the corresponding PWL payment factors.

The AAD program that was developed allows for the use of a compound payment equation that is made up of two straight lines with different equations. To illustrate this and to help match the EP curve of the PWL payment equation, it was decide to use two equations that met at AAD = 1.925. The program allows for an equation in the form:

(23)

It is easy to solve for the A and B coefficients using the values in table 42. Doing so resulted in the following AAD payment plan to match the PWL payment equation:

(24)

(25)

One should note that equations 24 and 25 each provide a payment factor of 75 when AAD = 1.925, thus providing the desired continuity in the two payment equations.

Computer simulation was used to compare the performance of a PWL payment plan based on equation 18 with that of an AAD payment plan based on equations 24 and 25. The results of these simulations are shown in table 43 and figures 59 and 60. The plans have very similar EPs, with the AAD plan being slightly higher in the mean offset range of about 1.5 to 2.5. However, table 39 shows that this represents PWL values of about 55 and below. The PWL standard deviations increase to a peak at a mean offset of about 1.6 to 1.7, and are higher than those for AAD for the middle portion of the plot. The PWL peak corresponds to a PWL of 50, which was shown previously to be the point of maximum variability for PWL estimates.

Table 43. Results of the simulation of matched PWL and AAD payment equations for sample size = 5.

Figure 59. EP curves for matched PWL and AAD payment equations for sample size = 5.

Figure 60. Standard deviations for individual payment factors for matched PWL and AAD payment equations for sample size = 5.

While there were some minor differences, the simulation study showed that when a single quality characteristic is used, it is possible to develop separate, but matched, PWL and AAD payment equations that provide similar EP curves and have reasonably close variability associated with the individual lot payment factors.

PAYMENT FOR MULTIPLE QUALITY CHARACTERISTICS

Since it is rare that only a single quality characteristic is used to determine the payment factor for a lot, it was also necessary to consider how the quality measures performed when multiple acceptance characteristics were used and to consider methods for combining these individual estimated quality measures into a combined, or composite, payment factor. While it was not within the scope of the original project, it was decided that, if multiple quality characteristics were to be investigated, it would also be necessary to consider potential correlations among these characteristics.

Simulating Correlated Variables

First it was necessary to develop computer routines for simulating correlated normal variables. These simulation routines were developed to simulate up to four correlated variables. The routines were then tested to verify that they were capable of generating samples with the intended level of correlation among the various variables.

Correlation is related to a pair of variables and the amount of correlation between the two variables is measured in terms of the correlation coefficient. Correlation coefficients can vary from +1.0 (as one variable increases, the other also increases with perfect correlation) to -1.0 (as one variable increases, the other decreases with perfect correlation). A correlation coefficient of 0.0 indicates that there is no correlation between the results of the two variables.

Figure 61 shows the output screens from the simulations of 5000 pairs of correlated normal variables, each with different selected correlation coefficients. The histograms on the left side of the plots show the two normal populations from which the correlated values were generated. The scatter plots on the right side of the figures show the data that were simulated. Figure 61(a) shows a situation where the two variables are not correlated. The resulting plot shows no pattern in the results when one variable is plotted against the corresponding second variable. In this simulation, the desired correlation coefficient was 0.000, while the correlation coefficient for the 5000 simulated pairs of values was -0.012, which is quite close to the desired value.

Figure 61(b) shows a similar plot for a desired correlation coefficient of +1.000. In this case, there is perfect correlation between the two variables. The increasing straight line at a 45-degree angle that is obtained when the two variables are plotted against one another shows this relationship. The correlation of the simulated value was also +1.000. A plot for a correlation coefficient of -1.000 would also be a straight line, but would decrease at a 45-degree angle. The remaining plots show comparisons of various levels of positive and negative correlation, and also show that the simulation routine does a good job of simulating the desired correlation.

Figure 61a. Illustration 1 of two normal variables with various values for the correlation coefficient.

Figure 61b. Illustration 2 of two normal variables with various values for the correlation coefficient.

Figure 61c. Illustration 3 of two normal variables with various values for the correlation coefficient.

Figure 61d. Illustration 4 of two normal variables with various values for the correlation coefficient.

Figure 61e. Illustration 5 of two normal variables with various values for the correlation coefficient.

Figure 61f. Illustration 6 of two normal variables with various values for the correlation coefficient.

Figure 61g. Illustration 7 of two normal variables with various values for the correlation coefficient.

Figure 61h. Illustration 8 of two normal variables with various values for the correlation coefficient.

The simulation routine that was developed is capable of simulating up to four correlated normal variables. When more than two correlated variables are simulated, it is not possible to clearly show the correlation relationships as done in figure 61 for the two-variable case. However, tables 44 and 45 show the results of simulations for three and four correlated variables, respectively. While the results cannot easily be shown graphically, the tables show that the simulated data have correlation coefficients that compare closely with the desired values.

Table 44. Demonstration of the simulation of three correlated normal variables with selected values for correlation coefficients.

Table 45. Demonstration of the simulation of four correlated normal variables with selected values for correlation coefficients.

Correlated Quality Characteristics

Earlier in this chapter, the results from two simulation studies-one using PWL and equation 18 to determine payment and the other using AAD and equation 19 to determine payment-were compared. This comparison was expanded to address the case where two individual variables were considered when determining payment. While it is possible, and even likely, to have more than two variables involved in the acceptance and payment decision, it is not easy to picture the results when more than two variables are considered. Therefore, only the two-variable case was considered in this project. The results from two variables should be equally applicable to cases with three or four variables.

Two Populations of Equal Quality: The routines discussed above were used to compare the EP values and the amount of variability in the individual payment factors for both PWL and AAD when two correlated variables were involved. In the analyses for both PWL and AAD, the combined payment factor was taken as the weighted average of the two individual payment factors. This general payment relationship is shown in equation 26.

(26)

where: W₁ = weighting for variable 1

W₂ = weighting for variable 2

PF₁ = individual payment factor for variable 1

PF₂ = individual payment factor for variable 2

A sample size = 5 was used in the analyses. Three different correlation coefficients between the two variables were considered: 0.0, -0.5, and +0.5. In each analysis, the two individual variables had the same actual population: 70 PWL or 1.238 AAD. Various weightings (1.00/0.00, 0.90/0.10, 0.75/0.25, 0.50/0.50, 0.25/0.75, 0.10/0.90, and 0.00/100.00) were used to determine the combined payment factor.

Since, in each case, the two individual populations were of equal quality (i.e., 70 PWL or 1.238 AAD), the same EP should be obtained regardless of the weightings used. For the PWL case, the EP for each individual variable should be 55 + (0.5 X 70) = 90. For the AAD case, the EP for each individual variable should be 105 - [24.75 X (1.238 - 0.798)] = 94.1.

The results from the analyses are presented in table 46 and figures 62 and 63 for PWL, and in table 47 and figures 64 and 65 for AAD. The results for the 1.00/0.00 and 0.00/1.00 weightings are equivalent to the one quality characteristic using variable 1 or variable 2, respectively.

From table 46 and figure 62, it is shown that the EP values do not change as either the correlation coefficients or the variable weights change. The EP values are all around the value of 90.0 calculated from the payment equation. Therefore, PWL provides an unbiased estimate for the combined payment factor regardless of the weights selected or the amount of correlation between the variables.

However, the same is not true with respect to the variability of the individual simulated combined payment factors, as indicated by the standard deviation values in table 46 and figure 63. There is a noticeable difference among the standard deviation values for the three different correlation coefficients, with the difference being greatest at the 0.50/0.50 weighting of the two variables. Also, within the values for each correlation coefficient, there is a similar trend of decreasing variability as the 0.50/0.50 weighting is approached.

Although they are both related to the same mathematical relationship for the variance of the sum of two random variables, two different phenomena are at work in these two trends. It can be shown mathematically that if a and b are constants and X and Y are random variables, then:

(27)

Table 46. Results of PWL simulation analyses for two correlated normal variables.

Table 47. Results of AAD simulation analyses for two correlated normal variables.

Figure 62. Expected combined weighted average payment factors for various weights and correlation coefficients based on PWL.

Figure 63. Standard deviations of weighted average payment factors for various weights and correlation coefficients based on PWL.

Figure 64. Expected combined weighted average payment factors for various weights and correlation coefficients based on AAD.

Figure 65. Standard deviations of weighted average payment factors for various weights and correlation coefficients based on AAD.

In equation 27, the last term is the covariance term and it is related to the correlation coefficient. The constants a and b can be related to the weightings (e.g., 0.50/0.50) that are multiplied by each of the random variables. Although equation 27 applies to the variability of the individual quality characteristic results, these results are used to determine PWL, which is linearly related to the payment factor by the payment equation. Therefore, equation 27 can be used to explain the standard deviation trends shown in figure 63.

In figure 63, for example, for a given set of weightings, such as 0.50/0.50, the standard deviation for +0.5 correlation is higher than for 0.0 correlation, which is higher than for -0.5 correlation. This follows directly from equation 27. For constant weightings, a and b in equation 27 remain the same for each correlation coefficient. Therefore, it is only the covariance term that changes the combined standard deviation value. For a correlation coefficient of 0.00, the covariance term in equation 27 also becomes 0.00. A positive correlation coefficient (and, therefore, a positive covariance term) results in a larger combined variability and, conversely, a negative correlation coefficient results in a smaller combined variability. Since the standard deviation also directly affects the AAD value, a similar standard deviation trend is shown for the AAD combined payment factors in table 47 and figure 65.

Equation 27 can also be used to explain the differences in standard deviation with respect to the weightings that are used. These weights are represented by the a and b constants in equation 27. For any given covariance value, the magnitudes of a and b will determine the differences in the combined standard deviations.

For example, take the case where the correlation coefficient is 0.00. In this event, the covariance term is eliminated from equation 27. Since the two variables have equal sample sizes, if measured in standard deviation units, they will also have equal variances for the purposes of calculating PWL and, therefore, payment. If we call this variance σ², then for the case of a 0.50/0.50 weighting, equation 27 leads to:

(28)

For example, figure 63 shows that the standard deviation for PWL, with a correlation coefficient of 0.00 and weighting of 1.00/0.00 or 0.00/1.00, is about 8.7. Based on the calculations above, the standard deviation for a 0.50/0.50 weighting should be about 0.707σ = 0.707 X 8.7 = 6.15, which is essentially the same value shown in table 46. Similar calculations verify the shapes shown in figures 63 and 65.

Therefore, for two populations with equal PWL values, the combined payment factor, based on the weighted average of the two individual payment factors, has the same mean payment as for the individual variables, but with smaller variability. This is an unexpected result of using a weighted average approach for combining individual payment factors into one combined payment factor.

To consider populations with additional PWL values, simulations were run for the case of populations with 90 PWL, 70 PWL, and 50 PWL. The simulations were run with weightings of 0.50/0.50 and with correlation coefficients between the two variables ranging from -1 to +1. For both the PWL case (figure 66) and the AAD case (figure 67), the horizontal lines show that the EP value is not sensitive to the correlation between the individual variables, at least when the weighted average is used to determine the composite payment.

However, figures 68 and 69 indicate that the variability of the payment estimates is indeed related to the correlation between the two individual variables. This same trend is evident for both PWL and AAD. This means that while the average payment stays the same, the variability about the average payment increases as the correlation goes from -1.0 to + 1.0 for the two lower quality populations (i.e., 70 PWL and 50 PWL, and 1.238 AAD and 1.688 AAD), while it decreases slightly as the correlation approaches 0.0 for the higher quality population (90 PWL and 0.798 AAD). The relationship is different for the higher quality population since the upper boundaries of either 100 PWL or 0.0 AAD will have a greater impact than for the lesser quality populations.

Two Populations of Different Quality: All of the two-variable analyses up to now have been for the case where the two populations have the same quality level (i.e., the same PWL or AAD). Tables 47 through 51 show the results of simulation analyses for cases where the two individual populations have different PWL or AAD values and 50/50 weighting of the two variables when determining the combined payment factor. Tables 48 and 49 show that while, as expected, the quality levels of the individual variables affect the EP values, the EP values do not appear to be impacted by a moderate degree of correlation (i.e., -0.5, 0.0, and +0.5). Tables 50 and 51, however, show that the variability of the payment values may increase slightly with either positive or negative correlation between the two individual variables. This trend is evident in each case, except when one of the variables has the lowest quality simulated (i.e., 50 PWL or 1.688 AAD).

The values in tables 48 through 51 are plotted in figures 70 through 73. As shown previously, the payment biases using PWL are small and are centered at the correct value, regardless of the quality levels of the two populations. This is not the case for the payment biases using AAD, which are larger than for PWL and are negative for higher quality levels and positive for lower quality levels. For both PWL and AAD, the standard deviation values are lower for higher quality populations and there is little difference associated with the correlation coefficients. As the level of quality becomes lower, the standard deviations become larger and a difference develops with respect to the correlation coefficients. This trend is true for both PWL and AAD.

Figure 66. Expected average payment factors for two populations with various correlation coefficients based on PWL.

Figure 67. Expected average payment factors for two populations with various correlation coefficients based on AAD.

Figure 68. Standard deviations of individual payment factors for two populations with various correlation coefficients based on PWL.

Figure 69. Standard deviations of individual payment factors for two populations with various correlation coefficients based on AAD.

Table 48. Bias in EP for two populations with equal PWL values and various correlation coefficients.

Table 49. Bias in EP for two populations with equal AAD values and various correlation coefficients.

Table 50. Standard deviations of individual payment factors for two populations with equal PWL values and various correlation coefficients.

Table 51. Standard deviations of individual payment factors for two populations with equal AAD values and various correlation coefficients.

Figure 70. Bias for the average payment for two populations with various actual PWL values.

Figure 71. Bias for the average payment for two populations with various actual AAD values.

Figure 72. Standard deviation for the individual average payment values for two populations with various actual PWL values.

Figure 73. Standard deviation for the individual average payment values for two populations with various actual AAD values.

SELECTION OF THE QUALITY MEASURE

While the analyses indicated that, in general, the AAD quality measure performed about as well as the PWL quality measure, there were instances where the AAD measure had greater payment variability and exhibited some trends regarding payment bias. These issues, in themselves, would not be sufficient to eliminate AAD as a potential quality measure. However, there are some other drawbacks to AAD that make it a less appealing quality measure than PWL.

It is necessary to measure both center and spread when characterizing a population (lot) of material. Because of this, there are potential difficulties with the AAD quality measure. One drawback with AAD acceptance plans is that since lot variability is not really measured, a given lot AAD could come from a number of different populations. For example, the population could be centered at the target, but have a relatively large standard deviation (i.e., larger than the one that was assumed when developing the AAD acceptance limits or payment equation). Another population could have the same AAD by being centered off the target and having the same standard deviation that was assumed when developing the acceptance values. A third population could have the same AAD value by having a mean far from the target, but also by having a relatively small standard deviation. The fact that dissimilar populations can have the same AAD is shown by the distributions in figure 38 in chapter 6.

While some of these drawbacks may also apply to PWL (or PD) acceptance plans, such as the fact that a given PWL can represent many different populations, there are fewer drawbacks because both sample mean and standard deviation are measured in the PWL method. Also, since the PWL method can be used with both one-sided and two-sided acceptance properties, it is more versatile than the AAD method, which cannot be applied to one-sided limits.

Therefore, PWL (or its complement PD) was selected as the best quality measure for use in QA specifications. All remaining analyses were conducted only for the PWL (or PD) quality measure.

Now that the quality measure has been selected, its use needs to be further analyzed with respect to the risks associated with the small sample sizes that are typically used in highway materials and construction applications. Risks related to PWL and PD estimates, and their related payment factors, are discussed in chapter 9.