Construction

Technical Advisories

Specification Conformity Analysis

T 5080.12

June 23, 1989

Par.

1. Purpose
2. References
3. Background
4. Application
5. Discussion
6. Concepts

Attachment 1, List of References

Attachment 2, Calculation Procedures and Example

1. PURPOSE. To provide guidance on the technical basis for, and the use of Specification Conformity Analysis (SCA) procedures (formerly called Quality Level Analysis (QLA)).
2. REFERENCES. A listing of references and additional publications pertaining to statistics and SCA procedures is provided as Attachment 1 to this Technical Advisory.
3. BACKGROUND
   1. The QLA (now called SCA) procedure was developed by the FHWA as a part of the Highway Condition and Quality of Highway Construction Surveys (1976-1982). It was used to estimate the degree of compliance with specifications and to provide an indication of construction quality on a nationwide basis. The terminology is being changed to SCA to better reflect the actual usage of the analysis procedure.
   2. The SCA procedure is being used in construction monitoring programs on a State-by-State basis where individual specification limits are held constant. The SCA procedure is also used by some States as an indicator of contractor performance in meeting specifications and/or as a basis for pay factor incentive and disincentive systems. In these cases, the specifications, sampling and testing programs, and contract documents have been written based on this type of acceptance. The concepts of the SCA procedure are consistent with those used in the AASHTO Guide Specification R9-86, "Standard Recommended Practice for Acceptance Sampling Plans for Highway Construction. "
4. APPLICATION. The use of SCA procedures can be an integral part of a construction monitoring program for measuring conformity with specifications and for developing background information for improving specifications or construction procedures.
   1. The SCA procedure can be effectively used to estimate the degree of conformity to specification requirements that is being achieved on a project or for a particular construction process. It can be used as a tool during project or in depth inspections, or as a special effort, to analyze construction measurements or materials test results and evaluate the degree of specification compliance. It can be used in Statewide inspections or reviews, when specification limits are uniform, to evaluate specification compliance over several projects to ascertain whether there are widespread uniformity or process control problems.
   2. The SCA procedure can identify specification requirements which are impractical or ineffectual in assuring good performance. In cases of consistently low percent conformity over several projects, the specification limits should be analyzed with respect to how much material variance, sampling variance, and testing variance is inherent in the construction process under question. If the specification limits do not allow reasonable room for these variances, they are likely impractical for current procedures, and conformity levels will always be low.
      • (1) A classic example of this situation can occur in the measurement of slump to determine the consistency of plastic concrete. Research has shown that for a reasonably well-controlled process, the overall standard deviation can be expected to be on the order of 1/2 inch. With specification tolerance limits of ±1/2 inch imposed, one could expect, over a period of time, to have approximately 32 percent of the test results out of specifications if the process is producing concrete with normal variability and the average slump is equal to the specified target value. Refer to Figure 3 on page 10.
      • (2) In order to allow for normal variability due to sampling, testing, and inherent materials variations, the tolerance limits in this example should be set at ±1 inch (equal to two standard deviations) when judging conformity on the basis of one test. With these tolerance limits, approximately 95 percent conformity can be expected over a period of time. This will avoid having changes made when the processis really under control. Only when the precision of sampling and testing methods and normal production processes are improved would it be practical to tighten these tolerances.
   3. The SCA procedures are useful for relating the degree of conformity to specifications with product performance. If an element is not performing as expected, then an analysis of specification conformity is appropriate. In cases where the calculated percent conformity consistently and easily equals 100, the appropriateness of the specification should be explored. The target value should be reviewed, and the production variances compared to the historical or expected variance established by research. Improved performance may be achievable by improving the specification target if the relationship of the variable being measured to performance is known or by reducing the allowable deviation from the target specification to encourage more uniform production. A change in specification limits may also be accompanied by an increase in sample size to reduce the risk of making an erroneous decision, i. e. , reduce the risk of accepting nonspecification material. It may also be necessary to test for another characteristic which better predicts the performance desired.
   4. The SCA procedures are useful for evaluating contractors' process control capabilities. Calculated percent conformity can serve as a general indicator of a contractor's equipment capabilities, workmanship, management, and overall desire to produce a specification product. The procedure consists of computing the percent conformance for selected construction phases or materials production and their respective properties, and maintaining a log of the percent conformance by contractor over a period of time or over a number of projects. By so doing, those contractors who adhere most closely to specification requirements can be identified.
   5. The SCA procedures are useful for monitoring the degree ofcontrol over a period of time. Project test data from several projects can be collected and stored over a long period of time. It is possible to investigate production variability for various materials and conditions by analyzing collected data independently of the specification limits for statistical measures such as averages and standard deviation, and by building a history of this data from actual construction projects. Similarly, cases can be identified where improvements in quality control are needed by comparing actual variations (standard deviations) to expected historical variations. The information gathered can be used to create or modify specifications. Care is needed regarding random sampling as discussed in paragraph 5c.
   6. The SCA procedures can be used to determine when to increase or decrease sampling and testing frequencies from the approved frequencies. If the calculated percent conformity is consistently high and it has been determined that the specifications are adequate to produce the performance desired, sampling frequencies could be reduced on a project basis. Conversely, if the calculated percent conformance is low and the same specification conditions exist as above, frequencies might be increased to reduce the risk of accepting nonspecification material.
   7. The SCA procedures are being used successfully by some States and on Direct Federal projects in specifications for the acceptance of materials, most notably asphalt concrete. The use of SCA procedures for this purpose must be implemented carefully. Material and production variations for the specific materials or work items, characteristics to be measured to predict performance, specification limits, process control and acceptance sampling and testing responsibilities, and pay factors are all items which must be carefully determined when developing specifications and acceptance procedures.
5. DISCUSSION
   1. In making use of the SCA procedure, it is helpful to havesome knowledge of the overall variability inherent in the highway construction processes, the variability inherent in the natural makeup of materials, the variability associated with sampling and testing, and the effect these variabilities have on performance. This information may be available in a particular State from research on specific materials or from research done on a broader basis. A good source of information on materials variations is the reports resulting from early research by the Office of Research and Development of the Bureau of Public Roads (now Federal Highway Administration) entitled, "Quality Assurance in Highway Construction" which were published in Public Roads, Vol. 35, Nos. 6-11, February 1969.
   2. The SCA procedure gives a statistical estimate of the degree of conformity to specification requirements, not an absolute measure. All work would have to be tested or measured to be 100 percent sure that all material or work meets specification requirements. The results of SCA calculations are sensitive to sample size. The larger the sample size, the higher the degree of confidence one can have that the calculated percent conformity reflects the true degree of conformity of the construction process to the specifications.

   3. It is important to keep in mind that these procedures assume that tests and/or measurements are taken on a statistically random basis. This means that each element of work has an equal chance of being selected for testing. An SCA performed on test results of samples taken on a judgment basis should not be construed to reflect the calculated percent conformity of the total project or process because the samples are probably biased, either toward good or poor construction. The results of an SCA made on test results of samples selected on a judgment basis reflect the calculated percent conformity being produced at a particular place or point in time and are biased by the judgment of the person doing the sampling. This does not mean, however, that the SCA procedures should not be used when random sampling has not been used. It does mean that caution is necessary when using data taken under these conditions, especially data collected for determining variances of production or construction processes.

      All test results should be available for use. If retesting is done without performing any corrective work on the product and the original test results are not included, the calculated percent conformity will be erroneous and misleading. It is important that the test results actually represent the population being analyzed.

   4. The SCA procedure is a statistical tool and its use in evaluating construction activities is based on certain assumptions. Since the SCA procedure only estimates the degree of conformity with a particular specification, it is not a direct measure of quality when viewed in terms of eventual performance. A high calculated percent conformity is more indicative of good performance when all the following factors are present: when the quality of design is high, when the specification and other contract requirements adequately reflect properties and/or limits that ensure a good quality of performance and workmanship, and when the measured attributes relate to performance. Table 1 illustrates the importance of each component of the process in achieving good performance.
   5. In summary, the SCA procedure, when properly used, gives an estimate of the degree of conformity to specification requirements for the construction process and/or material to which it is applied. The test results and/or measurements used in the SCA should be randomly selected and accurate for the results to be totally valid. In order to infer whether or not good performance will likely result, the measured attributes and the specification requirements must be related to the properties which control performance. Care must also be exercised when comparing SCA results among processes, projects, or States. Comparisons are invalid unless the same specification requirements have been used in the individual analyses.

TABLE 1

6. CONCEPTS
   1. Research has shown that statistical methods can be applied to test values and measurements of most highway construction materials and work items. This research has also shown that the variation in the production of these materials conforms closely to a standard curve that is commonly used in statistical analyses. This standard curve is known as a "bell shaped" or "normal" curve, and represents the frequency of encountering a particular value. The normal curve is a symmetrical, bell-shaped curve that is centered on the average of all values representing a single process or production, and has a base width that is a direct function of how widely the individual values vary. This base width is referred to as the total variability of the population.
   2. Earlier versions of the SCA calculation procedure were based on the range of the test values (the difference between the minimum and maximum value) as the measure of variability. The current version uses the standard deviation as the measure of variability. The range method was chosen initially because of the simple calculations and ease of use of the method since the calculations were performed manually. With the advent of computers and inexpensive calculators that have the standard deviation function, the disadvantage of lengthy calculations associated with the standard deviation method have now been removed. The standard deviation method is more statistically efficient, requiring smaller sample sizes to provide an equivalent degree of assurance. It is less sensitive than the range to possible outliers because it is computed from all the data values, rather than the two extreme values. The standard deviation method avoids the need for the subgrouping rules advocated with the range method and has better mathematical properties for varioussubsequent statistical analyses that might be performed.
   3. The construction or material resulting from a single process or production is generally known as the "population. " As the variability, or standard deviation, of the population increases or decreases, the base width of the normal curve increases or decreases correspondingly.

   4. Figures 1 and 2 (below) illustrate this point. Distribution curves for three different populations of densities are shown in Figure l that have the same average values (x) but different variabilities. The population variability is represented by the Greek letter sigma (), meaning "standard deviation. "

      Figure 1

      

      Figure 2

      

   5. The distribution curves for the three populations shown in Figure 1, are superimposed in Figure 2 to graphically represent the effect of increased variability. If the lower specification limit (LSL) for the example material illustrated in Figure 2 is established at 95 pounds per cubic foot (pcf), the area under the distribution curves to the right of the LSL represents that part of the production that is greater than the LSL. As illustrated in Figure 3 (below), the area under any distribution curve is defined to be unity (generally accepted to be between X ± 3 ), or 1.0. Since the area under the entire curve represents 100 percent of the population, that portion of the area under the curve that is to the right of the LSL represents the percent of the population that is in compliance with the specifications. Conversely, the area under the curve to the left of the LSL represents the percent of the population which is not in compliance with the specifications. By using established statistical procedures and with the average value (X) and variability () of each population known, the percent of each population that is greater than the LSL can be determined. For example, the percent of the population that is greater than the LSL for distribution curve (a) is 98 percent,since the LSL is -2 from the population average. For curve (b), the percent of the population that is greater than the LSL is 92 percent, and similarly for curve (c), the percent of the population greater than the LSL is 87 percent. Therefore, the calculated percent conformity for each of the example populations would be 98, 92, and 87 respectively. Attachment 2 shows the mechanics for making these kinds of computations.

      Figure 3
      Percentage of Area Within Given Standard Deviation Limits

      

   6. Therefore, by using established statistical procedures, the percent of the population that is within specifications limits (the calculated percent conformity) can be determined if the average and the variability of the population are known. However, the average and variability (standard deviation) of the population are generally not known and, therefore, must be determined either from research, historical data, or estimated from calculations based on a sufficient number of samples or measurements of the population.
   7. Estimating the average and variability of the population from construction data often means that the estimates must be based on small numbers of samples. The SCA calculation procedure as embodied in the software (QLA, version II) previously distributed to all field offices, uses a methodology which needs only a small number of samples from the population to make these estimates. Test data is usually readily available from project records and relatively simple calculations are then required. The procedure used to calculate the standard deviation of the samples (s) is based on the "beta" distribution which is most applicable to small sample sizes (n < 30). The "beta" distribution is also applicable for larger sample sizes. Indices are computed using the average and standard deviation of the sample test values and the upper and lower specification limits. These indices represent the number of standard deviations the upper and lower specification limits are from the average of the test values, commonly referred to as "Z" values. Using the number of samples (n) and the calculated indices, aspecial statistical table (Table 1 of Attachment 2) is consulted to obtain the percent of the population, from which the test samples were drawn, which is estimated to be within specification limits. This calculation yields the area under a normal curve, between upper and lower specification limits, described by the average and the standard deviation of the samples (s) from the population.

E. Dean Carlson
Associate Administrator for
Engineering and Program
Development

References

1. "Construction Inspection Techniques," Construction and Maintenance Division, Federal Highway Administration, 1986.

2. Doty, Leonard A. , "Reliability for the Technologies," Industrial Press, Inc. , New York, 1985.

3. "Handbook of Applications of Statistical Concepts to the Highway Construction Industry," Parts 1-2, Materials Research and Development, Inc. , MAT-DES-DEV-WGAI-7l-660-1, 1971.

4. "Highway Condition and Quality of Highway Construction Survey," Instruction Manual, Federal Highway Administration, 1985.

5. "Sampling Procedures and Tables for Inspection by Variables for Percent Defective," Military Standard 414, Department of Defense, June 1957.

6. Willenbrock, Jack H. , "A Manual for Statistical Control of Highway Construction, Volumes I and II," Federal Highway Administration, NHI, January 1976.

7. "Standard Recommended Practice for Acceptance Sampling Plans for Highway Construction, AASHTO Designation R9-86," American Association of State Highway and Transportation Officials (AASHTO), 1986.

8. McMahon, Thurmul F. , and Halstead, Woodrow J. , "Quality Assurance in Highway Construction," Public Roads, Vol. 35, Nos. 6-11, February 1969.

Attachment 2

Specification Conformity Analysis Calculation Procedure and Example

This attachment explains the procedure used to compute the percent conformity of a given set of test results using the standard deviation approach. It is shown here to both explain the manual calculation steps and to help the user understand how the computer program for computing percent conformity (QLA, version II) works. After the test data is input, the computer program will compute the percent conformity for each item and provide a convenient method of storing, editing, accumulating, and analyzing the data. More detailed information on the features of the program are included in the operations manual which accompanies it.

The calculation procedure illustrated in steps 2 and 3 can be facilitated through the use of a calculator with statistical capabilities. Specifically the calculator must be able to compute the sample standard deviation which is commonly abbreviated "s" or "_n-1. " If such a calculator is not available, the standard deviation can be computed using the tabular approach presented in this section.

1. Randomly select a set of test data representing a minimum of 3 production days, if possible, with a minimum of two tests per production day. If only 1 day of production is used, a minimum of three tests is necessary. The upper and lower specification limits for the tests need to be known.

Example

The following set of data was collected from project test records for 3 production days of asphalt concrete paving:

The job-mix formula (JMF) target for individual test results is 6.2 percent, and specification limits are JMF + 0.4 and JMF - 0.4 percent.

2. Determine the average of the test results:

x = 6.4 + 6.6 + 6.0 + 6.7 + 5.8 + 6.2 = 6.28
6

3. Compute the sample standard deviation (s): [SEE PRINTED COPY FOR SAMPLE STANDARD DEVIATION]

An illustration of computing the standard deviation using the tabular approach is presented below. Alternatively, calculators with statistical capabilities can be used to compute the s value (this is also known as _n-1).

Individual Test Results (x)	Mean
6.4	6.28	0.12	0.0144
6.6	6.28	0.32	0.1024
6.0	6.28	-0.28	0.0784
6.7	6.28	0.42	0.1764
5.8	6.28	-0.48	0.2304
6.2	6.28	-0.08	0.0064
		=	0.6084

[SEE PRINTED COPY OF TA FOR COMPLETE RESULTS]

4. Find the upper quality index (Q_u) by subtracting the average (x) from the upper specification limit (U.L.) and dividing by the sample standard deviation (s).

[SEE PRINTED COPY OF TA FOR SAMPLE]

Example

The upper specification limit is 6.6 (6.2 + 0.4).

[SEE PRINTED COPY OF TA FOR EXAMPLE]

5. Find the lower quality index (Q_L) by subtracting the lower specification limit (L.L.) from the average and dividing by the sample standard deviation (s).

[SEE PRINTED COPY OF TA FOR SAMPLE]

Example

The lower specification limit is 5.8 (6.2 - 0.4).

[SEE PRINTED COPY OF TA FOR EXAMPLE]

6. Estimate the percent of material (P_u) that is below the upper specification limit by entering Table 1A or 1B with Q_u and using the column appropriate to the total number of test results being analyzed.

Example

From Table 1A, the value of P_u corresponding to Q_u = 0.91, and n = 6 can be found to be 81 percent.

7. Estimate the percent of material (P_L) that will exceed the lower specification limit by entering Table 1A or 1B with Q_L and using the column appropriate to the total number of test results being analyzed.

Example

From Table 1A, the value of P_L corresponding to Q_L = 1.37, and n = 6 can be found to be 93 percent.

8. Where both U.L. and L.L. specification limits are applicable, find the percent of material within these limits by adding the percent below the upper limit (P_u) to the percent above the lower limit (P_L) and subtracting 100.

Example

P_t = (percent within limits) = (P_u + P_L) - 100

P_t = (81 + 93) - 100 = 74 percent

This value represents the statistically predicted percent of the population to be within the specification limits.

9. Where only one specification limit is applicable, U.L. or L.L., the percent within limits is that value obtained directly from Table 1A or 1B, by using either Q_u or Q_L as applicable.

Table 1A Specification Conformity Standard Deviation Method

Table 1B: Specification Conformity Standard Deviation Method