Aggregate Image Measurement System 2 (AIMS2): Final Report

APPENDIX A

Chapter 3 – Interlaboratory Study

Introduction

The Interlaboratory Study (ILS) was conducted to determine the repeatability and reproducibility of AIMS2 for multiple users and laboratories. The ILS was carried out in accordance with ASTM C 802 – 96, "Standard Practice for Conducting an Interlaboratory Test Program to Determine the Precision of Test Methods for Construction Materials." The ILS results were used to develop a precision statement for the test method using ASTM C 670 – 03, "Standard Practice for Preparing Precision and Bias Statements for Test Methods for Construction Materials."

ILS provides two different precision estimates of the test method; single-operator precision (within-laboratory precision) and multi-laboratory precision (between-laboratory precision). The single-operator precision provides an estimate of the variance that may be expected between duplicate measurements of the same sample made by the same operator in the same laboratory. The multi-laboratory precision gives an estimate of the differences that may be expected between measurements of the same material made in different laboratories by different users. The single-operator and multi-laboratory precision statements were determined in this study for the outputs of the AIM2S system: angularity, texture, 2D Form, sphericity, flat or elongated 3:1 ratio.

Aggregate Sources and Sizes

Three different aggregates (Crushed Gravel, Limestone, and Granite) were used for all sizes except that a sandstone source was used instead of granite for the size passing the 0.15 mm sieve (ASTM #100 sieve) and retained on the 0.075 mm (ASTM #200 sieve). Based on previous characterization of these aggregates, the Crushed Gravel (CG) has the lowest angularity and texture among the three aggregates, the Granite (GR) has the highest angularity and texture, while the Limestone (LS) is in the middle. A list of the materials and sources used in this study are shown in Table 3.1. The coarse and fine aggregates sizes are listed in Table 3.2. Coarse aggregates are defined as those retained on 4.75 mm sieve (ASTM #4 sieve), while fine aggregates are those passing the 4.75 mm sieve (ASTM #4 sieve). In Table 3.2, the aggregate size range gives the sieve size that all particles pass through and the sieve size that all aggregates are retained on.

As discussed, different aggregates sources were used for the 0.075 mm size (ASTM #200 sieve). These were Crushed Gravel, Limestone, and Sandstone. For simplicity in this study the 0.075 mm (ASTM #200 sieve) sandstone will be grouped with the granite.

In addition to the average shape characteristics for each sieve range, the AIMS2 software includes a method to determine the weighted average of a certain property of an aggregate blend. The weighing averaging factors are determined based on aggregate size (see Appendix A). The hypothetical gradation shown in Table 3.3 was used in determining the shape characteristics of the blend. Since the 0.075 mm (ASTM #200 sieve) fine aggregates were not from the same sources as the other aggregates sizes, it was not included in the combined results.

Each aggregate source was sieved according to the size ranges and randomly separated into samples which were shipped with each AIMS2 machine to the participating laboratories. Each coarse aggregate sample consisted of 60. All of the particles were placed on the tray and 50 of them were used in the analysis. Approximately 150 grams of each fine aggregate size, 2.36 mm (ASTM #8 sieve) to 0.15 mm (ASTM #100 sieve), and 50 grams of 0.075 mm (ASTM #200 sieve) aggregate were sent to the laboratories. A fine aggregate sample was spread onto the tray and 150 aggregate particles were used for the analysis.

Eight AIMS2 machines were used in this study. Given the number of participating laboratories (32 labs), three to four laboratories used the same exact machine and tested the same samples. This procedure satisfied the number of materials and participating laboratory requirements of ASTM C 802-96. Testing began with successfully calibrating the machines according to manufacture instructions. The user was instructed to scan the two replicate measurements on different days to provide meaningful replicate values. Data from each test was automatically saved into computer files.

Data Analysis

Following careful examination of the procedure followed to conduct measurements, three laboratories' data were removed from the ILS study due to user error by not following manufacture and procedure instructions. The within-laboratory and between-laboratory variances were calculated using data from the remaining 29 laboratories. A list of the raw data is show in Appendix B.

With an additional analysis of the raw data images, several 4.75 mm (ASTM #4 sieve) texture images were found to be of the aggregate edge instead of the aggregate surface. Figure 3.1 shows two texture images, one image including the aggregate edge and one image of the aggregate surface. The image of the edge of the aggregate contains both the surface of the aggregate and the surface of the tray. If several images are of the aggregate edge are within a sample data, these images can affect the AIMS2 results, in particularly the texture values. The images with aggregate edges were removed manually and the results were recalculated for the remaining images. The remaining coarse aggregate sizes were checked and the images did not have the same problems as the 4.75 mm (ASTM #4 sieve) aggregates.

Figure 3.1. Texture image with and without aggregate edge.

The data was checked for agreement of variances and interactions between material and laboratories. ASTM C 802-96 assumes that different laboratories have the same within-laboratory variances. The variance of the each laboratory was checked for an agreement of variances based on the ratio of the largest variance to the sum of variances. The laboratories with the variances above the upper 5% level were eliminated to bring the variances into agreement. The interactions between laboratory and material were checked by plotting the averages values obtained by each laboratory to aggregate type. A similar pattern of change was found from one material to another which indicated little to no interaction between laboratory and materials. An example of the analysis results of all 29 laboratory data is shown in Figure 3.2 for the angularity measurement of 25.0 mm (1in) size aggregates.

Figure 3.2. Interaction check for angularity versus material for 25.0 mm (1 in) aggregates.

The components of variance, variances, standard deviations, and coefficient of variations were calculated for each shape property for each aggregate type. The components of variance are the estimated amount of variation that can be attributed to the effects of the experiment from the factor.⁵ The averages, components of variance, and variances of the Gravel, Limestone, and Granite are shown in Table 3.4, Table 3.5, and Table 3.6, respectively. The standard deviation and coefficient of variation of the Gravel, Limestone, and Granite are shown in Table 3.7, Table 3.8, and Table 3.9 respectively. The combined data results for the weighted aggregate blend are listed in Table 3.10 and Table 3.11.

It should be noted that the analysis was conducted on the flat or elongated 3:1 ratio instead of the 5:1 ratio because the aggregate samples had a few or no particles that exceeded the 5:1 ratio. For example, the 25.0 mm (1 in) and 4.75 mm (ASTM #4 sieve) aggregates, it was found during the analysis that any small variation in measurements even by one particle would translate to a very high coefficient of variation if the 5:1 ratio was used. The coefficient of variation reported for the flat or elongated 3:1 ratio were calculated based on the average percent of particles that have a ratio less than (not more than) 3:1.

The standard deviations and coefficient of variations were plotted against the average of each materials source.  Figure 3.3 and Figure 3.4 are examples of the standard deviation and coefficient of variations relationships for the angularity measurement of 25.0 mm (1 in) aggregates. 

The precision statement of the data was established by analyzing the relationships of the standard deviations and/or coefficient of variations. The ASTM C 670-96 procedure includes two provisions for the data analysis. One provision is for a constant standard deviation case and the second provision is for a constant coefficient of variation case. The constant standard deviation case is where pooled within-laboratory standard deviation over all the materials becomes the single-operator standard deviation and the pooled between-laboratory standard deviation becomes the multi-laboratory standard deviation. In the case of a constant coefficient of variation, the average within-laboratory and between-laboratory coefficient of variation becomes the single-operator and the multi-laboratory coefficient of variation, respectively.

Neither of the constant standard deviation or constant coefficient of variation conditions was strictly satisfied in the analysis results.  However, from an engineering perspective, the variation of the standard deviation and coefficient of variation is considered small.  Therefore, it was decided to determine the precision statements for both a constant standard deviation and a constant coefficient of variation for the single-operator (within-laboratory) and multi-laboratory (between-laboratory) precision. 

Figure 3.3. Standard deviation versus average angularity of 25.0 mm (1 in).

Figure 3.4. Coefficient of variation versus average angularity of 25.0 mm (1 in).

The precision statements (1s%) based on the assumption of constant standard deviation are shown in Table 3.12 for the single-operator and multi-laboratory results.  The precision statements (1s%) based on the assumption of constant coefficient of variation are shown in Table 3.13.  The combined results precision statements (1s%) based on the assumptions of constant standard deviation and constant coefficient of variation are shown in Table 3.14 and 3.15, respectively.

Based on these precisions statements, the results of two properly conducted tests (d2s%) which are tested either by a single-operator or multi-laboratory are not expected to differ more than the values shown in Table 3.16, Table 3.17, Table 3.18, and Table 3.19. These numbers are based on the calculations described in ASTM C 670-96.

The machines were calibrated before each laboratory scanned the materials to eliminate possible sources of error. The 0.075 mm (ASTM #200 sieve) has larger than expected single-operator and multi-laboratory standard deviation. After investigation into the possible sources of error, the CHPR value, which is used to eliminate touching particles from the data before it is analyzed, was found to be the source of error.⁶ The limits of the CHPR value were believed allow several touching particles to be analyzed. This was determined by an inspection of the number of touching particles in the images from the analyzed data. Therefore, the 0.075 mm (ASTM #200 sieve) results should be further examined after developing a more robust method to eliminate touching particles. Once such a method is developed, precision statements for the standard deviation and coefficient of variation results will be developed for this size.

Conclusions

The analysis conducted in this chapter lead to the development of precision statements for the different shape indices and parameters given by AIMS2. In general, the experiments gave very reasonable coefficient of variation for the various indices for all sizes except the 0.075 mm size. The results from the constant coefficient of variation should be used to describe the precision statement since the standard deviation results have a slight trend of increase with an increase in average. Therefore, a precision statement based on constant standard deviation will be biased against materials with low average and work in favor of materials with high average. Overall, the maximum coefficient of variation was less than 5% for a single operator and less than 8% for multi laboratories when individual sizes were analyzed. The maximum coefficient of variation for the combined results of a blend was less than 3% for a single operator and less than 5% for multi laboratories. These are considered acceptable coefficient of variation values given the natural variation in aggregate samples from the same source.

Further tests will be necessary to determine the proper CHRP calibrated value for the 0.075 mm (ASTM #200 sieve) in order to remove touching particles in the analysis. The determination of this value is expected to reduce the variations in the measurements conducted on the 0.075 mm sieve, and reduce the precision coefficient of variation reported for this size.

The precision statements from the constant coefficient of variations were combined for aggregates sizes, excluding 0.075 mm (ASTM #200 sieve), for each aggregate shape characteristic. The was done by taking the square root of the sum divided by n-1 of the squares of all sizes except 0.075 mm (ASTM #200 sieve) for each aggregate shape property. Where n is the number of values sum. The precision statements for the single limit (1s%) and difference of two results (d2s%) are shown in Table 3.20 and Table 3.21, respectively.

⁵ Jay L. Devore (Ed.), Probability and Statistics for Engineering and the Sciences, 6th ed., 2004.

⁶ E. Mahmoud, L. Gates, E. Masad, S. Erdogafan, and E. Garboczi, "Comprehensive Evaluation of AIMS Texture, Angularity, and Dimension Measurements," Journal of Materials in Civil Engineering, vol. 22, no. 4, pp. 369–79, 2010.