Long-Term Pavement Performance Materials Characterization Program: Verification of Dynamic Test Systems With An Emphasis On Resilient Modulus

Chapter 5. PROFICIENCY PROCEDUREE

BACKGROUND

To perform a complete analysis of a laboratory’s ability to perform a particular test, all facets of the test process must be reviewed. A person with adequate experience in performing the test must be enlisted to do this review. The proficiency procedure brings together all of the experiments mentioned previously to ensure that accurate, repeatable test values are determined.

A large number of tests could be evaluated, but it is not practical to enumerate processes for each. The following section describes the general methodology to be used and provides three examples of screening criteria that could be used. Similar to previous sections of this document, the examples are based on LTPP Protocols P07 and P46. Similar procedures could be adopted for other test procedures as well.

Proficiency procedures should not be conducted until all mechanical and electrical system evaluation processes have been completed to the satisfaction of the evaluation team. Moreover, a well-written comprehensive test procedure must be available to document the test processes.

Several fundamental procedures should be evaluated during a proficiency testing review as follows:

• Material preparation.
• Test performance.
• Calculations.
• Data reporting.
• Data reasonableness.

The entire test procedure should be observed by personnel who are very familiar with the testing process. This should commence with sample preparation all the way through to data analysis and reporting. It should be noted that some acceptance criteria noted here are subjective, thus the necessity for a knowledgeable individual to perform this procedure.

Appendix A contains a checklist of items to look for in asphalt and soils/aggregate resilient modulus testing (figures 20 and 21).

DYNAMIC SYSTEM CONFIGURATION

All test systems should be turned on and the machine should be warmed up according to the manufacturer’s specifications.

APPROACH

Material Preparation

Inspection personnel should review material preparation activities, whether they be sample compaction (soils and aggregate) or sawing the specimen to size (asphalt). (See checklists, figures 20 and 21 in appendix A). Sample preparation worksheets should be reviewed to determine whether all applicable information is complete and accurate.

All sample/specimen preparation activities should be conducted according to applicable test procedures.

Test Performance

The inspector should view a test procedure being performed and perform a review of laboratory conformance with the test procedures.

Some items of special note that should be evaluated are as follows:

• Load pulse reasonableness.
• Deformation response reasonableness.
• Load versus deformation time lag.
• Review of data-computation process.
• Conformance to test parameters.
• Deformation device variation.
• Deformation balance.
• Reasonableness of test results.

For a resilient modulus test, the following items should be analyzed.

Load Pulse Reasonableness

Plot the load values (readings from the load cell) versus time for a representative cycle(s) at each load. Superimpose an ideal load over this typical load pulse. Compare the actual load pulse with the ideal load pulse. For resilient modulus testing, the procedure is set forth in the following paragraphs.

Construct a theoretical ideal loading pulse for each load sequence from the maximum load and the 0.1 s loading duration specified in the resilient modulus protocol. The peak theoretical load is matched in time with the peak recorded load of a given sequence. An acceptance tolerance band is then created around the theoretical load pulse and used to flag suspect data falling outside of the band. The development of the minimum and maximum values of the acceptance band is based on the following considerations:

• Acceptance tolerance range. A ±10 percent variation from the theoretical load is judged to be acceptable. In combination with the other checks, this range is effective at higher load levels and those near the peak. However, at low load levels, this range may create an unreasonably tight tolerance.
• Servovalve response time. A ±0.006 s time shift in load from the theoretical load pulse is reasonable to allow for the physical limitations on the response time of the servohydraulic system. This will provide a reasonable tolerance band that will be effective at intermediate loading and unloading portions of the load cycle.
• Resolution of the electronic load cell. The resolution of the electronic load cell generally used in resilient modulus testing for these materials is ±4.5 N. Therefore, a range of twice the minimum resolution of the load cell is used, i.e., ±9 N. This range provides acceptable tolerances for testing at low load levels.
• Logic. The minimum load allowed is 0 N.

For each time step in the load curve, the tolerance range from all of these components is computed. The maximum value of these three components is selected as the upper tolerance limit, while the minimum value is used for the lower limit at each time step. Over the entire range of loading, 5 points are allowed to be out of tolerance before the load cycle is considered failed. If pulse duration and shape are not improved within a reasonable number of iterations, problems such as friction in the servovalve piston, inadequate servoram size, problems with software controlling the load, etc., should be investigated.

Also, review the time history data for each load pulse. Ensure that one load cycle consists of 500 points. If the load cycle does not have 500 points, the system fails this check. The experiment should be repeated using a data acquisition rate of 500 points/s.

For this experiment, the user is looking for the following acceptance criteria:

• Generated haversine waveform is within tolerance.
• Load consists of 500 points per cycle.

Deformation Response Reasonableness

Plot the deformation values (readings from the deformation device) versus time for a representative cycle(s) at each load level. Superimpose an ideal deformation response over this typical deformation pulse. Compare the actual deformation pulse with the ideal deformation pulse. For resilient modulus testing, the approach is set forth in the following paragraphs.

Construct a theoretical ideal deformation pulse for each load sequence from the maximum deformation and the 0.1 s loading duration specified in the protocol. The peak theoretical deformation is matched in time with the peak recorded deformation of a given sequence. An acceptance tolerance band is then created around the theoretical deformation pulse and used to flag suspect data falling outside of the band. The development of the minimum and maximum values of the acceptance band are based on the following considerations:

• Acceptable tolerance range. A ±10 percent variation from the theoretical deformation is judged to be acceptable. The theoretical deformation is established by finding the maximum deformation and applying the checks set forth in the next paragraph to it using the equation for a haversine waveform. It should be noted that for a real specimen, the deformation response is specimen specific and may not follow a haversine waveform exactly.
• Logic. The minimum deformation allowed is 0.

For each step in the deformation curve, the tolerance range from these components is computed based on the maximum deformation point for the cycle. The maximum value of these checks is selected as the upper tolerance and the minimum value of these checks is selected as the lower limit for each time step. The actual deformation is then compared with the tolerances at each time step to determine conformance. Over the entire range of loading, 5 points are allowed to be out of tolerance before the deformation cycle is considered failed. If the system fails this check, then problems with system response and deformation response should be evaluated in a similar fashion as if the load pulse failed the check.

Perform this test for all deformation devices.

For this experiment, the user is looking for the following acceptance criteria:

• Generated deformation output is within tolerance.
• Load consists of 500 points per cycle.

Load versus Deformation Time Lag

Determine the maximum load point for a given cycle and extract the corresponding time stamp. Determine the maximum deformation for the same cycle and extract the corresponding time stamp. Subtract the maximum deformation point time stamp from the maximum load point time stamp. This value should be positive.
If the time delay is negative, it means that the maximum deformation is occurring prior to the maximum load, a practical impossibility.

Perform this analysis for each deformation device used for this experiment.

For this experiment, the user is looking for the following acceptance criterion:

• Maximum deformation is occurring after the maximum load.

Review of Data Computation Process

In this analysis, the raw data are manually reduced and the results compared to the summary data developed by the computer software system. For resilient modulus testing, the following values should be analyzed as is practical:

• Cyclic load.
• Maximum load.
• Contact (seating) load.
• Deformation response (for all deformation devices).
• Confining pressure (soils and aggregate testing).
• Temperature (asphalt testing).
• Deviator stress (soils and aggregate testing).
• Strain (soils and aggregate testing).
• Resilient modulus.

These values should be derived from the raw data using procedures stated in each protocol.

Special note should be paid to testing under LTPP Protocol P07. The data analysis routines in this protocol are very complicated. The data analysis routines used by LTPP have previously been verified by hand and are considered stable, thus no further evaluation should be conducted.

For this experiment, the user is looking for the following acceptance criterion:

• All manually calculated values should be within 5 percent of the automated calculated values.

Conformance to Protocol

After the summary data have been verified by hand, adherence to the protocol parameters should be analyzed. This analysis is undertaken to determine how close the laboratory is to the protocol test requirements.

The following items should be checked as appropriate for a particular protocol:

• Target deviator stress (soils and aggregate testing).
• Target confining pressure (soils and aggregate testing).
• Target temperature (asphalt testing).
• Target deformation response (asphalt testing).

For this experiment, the user is looking for the following acceptance criterion:

• All test parameters should be within 5 percent of the test requirements.

Coefficient of Variation of the Deformation Devices

The deformation devices should be stable within a given loading regime and test cycle. In this analysis, the coefficient of variation of the deformation devices is calculated at a given test sequence. To perform this analysis, select the deformation values for all collected cycles of data at a given test sequence. Determine the coefficient of variation of the values. Repeat for all test sequences and deformation devices.

For this experiment, the user is looking for the following acceptance criterion:

• Vertical deformation readings from each of the sequences should be such as to ensure that the deformation devices are recording values with averages that (for the collected cycles) have a coefficient of variation less than 2.5 percent.

Deformation Balance

This test is only conducted if more than one deformation device is mounted on the system. All deformation devices used for this comparison should be mounted in approximately the same location in the system, such as on top of the triaxial chamber or on the test specimen. In this analysis, the balance of the deformation devices is evaluated. If deformation devices are mounted in approximately the same location on the sample or in the system, it can be reasonably expected that the devices would experience similar deformation measurements.

For a given deformation cycle, extract the cyclic deformation value from the summary (calculated) data. The collected deformation readings will be checked to ensure that acceptable vertical deformation ratios are being measured. Acceptable vertical deformation ratios (R_v) are defined as R_v = Y_max/Y_min < 1.30, where Y_max equals the larger of the two vertical deformations and Y_min equals the smaller of the two vertical deformations. This analysis should be performed for each deformation device in order. If more than one deformation device is used, deformation transducer 1 should be used as the reference deformation value.

For this experiment, the user is looking for the following acceptance criterion:

• Deformation ratios of less than 1.30 should be observed for all the test sequences.

Reasonableness of Final Results

The final results of the test procedure should be reviewed for reasonableness. This check can only be conducted by personnel familiar with the test procedure.

For asphalt testing, the user should review the resilient modulus values versus temperature. The resilient modulus values should be decreasing versus increasing temperature.

For soils and aggregate testing, a basic check can be made of the resilient modulus versus confining pressure results. Generally, resilient modulus values at lower confining pressures should be lower than those at higher confining pressures (for a given deviatoric stress).

In this experiment, the bottom line is that the results should look reasonable to the user. Any anomalies should be investigated.

For this experiment, the user is looking for the following acceptance criterion:

• All final test results should be reasonable as determined by the review team.

In addition, the overall proficiency of the operator should be evaluated. Does the operator confidently run the test machine? Does the operator monitor progress of the test as it progresses? Is the operator capable of saving and retrieving test results?

All of these items should be evaluated in the most objective manner possible. For resilient modulus testing, the acceptance criteria used in other areas of this report can be used for proficiency testing. For instance, the pressure for a soil/aggregate test must be within ±2.5 percent of target. Example acceptance criteria are given in a later section of this document.

Calculations

Many test procedures have been automated to the point that data analysis and result preparation are handled automatically by computer. It is necessary, at least in the beginning of a testing program, to verify that the computer algorithm used to generate data is functioning properly and is indeed calculating correct values. Therefore, a manual analysis of the data results should be undertaken prior to accepting the data-calculation procedure. This can be a very time-consuming effort but is highly recommended to ensure accurate data.

To perform this procedure, the raw data should be broken down by hand and all test results calculated independently of the test system software or calculation algorithm. All values should generally match within 5 percent of those calculated using the test system calculation algorithm.

Data Reporting

The reviewer should inspect the data-collection forms to ensure the completeness and accuracy of the results. All sample numbering and naming should be checked for completeness and all calculated values should be checked for accuracy.

Data Reasonableness

As a final check, the resultant data should be checked for reasonableness. This should include reviewing the data patterns to ensure that suitable test results are obtained.

Data Analysis

All raw data (time, load, deformation, etc.) and summary data (calculated data) should be extracted from the test system. All of these data are used for the data analysis portion of this procedure.

Material Preparation

All calculations should be verified by hand to ensure accuracy.

DISCUSSION

If, using the acceptance criteria, the system fails this test, repeat the procedure. If the system fails the second test, the system fails the check. If the system passes the second test, then a third test should be run to determine acceptance or failure. The apparatus should be disassembled and re-assembled between each test.

Appendix A contains examples of a checklist for the LTPP resilient modulus procedures (figures 20 and 21).

A laboratory performing resilient modulus testing should consider participating in an inter-laboratory testing program to verify the calibration of the equipment and the procedures with respect to other laboratories. Also, it may be desirable to manufacture or procure a standard specimen to test on a continuing basis to detect gross changes in system performance over time.

EXAMPLE

The following is an example of the evaluation of results from a LTPP protocol P46 type 1 sample.

The proficiency check requires acceptable performance of the following major activities:

• Material preparation.
• Test performance.
• Calculations.
• Data reporting.
• Data reasonableness.

Material Preparation

The evaluation team observed the compaction procedures. In general, the compaction procedures observed were performed in accordance with the protocol. It was noted that the laboratory uses a standard compaction worksheet and all values on this worksheet, including specimen identification information, were verified for completeness and accuracy.

Test Performance

The individual load pulses and deformation pulses were reviewed for conformance with applicable standards. An example of the loading conformance check is shown in figure 10.

Table 22 contains a summary of the results of this analysis for load and deformation.

Figure 10. Graph. Example of load versus time check.

In summary, the load pulses appear reasonable. It will be noticed that lower and upper acceptance curves are shown on the loading figures. These curves are based on the same rationale as presented previously in this document. For load pulses that fail the check, the load pulse was outside of the acceptance bands.

The LVDT plots were also reviewed for reasonableness. Deformations were plotted versus the acceptance criteria; an example is shown in figure 11.

Figure 11. Graph. Example of deformation response analysis, type 1.

Most deformation curves were fairly flat at the maximum deformation point. This may be a case of the LVDT reaching its maximum stroke during operation or sticking during travel. All deformations fail the acceptance criteria rather badly; this phenomenon should be investigated more completely.

Next, a thorough evaluation of the data-computation process was undertaken. In this exercise, the raw data are manually reduced (per P46 specifications) to yield deformation and load values. These values are then used to calculate axial stresses, strains, and ultimately resilient modulus. All data for the resilient modulus test were obtained and independently analyzed. Table 23 contains the result of this comparison. In this table, all values are expressed as percent differences between calculated values and the independent review conducted by the evaluation team. As can be seen, all values showed very good correlation, indicating that the laboratory’s analysis algorithm is calculating values per the procedure.

Next, the deviator stress and confining pressure values obtained from the summary data file were compared to the test parameters to determine whether the test was performed in accordance with the procedure. Table 24 contains the results of this analysis.

As shown in this table, the maximum deviation from the target deviator stress is 2.8 percent. These percentages generally trend lower as the test progresses. This is considered an acceptable result. Likewise, the confining pressure values are very close to the target values identified in the procedure. These meet the acceptance criteria.

The coefficient of variation for the LVDTs was checked. Table 25 shows the results; all LVDTs were within the prescribed tolerances (coefficient of variation less than 2.5 percent for 14 of the 15 test cycles). This indicates that the LVDT measurements are very stable within a given test sequence.

Also, the LVDT deformation ratios were calculated as shown in table 26. This value is determined by taking the maximum cyclic deformation between the three LVDTs divided by the minimum deformation among the three LVDTs. A value less than 1.30 is an indication that the LVDTs are mounted properly on the sample and are recording consistent data. As can be seen from this table, the balance of the LVDTs is very poor for this test. Some values are measuring more than two times the deformation of the others. This is not an acceptable result, and this phenomenon should be investigated more thoroughly.

Finally, the resilient modulus results were reviewed. The results of the resilient modulus test are given in figure 12. The sample behaved in an expected manner and the overall resilient modulus results appear reasonable.

A completed checklist with the results from this procedure is shown in figure 13. The checklist in blank form is included in appendix A (figure 20).

Figure 12. Graph. Example of P46 type 1 (base/subbase) results.

N/A: Not applicable in this instance; NO: Procedure not done or criterion not met.
Note: The checklist in blank form is included in appendix A (figure 20).

Figure 13. Form. Sample checklist for base/subbase proficiency procedure.