U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
This report is an archived publication and may contain dated technical, contact, and link information |
|
Publication Number: FHWA-RD-03-093
Date: August 2006 |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Study of Long-Term Pavement Performance (LTPP): Pavement DeflectionsChapter 5. Data Entry ErrorsData ScreeningIn the course of screening for load-deflection errors, several types of generally correctable data entry errors were identified. These include the following:
Long-Term Sensor Positioning ErrorsProbably the most important—and by far the most common—form of data entry errors found in the FWD-associated database occurred during a simple, incorrect (and obviously inadvertent) manual recording of the actual sensor positions along the FWD’s raise-lower bar. In virtually all instances where this occurred, the FWD deflection sensors appeared to be functioning normally, and the only anomaly was an oversight rather than a fatal equipment malfunction or irreparable error. After a long search for a satisfactory method of screening for such errors, a method was devised whereby the deflection basin was transformed into nearly a straight line through a process called the SLIC transformation or transformed basin method. SLIC (pronounced “slick”) is an acronym for the authors of a 2000 paper—Stubstad, Lukanen, Irwin, and Clevenson.(1) Background and early development information on the transformed basin procedure are covered in this paper, which was prepared for the Transportation Research Board (TRB) and is largely based on the LTPP data-screening project reported in this report. A copy of this draft paper is included in the 2000 TRB preprint compact disk (CD), and the paper was subsequently published by TRB. An automated version of the SLIC procedure was eventually developed to screen the FWD database more thoroughly, whereas a visual-manual method was used when the sensor positioning errors were first identified with certainty in 1999. The technical details of the screening method are described in appendix B. The automated screening method uses a slightly different transformation of the deflection versus offset data than the visual method. Instead of a plot of ln-ln normalized deflection versus ln offset (as the visual method employs), the automated method predicts the position of sensors using two variable exponents (depending on sensor position) and a regression equation. This process has removed the previously identified bias from the method (particularly for the prediction of sensor d7 on the seven-sensor LTPP basin configuration), so that, on average, sensor position predictions will be equal to the protocol position in the LTPP database in cases where the sensor positions were not suspected to be in error. Using the automated SLIC method, the precision of the prediction model also improved appreciably. The visual method transforms a normal S-shaped deflection basin into a straight or smoothly curved line, assuming the measured deflections and the offset distances to the sensor holders are all correct. Table 5 presents the identified sensor positioning errors for each FWD used by the LTPP program, along with an associated period of time. Only the errors that are certain (i.e., those that can be verified through both the manual/visual and automated sensor prediction processes, together with other means and sources of evidence) are recommended for change in the database. In some cases, less serious sensor positioning errors may also exist; however, since these are not as certain, they were not flagged or recommended for change in the database.
1 inch = 25.4 mm * Same FWD, same period of time—LTPP field tests conducted in two different regions. Table 5 notes based on SLIC analyses, plus a variety of information sources:
Table 5 reflects the sensor positions that were used during each given period of time and with the specified FWD unit, as listed in the table. When concrete joints were tested, the protocol called for the FWD operator to move sensor #2 to a position 30.5 cm (12 inches) behind the loading plate. This was usually—but not always—carried out; however, this error category is dealt with elsewhere in this report. It is very important to review all the graphs shown in the referenced appendices for each example shown in table 5 and its footnotes. In each case, it can be argued “a picture speaks a thousand words.” Further—in each case—there can be no doubt whatsoever that the sensor positioning errors listed in table 5 are correct and highly accurate. As can readily be seen in table 5, long-term sensor positioning errors occurred infrequently but in all four regions, in many cases over relatively long periods of time. Such errors also occur, with unknown frequency, elsewhere—whether with State and other DOTs, National Road Administrations, or private consultants. The authors of this report have identified numerous instances where other FWD-sourced data was found to be incorrect due to sensor position reporting errors by equipment operators. In most of these cases, these errors were identified much earlier in the process, generally within a week or two of each occurrence. For the LTPP program, the good news is that changing the database sensor configuration tables to reflect the actual (as opposed to protocol) sensor positions, when such data entry errors generally occurred with an astronomically high probability, can easily rectify these inadvertent but very important errors. To further illustrate how the visual SLIC method works, examples of four transformed deflection basins are shown in figures 3 and 4. The deflection basins are from two sections tested 1 year apart. The data are the SLIC data from FWD SN 130 in line 5 of table 1, and footnote 5 to which this line refers. First we assume that the sensor positions were as reported in the database (in this case, the protocol positions) in all four of the illustrated cases. Based on the visual data in figure 3, it can clearly be seen that sensors 5 and 6 were misplaced along the raise-lower bar during the 1994 tests, while they were properly positioned during the 1995 FWD tests performed at the same LTPP sections about one year later. Under normal circumstances, each pair of lines should very nearly parallel one another. In figure 4, the two basins where sensors 5 and 6 were misplaced by 30.5 cm (12 inches) are replotted using their actual positions against the two correct (protocol) basins. Here it can be seen that the two pairs of lines are almost perfectly parallel; in fact, for all practical purposes, they overlap. It is of vital importance that the positions of the FWD’s deflection sensors are known quite precisely. It has been shown in numerous studies, including the TRB paper referred to earlier, that even very small errors in deflection reading accuracy affect the results of backcalculation appreciably.(1) It is not difficult to imagine, therefore, that if a given deflection sensor is even a small percentage of its plate distance away from its reported position, the results of backcalculation will be markedly affected. Figure 3. Graph. Sample deflection basins transformed with SLIC(input protocol positions). Figure 4. Graph. Sample deflection basins transformed with SLIC(input actual positions). The LTPP program has gone to great lengths to ensure, through periodic relative and reference calibration procedures, that the FWD's sensors are reading correctly, even to a better accuracy than advertised by the manufacturer (±2 percent ±2 mm). Since inadvertent and therefore unreported shifts in sensor position, such as those listed in table 5, generally change the measured deflections by at least 10 mm (usually more), this effect will be profound—unless the particular sensor or sensors in question are ignored in the backcalculation process. Statistical Calculations Supporting the Recommended Changes to Sensor PositionsThere are several standard ways to support the changes being recommended. In the referenced TRB paper, there is a two-sample T-test that can be used to determine whether the data from the dates in question, produced by the FWD on the dates in question, comes from the same population as the population of data produced by this same FWD on dates outside of the suspected interval.(1) The data in both cases are the predicted values of the sensor positions for the hypothesized erroneous sensor position. The P-value for this test is smaller than 10–52, a value so small that to continue to accept that hypothesis would be foolish. In plain language, if the sensor had not been moved during the suspect period, the probability of data so different for this period as that actually observed is nil; these data could not have happened. These calculations leave open the question of which hypothesis should replace the hypothesis that clearly requires rejection, that is, the hypothesis that the sensor remained at 152.4 cm (60 inches) throughout the time period identified. Several points need emphasis. One is that the automated method used to predict sensor positions was the same regardless of which of the above-described errors was identified. One of the suspected errors, shown in the next-to-last row in table 5 with 191 or more dates in error, was confirmed independently by other means. At that time, it was confirmed through a physical measurement that the d7 sensor was offset 121.9 cm (48 inches). Furthermore, there was a sensor holder in the raise-lower bar at 121.9 cm (48 inches). A position of 121.9 cm (48 inches) is therefore natural to hypothesize, and to compare with the hypothesis that the sensor is at 152.4 cm (60 inches). One standard method of comparison is the likelihood ratio. The technical details of the likelihood ratio computations are shown in appendix M. The likelihood ratio compares the ratio of the probabilities of data under competing hypotheses. If H48 is the hypothesis that sensor 7 was offset 121.9 cm (48 inches), and H60 is the hypothesis that sensor 7 was offset 152.4 cm (60 inches), then the likelihood ratio computes P48 (data given that H48 is true) divided by P60 (data given that H60 is true). Since the sensor error for FWD data with SN 8002–131 from May 1994 to April 1996 is not questionable, the likelihood ratio was computed for other sets of data. This was done for three other sample sets of data from table 5: one set with a small number of dates, one with a moderate number of dates, and one with a large number of dates, but different from the confirmed case containing 191 dates. An abbreviated summary of the likelihood ratio calculations appears in table 6.
1 inch = 2.54 cm In short, in the least compelling case, the data are over 2,000 times more likely to have occurred with the sensor 7 at 121.9 cm (48 inches) than with this sensor at 152.4 cm (60 inches). It should be re-emphasized that great effort was made to find an automatic prediction method that was apparently unbiased for predicting sensor 7, when sensor 7 was correctly positioned at 60 inches, and with good precision. Corroborating evidence confirms that the predictions of the position of sensor 7, with the same FWDs before and after the period in question, do not suggest any anomalies. Additionally, tests conducted during the period of time in question, on the same test sections but with different FWDs, also do not produce unusual (nonprotocol) sensor 7 predictions. Finally, in most cases there is additional evidence that the starting and ending dates are correct. For example, there generally was a calibration of the particular FWD in question between the ending date in table 5 and the first date the same FWD was used after calibration, when the sensor(s) were repositioned to their protocol positions. However, only the outputs of the predictions of the sensor(s) in question were used to construct table 5 (see the footnotes and appendices to table 5 for many other examples). All this evidence was further corroborated by extensive backcalculation in most of the table 5-listed cases of nonprotocol FWD sensor positions in the database. RNS–2 and RNS–2M Feedback ReportsThe occurrence of long-term sensor positioning errors was first reported in September 1999 (Feedback Report RNS–2). After that report was submitted, it was not universally believed that there could have been such errors made in sensor positions, at least to the extent indicated by RNS–2. Subsequently, a trial of the SLIC method was conducted, wherein it was quite clearly shown that one can, in fact, predict actual sensor positions (or sensor errors) using the SLIC method as long as some supporting information is included, such as the type of data included in the extensive LTPP database and shown in appendices C through L. Subsequently, the semiautomated method was developed to rescreen the entire pre-autumn 1998 database, to verify or reject the findings thereof. The result was an even more certain verification-almost to the letter-of the previous findings. In addition, two other previously undetected (though relatively short-term) sensor position errors were detected. The original eight occurrences (corresponding to nine of the lines in table 5, two of which are overlapping) and the two newly identified errors are all described in this chapter under the heading “Long-Term Sensor Positioning Errors.” Further, these findings are now supplemented by a variety of supporting information and graphs presented in the appendices. As a result of these findings, Feedback Report RNS–2M was submitted. This Feedback Report is also shown in appendix A. RNS–2M was accompanied by most of the associated information presented in this report, together with a copy of table 5 as presented here. The LTPP database user should use extreme caution when any load-deflection data from these ten periods of time, and the same FWD serial number, are used for analyses. Although presently the LTPP database still reports protocol sensor positions, these data are not correct; use of these data will result in incorrect analyses unless the sensor(s) in question are ignored when analyzing the FWD data. Development of a Screening Product Using the Transformed Basin MethodBased on the SLIC-based screening of the LTPP database, as discussed previously, a SLIC product was developed for use in the field. This product can conduct an automated SLIC screening of FWD data directly on the FWD’s output, as long as the data file has been closed and stored. Presently, this product is limited to Dynatest FWD field program generated load-deflection data, gathered and stored in Edition 25 output format. The original SLIC screening procedure tended to underestimate the position of the last sensor, sensor 7, in the LTPP database. Also, the semiautomatic version of that procedure described in the literature produced estimates that had an undesirable amount of variability.(1) Many models for the data were tried, and eventually a model that would estimate the position of the outer sensors without bias was found. A similar attempt was made to improve the estimate of the position of the inner sensors. This effort was also successful, but unfortunately a slightly different model was the most accurate, with (virtually) zero bias and good precision. These semiautomatic models are most useful for trying to correct errors in recording the position of the LTPP sensors. In these situations, the visual or automatic SLIC procedure has identified a period of time when the position of a particular sensor was apparently not properly recorded, and one must estimate the actual sensor position without being able to directly examine the FWD unit itself. Accordingly, the field product should have a different function, since the FWD operator has the opportunity to measure the sensor positions and/or check the sensors for other errors, and correct any sensor positioning error on the spot if necessary. Further, the operator can adjust the already recorded data to reflect the actual sensor positions of those measurements already made in cases when a sensor is positioned incorrectly. Accordingly, the most important function of the SLIC field product is to alert the operator that there is, or may be, an error in the positioning of one or more sensors, and for the operator to check them. The newer models and the older semiautomatic models were compared on large sets of data from LTPP where there was a high degree of certainty that the sensor positions were correct, and also where it was apparent that a sensor positioning recording error had been made. Various criteria were employed to detect that a sensor positioning error existed. As one might guess, no procedure is flawless, and the two kinds of errors inevitably compete with one another. One type of error is a false alert. This error occurs when the data meet the criterion for detection of a sensor positioning error, but in fact the sensors are positioned correctly. The other type of error is failure to detect. This type of error occurs when the screen for a sensor positioning error fails to be triggered by data that are measured with a sensor positioning error, or errors. The second type of error should be regarded as more serious, since this error produces data that may result in false information. The first type of error is an inconvenience to the operator, and may require the operator to do an unnecessary measurement that confirms the FWD sensors are correctly positioned and correctly functioning. To minimize the occurrence of failure-to-detect errors, the most reliable procedures were based on the R2 values from the original SLIC procedure, that is, the procedure used in the visual graphics approach (see, for example, figures 3 and 4). This procedure fits a quadratic function to y = ln (-ln(normalized deflection)) versus x = ln(offset), and then computes the value of R2. (Normalized deflection in this case is not load-normalized, but rather deflection-normalized, meaning that each offset deflection is divided by the center sensor deflection, resulting in a normalized deflection basin with a maximum center sensor deflection of 1.0.) R2 measures the ratio of squared deviations of predicted values (of y) from the mean of y to the squared deviations of actual data from the mean of y. The LTPP data usually produce R2 values (for the transformed basins) that are remarkably close to 1, frequently above 0.998. However, there are occasional values not so close to 1 from FWDs with correctly recorded sensor positions. Values less than 0.990 suggest that there may be a sensor positioning error, and values less than 0.980 are strong evidence that a sensor positioning error exists. In view of these facts, the SLIC field product is designed to alert the operator that there may be a sensor positioning error whenever the value of R2 falls below 0.990, and that there is strong evidence of an error whenever this value falls below 0.980. The graph of the data can be seen by the operator and, with practice and familiarity, this person will learn to see the graph as an indication of which sensor may be out of position. The SLIC transform produces a graph with a nearly straight, somewhat concave, downward smooth line, with no kinks (changes in concavity). If the operator can measure the actual FWD sensor positions and find an error, the values of the sensor positions can be changed on the spot (in the field data file) and the change can be viewed in the plotted SLIC transformed data. On the other hand, if the sensors are in their correct positions, an operator may have found a false alert error unless there is a physical problem with a given sensor or sensor holder, for example. In this case, the operator should record that finding in the field file so that the person doing the analysis will be aware that the data has an anomaly, and that the sensor positions were checked and confirmed. If the situation continues to persist on several sections, a relative calibration should be done to confirm that the suspect sensor is functioning correctly. False alerts will presumably happen more often than the failure-to-detect errors, which will necessarily go uncorrected. Note that, the more often an operator runs the SLIC sensor position check, the greater the likelihood that a false alert will occur, but the less likely it is that the more serious failure-to-detect errors will occur. Larger sets of data tend to produce fewer errors of either type. Nevertheless, the operator should be encouraged to perform a SLIC check at the end of each section tested or day of testing, before moving off a site or before opening a new data file. Please note that files that contain concrete joint data, where the joints are positioned between two of the sensors, should not be run through the SLIC field product, since this kind of data will almost always result in a false alert error due to the influence of the joint on the measured deflections. False alerts are also likely on badly distressed pavements, or on pavements where bedrock or loose gravel is close to the surface. Single Section Sensor Positioning ErrorsAs opposed to the type of long-term sensor positioning errors described above, there were several instances of day files where sensor number 2 (d2) was placed different from that recorded by the FWD operator. These cases almost always involved joint testing on PCC pavements (i.e., J4 and J5 or C4 and C5 tests), or tests conducted along the same line as these joint tests (e.g., J6). Three categories of short-term sensor position designation errors were identified and recommended for correction in the database:
Screening techniques, along with a visual review of the screened data, made it quite easy (albeit time-consuming) to identify the vast majority of errors in the first two categories. The last category (with the missing minus sign) has already been changed through a global correction in the database. Therefore, this category was not reported in detail as were the two others. The first two categories of errors affected approximately 0.4 percent of the total volume of FWD data in the pre-autumn 1998 database. These errors were reported in Feedback Report RNS–3 in September 1999 (see appendix A). Reportedly, the vast majority (135 out of 140) of the identified sensor positioning errors identified in Feedback Report RNS–3 have already been corrected in the level E load-deflection tables in the database. In fact, many of these errors were also identified through other means, as part of an independent global check for incorrect sensor configurations. The remaining five identified anomalies are presently being checked for correction, if necessary, in the appropriate database sensor configuration data tables. Lane Designation ErrorsSince the lane designations (e.g., F3, C1, J4) are manually input, it is not surprising that a limited number of lane designation errors were identified in the database. Examples of these included entering C4 instead of J4 on a jointed PCC section, or switching J4 and J5 at one of the tested joints. There were only about 350 instances of this type of error in the entire pre-autumn 1998 database, or less than 0.1 percent of the data. These errors were also reported in Feedback Report RNS–6 dated September 1999 (see appendix A). Recording Date ErrorsIn five instances, the same LTPP section day file was reopened on a second date of testing when it was not possible to complete testing of a given section during one day. As a result, only the last date of test was recorded in the day file, and therefore only this last date of test (even though two days of test actually occurred) was transferred to the level E database. These errors evidently occurred only five times in the entire pre-autumn 1998 database, thus affecting slightly less than 0.1 percent of the data in the database. The reason this type of error occurred is a property of the Dynatest Edition 20 field program, which permits reopening of a data file (for example when an operator breaks for lunch and reopens a test section file afterward). However, when a given file is reopened on a different date, the first date is precluded (overwritten) by the second date, although the time stamps are still correct for both. In each case when there should have been two separate test dates, the data records showing the incorrect dates were recommended for change and correction in the database in Feedback Report RNS–6 (shown in appendix A). Further, in all but one of these five cases, the affected regional coordinators agreed to the recommended changes and carried them out in the load-deflection database, which has been updated accordingly. The one controversial case has since been reexamined, and the originally recommended date change in RNS–6 was verified and re-recommended for change in the database. This case concerned LTPP test section 10–0210 in the North Atlantic Region, where some of the date stamps on test date June 30, 1997, should read June 29, 1997. Drop Height Designation ErrorsIn the drop height data field, the numbers 1, 2, 3, or 4 should appear, corresponding to one of the four possible drop heights of the Dynatest FWD equipment. However, in several instances, an X was placed in this field in the level E database. These instances were reported to FHWA in early 1999, and all Xs were duly changed to 1, 2, 3, or 4 in the database, as appropriate. Before this error was corrected, some 0.3 percent of the FWD data in the database were affected. Other Potential Data Errors or AnomaliesThere were also a few other relatively minor types of errors or anomalies identified in the FWD load-deflection level E database during the data screening processes described above. For example, there were a handful of instances where the time of test was evidently edited in the regional office from what was recorded in the field, but only partially; the remaining data should also have been edited to make it consistent with the temperature measurements and/or with the sequence of other events in the field. When and if possible, all obvious errors noticed during the screening process were recommended for correction accordingly. In a limited number of cases, the possibility (or even the probability) of somewhat vague errors or anomalies existed. However, when these errors were not obvious or certain, no changes or flags were recommended in the database. Apart from the unbound material test data, approximately 1 percent of the FWD load-deflection database appeared to include data anomalies that could not be verified with certainty, because there was neither any supporting information nor a plausible explanation for the unusual data. These types of data anomalies are discussed in chapter 6 in a general manner; however, the notes or remarks that were created are presently not available to LTPP load-deflection database users. That information is associated with individual, comma-delimited data files, not the relational Microsoft Access® database utilized in the various versions of DataPave.
|