U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

Report
This report is an archived publication and may contain dated technical, contact, and link information
Publication Number: FHWA-HRT-08-065
Date: November 2008

Long-Term Pavement Performance Compliance With Department of Transportation Information Dissemination Quality Guidelines

CHAPTER 5. EVALUATING INFORMATION QUALITY

The methods used by LTPP to evaluate and improve its data are the development of QC systems during data collection and processing, regular assessments of data quality, special studies of key aspects of the data, and data collection processes.

Data Quality Assessments

LTPP has developed two approaches to quality assessments. The first approach was a major program assessment conducted in 1996-1998. The second approach was the development of a quality control/quality assurance system based on the ISO 9001 quality management principles.

The objective of the 1996 assessment was to evaluate the program goals, objectives, and future direction. This was completed by evaluation of the impacts on deviations from the program's plans, number of test sections, data collection deficiencies, and resources. The ultimate objective of the assessment was to develop a revised strategic plan that focused on high-payoff product objectives that met States' needs, improved program efficiency, and provided better quality data for product development. Some of the aspects of this assessment related to the Federal data quality guidelines, including the following:

With the letting of the four regional LTPP data collection contracts in 2001, FHWA required documentation for the preparation of formal data collection and a processing QC system. After central review by a contractor with ISO 9001 certification, these QC documents were transformed into quality management documents based on ISO 9001 principles. Some of the relevant features of this management process, as related to this portion of the Federal data quality guidelines, include the following:

Quality assurance audits are performed on highway agency operated FWD reference calibration facilities used by LTPP. These facilities were developed under cooperative agreements with select highway agencies. The facilities use LTPP-provided equipment and follow LTPP test protocols. Annual audits are performed on conformance to test protocols on the operators of these facilities. Audit results are documented and certificates of compliance are issued.

Evaluation Studies

LTPP has used evaluation studies to analyze data quality issues that can not be identified by mere inspection of the data. Evaluation studies have also been performed in the development, refinement, or implementation of new or advanced data collection systems. The following are some of the evaluation studies conducted by the LTPP program:

LTPP has established the basis for future evaluation studies that may not be able to be conducted with current funds. For important data elements based on measurements from instrumentation, when possible, equipment calibration factors are stored in the database. This permits evaluations of the effect of changes in the calibration factors over time.

Quality Control Systems

LTPP has invested significant resources in developing data QC systems to address the variety of data sources and measurement technologies employed in the program. The goal of these QC systems is to provide a preventive system of error identification to avoid data errors before entry of data into the database. Within the LTPP program, data quality control is defined as the processes and procedures used to inspect data and data collection equipment prior to entry of data into the database.

The major categories of QC systems developed by LTPP include equipment calibration procedures, equipment calibration checks, operator training and certification, post data collection reviews, data screening from external program sources, and formal quality control management procedures.

Equipment Calibration

In the development of the data collection plan, it was determined that the LTPP program needed to own and operate specialized data collection equipment judged by panels of experts to be critical to its success. While the LTPP program tried to rely on existing technology, in some cases, it had to develop its own measurement technology. In some cases, LTPP had to develop calibration procedures for its specialized equipment and, in other cases, used exiting procedures. Some highlights of LTPP equipment calibration procedures include the following:

In those cases where it is not possible to directly calibrate a device, equipment calibration checks are used to ensure proper function. For example, measurements by temperature sensors are conducted on items of known temperature such as ice and boiling water, and if found to be outside an established range, are either returned to the manufacturer for adjustment or replacement.

It is LTPP policy to identify data collection equipment operators or data collectors in the database to allow evaluation of operator biases and errors.

Data Collection Operator Training and Certification

Due to the complexity and subjectivity of many of the LTPP data collection functions, LTPP has established formal data collection training classes and certification evaluations.

LTPP requires that collection and interpretation of pavement distress data be performed by someone who has an active certification from a LTPP distress rater accreditation workshop. Raters must meet minimum experience and time-based recertification requirements in order to maintain their certification.

The LTPP regional data collection contractors are required to train and certify operators of equipment used to collect LTPP data as a part of their formal data QC management plan. Regional equipment operator training is documented, as required. Evaluation of the performance of new personnel by the regional data collection management staff is also documented, as required.

To promote consistency among regional data collection contractors, national meetings of regional data collection operators have been conducted on each major data collection topic. When the program began, annual meetings of data collectors were held. Due to program budget cuts, national meetings were scheduled on a priority basis, and the use of teleconferences was increased.

When new data collection technologies are being implemented, the LTPP program has used the following process:

Screening of Data from External Program Sources

The data collection plan relies upon multiple data sources. These data are screened prior to entry into the database. For data submitted on paper forms, the first level of screening is for completeness and logic checks on the provided information. Like all other data, these data are also screened after entry into the database using automated methods.

A large amount of data is received in electronic format. Two of the largest modules of data from other agencies are traffic monitoring data and climate data. The following screen method demonstrates the steps used by LTPP on these data:

Data Error Correction

The LTPP program has established standard and formal methods for data error correction. It has been LTPP's policy to not load known "bad" data into the database. Although steps are taken to prevent entry of erroneous data, there have been numerous instances when data in the database were found to contain errors, or, in some cases, unknown facts have caused previously entered data to be invalid.

When data errors are found, the standard mechanism is to correct the error if possible or remove the data from the database if the error cannot be corrected.

Error correction procedures are contained in formal documents issued by directive by data type. An example of an error correction policy is contained in the following excerpts from LTPP Directive I-85 on Manual Upgrades to QC Checks. This discussion describes the steps to be taken when data fail an automated check.

When a record does not pass a QC check, the first action that should be taken is to determine the cause, examine the data in the record or other related records, and try to rectify the situation if possible. Some types of possible errors that can be corrected include the following:

The above policy and guidelines apply to data after entry into the LTPP database. Error correction guidelines are also contained in the data collection and processing documents for FWD measurements, profile measurements, seasonal monitoring measurements, AWS measurements, and traffic data.

 

< PreviousContentsNext >>

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101