Traffic Data Edit Procedures Pooled Fund Study

Traffic Data Quality "TDQ"

FINAL REPORT

By Mark Flinner, Minnesota Department of Transportation and Henry Horsey, Intelligent Decision Technologies, Inc.

For Transportation Pooled-Fund Study SPR-2 (182)

Excerpt from the full report.

Introduction/Executive Summary

Background and Problem Statement

Challenges facing staff from state and local agencies who are responsible for providing traffic data for safety analysis and highway design include:

• The increasing complexity and number of traffic data collection sites make it more difficult to perform quality control using older, manual methods.
• Traffic data from highly maintained traffic data collection devices should always be of good quality. However, these traffic data may be atypical due to events or diversions and may be discarded by simple, "outlier" based edit programs.
• Many things can go wrong with traffic data collection devices, depending upon what they are designed to do. Many if not all, equipment malfunctions will affect the integrity of the resulting traffic data.
• The use of simple minimum/maximum bounds and simple average values leaves too many instances of poor data undetected.
• Many recent efforts to create traffic data screening systems need to be identified, understood then logically merged in order to develop a more comprehensive understanding of the "state-of-the-art".
• Traffic data screening systems should preserve the role of the traffic data analyst. The analyst is the expert, i.e. the one who seeds the system with site and time specific comparison values, and performs follow up analyses or field studies before making a final judgment as to the acceptability of the data.

Introduction to the Project

The Traffic Data Edit Procedures (TDEP) pooled fund study SPR-2 (182) began in 1995 when fourteen states and the Federal Highway Administration (FHWA) expressed the desire to work together to:

1. Learn from other experts in the field who are in charge of the operation and data quality control of traffic data collection equipment
2. Document and qualify existing and potential traffic data screening methods
3. Refine the list of traffic data screening methods into a logically integrated "rule base"
4. Test the rule base in the context of actual data and expert analysis
5. Develop software to assist in the evaluation of the rule base and to put revised software into production

Due in part to Minnesota's earlier experience assisting in the development of the Traffic Data Analysis Expert System and Editor, an "expert system" designed to work with continuously collected volume data from automatic traffic recorders (ATRs), Minnesota was chosen to lead this study. The participating states are Minnesota, Wisconsin, South Dakota, Indiana, New York, Connecticut, North Carolina, South Carolina, Georgia, Florida, New Mexico, California, Idaho, and Montana. Arizona and other states attended various meetings as they could.

In the subsequent four years traffic data analysis staff from the participating states and representatives from the FHWA met to:

• Define the scope of the study,
• Hire a consulting team to assist with the study (Intelligent Decision Technologies Inc. and In Motion Inc.)
• Determine how to identify acceptable quality data, and
• Develop a logical structure for evaluating traffic data in the context of already existing state systems and traffic data screening expertise.

Accomplishments and Key Results

Major accomplishments of the study include the following:

Publication of Candidate Traffic Edit Topics, Task A2 Report, that included the following work:

• The identification of primary traffic volume, vehicle classification, and weigh-in-motion (WIM) data products that would be affected by improved data screening tools.
• A definition of acceptable data quality and a method for identifying acceptable data to support selected data products.
• A compilation of all data screening tools used by one or more of the participating states as they are applied to short or continuous volume, vehicle classification and/or WIM data for the selected data products.

Publication of Expert Knowledge Base, Task A3 Report that included the following work:

• The creation of a set of logically consistent, state-of-the-practice rules for traffic data screening derived from five, multiple day knowledge engineering sessions attended by over sixty traffic data screening experts. Some rules from the Task A2 document were not passed into the next phase due to an analysis of their unique contributions to the overall screening strategy.
• The development of traffic data screening algorithms, definitions and pseudo-code statements to support the development of rule base testing software.
• The creation of a plan to measure rule base effectiveness on the basis of intense analyst data review in conjunction with an analysis of software generated false negatives and false positives using raw data and analyst verified model data.

With the publication of the Task A3 report, the first three objectives of the study were fully met. A prototype version of software intended to test the effectiveness of the coded portions of the rule base was developed called "TDQ" (Traffic Data Quality). The resulting prototype is described in this report and was successfully run in September 1999 on a Minnesota Department of Transportation data set.

However, the lack of funding for contractor support and other state management priorities resulted in the TDQ prototype not being tested again until the summer of 2000. At that point in time, the prototype was unable to complete an analysis. The lack of funds precluded troubleshooting and resolving the problems encountered in the summer of 2000.

This study was unable to meet its last two objectives. While the rule base developed in this effort represented the consensus expertise of traffic analysis experts from around the country, it has not been validated by rigorously testing against actual data from a spectrum of traffic monitoring sites. Prior experience with knowledge-based systems suggests that such testing would have resulted in further modification of the rule base and a better understanding of the types of traffic monitoring sites where the rule base would be most applicable. Thus the fourth objective of the study was not met.

The fifth objective of the study, to put the TDQ software into production was likewise not met. This objective could only be met after extensive testing and modification of the rule base, as well as subjecting the system to a variety of other testing including user interface testing, data integrity testing, stress testing, and system integration.

This study has elevated the standard for traffic data analysis and has helped to increase the consistency of data analysis methods among the participating states. Such improvements in data analysis will result in a more accurate understanding of roadway usage and traffic trends to support investments in roadway construction and maintenance. We invite others to explore ways to implement what has been learned in this study and to develop a more complete understanding of how rule bases can assist the traffic monitoring community in providing improved data to their clients.

It is appropriate to thank several people for contributions to this project that might otherwise go unnoticed. Mark Flinner of the Minnesota DOT and the Project Technical Liaison, deserves a great deal of credit for the existence of this software. Mark's patient advice and endless willingness to review our work has been essential. Joe Racosky of In Motion, Inc., did a great job of documenting much of the knowledge that is built into TDQ. Much of the technical design and virtually all of the development of the programs are the work of Tom Conlon and Simon Damberger, members of the IDT development team. A special thanks is also due to the support staff at IDT, particularly Barb Johnson and Virginia Lilly. Finally, Fred Newcomer, the TDQ Development Manager at IDT deserves considerable credit for shepherding the project through its trials and tribulations to the release of 0.6.

Traffic Data Edit Procedures - TDQ prototype software Rule List