U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
FACT SHEET |
This fact sheet is an archived publication and may contain dated technical, contact, and link information |
|
Publication Number: FHWA-HRT-15-070 Date: August 2015 |
Publication Number: FHWA-HRT-15-070 Date: August 2015 |
The Exploratory Advanced Research Program Fact SheetPDF Version (363 KB)
PDF files can be viewed with the Acrobat® Reader® Exploratory Advanced Research . . . Next Generation Transportation Solutions |
The second Strategic Highway Research Program (SHRP 2) Safety research area has produced unprecedented data for highway researchers in the form of the naturalistic driving study (NDS). With approximately 2 petabytes of data available to researchers, most of which is video data, new automated tools for data extraction and analysis are required. Developing a prototype tool was the goal of the Exploratory Advanced Research (EAR) Program project, “Machine Learning for Automated Analysis of Large Volumes of Highway Video.” The National Robotics Engineering Center, at Carnegie Mellon University (CMU), conducted this research, which was funded by the Federal Highway Administration (FHWA) in 2012. |
A typical challenging situation for identifying discreet
objects in a cluttered urban environment.
The SHRP 2 NDS involved almost 3,000 volunteers going about their regular activities with extensive instrumentation, including video and still cameras, a Global Positioning System, and many other sensors attached to their personal vehicles for up to 2 years. The NDS data will provide unparalleled insight into the connections between driver behaviors, road design, traffic, and many other topics that have previously been difficult, if not impossible, to directly investigate, or were limited in scale by the intimidating logistics of collecting and analyzing this much data. The major challenge is decoding the video data. For example, one researcher might be interested in how drivers react to particular elements of the overall roadway environment whereas another researcher might be focused on the video from the camera pointed directly at the driver’s face and upper torso.
The traditional way to extract features of interest from video is for technicians to physically sit in front of a monitor and manually scroll through frame by frame, logging the location where each feature is found. With approximately 1.2-million hours of video in the NDS dataset, it is estimated that it would take almost 600 technicians a full year, working 40 hours per week, to go through all the video. Automation of these feature extraction tasks would dramatically reduce the time and cost involved.
In accordance, the EAR Program is funding a total of six projects, including the CMU research project, that are directly or indirectly focused on making the SHRP 2 NDS data more accessible. The objective of these research projects is to advance efficient and cost-effective methods and tools to analyze large amounts of safety-related video data. Such tools will reduce the time and costs required to pull needed features from these rich datasets, and thus expand the pool of researchers able and willing to use such data in their research. The CMU research project aims to address the challenge by automating the feature extraction so that the details that researchers need can be quickly and accurately extracted.
The research team for this CMU study implemented a machine-learning–based approach. This approach feeds a multitude of features into learning algorithms that are trained on very large datasets and capable of learning with high accuracy exactly which features are important, which provide less useful data, and how features can be used in combination to accurately recognize and classify the desired targets. This means that the great diversity provided by large and complex datasets adds to the power and robust performance of the system.
To extract useful information from petabytes of data generated by moving or static video cameras, the research team integrated state-of-the-art algorithms to exploit contextual clues, reduce the amount of expert effort required to guide learning, and manage the computational effort required. When humans interpret sensor data, they rely on contextual cues, such as inferring from a patch of image showing a car that it is likely to lie above an image of a roadway. Roads and vehicles offer many other similar contextual cues that can be exploited within a scene. Researchers traditionally encode these contextual relationships in a graphical model; however, predictions may be inaccurate in these models. This study focuses on a novel set of methods to automate the interpretation of ambiguous video data by building a sequence of context dependent predictions. Instead of a traditional graphical model, this approach defines an inference procedure (e.g., wheels are found near the corners of a car), which can then be used to classify features moving forward.
The research team implemented powerful labeling tools to enable fast human analysis and labeling of data, anomaly detection to prioritize data access and focus labeler time on significant events, and semi-supervised learning to build predictive models and to minimize the amount of labeled data required. In addition, with datasets running into petabyte sizes, there is a critical need for computational systems that can be scaled to match. The researchers considered two different approaches for managing such computation- and data-intensive machine learning. One approach is parallelization, involving the distribution of work across multiple cores in a network. The other approach is a new class of techniques, known as anytime predictors, used to provide a sequence of results that improves as more time is provided to analyze test data.
FHWA’s Exploratory Advanced Research (EAR) Program focuses on longer term, higher risk research with a high payoff potential. The program addresses underlying gaps faced by applied highway research programs, anticipates emerging issues with national implications, and reflects broad transportation industry goals and objectives.
To learn more about the EAR Program, visit the Exploratory Advanced Research Web site at www.fhwa.dot.gov/advancedresearch. The site features information on research solicitations, updates on ongoing research, links to published materials, summaries of past EAR Program events, and details on upcoming events.
The primary outcome of this project is a software framework to support efficient application of machine-learning-based feature extraction techniques for very large sets of transportation video data, as well as specific target detectors to support various transportation analysis tasks. An additional benefit will be to expand industry understanding of a highly effective data-processing approach that can be widely used to process large volumes of future data. The goal is to be able to apply this approach to both moving targets (e.g., cars and pedestrians) and static targets (e.g., traffic signs).
For more information on this project, contact Lincoln Cobb, FHWA Office of Safety Research and Development, at 202-493-3313 (email: lincoln.cobb@dot.gov).
Publication No.
FHWA-HRT-15-070
HRTM-30/08-15(WEB)E