U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590

Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-11-056    Date:  October 2012
Publication Number: FHWA-HRT-11-056
Date: October 2012


Layered Object Recognition System for Pedestrian Sensing


The research team developed a real-time in-vehicle vision-based stereo system that detects and recognizes pedestrians in the camera’s field of view. The system uses a layered or hierarchical approach that progressively operates on all or part of the input image data, with each step increasing in computational complexity and reducing the image area that needs to be processed by subsequent steps. The system integrates multiple cues including depth, appearance, and motion. The key steps are as follows:

  1. Large-scale object extraction using stereo depth templates.

  2. SC recognition of multiple classes including ground, buildings, trees, and poles and separation of those classes from vehicles and pedestrians.

  3. Appearance classification using a cascade of classifiers that explicitly recognizes pedestrians and discriminates against other objects such as vehicles and bushes.

  4. Pedestrian tracking using shape and appearance matching.

Based on offline and live experiments using a Toyota® Highlander with a stereo camera head and a personal computer processing unit, the following conclusions were made:

If successful, the following recommendations for future work could lead to a commercially viable system:

Implement the developed system on an embedded platform. Potential candidates include the Acadia II™ application-specific integrated circuit in combination with a field-programmable gate array (FPGA) or the automotive-grade multiple digital signal processor and FPGA system jointly developed by Autoliv Electronics and Sarnoff Corporation.


Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101