U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
SUMMARY REPORT
This summary report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-15-016    Date:  March 2015
Publication Number: FHWA-HRT-15-016
Date: March 2015

 

The Exploratory Advanced Research Program

Making Driving Simulators More Useful for Behavioral Research

Simulator Characteristics Comparison and Model-Based Transformation

PDF Version (2.42 MB)

PDF files can be viewed with the Acrobat® Reader®

Notice

This document is disseminated in the interest of information exchange under the sponsorship of the Department of Transportation. The United States Government assumes no liability for its contents or use thereof. This report does not constitute a standard, specification, or regulation.

The United States Government does not endorse products or manufacturers. Trade and manufacturers’ names appear in this report only because they are essential to the object of the document.

Quality Assurance Statement

The Federal Highway Administration (FHWA) provides high-quality information to serve government, industry, and the public in a manner that promotes public understanding. Standards and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its information. FHWA periodically reviews quality issues and adjusts its programs and processes to ensure continuous quality improvement.

 

Technical Report Documentation Page

1. Report No.

FHWA-HRT-15-016

2. Government Accession No. 3 Recipient's Catalog No.
4. Title and Subtitle

Making Driving Simulators More Useful for Behavioral Research— Simulator Characteristics Comparison and Model-Based Transformation Summary Report

5. Report Date

March 2015

6. Performing Organization Code
7. Author(s)

Brian Philips and Tom Morton

8. Performing Organization Report No.

 

9. Performing Organization Name and Address

National Advanced Driving Simulator
The University of Iowa
2401 Oakdale Blvd.
Iowa City, IA 52242

Woodward Communications, Inc.
1420 N Street, NW, Suite 102
Washington, DC 20005

10. Work Unit No. (TRAIS)

11. Contract or Grant No.

DTFH61-09-C-00003
DTFH61-09-F-00027

12. Sponsoring Agency Name and Address

Office of Safety Research and Development
and Office of Corporate Research, Technology, and Innovation Management
Federal Highway Administration
6300 Georgetown Pike
McLean, VA 22101-2296

13. Type of Report and Period Covered

Final Report, November 2008–June 2013

14. Sponsoring Agency Code

HRTM-30

15. Supplementary Notes

Notes FHWA Contracting Officer's Representatives (CORs): Brian Philips and Chris Monk
FHWA's Contracting Officer's Task Manager (COTM): Zachary Ellis, HRTM-30

16. Abstract

A central issue in making simulators useful for highway and traffic engineers concerns how well driver behavior in the simulator corresponds to driver behavior in the real world. Simulator fidelity plays a central role in matching behavior in the simulator to behavior on the road. Simulator fidelity often refers to the features and appearance of the simulator. The degree to which behavior in the simulator matches behavior on the road defines behavioral fidelity. This project characterized the physical fidelity and behavioral fidelity of four simulators. These four simulators represent a broad range of fidelity and cost. Data collected from these four simulators begin to address the question of how simulators can support highway and traffic engineers. Overall, the results show that simulators with high physical fidelity demonstrate high behavioral fidelity and me likely to provide good estimates of mean speeds in typical engineering applications such as roundabouts and roadway treatments designed to moderate drivers' speed. A detailed analysis of both physical fidelity and behavioral fidelity suggests the need to carefully assess the match between simulator features and the properties of the roadway design issue. A model-based transformation was developed to relate data collected in the simulators to data collected on the road. Future research should examine physical fidelity in more detail and its relationship to behavioral fidelity across a broader range of driving behavior parameters.

17. Key Words

Driving simulator, physical fidelity, behavioral fidelity, roadway design, roundabout, simulator fidelity, simulators.

18. Distribution Statement

No restrictions. This document is available through the National Technical Information Service, Springfield, VA 22161.

19. Security Classification
(of this report)

Unclassified

20. Security Classification
(of this page)

Unclassified

21. No. of Pages

36

22. Price
Form DOT F 1700.7 Reproduction of completed page authorized

SI* (Modern Metric) Conversion Factors

 

Executive Summary

Highway and traffic engineers face considerable challenges in creating designs that are consistent with drivers’ capabilities and expectations; however, failing to consider driver behavior can cost lives and millions of dollars if roadways require revision after they are built. The use of driving simulators to guide designs or to evaluate design choices is a promising approach, but discrepant results across studies undermine the utility of these findings. This is particularly true when simulator results fail to match on-road data. One potential source of this mismatch is when the simulator does not have the appropriate fidelity, or realism, to address the design issue of interest. Appropriate simulator fidelity, which includes the simulator hardware and software as well as the modeling of the virtual environment, is an important component of obtaining data useful for highway design. For example, one could envision a staged approach to simulator fidelity, similar to that used in software prototyping, in which a low-fidelity desktop simulator could be used for rendering scene and roadway elements, whereas a high-fidelity simulator could be used for speed estimates. Choosing the appropriate level of simulator fidelity to address a particular design issue represents a critical challenge.

The aim of this project was to address this challenge and to help engineers identify the appropriate simulator platform for particular design questions, as well as to identify a mathematical transformation that can equate simulator data to real-world outcomes. In particular, the research team identified highway design needs and matched them to specific simulator characteristics to facilitate the appropriate choice of simulator for a particular design problem. As part of this research, the research team developed and demonstrated a proof of concept approach to characterizing simulator fidelity to allow for comparison between simulators and the real world. The research team also developed a driving environment that contained virtual recreations of two roundabouts from Maryland and Arizona, as well as a gateway from a rural road to a small town in Iowa. The researchers manipulated this virtual environment to vary the visual complexity of the driving environment and tested it on four simulator platforms, three of which were tested with and without motion. They compared driver judgment of fidelity and performance across the simulator platforms. No consistent effect of motion was found, but a moderate effect of visual complexity was apparent in the data. Performance data showed good relative and absolute matches to on-road speed data. The researchers used the data to develop linear regression and process models that could be used to transform the simulator data to match the on-road data. These models will provide the foundation for future work that will allow designers to transform results for simulator studies to make design decisions and to predict changes in driver behavior and performance on the basis of evaluations conducted on simulators. For example, these models can relate speed through a roundabout observed in a simulator to speed that is likely to be observed on the road.

Following completion of this project, additional work is necessary to improve and refine the tools developed so far. One area that requires refinement is the characterization of simulators. This is because those characteristics that matter most are not always the easiest to measure. Additional work is needed to define the critical measures that differentiate simulator fidelity related to roadway design. Additional work is also needed to characterize what constitutes a typical vehicle and how much variability exists among vehicles on critical measures. These data could then be used to enhance the psychophysical scaling required to determine when a simulator is noticeably different from a typical vehicle and the extent to which different vehicle types influence highway design decisions. These differences must also be investigated to determine whether future studies need to include not only a range of drivers, but also a range of vehicle types.

This research would also be enhanced through its application to real-world design problems to provide the opportunity for continued evaluation and refinement. For example, use of a simulator to support a State Department of Transportation project, from inception to evaluation, would enable a thorough evaluation of the utility of the simulator in all phases of the design process. Through final evaluation of the real-world design implementation, the predictions of the simulator across a broader range of performance metrics could be assessed, and model refinements could be made.

Another promising line of research would be to draw on naturalistic data to identify critical design issues and scenarios that can be further examined through simulator studies. These studies would provide additional data to improve the transformations of simulator to real-world data. A further opportunity would be to examine the minimum fidelity of simulator needed at each phase of the design process and across design problems. If lower fidelity simulators can be used to successfully address design decisions, then their use may be opened up to a broader group of highway designers who cannot necessarily afford more expensive simulation platforms.

The model-based transformations used in this study highlight the promise of driver modeling in helping to address highway design decisions. Ongoing projects continue to explore the use of driver models to enhance driver safety through a systematic evaluation of design options; however, this requires a reliable and validated model of the driver. Additional work along these lines is therefore needed, particularly as it relates to roadway geometry and visual complexity. These theory-based models can be used to accumulate an understanding of simulators and driver behavior related to a set of stimuli. A comprehensive approach that integrates a driver model with the Interactive Highway Safety Design Model would provide benefits to highway designers as an efficient way of using previous data to assess new design decisions.

 

Table of Contents

INTRODUCTION

PHYSICAL FIDELITY

BEHAVIORAL FIDELITY

MODEL-BASED TRANSFORMATION OF SIMULATOR DATA

CONCLUSIONS AND RECOMMENDATIONS

ADDITIONAL INFORMATION

REFERENCES

 

List of Figures

 

List of Acronyms and Abbreviations

EAR Exploratory Advanced Research  
FHWA Federal Highway Administration  
NADS National Advanced Driving Simulator  
WTI Western Transportation Institute  

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101