The Exploratory Advanced Research Program

Making Driving Simulators More Useful for Behavioral Research

Simulator Characteristics Comparison and Model-Based Transformation

CONCLUSIONS AND RECOMMENDATIONS

As described in the previous section, behaviors seen in the driving simulator and on the roadway in this research project are generally in agreement, but there are still some mismatches to be addressed. For example, the distribution of speeds observed in the simulator should ideally match speeds observed on the road, but this is not always the case. Simulator characteristics can explain some of the behavior differences, but there are other important characteristics that can lead to differences, including familiarity with the route and individual driver motivations.

A central challenge in making simulators useful for roadway design concerns how well driver behavior in the simulator matches driver behavior on the road. The results of this study begin to address the question of how simulators can support highway and traffic engineers. Overall, the results show that simulators with high physical fidelity demonstrate high behavioral fidelity and are likely to provide good estimates of mean speeds in typical engineering applications, such as roundabouts and roadway treatments designed to moderate drivers’ speed. The use of the data from simulator studies can also be further refined through the use of the transformations developed as part of this research. The detailed analysis of both physical and behavioral fidelity included in this study suggests important opportunities to improve simulator fidelity and the need to carefully assess the match between simulator features and the properties of the roadway design issue.

In general, the NADS and WTI simulators showed the highest level of fidelity across the range of metrics examined; however, results also showed that no single metric can serve as a proxy for overall simulator fidelity. This illustrates how simulators can differ across different dimensions that affect the level of fidelity. The broad concept of an overall level of fidelity is also misleading and should instead be addressed in a multidimensional manner. In addition, there is a need, when considering fidelity, to consider the type of vehicle the simulator is designed to reproduce and the type of measure that is relevant in a given scenario. For example, the effect of using a motion base was minimal in the scenarios used in this study because there were few occasions of strong lateral or longitudinal g-forces. Indeed, the effect of simulator platform (e.g., miniSim versus NADS) was often less influential than was the effect of the details included in creating the virtual roadway (e.g., visual complexity).

The results of this research project confirm the importance of understanding how different dimensions of physical fidelity work together to provide overall fidelity. As noted earlier in this summary, even without perfect fidelity, drivers adapt and make use of the cues available to respond to the changes in the driving environment. Fewer cues for speed estimation may lead simulator drivers to attend to the speedometer more than they would on the road. Some of these differences in physical fidelity can degrade driver response to the point that behavioral fidelity is compromised, whereas in other cases drivers adapt compensating behaviors that allow for realistic responses but may not fully reflect how the driver would respond in the real world. Attending to the speedometer to maintain the instructed speed may distort how drivers respond to other elements of the roadway. These cases therefore require care when interpreting the results.

The interaction between physical fidelity levels and resulting behavioral fidelity also needs to be considered. First and foremost, the driver experiences the simulator software through the visual display, motion system, sound system, steering torques, and pedal forces. If the steering forces are not produced by the vehicle dynamics model, then the vehicle dynamics that the driver experiences will differ substantially from the true one. In a similar vein, if the simulator does not present vestibular cues, then much of the vehicle dynamics (accelerations) will not be perceived directly by the fast vestibular system but instead through slow visually perceived speed changes; therefore, drivers will perceive dynamics as much more sluggish than what the software might portray.

This research effort provides a valuable contribution to the understanding of the use of simulators for evaluating roadway designs. Prior efforts have focused on addressing research design projects on single platforms in a fixed configuration and have failed to address the discrepancies among outcomes on different platforms. This project directly addresses those issues to provide guidance to the research community and highway designers. The research shows how the simulator configuration affects speed in the simulator relative to the real worldreal world. In addition, it shows that model-based transformations can be used to estimate speed adjustments based on the simulator configurations for the platforms tested and can potentially be extrapolated to other configurations. The results show that using a high-fidelity simulator, such as the NADS, FHWA, or WTI simulator, with attention to accurately rendering the visual complexity of the roadway, will lead drivers in the simulator to drive at speeds quite comparable to those observed on actual roadways.

Overall, the research team for this project developed a set of tools that provide the foundation for future work that allows designers to transform results for simulator studies to make design decisions and to predict changes in driver behavior and performance based on evaluations conducted on simulators. This project is an important first step in understanding not only the translation of simulator data to real-world contexts, but also the hidden and complex issues that underlie this type of study, comparing multiple simulators with each other and with a real-world data set. Some of the key contributions of this project include the following:

• Proposed set of metrics and methods to characterize the physical fidelity of simulators.
  
  
• Proposed set of analytic methods and graphics to characterize the behavioral fidelity of simulators.
  
  
• Identification of simulator characteristics that may be most relevant to measuring speed in road design (e.g., visual complexity, field of view, and motion base).
  
  
• Linear model specific to each simulator to predict mean speed and speed variance for real-world context (for the scenarios selected).
  
  
• Driver model specific to each simulator to transform curve negation speed in the simulator to the real-world curve context.

Future Research

Understanding Real-World Values

The metrics and methods used in this study to characterize physical fidelity of simulators highlight a need to consider differences among actual vehicles in assessing simulator fidelity. When considering differences between real-world cues and those provided by the simulator, a more extensive survey of typical values of vehicle characteristics should be conducted, as it was revealed that parameters can vary from vehicle to vehicle in the same class. Understanding the range of possible real-world values will enable more accurate descriptions of simulator vehicles relative to their real-world counterparts than is possible when a single vehicle is chosen to represent a class of vehicles. Normalizing for these differences in variation could greatly improve the mapping from simulator characteristics to physical and behavioral fidelity.

Tuning Vehicle Characteristics

Looking to the future, a specific research direction to address vehicle characteristics could investigate methods to tune simulators with respect to a standard vehicle or a generic compiled vehicle (e.g., based on the average of a set of typical vehicles) or develop methods to quickly adjust tuning parameters to the type of vehicle (and expected “feel”) of a given driver. In addition, the approach used in this report could be expanded to include a wider range of typical vehicles, rather than a single example.

Considering Virtual Fidelity

When describing overall simulator fidelity, weighting the factors that can be easily described provides insight into how simulators compare to each other and the real world; however, this does not provide a comprehensive understanding, because not all features that matter are easily measured and not all features contribute equally. When simulators are considered relative to a specific type of evaluation, weighting the factors that contribute based on their influence on the behavior of interest is a better approach to understanding the relative fidelity to address research questions. This would mean that a simulator might be a “high” fidelity simulator for accessing speed through a roadway design but a “low” fidelity simulator for another design problem. The results of this project point to the importance of considering the fidelity of the virtual environment in this assessment—visual complexity had a larger effect than the motion base. Along these lines, the presence of traffic might have a surprisingly strong effect on driver behavior.

Quantifying Simulator Characteristics

Further research is needed to quantify the variation of simulator characteristics, the degree to which drivers can adapt to different vehicle simulator characteristics, and the degree to which these characteristics influence behavior as well as driving strategies and operator workload. In addition, a larger number of simulator configurations should be compared to make the mapping from simulator characteristics to behavior tractable. To achieve this, researchers will need to develop focused studies that explore which simulator characteristics humans can adapt to and which distort behavior. Based on the results of this study, systematic variation of simulator features, such as visual complexity, as well as sound and vibration might be particularly fruitful.

Collecting Additional Real-World Data

To further understand the comparison between the simulator and the real world, additional real-world data is required to provide a more continuous and complete description of driver behavior. This project was limited to speed observed on the road at widely spaced points, and the current on-road data are sparse with no lane position data. Continuous speed data along with accelerator pedal modulation and lane-position data would provide a much richer basis for comparing behavior in the simulator to that observed on the road. Collecting instrumented vehicle data in 3 segments by using 15 subjects per condition so that a model can be developed would provide a strong foundation for continuing this research.

Examining Naturalistic Data

Naturalistic data provide another promising avenue for future research. Naturalistic data associated with crash and near-crash situations observed on the road could be replicated in the simulator where a more detailed assessment of driver behavior and potential countermeasures would be possible. This would provide a more comprehensive basis for using driving simulators to enhance traffic flow and also improve road safety. Rather than a focus on replicating speed observed on the road, the focus could be on replicating in the simulator the behavior that precipitates crashes.