U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
Public Roads
This magazine is an archived publication and may contain dated technical, contact, and link information.
Public Roads Home | Current Issue | Past Issues | Subscriptions | Article Reprints | Guidelines for Authors: Public Roads Magazine | Sign Up for E-Version of Public Roads | Search Public Roads
Back to Publication List        
Publication Number:  FHWA-HRT-15-003    Date:  March/April 2015
Publication Number: FHWA-HRT-15-003
Issue No: Vol. 78 No. 5
Date: March/April 2015

 

The Science and Art of Putting Drivers Under The Microscope

by Alicia Romo and C. Y. David Yang

Refining the questions, improving the collection of data, and evaluating the methods used to study the behavior of road users can help reduce driving errors and improve safety.

Cameras on this vehicle’s dashboard track a study participant’s eye movements while she drives on an instrumented roadway. Identifying sources of driving distraction through eye fixation was the purpose of this experiment.
Cameras on this vehicle’s dashboard track a study participant’s eye movements while she drives on an instrumented roadway. Identifying sources of driving distraction through eye fixation was the purpose of this experiment.

According to statistics from 2010 reported by the National Highwa y Traffic Safety Administration, the $871 billion economic loss and societal harm caused by vehicle crashes cost the average U.S. citizen $900 per year. Studies into the causation of crashes have concluded that approximately 90 percent of incidents can be attributed to driver error.

These statistics underscore the need for research to understand driver limitations, with the goal of increasing roadway safety and reducing the cost of crashes. Associate Administrator Michael F. Trentacoste, head of the Office of Research, Development, and Technology at the Federal Highway Administration, says, “Understanding the behavior of road users is the key to designing effective countermeasures to improve transportation safety.”

Significant research findings on driver behavior require selecting an appropriate research method to begin with and then ensuring credible interpretations of the results. To select a method and thus to provide solutions to problems and questions, researchers need to understand the types of data gathered under different methods. The usefulness of each method to performing analyses and obtaining valid findings has an impact on the quality of the research.

The question is: How should a researcher select a method to study a given driver behavior? More important, how do researchers determine whether their research approach is appropriate?

Researchers need to improve the selection of research tools to conduct studies, refine questions, collect relevant data, and find effective solutions to reduce driver errors and thus improve transportation safety. They need to examine the various methods to select those that can answer the research questions.

FHWA’s Human Factors Team is working on identifying factors to evaluate a number of research methods and assist researchers in understanding the types of data needed for a given study. The following assessment of methods is essential to explaining the similarities and differences among the various approaches and communicating the value of research across disciplines.

Research Methods Used to Study Driver Behavior

Safety researchers use various methods to generate and test new ideas, develop causal explanations of crashes, and understand the complex behavior and interactions of road users. The FHWA study explored the following six major methods that researchers use to collect data and investigate questions linked to driver behavior and transportation safety: surveys, crash data, driving simulators, test tracks or instrumented roadways, field operational tests, and naturalistic driving studies.

Surveys. A survey is a nonexperimental research method that is used in transportation research. The survey method involves using either self-reporting questionnaires or interviews with participants to collect information on their experiences. Researchers use surveys to understand driving attitudes and safety awareness, investigate travel behavior, rate driver acceptance of countermeasures and mitigations, and understand expectations for new technologies and preferences regarding them. Using predetermined questions enables researchers to understand and assess the reactions of road users to countermeasures of interest, such as a new traffic calming strategy or a new tolling policy. In addition, driving behavior questionnaires can be used to predict crash avoidance and crashes based on self-reported errors and violations.

For research about driving behavior at intersections with red-light cameras, this study participant is filling out a hardcopy survey.
For research about driving behavior at intersections with red-light cameras, this study participant is filling out a hardcopy survey.

Crash databases. Crash databases are not considered experiments, but because of their significance to performing safety analysis, using crash data is considered a research method and therefore was reviewed in this study. Researchers collect most crash data by using documentation from police accident reports. The crash data from these reports provide statistics on high-crash locations and the frequency and severity of crashes. Researchers use crash data to investigate and predict when and where crashes will occur and identify roadway designs and road users vulnerable to crashes.

Driving simulators. By setting up studies in virtual driving environments, researchers can gather various kinds of behavioral data. They can use driving simulators to generate crash scenarios; investigate driver impairments; and test the limitations of vehicle designs, roadway configurations, and invehicle technologies. They can measure driving behavior and performance by controlling various factors in a study’s experimental design.

Test tracks or instrumented roadways. A test track or instrumented roadway enables experimental research to be performed on closed or open roads. Participants complete driving tasks in a research vehicle while researchers record their behaviors and interactions. Researchers can use test tracks to analyze dangerous and high-speed scenarios, last-second braking and steering performance, rear-end collisions, and alcohol impairment. Experiments on an open road permit interaction with other road users in real and controlled traffic settings.

A female researcher is performing data analysis on a computer. Line graphs are shown on the computer monitor in front of her.
This researcher is performing data analysis to compare historical crash trends.

Field operational tests. Often, these tests help researchers measure the effects of new technologies in the field while participants are being monitored as they drive on streets and highways. This kind of experiment uses a fleet of vehicles equipped with various systems and devices. Researchers view video and driving recordings to observe how participants adapt to technologies over a period of several weeks to a few months. In technology-based studies, researchers compare scenarios to analyze behavior changes, side effects, and usability issues. Some have used field operational tests to develop driving education and training programs. With data from field operational tests, researchers also can study risky behavior and driving performance of groups such as teen drivers, truck drivers, and motorcyclists.

Naturalistic driving studies. This kind of study offers an indepth investigation of everyday driving by observing the actual behavior of motorists going about their daily lives. Researchers collect large amounts of data, including nonnumeric (video recordings) and numeric information (such as speed, acceleration, and braking force). They provide the participants with devices or equip their vehicles or bicycles to record data in an unobtrusive manner.

Sitting in a driving simulator, this research participant is using an automated adaptive cruise control technology in a virtual environment.
Researchers in a control room are monitoring the driving performance of the participant in the driving simulator.
(Top) Sitting in a driving simulator, this research participant is using an automated adaptive cruise control technology in a virtual environment.

(Bottom) Researchers in a control room are monitoring the driving performance of the participant in the driving simulator.

Data involving participants’ ordinary trips are collected for several days or months to as long as a few years. Measures of the outcomes help in tracking evidence on road users’ behaviors and interactions, as well as drivers’ cognitive and physiological states, which are investigated through multidisciplinary approaches. Some naturalistic driving studies focus on predicting risk exposure, estimating crash or near-crash count rates, and identifying critical events in crashes. Different from other research methods, the exploratory nature of naturalistic driving studies also permits researchers to investigate open-ended questions related to behavior, such as how do drivers interact with roadway features and how these interactions demonstrate the effectiveness of countermeasures.

Each of these six research methods offers different perspectives. For some studies, researchers may consider using multiple methods. Under-standing the differences among the methods enables researchers to enhance the value of each dataset in providing crucial information without ignoring the limitations of each.

A motorist participating in a research study is driving a field research vehicle along a test track—an instrumented roadway—that in this case replicates an intersection.
A motorist participating in a research study is driving a field research vehicle along a test track—an instrumented roadway—that in this case replicates an intersection.

According to Sue Chrysler, former director of research at The University of Iowa’s National Advanced Driving Simulator, “There is no perfect method that can account for every aspect of behavior research. There are various factors that can influence the type and quality of data being collected and, ultimately, the study findings.”

Factors in Selecting Research Methods

The FHWA study determined that five factors can explain the similarities, differences, advantages, and disadvantages among the six research methods. The five factors are driving states, risk, level of control, measurements, and validity.

Driving states. The FHWA Human Factors Team categorized driving phenomena into the following driving states: normal, conflict, near-crash/pre-crash, and crash. Normal driving represents driving tasks that are repeated as long as the motorist is on the road. The conflict state involves two or more vehicles with one of them performing an unusual action that jeopardizes the safety of others. This event may require an evasive maneuver by the other driver or drivers. Near-crash/pre-crash represents the last-second braking and steering maneuvers triggered by an imminent threat of a crash. Crashes are failed maneuvers that result in a collision of a vehicle with another vehicle or object.

Risk. A level of risk is inherent in every experiment that is performed for a research study. This factor includes exposures such as discomfort, simulation sickness, getting involved in an actual crash, and other risks associated with participating in an experiment. Of all the research methods discussed, survey has the lowest risk, whereas naturalistic driving studies and field operational tests are considered the riskiest. In those two research approaches, participants are exposed to real-life situations for a longer period than in the other methods, and actual crashes could therefore occur. Researchers who are performing behavioral studies must be aware of principles for protecting human subjects and comply with those principles. Knowing the risks of each experiment, the researchers can make the necessary preparations in advance to maximize the safety of participants.

Relationship Between Research Methods and Impacting Factors

Graphic with elements formed into a box. On the far left is a vertical bar labeled “Risk” and running from “Low” at the bottom to “High” at the top, with colors ranging from green at the bottom through yellow to red at the top. Next to the Risk bar is a “Validity” bar on the vertical axis, and it has a vertical arrow labeled from bottom to top: “Conclusion,” “Internal,” “Construct,” and “External.” At the top of the graphic is a horizontal label, “Driving States,” and beneath that are four labeled columns, from left to right, “Normal,” “Conflict,” “Near/Pre-Crash,” and “Crash.” The six research methods are typed across those columns. The research method Naturalistic Driving Studies runs from halfway through Normal to just into Crash; Field Operational Tests runs from just into Normal to just touching Crash; Test Tracks/Instrumented Roadways runs from just touching Normal to just touching Crash; Driving Simulators runs from just touching Normal to just into Crash; Crash Databases runs from halfway through Near/Pre-Crash to the end of Crash; and Surveys runs from just into Normal to just into Crash. The right vertical axis has a two-way arrow labeled “Measurements Variability” and running from “Less” at the bottom to “More” at the top. To the right of Measurements is another arrow pointing down and labeled “Level of Control,” running from “Low” at the top to “High” at the bottom. At the bottom of the graphic, completing the box, is an arrow running from the left to the right and not labeled.
Six research methods are shown inside a framework according to the types of behavior captured by each method. The color of the vertical bar ranges from green to red to represent a low to high variation of risk to participants. Following this color code, Naturalistic Driving Studies and Field Operational Tests are labeled in a red color to denote such methods as progressively riskier to participants. Test Tracks/Instrumented Roadways are coded as yellow to denote a medium risk. The green color expresses the lower risk of Driving Simulators, Crash Databases, and Surveys.

Level of control. Researchers exert experimental control in manipulating the variables that are presumed to influence the occurrence of a crash or affect certain behavior (for example, roadway scenarios, speed of drivers, and distractions). The level-of-control factor demonstrates the efficacy of a research method in capturing what it is intended to capture and is inversely tied to the realism of the experiment. Researchers have a high level of control when using surveys to collect data. This is because they can tailor questions to target specific driving behavior. However, there is no guarantee that participants’ responses are a true reflection of their actual behavior. Experiments on test tracks and instrumented roadways permit the isolation of scenarios that create situations of interest. Those have a medium level of control. Naturalistic driving studies have a minimum level of control because they are designed to observe behavior “as is.” Sometimes it is advantageous to use more than one experimental method due to the difference in the level of control.

Measurements. During experiments, researchers measure a behavior of interest and track their observations. According to the International Organization for Standardization: Accuracy (trueness and precision) of measurement methods and results--Part 1: General principles and definitions, published in 1994, the variation in the phenomena captured determines whether a study can be repeated under the same conditions so that other researchers can observe similar behavior or whether the phenomena change when reproduced in different settings. Minimum variability in observations enables researchers to track or explain behavior consistently.

An example of this is a survey, which can keep questions consistent and produce minimum variation in the responses. Otherwise, research outcomes vary if observations occur sporadically in different environments, such as data gathered from field operational tests or naturalistic driving studies. Driving simulators have both characteristics. Researchers using driving simulators can repeat the same experiment with different participants to observe cause-effect behavior. They also can reproduce experiments by testing the same behavior in different types of simulators.

Validity. The final factor describes the quality of the data and the credibility of the findings. This factor has four types: conclusion, internal, construct, and external validity. Conclusion validity refers to the appropriate statistical procedures to detect differences and correlations. Internal validity describes the degree to which alternative explanations can be ruled out. Construct validity evaluates whether the data consistently and accurately represent what they are assumed to capture. External validity measures the application of results across persons and places.

Surveys and crash data can control for factors not relevant to a study, such as personal information of participants, and set apart situations that establish a cause-effect relation of driving responses. Thus, they have the highest internal validity. Participants in driving simulation experiments may depict themselves as better drivers and alter their driving behavior, thus compromising the construct validity of the study. In the case of naturalistic driving studies, the validity of conclusions is threatened by the small sample size of events and limited number of participants who exhibit a specific behavior of interest. For example, searching for crash events caused by talking on a cell phone while distracted may result in few cases.

These five factors are empirical parameters that evaluate the quality of data and the research findings based on the analysis performed. Other considerations for researchers are the resources needed to conduct experiments and the time required for data collection. When comparing methods according to their cost, surveys have the lowest cost, simulator and observational methods involve medium-cost expenses, and onroad experiments are higher priced.

The time for a study depends on what the specific question is and the measure wanted. Surveys and crash data analyses are usually the fastest to conduct. Experiments that use driving simulations are usually medium in duration. Onroad studies, such as those involving test tracks  or instrumented roadways, field operational tests, and naturalistic driving studies, require longer durations. Onroad studies are preceded by a lengthy process that involves activities such as obtaining permission to install devices, material facilitation and installation, and insurance coverage and liability for participants.

Selecting Research Methods: An Example

The following graphic example of selecting appropriate research methods is a hypothetic scenario in which the researchers want to understand driver behavior in work zones prior to crashes. The researchers must identify the number of methods available and their ability to measure certain variables of interest. The decision criteria summarize the characteristics of methods based on the factors previously described. Such criteria enable researchers to anticipate undesirable variables or unexpected results and select the best fit for their study

In this hypothetical scenario, the researchers must collect data and measure behavior described by the questions under near-crash/pre-crash driving states. Based on the particular research questions assigned to this driving state, all methods are capable of identifying factors leading to crashes in work zones. At this point in the hypothetical scenario, it is unclear which method will yield the best quality results for the purpose of the study.

Researchers could make a straightforward selection based on the level of risk. For instance, avoiding congested traffic conditions and exposure to real situations will result in selecting surveys, a driving simulator, or a test track or instrumented roadway. If the researcher requires capturing road-user behavior in the field, other criteria must be considered, based on the preference to control an experiment and the consequent restriction in the variability of measurements.

Being able to reproduce the behavior of interest suggests collecting data without interruptions (option 1). A detailed study will be able to capture risky situations only applicable to certain conditions or situations (option 2). Based on the choice of the researchers, these options then will recommend a particular method or narrow it down to a couple of methods.

Concluding Remarks

The proper selection of research methods provides quantifiable solutions to improve transportation safety and eliminates wasteful spending of money, time, and effort. Research procedures must be adequate and need to address the research questions at hand. According to Monique Evans, director of FHWA’s Office of Safety Research and Development (R&D), “Effective transportation strategies to reduce human error are developed by solid research methods and reliable data. Selecting an appropriate method is a critical step in the earliest stage of a research project.”

Decisionmaking Process for Selecting Research Methods

Flowchart. On the left is a box with the words: “Purpose of Study: To identify factors leading to crashes in work zones.� To the right of that box is a box with the words: “Example of Research Questions for Near-Crash/Pre-Crash Driving State: What type of driving performance or road conditions trigger risky situations in work zones? What is the vehicle’s deceleration/acceleration rate in work zones? What is the change in time-to-collision when encountering slow/stopped traffic in a work zone? What evasive maneuvers are performed to avoid crashes in work zones?� From the Purpose of Study box an arrow points down to a table with two columns labeled “Available Research Methods� and “Can this research method capture such behavior?� Under the column labeled “Available Research Methods� are the following methods: Survey, Crash Data, Driving Simulator, Field Operational Test, Test Track/Instrumented Roadway, and Naturalistic Driving Study. Under the column labeled “Can this research method capture such behavior?� the answer is “Yes� under all six methods. A horizontal arrow leads to the right from this table to a box with the following three questions: “Do I want to expose the participants to actual traffic conditions? Do I want participants to drive in actual work zones? Will there be a high potential for drivers getting into near-crash situation or crash?� Two horizontal arrows lead to the right from this box; one is labeled “No� and leads to a box with the words, “Select method based on duration and/or cost� and from there another horizontal arrow leads to a box, “Survey or Driving Simulator or Test-Track/Instrumented Roadway.� From the “Yes� box, a downward arrow points to a box that says, “Select an option based on criteria in table below,� with a downward arrow to a large table with three columns, “Criteria,� “Option 1,� and “Option 2.� The table has
eight rows. Row 1 reads “Data may have small variability� and Option 1 has “no,� Option 2 has “yes.� Row 2 reads “Researchers need to handle and process large and complex datasets� (Option 1 yes, Option 2 no). Row 3: “Driving conditions are repeated for all participants (1 no, 2 yes). Row 4: “Data from multiple work zone sites is collected� (1 yes, 2 no). Row 5: “Drivers may maintain certain behavior during experiment (e.g., follow the same vehicle/change lanes, maintain gap with vehicle ahead)� (1 no, 2 yes). Row 6: “Experiment will take place during certain time period, days of a week, etc.� (1 no, 2 yes). Row 7: “Examine specific work zone warning signs, pavement markings, traffic devices (e.g., barrels, cones, concrete barriers, etc.), configurations (e.g., tapper in a shoulder, etc.)� (1 no, 2 yes). Row 8: “Collect data in specific circumstances (e.g., heavy traffic condition, presence of construction workers, mobile and short-duration work zones, no construction activity, speed photo enforcement, police present, etc.)� (1 no, 2 yes). To the left of this table is a horizontal arrow labeled “Option 1� leading to a series of boxes. The top box contains the following questions: “Do I want to analyze behavior without having to develop a ‘baseline’ or using surrogates to compare observations? Do I want to avoid dealing with sampling events under continuous driving and selecting highly coordinated variables? Do I want events of interest to happen frequently?� Under that box are yes and no options. Under “Yes� is “Driving Simulator.� Under “No� is “Select method based on duration and/or cost� leading to “Field Operational Test or Naturalistic Driving Study.� To the right of the large table is a horizontal arrow labeled “Option 2� and leading to a box with the questions, “Are the results only applicable to the region/work zone in the study? Can I accept results that may have some deviation from actual driving behavior in a work zone?� That box leads down to a choice between “No� with an arrow to “Crash Data� or “Yes� with an arrow to “Select method based on duration and/or cost� and an arrow to “Driving Simulator or Test-Track/Instrumented Roadway.�
This flowchart illustrates a decisionmaking process for researchers to select a research method for a hypothetical study focusing on factors leading to crashes in work zones.

Using this systematic approach, researchers can make informed decisions on which method(s) will permit a targeted investigation of transportation issues and facilitate solutions.

Improving safety is a core mission of the U.S. Department of Transportation and FHWA. Examining and understanding behaviors of road users has a strong correlation to transportation safety. As the collection of behavior data becomes more sophisticated, researchers must employ proper methods to gather pertinent data about road users in order to implement effective transportation solutions.


Alicia Romo, Ph.D., is a research associate working at FHWA’s Office of Safety R&D. Romo was selected for a National Research Council fellowship in 2013 to develop research strategies for safety studies. Her research interests include human factors, modeling of car-following driving behavior, and control systems. She received her B.S. and Ph.D. degrees in civil engineering from the University of Texas at El Paso.

C. Y. David Yang, Ph.D., is the leader of the Human Factors Team in FHWA’s Office of Safety R&D. He joined FHWA in 2008. Yang is the chair of the Transportation Research Board’s User Information Systems Committee and serves on the editorial board of the Journal of Intelligent Transportation Systems. He received his B.S., M.S., and Ph.D. degrees in civil engineering from Purdue University. His doctoral dissertation used principles of human information processing and human factors to develop design recommendations for Advanced Traveler Information Systems.

For more information, contact Alicia Romo at 202–493–3369 or alicia.romo.ctr@dot.gov, or C. Y. David Yang at 202–493–3284 or david.yang@dot.gov.

 

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101