U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
REPORT
This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-16-056     Date:  December 2016
Publication Number: FHWA-HRT-16-056
Date: December 2016

 

Cooperative Adaptive Cruise Control Human Factors Study: Experiment 1—Workload, Distraction, Arousal, and Trust

 

CHAPTER 2. METHOD

This chapter provides an overview of the approach to assessing workload, arousal, distraction, and crash avoidance as well as extensive details on the experimental design and procedures.

APPROACH

Four groups drove the simulated vehicle over the same 39-mi stretch of limited access roadway. Three of the four groups drove a CACC-equipped vehicle in a platoon with other CACC-equipped vehicles, while the fourth group manually controlled their following distance within a platoon in which all the other vehicles used CACC. For most of the distance traveled, the roadway and the behavior of other vehicles was the same for all groups.

Workload Assessment

Driver workload was assessed by administration of the National Aeronautics and Space Administration Task Load Index (NASA-TLX).(12) This subjective workload measure is typically administered through a series of two paper-and-pencil ratings after a task (e.g., a flight or drive) is completed. For this method, participants rate mental demand, physical demand, temporal demand, effort, and frustration by placing a mark on a rule where one end of the rule is labeled “low” and the other “high.” Similarly, respondents rate their performance by marking on a rule where one end is labeled “good” and the other “poor.” After providing these ratings on each of these workload factors, respondents are asked to consider all possible pairings of the six factors and indicate which member of the pair was the most important contributor to their workload. Because the present interest was to examine workload during different phases of a single drive, the typical paper-and-pencil method for administering the NASA-TLX would have been unwieldy. It would have required either stopping the drive while the participant completed the forms or relying on participants’ recall of each phase after the drive was completed. To get a more immediate workload rating while allowing the participants to continue the drive, the NASA-TLX was verbally administered. Although the verbal administration technique has not been formally validated, participants seemed to accept the method, and, as is discussed in chapter 3, the results appear to be interpretable.

Workload was assessed four times. The first assessment was during a practice drive and was intended to familiarize participants with providing verbal responses to the NASA-TLX protocol. The second workload assessment was 5 min into the main scenario, just after a vehicle merged into the platoon between the participant’s vehicle and the vehicle the participant had been directly behind. This NASA-TLX assessment was intended to determine the workload imposed by a vehicle halving the following distance between the participant and the vehicle ahead. The third assessment occurred 15 min into the drive and was intended to assess the workload associated with driving in a CACC platoon when no changes in the platoon had occurred for 10 min and 11.7 mi of uneventful driving. The final assessment was near the end of the drive and immediately followed the events that varied between groups. These events are described in detail in the following subsections. The first assessment (i.e., the practice assessment) was not analyzed; however, the second and third assessments enabled comparison of workload between the three CACC groups (which should not have differed from each other at these points) and the control group which had to manually maintain gap. The final workload assessment was intended to assess the workload associated with the various events. Although participants continued to drive while workload was assessed, none of the other performance measures reported were collected during workload assessment.

Physiological Arousal

Physiological arousal was assessed by measuring eyelid closure, pupil diameter, and skin conductance. These measures were assessed at five 30-s periods during the drive: (1) before the first merge event, (2) after the start of the first merge event and before the NASA-TLX assessment, (3) 15 min into the drive and before the NASA-TLX administration, (4) just before the final event, and (5) during the final events. Thus, these measures were intended to assess changes in arousal as a result of the initial merge event, after 10 min of uneventful driving, and as a result of the final events.

Distraction

Automation is generally believed to reduce driver workload. A positive result of automation would be a more relaxing and rewarding driving experience. A less positive result might be that the driver feels free to engage in more non-driving tasks that might subsequently result in less attention to the driving task. The shift in attention away from the driving task might be termed “distraction.” However, distraction is a psychological construct that may have many different operational definitions and associated theoretical measurement methods.(13) In this study, participants were allowed to play the radio, use their cell phones, or otherwise engage in non-driving-related activities. Engagement in non-driving activities was neither encouraged nor discouraged. Participants were instructed to drive as they normally would with the exception of CACC usage and gap maintenance. The extent to which participants engaged in voluntary non-driving activities and how these correlated with physiological arousal and crash avoidance was observed. Aside from assessing these correlations, no attempt was made to measure the extent, if any, to which these activities might be distracting.

Crash Avoidance

At 34.6 mi from the start, the behavior of other vehicles varied between groups. Two groups experienced a non-CACC-equipped vehicle cutting in front of the platoon and overturning. Of these two groups, one was using CACC and the other, the control group, was manually maintaining the gap. This manipulation was intended to test whether CACC-equipped drivers would be more or less likely to avoid a crash when sudden hard braking was required. If the convenience of CACC induced drivers to become complacent, distracted, or unaware or to have a very low level of arousal, then the CACC group might experience more crashes than the control group, which, because they were forced to monitor gap distance manually, might be expected to be more aware, aroused, and attentive. However, the CACC system had partial braking up to about 0.4 g, and in this scenario, braking was initiated before the brake lights of the lead car came on. Also, when the CACC system began to brake, a loud series of beeps was intended to alert the driver of the need to take longitudinal control. This beep carried a meaning similar to that of a forward collision warning. With the automated assistance of the CACC system, the CACC group might gain the slight reaction time advantage they would need to avoid a crash. This test of crash avoidance would be specific to the warning and brake response programmed into the system, and, thus, the results cannot be expected to generalize to all potential CACC implementations. Nonetheless, they should give a clue as to whether similar CACC systems will be more likely to be a boon or detriment to safety and may provide a starting point for exploration of CACC warning parameters and the effects of automated braking.

Equipment and Materials

The Driving Simulator

The experiment was conducting in the Federal Highway Administration (FHWA) Highway Driving Simulator. The simulator’s screen consisted of a 200-degree portion of a cylinder with a radius of 8.9 ft. Directly in front of the driver, the design eye point of the simulator was 9.5ft from the screen. The stimuli were projected onto the screen by 5 projectors with resolutions of 2,048 horizontal by 1,536 vertical pixels. Participants sat in a compact sedan. The simulator’s motion base was not enabled in this experiment. The car’s instrument panel, brake, and accelerator pedal all functioned in a manner similar to real-world compact cars. The steering wheel did not function as intended in this experiment because the force feedback mechanism was not functioning. As a result of the steering system malfunction, the vehicle’s response to steering inputs was immediate and not modulated by vehicle dynamics so that participants needed to continually correct responses to prior steering inputs. The effect of the steering malfunction was to generate an unintended level of driver workload.

The simulator was equipped with a hidden intercom system that enabled communications between the participant and a researcher who ran the experiment from a control room. The researcher in the control room could also view the face video from the eye-tracking system and thereby monitor the participant’s well-being.

Eye-Tracking System

The simulator was equipped with a four-camera dashboard-mounted eye-tracking system that sampled at 120 Hz.(14) The system tracked horizontal gaze direction from approximately the right outside mirror to the left outside mirror and vertical gaze direction from the bottom of the instrument panel to the top of the windscreen. Gaze direction accuracy varied by participant. The mean accuracy of gaze position across all participants was 1.4 degrees (radius) with a 0.9-degree standard deviation for the left eye and 1.7 degrees with a 1.3-degree standard deviation for the right eye. In this study, the eye-tracking system was primarily used to determine which vehicle displays the participant was looking at. The following display locations were tracked:

In addition to tracking the direction of gaze, the eye-tracking system computed eyelid opening and pupil diameter. These measures were also recorded at a 120 Hz.

Multifunction Display

The model of sedan used for the simulator was not originally equipped with cruise control. For this experiment, a 7-inch diagonal liquid crystal display touch screen was mounted on the center console above the radio. For the CACC conditions, the touch screen contained three interactive elements. On the left side of the screen, the selected speed was displayed, and two arrows could be touched to modify the selected speed. Touching the upward-pointing arrow increased the set speed by 1 mi/h, and the downward-pointing arrow could similarly be used to decrease the set speed. In the center of the screen were three bars that when touched would cycle through the three available gap selections (near, medium, and far). On the right side of the screen was a touch screen button that toggled the CACC system on and off. Although set speed and gap adjust buttons altered the displayed settings, those display settings did not affect the simulated performance of the CACC system. Throughout the experiment, the set speed remained 70 mi/h, and the gap target remained 1.1 s. The participants were informed that the gap and speed setting adjustments were non-functional and intended only to assist in explaining the ACC concept. No participants were observed trying to change these settings. The engage button on the right side of the display was fully functional. When CACC was engaged, the set speed arrows, near gap adjust bar, and engage button were green. When the CACC system was not engaged, all three elements were a shade of red or magenta.

For the control group, the multifunction display appeared as shown in figure 1. The control group display was not interactive. A black bar on the colored ribbon displayed the current gap between the control participant’s front bumper and the rear bumper of the vehicle ahead. Control participants were asked to try and maintain a 1.1-s gap and keep the black bar in the green region of the ribbon.

In this figure, a ribbon display is shown that stretches from top to bottom of the multifunction display. On the left side of the ribbon are numbers that range from 0.0 at the top to 2.0 at the bottom. On the left side of the ribbon are red, yellow, and green shading. The green shading spans the area between 0.8 and 1.3. There are two yellow areas. The upper yellow area ranges from 0.5 to 0.8, and the lower yellow area ranges from 1.3 to 2.0. The area between 0.5 and 0.0 is shaded red. Participants were instructed to keep the small black pointer in the green area. In the screen capture, the pointer is in the green area at 1.1.

Figure 1. Screen capture. Appearance of control group multifunction display.

Skin Conductance Sensor

Galvanic skin response (GSR) was measured with silver-chloride salt electrodes placed on the palmar-side base of two fingers on the participant’s left hand. The electrodes were connected to a small sensor with a Bluetooth® transmitter strapped to the left wrist.

Simulation Scenarios

Participants drove in a dedicated lane (i.e., lane adjacent to the median) on a simulated eight-lane interstate highway (four lanes in each direction). This lane was separated from the other lanes by F-type barriers. A typical portion of the roadway is depicted in figure 2. The center dedicated lane was accessed from the left side of the roadway from a ramp with a ramp meter. The ramp meter is depicted in figure 3. The simulation began with the participant’s vehicle in the third position within a platoon of four vehicles. When the ramp meter turned green, the platoon accelerated and merged into the CACC lane and cruised at 70 mi/h. Vehicles in CACC mode were set to maintain a 1.1-s gap. There were a few simulated vehicles in the lanes to the right of the dedicated CACC lanes that could be viewed when looking over the barrier.

This figure shows a screen capture of a typical section of the simulated roadway. The back end of a gray sport-utility vehicle is shown in the distance in the middle of a single freeway lane. The lane has a yellow edge line on the left and a white edge line on the right. There is a concrete barrier a short distance to the right of the right edge line. To the left of the left edge line is a breakdown lane. To the left of the breakdown lane is a grassy median.

Figure 2. Screen capture. Typical section of the simulated roadway.

 

This figure shows a screen capture of an entrance ramp meter. An on-ramp is shown, and it slopes down to a single travel lane. There is a concrete barrier on either side of the ramp. Atop the left-side barrier is a pair of two-lens signals. Both signals are showing the lower (green) lens illuminated. The signals are mounted one above the other. Between the two signals is a “Stop Here on Red” regulatory sign (R10-6a).

Figure 3. Screen capture. Entrance ramp meter.

For the first 5.8 mi or 5 min, the platoon proceeded as formed. At 5.8 mi, a CACC vehicle merged into the platoon from the left in front of the participant driver. The merge was from a ramp identical to the initial ramp. Initially, the gap between the participant and the merging vehicle was about 0.5 s or 51 ft. At the 34.8-mi point, one of the following critical events occurred:

Calibration of CACC Vehicle Size

In previous testing in the simulator and in pilot testing for the present experiment, most individuals showed a reluctance to follow other vehicles with a 1.1-s gap and indicated that they never followed that closely. Because the literature suggests that a 1.1-s gap is greater than most people consider safe, an experiment was conducted in which six drivers from the FHWA research center (Federal employees and contractors) drove an instrumented vehicle in the field while following a Jeep® Grand Cherokee and followed the same simulated vehicle in the driving simulator.(15) The field data collection was conducted on limited-access managed lanes with minimal traffic. The simulated roadway was the CACC-managed lane used in this experiment. Each of the drivers was asked to follow both the real and simulated vehicles with the following instructions:

In the field, the participants drove an instrumented Cadillac® SRX that was equipped with ACC. In the simulator, the eye point of the simulator cab was positioned to approximate the eye height of a Jeep® Grand Cherokee. The procedures in the field and in the simulator were the same. Participants first drove for 5–7 min to accustom themselves to the vehicle/simulator. They then caught up to the lead vehicle, which was traveling with cruise control set to 65 mi/h, and were instructed to follow at a comfortable distance. After following constantly for about 1 min at what the participants said felt was a comfortable distance, participants were asked to back off a substantial distance (greater than 4 s). Next, the participants were asked to accelerate and follow while maintaining the minimum safe distance (the shortest gap they believed to be safe). This procedure of catching up to follow at comfortable and minimum safe distances was repeated at least twice. After backing off to more than 4 s again, participants were asked to engage the ACC/CACC system that was set to follow with the “near” setting. The near setting sought a 1.1-s gap. Once they had followed at the near distance for at least 1 min, the system was again disengaged, and the participants backed off to a distance of more than 4 s. The final request was to accelerate to and maintain the same following distance they had driven with the ACC/CACC system engaged. On all trials, steady state following was recorded for approximately 1 min.

With the simulated lead vehicle set to have a visual angle subtended precisely the same as it would be in the real world, participants maintained a following distance about 1.3 times the distance they had maintained in the field. This suggested that the lead vehicle’s size needed to be reduced to induce the same perceived following distance the participants maintained in the field. As a first approximation, the lead vehicle size was reduced to 75 percent of the “correct” size based on 1:1 visual angle correspondence. Several weeks later, the original six participants returned to the simulation laboratory and followed the original procedure but following a reduced size lead vehicle. Participants were not informed about the changes that had been made to the leading vehicle. In the second simulation drive, the participants nearly duplicated the comfortable and minimum safe distances they had driven in the field. The results of this testing are shown in figure 4.

This line graph shows the results of field and simulator gap maintenance testing. The x-axis is labeled “Gap Instruction” with two categories: comfortable and minimum safe. The y-axis is labeled “Mean Time Gap” and ranges from 0.0 to 2.0 s. Three lines are plotted on the graph: field, sim 1:1, and sim 0.75:1. The data points are as follows for each line for comfortable and minimum safe, respectively: field: 1.4 and 0.7; sim 1:1: 1.9 and 0.9; and sim 0.75:1: 1.4 and 0.8.

Figure 4. Graph. Results of field and simulator gap maintenance testing.

As a result of this testing, it was decided to reduce the size of the other vehicles in the CACC platoon to 75 percent of the size of a 1:1 depiction. Figure 2 shows a leading vehicle with the original (1:1) scaling. Figure 5 shows a leading vehicle scaled to 75 percent of the original size. Although extensive testing was not done to determine the source of the following distance misperception, the Ponzo illusion appears to be a likely candidate explanation.(16) The Ponzo illusion is illustrated in figure 6, where all three vehicle pictures are the same size, but the upper vehicle appears to be larger than the lower vehicles.

This screen capture shows a reduced-size lead vehicle depicted with a 1.1-s gap. It is very similar to figure 2. The difference is that the sport-utility vehicle in the distance appears to be further away. In actuality, the vehicle is the same distance away as in figure 2, but the size of the vehicle has been reduced by 25 percent.

Figure 5. Screen capture. Reduced-size (75 percent) lead vehicle depicted with 1.1-s gap.

 

This figure depicts a Ponzo illusion. There are three identical sport-utility vehicles shown one behind the other from a perspective of above and behind slightly to the right. The vehicles are in a travel lane with a yellow edge line on the left and a white edge line on the right. The edge lines converge in the horizon. The first vehicle in the front appears to be much larger than the vehicle behind it, which appears larger than the vehicle behind it. If a ruler is used to measure the size of the three vehicles, it will be discovered that each vehicle has exactly the same dimensions. The visual illusion caused by the converging lines that causes objects closer to the horizon to appear larger is called a Ponzo illusion.

Figure 6. Screen capture. The Ponzo illusion. (The vehicles in the picture are all the same size.)

Procedure

Upon arrival at the research center, participants were asked to review and sign an informed consent statement. This was followed by a health screening to ensure that the participants were not at increased risk of simulator sickness as a result of illness or lack of sleep. Participants were asked to show a valid driver’s license. A Snellen chart was used to verify visual acuity equal to or better than 20:40, with correction if necessary. A slideshow presentation with embedded videos was shown to explain the CACC concept. Participants assigned to one of the CACC conditions were presented with the warning tone that is triggered when more braking was needed than the CACC system could provide. The CACC related instructions were as follows:

1. Set the gap to “near.”

2. Set the speed to 70 mi/h.

3. Control steering—follow the car in front.

4. Allow the system to accelerate and brake up to a limit.

Except for the previous numbered instructions, the control group instructions were the same. The instructions unique to the control group were as follows:

1. Aside from maintaining 1.1-s gap, drive normally.

2. Stay alert for unexpected events.

The slideshow presentation concluded with an explanation of the NASA-TLX, which was verbally administered while they were driving.

Following the slideshow presentation, participants were fitted with the GSR sensor and seated in the simulator cab where the controls and displays were reviewed, and the instructions were repeated. While seated in the cab, participants were asked to complete the simulator sickness questionnaire (SSQ) to provide a symptom baseline. Finally, the eye-tracking system was calibrated to the participants; the procedure generally took 5 to 10 min.

With the preliminaries completed, participants were asked to perform a brief (less than 10 min) practice drive. On the practice drive, participants were asked to merge onto the dedicated CACC lane, which was free of traffic, and to accelerate to 70 mi/h. They were asked to gently brake and then accelerate, which was followed by a request to brake hard and then accelerate. To enable adaptation to the lateral control, participants were asked to gently change from the travel lane into the breakdown lane and then change back into the travel lane. This was followed by a request to quickly change into and out of the breakdown lane.

Participants in a CACC condition were then asked to engage the CACC system. With no vehicles ahead, the CACC system accelerated to 75 mi/h until it closed on a platoon of CACC vehicles traveling at 55 mi/h. The platoon traveled at 55 mi/h for 2 min and then accelerated to 70 mi/h.

Participants in the control condition were asked to accelerate to 70mi/h and maintain that speed until they closed on a platoon of CACC vehicles. They were then asked to follow with a 1.1-s gap and refer to the ribbon gap display as necessary. The platoon behaved in the same manner as for the CACC conditions.

After traveling in the platoon at 70 mi/h for 2 min, the NASA-TLX was administered to all participants. This administration was intended to further familiarize participants with the workload assessment tool, which is not typically verbally administered. With the conclusion of the workload assessment, participants were asked to take the next available off-ramp and come to a complete stop.

After completion of the practice drive, participants were asked to exit the vehicle and complete the SSQ.

The experimental session began with the participants seated in the third vehicle of a platoon of four vehicles. The platoon was stopped on a ramp in front of a ramp meter showing a red indication. When the ramp meter turned green, the vehicles ahead began to accelerate down the ramp toward the CACC travel lane. At this time, participants in the CACC condition were asked to release the brake and press the “ENGAGE” button on the multifunction display. With CACC engaged, the participant’s vehicle followed the two preceding vehicles in the platoon with a 1.1-s gap. Participants in the control condition were asked to follow the preceding vehicles and try to keep the gap close to the 1.1-s target.

About 5 min into the drive, a CACC vehicle came down a ramp on the left and merged into the gap directly in front of the participant’s vehicle, which momentarily cut the gap to half of what it had been. The CACC-equipped vehicles behind the merged vehicle responded by decelerating with engine braking until the gap was again 1.1 s. If necessary, a researcher would remind control participants to return to the 1.1-s following distance. As soon as the platoon stability was reestablished, which generally took about 30 s, the NASA-TLX was administered to assess workload during the merge event (“during the preceding minute or so”).

From the conclusion of the NASA-TLX, about 10 min elapsed before another NASA-TLX was administered again. This administration was intended to assess workload during uneventful cruising in a CACC platoon (also described as during the last minute or so). The cruise was again uneventful for the next 31 min until the critical event (described previously in section entitled The Simulation Scenarios). At the conclusion of the critical event, a final NASA-TLX was administered, after which the participant was asked to take the next exit ramp and come to a stop.

After exiting the simulator, participants were asked to complete a final SSQ, debriefed, and paid for their participation.

Experimental Design

The primary between-group independent variable was whether the participant vehicle was equipped with CACC. The experimental design called for 36 participants to drive with CACC and 12 to drive without cruise control but within a platoon of simulated CACC vehicles. Participants driving with CACC were assigned to one of three critical events, with 12 participants assigned to each event.

Thus, there were the following four distinct participant groups:

There was one within-subjects variable, referred to as a “period,” with five levels that was intended to distinguish the effects of CACC on driver behavior. The five levels are defined as follows:

Participants

Participants were 51 licensed drivers recruited from the Washington, DC, metropolitan area. The goal was to recruit an equal number of males and females over and under the age of 46, which is the median age of participants in the FHWA recruitment database. Participants were required to be at least 18 years of age and were screened for susceptibility to motion and simulator sickness. One participant withdrew before completion because of simulator sickness symptoms. Another participant’s data were lost as a result of simulator operator error. Table 1 shows the age group and gender counts by treatment group for the participants who provided useable data. The mean age of the younger participants was 30.4 years (ranging from 21 to 38 years). The mean age of the older participants was 60.4 years (ranging from 49 to 76 years).

Table 1. Demographic breakdown of participants by treatment group.
Condition Young Females Young Males Older Females Older Males Total
Control 3 3 3 3 12
CACC with crash avoidance 3 4 3 3 13
CACC with cut-in 3 3 3 3 12
CACC with communications failure 2 3 3 4 12
Total 11 13 12 13 49

 

Participants were paid $60 for their participation.

 

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101