EXPERIMENTAL DESIGN

The experiment was a 2x2x2x2x2x4 mixed factorial design. Table 2 shows the within-subjects variables distributed in a Latin square design. The experiment considered three dimensions of ATIS design: (1) display location (centralized versus distributed), which was a between-subjects variable, (2) message style of ATIS information (command versus notification), which was a within-subjects variable, and (3) mode of presentation (auditory versus visual), which was a within-subjects variable. The order of these variables was counterbalanced in a Latin square design. Each row in table 2 represents the experimental conditions that a driver experiences. Each driver experiences 16 scenarios and, during each scenario, 4 roadway events occur. The letters and numbers represent levels of independent variables.

Each line represents a series of 16 scenarios with each scenario representing a separate experimental condition. A scenario is defined as a sequence of four driving events lasting approximately 6 min. A driving event is defined as a circumstance that requires a driver decision and action. When ATIS information is available, drivers are notified of the upcoming events by ATIS warning messages. Each subject experienced 16 scenarios. The design was constructed so that each set of 16 scenarios was experienced by 4 subjects: 1 young male, 1 older male, 1 young female, and 1 older female. This pattern of experimental conditions was replicated so that 16 drivers experienced the scenarios in table 2 with a centralized display and 16 drivers experienced these conditions with a distributed display.

Independent Variables

Table 3 summarizes the independent variables examined in this experiment. The independent variables included the availability of roadway and ATIS information, display location, message style, display modality, and driver age. The availability of information was a within-subjects variable with four different levels: (1) ATIS and roadway information, (2) ATIS information only, (3) roadway information only, and (4) neither. Roadway information was presented as a changeable-message sign in a style that matches most standard warning signs. The style of all the roadway signs is similar to the notification style for the ATIS messages. During a scenario with the information availability condition "ATIS and roadway information," drivers receive four ATIS messages and see four changeable-message signs, one for each roadway event they encounter. During the "ATIS only" condition, drivers receive four ATIS messages and see only one changeable-message sign. During the "Roadway only" condition, drivers receive only one ATIS message and see four changeable-message signs. During the "Neither" condition, drivers receive only one ATIS message and see only one changeable-message sign. In the "Neither" condition, drivers encounter two events without any warning, they receive an ATIS message for one, and they see a changeable-message sign for the remaining event. This was done to mimic the fact that a completely unreliable ATIS or road-sign system is not realistic. Table 3 summarizes the independent variables.

Table 3. Independent variables included in the experiment.

For each scenario, drivers experienced a random selection of four of the events shown in table 3. Table 4 shows the ATIS warnings for each of six different events. Depending on the experimental condition, drivers received a message from table 1 either as a command or as a notification message, formatted as an auditory or visual warning.

Table 4. ATIS warnings for each of six different events.

Dependent Variables

This experiment collected data on two types of dependent measures. One type included direct measures of driving safety and compliance with warnings. The other type included intervening variables that may illuminate the cognitive processes that influence the direct measures of compliance and driving performance. The intervening dependent variables measured drivers' attitudes, situational awareness, and message acknowledgment.

Drivers' attitudes were measured by subjective ratings given at the end of each scenario. Subjective scales measured drivers' attitudes, including: (1) trust in the ATIS system to identify and notify them of roadway events and hazards, (2) self-confidence in their ability to accurately identify roadway conditions and hazards, (3) mental effort, (4) physical effort, and (5) perceived driving performance. Mental and physical effort were included to estimate the influence of display characteristics on driver workload. The scales were presented on a touchscreen display at the end of each 6-min trial.

Subjects' situational awareness was measured once during each scenario using the Situation Awareness Global Assessment Technique (SAGAT) (Endsley, 1995a). Situation awareness has been defined by Endsley (1995a, p. 36) as "the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future." SAGAT has been studied extensively and has proven to be a valid and effective measure of SA (Endsley, 1995b). Using this technique, the simulation was "frozen" while the subjects responded to the questions. Examples of questions include: "What is your current speed?," "What product was the last billboard advertising?," "In the next 30 seconds, which action will you perform?" For each question, the drivers' were given two choices from which to select their answer (forced choice). Four questions were randomly drawn for each query from a pool of 14 questions (see appendix C for a complete list). After each question, subjects rated their confidence in their answer on a 0-100 scale. Each situation awareness query contained four questions and was presented once in each 6-min trial.

Drivers were asked to acknowledge each ATIS and roadway warning by pressing a steering-wheel button. Drivers used the button on one side of the steering wheel to acknowledge ATIS messages and the button on the other side to acknowledge roadway signs. The acknowledgment latency and accuracy were measured to estimate the focus of the drivers' attention. If drivers focus attention on the ATIS, then acknowledgment latency and accuracy should favor the ATIS acknowledgment. The acknowledgment side was counterbalanced to guard against any right/left bias. To avoid learning effects, acknowledgment side was introduced as a between-subjects variable.

Measures of driving safety included:

These measures began 5 s before the time that the drivers received the ATIS message or roadway information and lasted until 5 s after the event had ended. For driving performance, lane position, speed, brake and accelerator actuation, turn-signal actuation, steering-wheel use, and crashes were all collected during this time. The purpose of collecting these data was to determine how ATIS messages might distract from the primary task of driving.

For this study, warning compliance has been defined as the degree to which drivers correctly respond to messages. For example, drivers who move to the left lane after a message instructing them to merge left would have a higher level of compliance compared to a driver who fails to comply with the message and stays in the right lane. Similarly, if a driver slows from 45 to 25 mi/h (72.4 to 40.2 km/h), compliance would be greater than one who remains at 45 mi/h (72.4 km/h). Complete compliance with the warning is rated at 100 percent. It is possible to be more than 100 percent compliant, such as when a driver swerves to the far side of the left lane to avoid an accident in the right lane.

Figure 3 shows several measures of compliance. The analysis addressed three measures of compliance. The first measure is the time to 10 percent of maximum compliance, shown on the left side of figure 3. The second measure reflects how quickly compliance increases and is labeled rise time. Rise time is the time it takes compliance to change from 10 percent of maximum compliance to 90 percent of maximum compliance. The third measure of compliance, known as integrated compliance, represents the area under the curve in figure 3. It is calculated by numerically integrating the level of compliance from the onset of the message to the end of the event.

Procedure

Figure 4 shows the experimental protocol. When subjects first arrived, they were briefed on what they would be asked to do and were then asked to sign an informed consent form if they chose to participate. Subjects were then given a brief pre-test that evaluated their general attitudes toward technology. Following the pre-test, drivers were trained on the experimental procedures. To ensure that the drivers understood the instructions, they were given a short test. The experimenter reviewed any questions that were answered incorrectly. During the training, drivers were told to drive normally and obey speed limits. They were also instructed to respond to ATIS and roadway warnings by pressing the appropriate steering-wheel buttons. Subjects who failed to respond accurately to more than 25 percent of the events repeated the training scenario. If after two repetitions they were not able to respond accurately, they were paid and escorted out of the laboratory.

The training scenario was followed by four 30-min "blocks" composed of four 6-min scenarios and a short break. This generated data for a total of sixteen 6-min scenarios. Each 6-min scenario contained four main driving events. These events included: disabled vehicles, pedestrian zones, high-occupancy vehicle (HOV) lanes, accidents, and construction zones. Drivers experienced a random selection of four of these events during each 6-min scenario.

During each scenario, the subjects were in control of steering, acceleration, and braking. Since the subjects were in complete control of the vehicle during the scenarios, the scenario length varied slightly depending on the driver. In each scenario, ATIS and roadway information was presented to alert drivers to each of the four roadway events. Other roadway characteristics that drivers might typically see while driving (traffic lights, pedestrians, billboards, etc.) were also displayed. Drivers also faced typical driving hazards. They faced oncoming traffic and passing traffic. As they drove in the right lane of a two-lane rural road, cars approached and passed in the left lane.

Two sets of questions were administered during each scenario. One set, concerning drivers' situational awareness, was presented randomly during one of the four roadway events and the other set, containing subjective measures of the drivers' experiences, was presented at the end of each scenario. After completing the 16 scenarios, the subjects were given a post-test identical to the pre-test that evaluated subjects' attitudes toward technology. Subjects were then debriefed, paid for their participation, and escorted out of the laboratory.

Previous

Next