• Federal Highway Administration >
• Publications >
• Research Publications >
• 96143 >
• Sec2.Cfm >
• Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies

Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies

CHAPTER 2. EXPERIMENT 1 AND 1B

EXPERIMENT 1 METHOD

EXPERIMENT 1B METHOD

EXPERIMENT 1 RESULTS

EXPERIMENT 1B RESULTS

DISCUSSION

EXPERIMENT 1 METHOD

Subjects

A total of 109 subjects participated in this study. Subjects ranged in age from 18 to 85 years old. A total of 57 of the subjects were male and 52 were female (figure 2). All subjects were licensed drivers who drove at least once a week in the Seattle area. Twelve additional subjects began the study but did not complete it. One subject had a child to care for, two older subjects had difficulty understanding the concepts being presented, and the nine remaining subjects did not finish in the time they had available due to a scheduling conflict at the group's facility. Each subject was paid for their time and thanked for participating.

Subjects were recruited from organizations in the Seattle metropolitan area. These organizations included the University of Washington, senior citizens' centers, churches, and other service organizations. Subjects were paid $10 per hour for their participation and could choose either to keep the payment or to donate it to an organization of their choice.

Apparatus

The apparatus for this experiment consisted of a Dukane overhead projector, an InFocus LCD monitor, a Panasonic TV/VCR, and stopwatches (Health Tech, Spalding, and Micronta brands). The video image from the television/videocassette recorder (TV/VCR) was output through the liquid crystal display (LCD) monitor and overhead projector in order to create a display large enough for small group viewing. Audio output from the TV/VCR was adjusted in each session so that all subjects could hear the recorded sound.

Materials

Materials for this experiment (see appendix B, pp. 173–250) included two videotapes showing the TravTek system, a consent form, a driver demographic characteristics questionnaire that included technology use items, a questionnaire designed to assess user acceptance information, and photocopied maps of the Orlando area (figures 3 and 4).

The first video was an edited version of the American Automobile Association's (AAA) TravTek system orientation video. This video explained the benefits and options of the ATIS and lasted approximately 15 min. The other video was a split–screen presentation that contained out–the–window views of a filmed route from the Harry P. Leu Botanical Gardens to Church Street Station in Orlando, Florida, using a TravTek system–equipped car. A video overlay of the TravTek system screen was presented in the lower left corner of this video while, periodically, a full–screen view of the TravTek system was shown. Voice messages provided by the TravTek system were recorded as well. The route, which included residential streets, four–lane State roads, and a portion of the Interstate, was approximately 14 min long.

Experiment 1 used a quasi–experimental design. Independent and attribute variables that were measured include: AGE, GENDER, demographic variables, and technology use. Technology use items included questions that asked about subjects' use of vehicle technologies (e.g., anti–lock brakes, air bags) and household technologies (e.g., VCR, microwave oven, personal computer [PC]). Table 2 summarizes the independent variables.

Table 2. Independent variables in experiment 1

Several dependent variables were assessed by the TravTek system questionnaires, all of which are shown in appendix B:

• TravTek System Capabilities

• TravTek System Feature Desirability

• TravTek Demonstration Fidelity

• TravTek: Modifying Your Trip to Avoid Traffic

• TravTek: Trust & Self–Confidence

• TravTek User Acceptance Issues

• TravTek Perceived Usefulness

• TravTek Perceived Ease of Use

• TravTek User Test Questions

Table 3 summarizes the dependent variables in experiment 1.

Table 3. Dependent variables in experiment 1.

To keep the data organized, the questionnaire was copied onto several different colors of paper, each color representing a specific video portion of the experiment. Two maps of the Orlando area (taken from portions of the Gousha Fastmap of Orlando) were copied onto different colors of 216 x 279 mm paper and increased 10 percent in size. The first map consisted of an origin (a middle school) and destination (a Greyhound bus station) marked with a large red "X". The other map was the route shown in the associated TravTek system video. The origin (Harry P. Leu Botanical Gardens) and destination (Church Street Station) were also marked with a large red "X". Subjects were provided with markers to draw their routes on the paper maps (figures 3 and 4).

Examples of the information presentation formats used in the TravTek system were copied onto an overhead transparency. Two sample electronic maps, two text or icon displays, and one voice message were presented on this overhead. A copy of these examples is shown in figure 5.

Procedure

Subjects received a brief description of the ATIS research and of what they would be doing during the study. They read and completed informed consent sheets stating their rights as subjects and completed the questionnaire that asked demographic and technology use questions (see appendix B, p. 173). After filling out these items, subjects were told how to use their stopwatches. When all subjects had indicated that they could operate their stopwatches, they were walked through an example map routing task (from the middle school near Glenridge Way to the Greyhound Bus Station in Orlando). Subjects were shown a copy of the map (the experimenter identified the route origin and destination) and then told them that they would be asked to turn their maps over, start their stopwatches, draw the route they would take if they were actually visiting the Orlando area, turn off their stopwatches when finished, and record the time shown on the stopwatch display. The experimenter told them that even though they were timing the routing task, accuracy was more important. Steps were reviewed until each subject understood the procedure. The routing task was meant to directly involve the subjects in the experiment as a contrast to the passive observation of the videotape. This task was used only to orient the drivers; the data collected from the self–timing procedure was not used in hypothesis testing. As a result, the possibility that the self–timing procedure may add additional error into the data was not a concern. This was deemed to be the most efficient procedure. In addition, this task provided subjects with an immediate comparison to the automated routing functions of the TravTek system.

A few minutes were taken to familiarize subjects with what they needed to look for in the video– tapes as some subjects had heard of the TravTek system but none had ever used it. The overhead with examples of TravTek system's information presentation formats (figure 5) was reviewed.

As a final step, the list of potential feature headings (bold items) in the TravTek system capabilities section was read aloud to subjects while they followed along on their questionnaire copies (appendix B, p. 178). Subjects then had the opportunity to ask for clarification of item definitions before they viewed the videos.

When all questions had been answered, the experimenter instructed the subjects to turn over their experimental packets and watch an edited AAA TravTek system video, paying particular attention to the types of features that were previously reviewed. The experimenter told subjects that the video image and the TravTek system voice might not always be clear, but to try not to be distracted by either. The unclear image and voice presentations were not random. The unclear voice refers to the synthesized voice used for audio messages that was not as clear as natural speech. The lack of image clarity was due to loss of image quality from using a second generation source. This loss of quality led to slightly blurred symbology and words. This might lead to decreased user acceptance relative to actual traffic drivers who had more time to adapt to the synthesized voice and a clearer visual image. As soon as the first video ended, subjects were instructed to complete the questionnaire as quickly as possible, but were told that accuracy was most important. They were also told to work independently and were allowed to leave the room to take a break as soon as they finished the first set of questions. They were asked to remain quiet if they chose to remain in the experimental testing room and not to look ahead in their experimental packets. The experimenter answered any questions and picked up the completed questionnaires.

All subjects completed the TravTek system questionnaires (TravTek System Capabilities through TravTek User Test Questions, appendix B, pp. 178–214) after completing both the routing task and watching the video. Subjects were given another break during which the experimenter collected the questionnaires and prepared for experiment 1B.

Subjects were tested in approximately 14 small groups. The group size ranged from 3 to 12 subjects. Some groups were composed of a single age group (younger, older) of subjects and some were composed of mixed age groups. Most groups contained both male and female subjects. Education level varied some within each group. There was relatively little ethnic diversity in each group.

The total time for subjects to complete the study ranged from 1 h and 53 min to approximately 3 h and 15 min. Younger groups of subjects, as a whole, took less time than older subjects (although there were exceptions for individuals). The longer completion time primarily for older subjects may be correlated with greater fatigue relative to younger and faster subjects. Indeed, we did not present a third video and set of questions as being beyond the endurance of older drivers. Table 4 shows the experimental tasks and the time range for each activity.

Table 4. Time table for experiment 1.

Top

EXPERIMENT 1B METHOD

Subjects

The same subjects that participated in experiment 1 also participated in experiment 1B. Two of the subjects in experiment 1 had to leave before completing experiment 1B due to other commitments. In addition, 21 commercial vehicle operators participated only in experiment 1B. All commercial drivers were male and less than 54 years old. A few ethnic groups were represented. Education level had some variation (less than high school through some college).

Apparatus

The apparatus for this experiment consisted of a Dukane overhead projector, an InFocus LCD monitor, and an AST 486/25 laptop computer with a color display. The computer screen image was output through the LCD monitor and overhead projector in order to create a display large enough for group viewing.

Materials

The informed consent sheet and the demographic and technology use questionnaires were completed in experiment 1 by private drivers. Commercial drivers completed an informed consent form and a similar demographic questionnaire (appendix B, p. 245, section G). Zagat-Axis CityGuide for Windows (1991) software for New York City was installed on the laptop computer for demonstration purposes. The CityGuide system uses a mapping data base from Etak, Inc., and survey data from Zagat Survey. The program can be used to plan routes and access information about hotels, restaurants, and landmarks. A questionnaire was developed for this experiment that paralleled the items asked in experiment 1 (appendix B, p. 240). The independent and attribute variables that were measured include AGE, GENDER, demographic variables, and technology use. Table 5 summarizes the independent variables of experiment 1B.

Dependent variables were assessed by questionnaires that paralleled the TravTek system questionnaires and are also shown in appendix B:

• CityGuide System Capabilities.

• CityGuide System Feature Desirability.

• CityGuide Demonstration Fidelity.

• CityGuide: Trust & Self-Confidence.

• CityGuide User Acceptance Issues.

• CityGuide Perceived Usefulness.

• CityGuide Perceived Ease of Use.

• CityGuide User Test Questions.

Table 5. Independent variables in experiment 1B.

Table 6 summarizes the dependent variables in experiment 1B.

Table 6. Dependent variables in experiment 1B.

Procedure

Subjects were tested in the same groups described in experiment 1, except for the commercial vehicle operators, who only participated in experiment 1B. Following completion of experiment 1, a researcher demonstrated the use of the CityGuide system software. The 15-min demonstration consisted of a brief introduction on how the software is used and two scenarios. Scenario 1 was used to demonstrate how to find a route between LaGuardia Airport and a hotel near the Metropolitan Museum of Art. Subjects were shown text instructions and a route map for the route generated by the CityGuide system, and were also given information about the hotel. Examples of the instructions, hotel information and city map are shown in figures 6 through 8.

In scenario 2, the researcher described how to find the Gershwin Theater and a restaurant near the theater. The researcher followed the same script each time and the demonstration was presented to maintain consistency. Subjects were given an opportunity to ask questions following the demonstration. They were then asked to complete the CityGuide system questionnaires (appendix B, pp. 213-239). When subjects were finished with this questionnaire, they were paid and thanked for their participation.

Commercial vehicle drivers filled out informed consent sheets and a demographic survey (that contained most of the items from the private driver demographic survey as well as the items specifically related to commercial vehicle operation) prior to the demonstration (see appendix B, p. 240).

Before the demonstration began, the experimenter instructed the commercial vehicle drivers to view the CityGuide system with regard to how it might be used by commercial drivers (even though the scenarios focused on private vehicle applications).

Top

EXPERIMENT 1 RESULTS

Examined in the analyses were 5 objective rating dependent variables and 155 subjective rating dependent variables. The five objective dependent variables were the percent correct scores for the TravTek system capabilities items: (1) trip planning, navigation, and routing; (2) services and attraction information; (3) in–vehicle road sign information; (4) safety and warning information; and (5) a total for all system capabilities. These scores indicated the drivers' understanding of the TravTek system. A factor analysis of the percent correct scores from a specific binary–form questionnaire item category was used to create two composite variables. The remaining subjective rating dependent variables were factor–analyzed as related groups of questionnaire items and used to create 18 composite variables. These composite variables then succinctly represent the several individual variables in a way that provides greater statistical reliability compared to the individual variables.

The composite variables provide the basis for examining the relationships in figure 9. The directional links in figure 9 are hypothetical; they are based upon Battelle's analysis of the consumer acceptance model described in chapter 1. Those models too are global and not immediately useful. The new model uses local concepts that can be evaluated directly from the set of questions administered to drivers. This experiment tests the hypothetical relationships represented in figure 9. The goal of experiment 1 is to understand what variables drive consumer acceptance; that is, what feature patterns (shaded box) do drivers want?

The analyses were conducted in three phases using the SPSS/PC+, version 5.0, software package. In the first phase, descriptive statistics were calculated for items taken from a survey used in the TravTek system demonstration project. In the next phase, a repeated–measures analysis of variance (ANOVA) was used to examine the relationships between AGE, GENDER, VIDEO (1 = the AAA TravTek system tutorial, 2 = an on–road TravTek system demonstration) and the total percent correct score for the TravTek system capabilities items. The variable for AGE consisted of two levels: younger drivers (18 to 54 years) and older drivers (55 to 85 years). In the final phase of the analyses, three parts aimed at identifying the relationships among the variables shown in figure 9. Results from each phase of the analyses are described below. ANOVA tables are presented in appendix C (pp. 251–261).

• Phase 1. TravTek System User Test Questions –– Descriptive Statistics

• Phase 2. Age, Gender, Video and Mean Percent Correct Scores on the TravTek System User Test

• Phase 3. Identifying Relationships Among Variables

• Feature Patterns

• Composite Variable Evaluations

• Relationships of Feature Patterns with Specified Variables

• Fidelity and Attention

Phase 1. TravTek System User Test Questions – Descriptive Statistics

In the first phase of the analyses, a subset of questionnaire items was taken from Your TravTek System Driving Experience, the survey given to drivers who participated in the TravTek System Demonstration Project in Orlando, Florida. These items are listed in table 7 and are shown in appendix B under the heading TravTek User Test Questions (p. 207). The items are related to information presentation formats (i.e., guidance map, route guidance, voice guidance), usefulness of the system in various driving situations, value of the TravTek system in terms of how much a driver would pay for it, and traffic-related factors. Relevant means were calculated for these items and are shown in figure 10 through figure 30. Then a repeated-measures analysis ANOVA was performed for each of these items.

Table 7. TravTek system user test questions.

Figure 10 (TRAV1A) shows mean ratings for the TravTek system's guidance display ease of learning as a function of AGE and VIDEO. Error bars in this and subsequent figures indicate standard deviations. The ANOVA resulted in a significant main effect for AGE, F(1,102) = 18.97, p < 0.001, and a significant main effect for VIDEO, F(1,102) = 5.78, p < 0.018. Younger drivers (mean = 4.8) rated the TravTek system's guidance display easier to learn than older drivers (mean = 3.9). Ease of learning ratings increased from video 1, the AAA TravTek system tutorial videotape (mean = 4.3) to video 2, the on-road TravTek system demonstration videotape (mean = 4.6).

Figure 11 (TRAV1B) shows mean ratings for the TravTek system's guidance display ease of use as a function of GENDER and AGE. A significant GENDERxVIDEO interaction occurred, F(1,99) = 5.26), p < 0.024. Female drivers' ratings increased more from video 1 (mean = 4.0) to video 2 (mean = 4.6) than male drivers' ratings from video 1 (mean = 4.2) to video 2 (mean = 4.4). A main effect for AGE, F(1,99) = 16.86, p < 0.001, and a main effect for VIDEO, F(1,99) = 20.68, p < 0.001, were also significant. Younger drivers (mean = 4.6) rated the system's guidance display easier to use than older drivers (mean = 3.9). Ease of use ratings increased from video 1 (mean = 4.1) to video 2 (mean = 4.5).

Figure 12 (TRAV1C) shows mean ratings for the TravTek system's guidance display usefulness. The ANOVA resulted in a significant main effect for AGE, F(1,99) = 7.71, p < 0.007, and a significant main effect for VIDEO, F(1,99) = 7.05, p < 0.009). Younger drivers' ratings were higher (mean = 4.7) than older drivers' ratings (mean = 4.2). Usefulness ratings increased from video 1 (mean = 4.4) to video 2 (mean = 4.6).

Figure 13 (TRAV2A) shows mean ratings for the TravTek system's route map ease of learning. A three-way interaction for AGE, VIDEO, and GENDER was significant, F(1,102) = 5.26, p < 0.024. Older female drivers rated the route map easier to learn after video 2. Both age groups of male drivers and the younger age group of female drivers rated the route map as less easy to learn after video 2. The two-way AGExVIDEO interaction, F(1,102) = 5.12, p < 0.026, captures most of the variance of the three-way interaction. Younger drivers' ratings of ease of learning decreased from video 1 (mean = 4.7) to video 2 (mean = 4.4). Older drivers' ratings increased from video 1 (mean = 3.9) to video 2 (mean = 4.1).

Figure 14 (TRAV2B) shows mean ratings for the TravTek system's route map ease of use. A significant AGExVIDEO interaction occurred, F(1,99) = 4.07, p < 0.046. Younger drivers rated the route map as easier to use after video 1 (mean = 4.6) than after video 2 (mean = 4.2). Older drivers rated the route map as easy to use after video 1 (mean = 3.8), but more easy to use after video 2 (mean = 3.9). The main effect for AGE was also significant, F(1,99) = 8.51, p < 0.004, with younger drivers giving higher ratings than older drivers.

Figure 15 (TRAV2C) shows mean ratings for the TravTek system's route map usefulness. A three–way interaction for AGE, GENDER, and VIDEO was significant, F(1,100) = 6.68, p < 0.011. Older female drivers rated the route map as slightly more useful after video 2. Both age groups of male drivers and the younger age group of female drivers rated the route map as less useful after video 2. The two–way AGExVIDEO interaction, F(1,100) = 6.68, p < 0.011, captures most of the variance of the three–way interaction. As figure 15 (TRAV2C) illustrates, younger drivers' ratings of usefulness decreased from video 1 (mean = 4.8) to video 2 (mean = 4.3), whereas older drivers' ratings of usefulness increased slightly from video 1 (mean = 4.1) to video 2 (mean = 4.2). The main effect for AGE was significant, F(1,100) = 4.85, p < 0.030. Younger drivers (mean = 4.5) rated the route map usefulness higher than older drivers (mean = 4.2).

Figure 16 (TRAV3A) shows mean ratings for the TravTek system's voice guide feature ease of learning. The ANOVA resulted in a significant main effect for AGE, F(1,102) = 5.80, p < 0.018, and a significant main effect for VIDEO, F(1,102) = 10.06, p < 0.002. Younger drivers (mean = 4.8) rated the TravTek system's voice guide feature as easier to learn relative to older drivers (mean = 4.4). Ease of learning scores increased from video 1 (mean = 4.5) to video 2 (mean = 4.8).

Figure 17 (TRAV3B) shows mean ratings for the TravTek system's voice guide feature ease of use as a function of AGE and VIDEO. The ANOVA resulted in a significant main effect for AGE, F(1,100) = 6.08, p < 0.015, and a significant main effect for VIDEO, F(1,100) = 12.00, p < 0.001. Younger drivers (mean = 4.8) rated the TravTek system's voice guide feature as easier to use than older drivers (mean = 4.3). Ease of use ratings increased from video 1 (mean = 4.4) to video 2 (mean = 4.8).

Figure 18 (TRAV3C) shows mean ratings for the TravTek system's voice guide feature usefulness ratings. A significant main effect occurred for VIDEO, F(1,100) = 9.30, p < 0.003, with usefulness ratings increasing from video 1 (mean = 4.1) to video 2 (mean = 4.6).

Figure 19 (TRAV4) shows mean ratings for display preference. A significant main effect occurred for VIDEO, F(1,99) = 39.01, p < 0.001, with ratings after video 1 (mean = 3.2) tending toward a preference for the route map. Ratings after video 2 (mean = 4.0) tended toward a preference for the guidance display.

Figure 20 (TRAV5A) shows mean ratings for the TravTek system's overall ease of learning. A main effect for AGE was the only significant result, F(1,102) = 23.36, p < 0.001. Younger drivers' ratings (mean = 4.7) indicated that they found the TravTek system easier to learn than older drivers (mean = 3.9).

Figure 21 (TRAV5B) shows mean ratings for the TravTek system's overall ease of use. The ANOVA resulted in a significant main effect for AGE, F(1,100) = 13.95, p < 0.001, and a significant main effect for VIDEO, F(1,100) = 5.35, p < 0.023. Younger drivers' ratings of overall ease of use were higher (mean = 4.6) than older drivers' ratings (mean = 3.9). Ease of use ratings increased from video 1 (mean = 4.2) to video 2 (mean = 4.4).

Figure 22 (TRAV5C) shows mean ratings for the TravTek system's overall usefulness. A main effect for AGE was the only significant result, F(1,100) = 4.74, p < 0.032. Younger drivers' mean ratings (mean = 4.6) were higher than older drivers' mean ratings (mean = 4.2).

Figure 23 (TRAV6A) through figure 25 (TRAV6C) show the percentages of drivers responding "yes" (as a function of AGE and VIDEO) that they would find the TravTek system useful for (a) at-home daily driving, (b) out-of-town vacation driving, and (c) out-of-town business driving. In general, a higher percentage of younger drivers indicated that the TravTek system would be useful in each situation than was indicated by older drivers.

Figure 23 (TRAV6A) shows the percentage of drivers indicating that the TravTek system would be useful for at-home daily driving. The percentage of younger drivers indicating that the TravTek system would be useful for video 1 was 24.3 percent and for video 2 was 27.6 percent. The percentage of older drivers indicating that it would be useful for video 1 was 12.6 percent and for video 2 was 14.3 percent.

Figure 24 (TRAV6B) shows the percentage of drivers indicating that the TravTek system would be useful for out–of–town vacation driving. The percentage of younger drivers indicating that the TravTek system would be useful for video 1 was 57.6 percent and for video 2 was 59.6 percent. The percentage of older drivers indicating that it would be useful for video 1 was 39.6 percent and for video 2 was 37.5 percent.

Figure 25 (TRAV6C) shows the percentage of drivers indicating that the TravTek system would be useful for out–of–town business driving. The percentage of younger drivers indicating that the TravTek system would be useful for video 1 was 60.2 percent and for video 2 was 60.2 percent. The percentage of older drivers indicating that it would be useful for video 1 was 35.0 percent and for video 2 was 36.9 percent.

Figure 26 (TRAV7) shows the amount drivers indicated they were willing to pay for the TravTek system after each video. The mean amounts younger drivers were willing to pay following video 1 was $850 and following video 2 was $837. The mean amounts older drivers were willing to pay following video 1 was $656 and following video 2 was $746.

Figure 27 (TRAV8A) through figure 30 (TRAV8D) show mean ratings of the importance of traffic-related factors. In general, subjects ranked highway/traffic safety and relief of highway congestion as more important factors. They ranked energy conservation and environmental quality as less important factors.

Figure 27 (TRAV8A) shows the mean rating for the importance of energy conservation. A significant GENDERxVIDEO interaction occurred, F(1,96) = 5.93, p < 0.017. Male drivers' ratings remained the same from video 1 to video 2 (mean = 2.8). However, female drivers' ratings increased from video 1 (mean = 2.7) to video 2 (mean = 3.0), indicating a slight decrease in the mean rating of the importance of the problem.

Figure 28 (TRAV8B) shows the mean rating for the importance of environmental quality. Younger subjects' mean rating for video 1 was 2.9 and for video 2 was 3.0. Older subjects' mean ratings for video 1 was 2.7 and for video 2 was 2.9.

Figure 29 (TRAV8C) shows the mean rating for the importance of highway/traffic safety. Younger subjects' mean rating for video 1 was 1.9 and for video 2 was 2.0. Older subjects' mean rating for video 1 was 1.8 and for video 2 was 1.7.

Figure 30 (TRAV8D) shows the mean rating for the importance of relief of highway congestion. Younger subjects' mean rating for video 1 was 2.0 and for video 2 was 1.9. Older subjects' mean rating for video 1 was 2.0 and for video 2 was 1.8. These non-significant results are presented to facilitate their later comparison with results of the TravTek System Demonstration Project (Orlando site).

Experiment 1 Results

Phase 2. Age, Gender, Video, and Mean Percent Correct Scores on the TravTek System User Test

The second phase of the analyses examined AGE, GENDER, and VIDEO relationships for the five objective dependent variables, the mean percent correct scores for each of the TravTek system capabilities items. The mean percent correct scores for each of the sets of items showed no significant effects. However AGE and VIDEO influenced the mean percent correct scores for the system as a whole and are presented in figure 31 as the scores for all system capabilities for younger and older drivers after each video presentation. For younger subjects (18 to 54 years), mean percent correct scores were 72.8 and 69.7 for video 1 and video 2, respectively. For older subjects (55 to 85 years), mean percent correct scores were 64.6 and 64.2 for video 1 and video 2, respectively. An interaction between AGE and VIDEO indicates that younger subjects' scores decreased from video 1 to video 2, while older subjects' scores changed very little, F(1,102) = 8.50, p < 0.004.

Experiment 1 Results

Phase 3. Identifying Relationships Among Variables

The last phase of the analyses was conducted in three steps aimed at identifying relationships among the variables shown in figure 9. During the first step, the feature patterns were determined via a factor analysis of driver responses on the TravTek System Features Desirability questionnaire. Mean values and other descriptive results for individual features were also developed. During the second step, composite variables were developed from individual questionnaire items. The first-order relationships among these composite variables were then explored correlationally. During the third step, multiple correlational analysis evaluation of the relationships between the individual feature patterns was done. This final step resulted in an understanding of the connection between the feature patterns, demographic variables (AGE and GENDER) and the derived composite variables (i.e., tolerance patterns, system trust, etc.). Results of the three steps of this analysis are described below.

Experiment 1 Results

Feature Patterns

The feature patterns were derived and verified from the results of the respective factor analyses of the 52 unfamiliar– and 52 familiar–city responses on the TravTek System Feature Desirability questionnaire (appendix B, p. 183). The primary focus was on the unfamiliar–city patterns because of expectations that drivers would require the most comprehensive sets of features in unfamiliar cities (and unfamiliar portions of familiar cities). The derivation and verification processes are described in the following subsections.

Deriving the Feature Patterns

As a first step in the analyses to derive the feature patterns, the mean values for the TravTek system feature desirability items were calculated. Features with mean desirability ratings greater than or equal to 1.5 made up the most desired features category. Table 8 lists these 14 features. Features with mean desirability ratings less than or equal to 0.5 made up the least desired features category. Table 9 lists these eight features. To summarize, the most desired features were current position, congestion information, other real–time traffic information, and emergency aid requests. All but three of the most desired features were for unfamiliar–city applications. Voice advertising information was the only unfamiliar–city feature pattern in the least desired features category. Driver comments during debriefing indicated that they did not want to be distracted by voice messages for advertising. The majority of least desired features related to the coordination of travel and the advertising information. The position/location, parking information, and only signs relevant to the driver's pre–planned route features were the remaining least desired features.

These results do not explain why certain feature patterns are more or less desirable. More specifically, they do not reveal how variables such as driver characteristics, attitudes, and understanding influence feature pattern desirability. This factor analysis method was used to first identify feature patterns and then identify variables that influence these feature patterns. This factor analysis approach reduced the numbers of individual analyses from the total number of feature patterns (52 for unfamiliar–city driving and 52 for familiar–city driving) to a more manageable number of feature patterns (six for unfamiliar–city driving). Reducing the number of analyses avoided a large experiment–wide error rate and provided a more parsimonious model for driver acceptance. Moreover, it was expected that these feature patterns would represent integrated functional groupings that drivers would expect in the actual final ATIS design.

Table 8. TravTek system most desired features.

Table 9. TravTek system least desired features.

Table 10 summarizes the method for determining the feature patterns from the unfamiliar–city responses following an approach successful in previous time–course investigations (Harmon, 1976; Bittner, 1992). This method featured a Scree–test cutoff (Harmon, 1976) that is typically more parsimonious than a unity eigenvalue cut–off (hence the minimum eigenvalue was 1.71 in this case). Additionally, the Varimax procedure was used to orthogonally rotate the resulting factors to facilitate interpretation (Harmon, 1976).

Table 10. Method for determination of feature patterns.

The results of the principal factor analysis were six feature patterns summarized in table 11. It can be seen that a Basic Map display is indicated by the first feature pattern (Factor I) with the other feature patterns representing various overlays of features (e.g., Voice, Text/Icon). Summarized in this table are the variables most associated with each feature pattern. The numbers in parentheses show the correlations between individual variables and the relative feature patterns. These feature patterns include mixes of feature patterns drawn from IRANS, IMSIS, etc. This was consistent with driver comments that clusters of feature patterns from IRANS and the other systems go together functionally.

Table 11. Desired feature patterns.

Verifying the Feature Patterns

The derived feature patterns were "verified" for the familiar–city responses by comparison of unfamiliar– and familiar–city factor scores. The results of the familiar–city responses were first factor analyzed following the unfamiliar–city approach described in table 12. The familiar–city factor analysis revealed six factors that, on–the–surface, appeared consistent with those summarized in table 11.

To more conservatively evaluate this consistency, the separate feature pattern scores of unfamiliar– and familiar–city results were cross–correlated. Table 12 summarizes the results of the cross–correlation of the respective sets of six unfamiliar– and familiar–city factor scores. Although the factors are not in identical order, the dominant weights in each row and column indicate that the unfamiliar–city factor scores generally had substantial overlaps with those for the familiar–city factor scores (r > 0.52, p < 0.001). The ordering is not important as it only represents the relative importance of the various feature patterns that are expected to vary between unfamiliar and familiar cities. Correspondence requires only moderate substantial correlations between similar feature patterns.

Table 12. Cross–correlations between unfamiliar–city and familiar–city factor scores of the feature patterns.

Further, with the exception of the first of the unfamiliar–city feature patterns (Basic Map) that is somewhat split between the Familiar–city Basic Map (Feature Pattern I) and Map Simplification (Feature Pattern V), it is clear that the individual unfamiliar–city feature patterns typically are uniquely identified by single, familiar–city patterns. The split was consistent with drivers desiring simplified feature patterns in a familiar–city (when traversing familiar streets). These results generally support the validity of unfamiliar–city patterns as representative for both desired familiar– and unfamiliar–city feature patterns.

Experiment 1 Results

Composite Variable Evaluations

Composite variables were evaluated in two stages. First, using factor analytic methods, composite variables were derived from the following six sets of relevant questionnaire responses:

• Fidelity (appendix B, pp. 188–189, items 2, 4, 5, 8, 9).

• Attention (appendix B, pp. 188–189, items 1, 3, 6, 7).

• Capabilities Understanding (appendix B, pp. 178–182, 4 sets of items and total).

• System Trust (appendix B, pp. 192–195, items 1a–7a).

• Self–Confidence (appendix B, pp. 192–195, items 1b–7b).

• Tolerance Patterns (appendix B, pp. 190–191, all but item 1).

Second, the first–order relationships among the derived composites were then explored in terms of the model shown in figure 9. The factor analyses and correlations are described in the following subsections.

Derivation of the Composite Variables

Table 13 summarizes the method used for deriving the composite variable factors. This method is analogous to that employed earlier to derive the feature patterns.

Table 13. Method for determination of composite variables.

Results of applying this method to each of the six sets of composite variables are summarized in the following:

Fidelity––The PFA of the five fidelity questionnaire items revealed a single factor variable, with an eigenvalue greater than unity, that explained 65.6 percent of the total variation. This composite variable was given the short title FIDELITY for identification purposes in the analyses that follow.

Attention––The PFA of the four attention questionnaire items revealed a single factor variable, with an eigenvalue substantially greater than unity, that explained 64.2 percent of the total variation. This composite variable was given the short title ATTENT.

Capabilities Understanding––The PFA of the summary scores for the four sections of capabilities understanding questionnaire items revealed two factor variables, with eigenvalues greater than unity (i.e., 1.5 and 1.1) that together explained 64.8 percent of the total variation. The first of these reflected a general understanding of all but the safety–related items, while the second reflected safety–related items. These composite variables were given the short titles UNDRSTD1 and UNDRSTD2, respectively.

System Trust––The PFA of the seven system trust questionnaire items revealed a single factor variable, with an eigenvalue greater than unity, that explained 49.7 percent of the total item variation. This composite variable was given the short title SYSTRUST.

Self–Confidence––The PFA of the seven self–confidence questionnaire items resulted in a single factor, with an eigenvalue greater than unity, that explained 51.9 percent of the total variation. This composite variable was given the short title SELFCON.

Tolerance Patterns––The PFA of the seven tolerance questionnaire items revealed two factor variables, with eigenvalues greater than unity (i.e., 2.1 and 1.5), that together explained 51.1 percent of the total item variation. The first of these was related to the proportions of trips that prediction failures could be tolerated, whereas the second was related to tolerances for various errors in the arrival times. These composite variables were given the short titles TOLPAT1 and TOLPAT2, respectively.

These PFA results were generally in keeping with the conceptual expectations, although two feature patterns occasionally emerged where one might have been expected (e.g., with regard to UNDRSTD1 and UNDRSTD2). The results, however, were consistent with somewhat broadened concepts (e.g., some older drivers give relatively greater attention to safety–related information). This theoretical consistency pointed toward the evaluation of the first–order correlations among the various composite variables and the multivariate evaluation of their relationships with feature patterns. The results of these evaluations are described below.

Composite Variable First–Order Relationships

Table 14 summarizes the first–order correlations among the factor scores for the feature patterns computed for the eight composite variables. The correlations between the variables indicate that they tend to be only moderately related (i.e., r > 0.54). However, these first–order correlations point out relationships that impact driver judgments of the desirability of various feature patterns.

Table 14. Composite variable correlations.

Experiment 1 Results

Relationships of Feature Patterns with Specified Variables

The third step was directed at the overall relationships among each of the six feature patterns and the selected variables shown in figure 9. Hence, six multiple correlation analyses were conducted that evaluated the joint relationships of each of the feature "patterns" scores with the following:

• Demographic variables (AGE, GENDER, and their interaction AGExGEN).

• Capabilities understanding variables (UNDRSTD1 and UNDRSTD2).

• System trust variable (SYSTRUST).

• Self-confidence variable (SELFCON).

• Tolerance pattern variables (TOLPAT1 and TOLPAT2).

• VIDEO (whether after first or second video presentation).

First, an initial multiple correlation was performed to identify relationships among the feature pattern's scores and all of the previously listed variables. Each of these initial analyses will be shown in a table. Then, the initial multiple correlation models were evaluated using a step–down procedure. Each of these final correlation models will also be shown in a table. A description of the table headings is given below:

• VARIABLE = the variable name.

• "B" = the raw weight of the variable in the model.

• "SE B" = its standard error.

• "BETA" = the standard score model weight.

• "T" = the t-test value for the term (T).

• "SIG T" = the significance (p) value.

"B" is the raw weight of the variable in the regression equation:

y_i = constant(additive) + _j B_jx_ji

where Y_i is the driver's score on a variable, B_j is the jth variable's "B" weight, and X_ji is the ith driver's score on variable j.

Results for the six feature patterns are presented below in the order of their earlier numbering (i.e., Factors I to VI).

Basic Map Feature Pattern (Factor I)

The initial analysis revealed a very highly significant (p < 10^–6) multiple correlation among the 10 independent variables and the Basic Map feature pattern: R = 0.489. Table 15 summarizes the model resulting from this analysis in terms of the raw weight of a term in the model (B); its standard error (SE B); the standard score model weight (BETA), the t–test value for the term (T) and its associated significance (p) value (SIG T). In addition to the additive constant (Constant), several model variables were initially significant (ps < 0.05): UNDRSTD2, UNDRSTD1, and AGE. Others (e.g., SYSTRUST) approach significance (p < 0.06) and some appeared clearly unrelated to the model (e.g., VIDEO with p > 0.7). These results suggested the examination of simplified multiple correlation models that might better reveal the relationships with the Basic Map feature pattern.

Table 15. Basic map feature pattern initial analysis summary.

Simplified multiple correlation models were evaluated using a step–down procedure that progressively eliminated variables with the largest significance levels, those greater than p = 0.10 (Norysis, 1992). This procedure revealed a very highly significant (p < 10^–6) multiple correlation among the five remaining independent variables and the Basic Map feature pattern: R =  0.472. Table 16 summarizes the model resulting for this analysis and shows that, in addition to the additive constant (Constant), significant (p < 0.05) model variables included UNDRSTD2, UNDRSTD1, SYSTRUST, and AGE.

Table 16. Basic map feature pattern final analysis summary.

Figure 32 illustrates the relationships among these variables in the context of the other potential influences. By multiplying the B weight of AGE by the AGE code, these results indicate that the Basic Map feature pattern was seen by older drivers, as a whole (AGE = 2 x –0.53 = –1), to be less desirable than by younger drivers (AGE = 1 x –0.53 = –0.53). However, AGE effects can be offset with greater understandings of the system features (as indicated by positive B weights for UNDRSTD2 = 0.16 and UNDRSTD1 = 0.20). Increases in SYSTRUST would also add to the desirability of this basic pattern (as indicated by a positive B weight = 0.13). These results point out the importance of education and experience for enhancing the desirability of the Basic Map feature pattern.

Voice Feature Pattern (Factor II)

Initial analysis revealed a very highly significant (p < 10^–5) multiple correlation among the 10 independent variables and the Voice feature pattern: R = 0.445. Table 17 summarizes the model resulting from this analysis in the same terms as described above. In addition to the additive constant (Constant), significant (p < 0.05) model variables initially included VIDEO and SELFCON. Others (e.g., UNDRSTD2) range from the suggestive (p < 0.09) to the clearly unrelated (e.g., UNDRSTD1 with p > 0.7). These results also suggested the examination of simplified multiple correlation models for the Voice feature pattern.

Table 17. Voice feature pattern initial analysis summary.

Simplified multiple correlation models were evaluated using the same (Norysis, 1992) step–down procedure described earlier. This procedure revealed a highly significant (p < 10^–5) multiple correlation among three remaining independent variables and the Voice feature pattern: R = 0.424. Table 18 summarizes the model resulting for this analysis and shows that, in addition to the additive constant (Constant), significant (ps < 0.001) model variables included VIDEO and SELFCON.

Table 18. Voice feature pattern final analysis summary.

Figure 33 illustrates the relationships among these variables in the context of the other potential influences. Safety understanding (UNDRSTD2) remained insignificant albeit suggestive (p < 0.07) that enhancing the TravTek system safety feature understanding (B = 0.11) would increase the desirability of the Voice feature pattern. Contrasting with this, the results indicate that drivers with high levels of SELFCON tend to find the Voice feature pattern less desirable (B = –0.21). However, strongly overriding both of these is the strong influence (B = 0.71) of video 2 over video 1 in increasing the desirability of the Voice feature pattern. This influence, as indicated by post–study driver comments, resulted from the strong illustration of voice guidance in video 2 (Orlando trip). This result points out the importance of a specific voice illustration for increasing the perceived desirability of the Voice feature pattern.

Text/Icon Feature Pattern (Factor III)

The initial analysis revealed a highly significant (p < 0.001) multiple correlation among the 10 independent variables and Text/Icon feature pattern: R = 0.376. Table 19 summarizes the model resulting from this analysis. Significant (p < 0.03) model variables initially included SELFCON, GENDER, AGExGEN, and SYSTRUST. Others (e.g., AGE) ranged from the nearly significant (p < 0.06) to the clearly unrelated (e.g., VIDEO with p > 0.69). These results also suggested examination of simplified multiple correlation models for the Text/Icon feature pattern.

Table 19. Text/Icon feature pattern initial analysis summary.

Simplified multiple correlation models were evaluated using the same (Norysis, 1992) step-down procedure described earlier. This procedure revealed a very highly significant (p < 0.001) multiple correlation among three remaining independent variables and the Text/Icon Feature Pattern: R = 0.356. Table 20 summarizes the model resulting for this analysis.

Table 20. Text/Icon feature pattern final analysis summary.

Figure 34 illustrates the relationships among these variables in the context of the other potential influences.

Significant (p < 0.02) model variables included UNDRSTD2, SELFCON, GENDER, and AGExGEN. Of these, greater SELFCON was associated with decreased desirability for the Text/Icon feature pattern (B = -0.21), while greater understanding of system safety features (UNDRSTD2) was associated with enhanced desirability (B = 0.17). There was also a tendency for increased desirability for the Text/Icon feature pattern with increased SYSTRUST (p = 0.06). More striking than these, however, are the compound effects of GENDER (1 = male and 2 = female), AGE (1 = younger and 2 = older), and AGExGEN. Taking these and the additive constant (Constant) into account, the net effects are as shown in table 21. Because AGE and GENDER are fixed, these results point out the importance of increasing system safety understanding (UNDRSTD2) and SYSTRUST to increase the desirability of the Text/Icon feature pattern, particularly for older females.

Table 21. Gender and age interaction on desirability.

Coordination of Travel Information Feature Pattern (Factor IV)

Initial analysis revealed a very highly significant (p < 0.003) multiple correlation among the 10 independent variables and the Coordination of Travel Information feature pattern: R = 0.390. Table 22 summarizes the model resulting from this analysis in the same terms described above. Significant (ps < 0.02) model variables initially included UNDRSTD1, SYSTRUST, and VIDEO. Many other variables appeared clearly unrelated (e.g., TOLPAT1 with p > 0.9). These results also suggested examination of simplified multiple correlation models for the Coordination of Travel Information feature pattern.

Table 22. Coordination of travel information feature pattern initial analysis summary.

Initial analysis revealed a very highly significant (p < 0.003) multiple correlation among the 10 independent variables and the Coordination of Travel Information feature pattern: R = 0.390. Table 22 summarizes the model resulting from this analysis in the same terms described above. Significant (ps < 0.02) model variables initially included UNDRSTD1, SYSTRUST, and VIDEO. Many other variables appeared clearly unrelated (e.g., TOLPAT1 with p > 0.9). These results also suggested examination of simplified multiple correlation models for the Coordination of Travel Information feature pattern.

Table 23. Coordination of travel information feature pattern final analysis summary.

Figure 35 illustrates the relationships among these variables in the context of the other potential influences. The first two of these results shows a decreased desirability with increased general understandings of features (UNDRSTD1, B = –0.24) and video 2 (Orlando trip) (B = –0.32).

The decreased desirability after video 2 can be posited as due to undemonstrated features (associated with the Coordination of Travel Information feature pattern) being obscured by the features that were demonstrated. This explanation is consistent with earlier results indicating the desirability for the Voice feature pattern increased after the associated features were demonstrated in video 2. Of course, higher levels of SYSTRUST could offset the obscuring effects of the demonstrated features. Salient demonstrations of the Coordination of Travel Information feature pattern might increase the desirability, although this remains to be verified in later research.

Map Simplification Feature Pattern (Factor V)

Initial analysis revealed a significant (p < 0.03) multiple correlation among the 10 independent variables and the Map Simplification feature pattern: R = 0.309. Table 24 summarizes the model resulting from this analysis in the same terms as described above. In addition to the additive constant (Constant), model variables initially included UNDRSTD1 and TOLPAT1. Others (e.g., VIDEO) ranged from the suggestive (p < 0.09) to the clearly unrelated (e.g., GENDER with p > 0.8). These results also suggested examination of simplified multiple correlation models for the Map Simplification feature pattern.

Table 24. Map simplification feature pattern initial analysis summary.

Simplified multiple correlation models were evaluated using the same (Norysis, 1992) step-down procedure described earlier. This procedure revealed a significant (p < 0.03) multiple correlation among four remaining independent variables and the Map Simplification feature pattern: R = 0.298). Table 25 summarizes the model resulting for this analysis and shows that UNDRSTD1, TOLPAT1, and AGE were significantly associated with the Map Simplification feature pattern.

Table 25. Map simplification feature pattern final analysis summary.

Figure 36 illustrates the relationships among the variables in the context of the other potential influences. These results show a decreased desirability with increased general understandings of features (UNDRSTD1, B = -0.17), perhaps as the advantages of a fuller spectrum of features become apparent. Simplification could have advantages for the older drivers (AGE) or others who would have a greater tolerance for system prediction failures (TOLPAT1). These results point to the importance of education to promote appreciation for a greater spectrum of features. Also, the results show the importance of the Map Simplification feature pattern for older and more prediction-tolerant drivers.

Monitoring & Emergency Response Feature Pattern (Factor VI)

The initial analysis revealed a marginally nonsignificant (p = 0.08) multiple correlation among the 10 independent variables and the Monitoring & Emergency Response feature pattern: R = 0.283. Table 26 summarizes the model resulting from this analysis. Examining this table, it was apparent that VIDEO was very highly significant (p < 0.0012), although the overall model was not significant. This, together with the body of clearly unrelated variables (e.g., SYSTRUST with p > 0.9), suggested the examination of a simplified multiple correlation model.

Table 26. Monitoring & emergency response feature pattern initial analysis summary.

Simplified multiple correlation models were evaluated using the step–down procedure described earlier (Norysis, 1992). This procedure revealed a significant (p < 0.0003) multiple correlation between VIDEO and the Monitoring & Emergency Response feature pattern: R = 0.248. Table 27 summarizes the model resulting for this analysis and shows that VIDEO is negatively associated (B = –0.50) with the Monitoring & Emergency Response feature pattern, indicating a decreased desirability after video 2.

Table 27. Monitoring & emergency response feature pattern final analysis summary.

Figure 37 illustrates the relationships between the variables in the context of the other potential influences. This decreased desirability can be posited as due to undemonstrated features (associated with the Monitoring & Emergency Response feature pattern) being obscured by the demonstrated features. Subsequently, the Monitoring & Emergency Response feature pattern desirability might be increased by salient demonstrations (as suggested earlier for the Coordination of Travel Information feature pattern).

Experiment 1 Results

Fidelity and Attention

The relationships among FIDELITY, ATTENT, CAPABILITIES UNDERSTANDING, SELFCON and SYSTRUST were not directly considered in the section that describes the Relationships of Feature Patterns with Specified Variables. This was, as may be recalled, because these variables were posited as only influencing the results through other variables. Therefore, only the direct relationships required analysis. Table 14 showed correlations among these variables. Of the five relationships predicted in figure 9, only the following three relationships were significant:

1) FIDELITY and ATTENT (r = 0.535, p < 0.001), 2) FIDELITY and SYSTRUST (r = 0.396, p < 0.001), and 3) FIDELITY and SELFCON (r = 0.151, p < 0.01). Figure 38 illustrates these relationships. Contrary to links hypothesized in figure 9, FIDELITY and ATTENT do not drive CAPABILITIES UNDERSTANDING.

Top

FHWA-RD-96-143