• Federal Highway Administration >
• Publications >
• Research Publications >
• 96143 >
• Sec2c.Cfm >
• Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies

Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies

DISCUSSION

Experiments 1 and 1B are part of an integrated series of studies directed at exploring questions of user acceptance and evaluation methodology. Initially discussed in this section are the individual results for examinations of two ATIS related systems: the TravTek system and the CityGuide system. This discussion addresses the effect of driver attitudes and system understanding on feature pattern desirability. In addition, it considers how driver characteristics influence feature pattern desirability, and how simulation fidelity influences driver understanding and preferences. The final section considers the integrated results of these experiments with regard to the broader questions concerning (1) feature pattern influence on user acceptance and (2) issues involved with evaluation methodology.

TravTek System Feature Pattern Desirability, Driver Attitudes, and System Understanding

A critical issue of user acceptance concerns the combinations of ATIS feature patterns that drivers would like to have. This experiment successfully determined the desirable TravTek system feature patterns. Six feature patterns emerged (table 11). These patterns were unrelated to the functional divisions of ATIS (into IRANS, IMSIS etc.) used by human factors professionals (Lee, Morgan, Wheeler, Hulse, & Dingus, 1997; Wheeler et al., 1997). Rather, they involved the: (1) Basic Map feature pattern; (2) Voice feature pattern; (3) Text/Icon feature pattern; (4) Coordination of Travel Information feature pattern; (5) Map Simplification feature pattern; and (6) Monitoring and Emergency Response feature pattern.

Table 44 summarizes the statistical relationships among the six TravTek system feature patterns and the variables that were hypothesized to influence the desired feature patterns (figure 56). This table also considers independent variables of AGE, GENDER, and VIDEO, where VIDEO refers to the differential effect of an AAA tutorial videotape and an on-road demonstration videotape. Figure 56 shows this information graphically.

Table 44. Summary of the TravTek system multiple–correlation results.

The model shown in table 44 and figure 56 demonstrates the importance of subjective variables on feature pattern desirability. It also shows that the importance of various variables depends on the specific feature patterns being considered. For example, system trust (SYSTRUST) significantly influenced the desirability of three feature patterns, most importantly the Coordination of Travel Information feature pattern (Feature Pattern IV). Self–confidence (SELFCON) also had a weak positive relationship to the Basic Map feature pattern (Feature Pattern I), which at first glance appeared to mildly contradict other research on automation (Lee & Moray, 1994; Experiment 2 of this report) that shows a negative relationship between SELFCON and the use of automation. In contrast, SELFCON was negatively related to two other feature patterns. Drivers with high SELFCON do not find voice instructions from ATIS to be as attractive as those with lower SELFCON. The apparent paradox of these differences in the influences of SELFCON may be the result of perceptions of the value of the feature patterns for promoting driver self–reliance. The Basic Map feature pattern (Feature Pattern I) may be perceived as directly supporting SELFCON since it reduces the need for stopping for directional assistance, whereas the reverse is true for the other feature patterns. The effects of the prediction failures (TOLPAT1) and arrival time mis–estimates (TOLPAT2) are consistent with expectations. Older drivers and others with less processing capabilities, who desire the Map Simplification feature pattern (Feature Pattern V), might be more tolerant of occasional prediction failure as they tend to have more experience with it in their lives. In contrast, intolerance for arrival time mis–estimates (TOLPAT2) is weakly negative related to preferences for the Text/Icon feature pattern (Feature Pattern III), perhaps reflecting the personality style of those preferring the exactness of the Text/Icon feature pattern (Feature Pattern III) (vs. Map analog). More specifically, the Text/Icon feature pattern (Feature Pattern III) is believed to be preferred by people who want to have access to more exact details of the driving situation so as to have greater control of the driving situation. Thus, drivers' tolerance for inaccuracies depends on both the type of inaccuracies (inaccurate predictions compared to inaccurate estimates of arrival times) and the specific feature pattern.

Similar to the differential effect of inaccuracy types, drivers' understanding consists of two factors, one reflecting understanding safety–related features (UNDRSTD2) and one reflecting understanding features not related to safety (UNDRSTD1). As UNDRSTD2 increases, so does the desirability of Basic Map, Voice and Text/Icon feature patterns (Feature Patterns I, II, and III). Additionally, the negative relationship between UNDRSTD1 and the Map Simplification feature pattern (Feature Pattern V) also makes sense, as a greater appreciation of features would be expected to reduce the desirability of a feature pattern which simplifies them (except for older drivers and others who might appreciate the lessened processing load with the Map Simplification features). The marginal (p < 0.0008) negative relationship between drivers' UNDRSTD1 and Coordination of Travel Information (Feature Pattern IV) may arise because the set of appreciated capabilities may be somewhat overwhelming, thereby reducing the desirability of some of those capabilities that are more tangential.

This discussion demonstrates that a variety of subjective variables influences desirability of feature patterns. These variables cover a broad spectrum of potential influences, and include SELFCONC, SYSTRUST, TOLPAT1, TOLPAT2, UNDRSTD1 and UNDRSTD2. This broad spectrum of variables does not influence the desirability of all features equally. Instead, they affect user acceptance in a complex manner that depends on the specific features being considered.

Driver Characteristics and Feature Pattern Desirability for the TravTek System

Experiment 1 examined the effect of driver characteristics, AGE and GENDER, on user acceptance. In general, younger drivers were more comfortable with TravTek system ATIS technology. They found the TravTek system easier to learn and saw more value in it than did older drivers (table 45).

Table 45. Age differences in the TravTek system feature acceptance.

The model shown in table 44 and figure 56 illustrates the effect of driver characteristics on feature desirability. AGE had a strong negative effect on the desirability of the Basic Map feature pattern (Feature Pattern I). In part this could be because older drivers are less able to process a full array of map information. Supporting this interpretation, older drivers particularly appreciated the Coordination of Travel Information feature pattern (Feature Pattern IV) and Map Simplification feature pattern (Feature Pattern V). These results are consistent with other results concerning information processing in the aging driver (Barfield et al., 1993).

Overall, GENDER had a modest effect on the desirability of the Text/Icon feature pattern (Feature Pattern III). Across younger and older drivers, these features were more desirable for women. However, the interaction of AGExGENDER indicated that older female drivers were negatively disposed to the Text/Icon feature pattern (Feature Pattern III), whereas younger female drivers found it particularly attractive (table 46). The Text/Icon feature pattern (Feature Pattern III) needs to be a selectable option to accommodate both female driver age groups.

Table 46. Text/Icon feature pattern AGEXGEN subset of the final analysis summary.

Influence of Demonstration Fidelity for the TravTek System

A fundamental question that guided this experiment addresses the development of empirical methods to study driver acceptance of ATIS. To address this question, experiment 1 included a series of questions from the TravTek System Evaluation questionnaire used in the Orlando study. A major goal of including these questions was to compare ratings of drivers who have driven the TravTek system vehicle in Orlando to the Seattle drivers who have only seen two videos. Such a comparison would reveal how well a simple videotape representation of the TravTek system compares to direct experience with the actual TravTek system. Systematic differences can serve as a guide to future empirical studies of driver acceptance. However, this comparison must be deferred until either these data are made available to the TravTek System Project team or until the Orlando TravTek System data are made available to the Battelle team.

In lieu of comparing the actual TravTek system and videotape experiences, this experiment examined the effect of watching two different videotapes. The first videotape (edited version of the AAA instructional tape) provided a static tutorial of the TravTek system functions and features. The second videotape (a driver's view of a trip through Orlando) provided a dynamic, on–road demonstration of the TravTek system being used to select a destination and then guiding the driver through the city. The differences between the two videotapes seemed to influence driver acceptance. Specifically, voice and other specific system features received higher ratings after drivers had seen video 2 (Orlando trip). However, perhaps reflecting the difficulty of a global evaluation, ratings of the overall TravTek system did not increase after video 2. Drivers may have considered video 1 (AAA instructional tape) to be a sufficient learning experience. Supporting this, video 2 did not reliably increase the dollar amount drivers were willing to pay for a TravTek system.

The two videotapes did not influence ratings of ease of use or dollar amount drivers were willing to pay, but did produce the strongest effect across the feature patterns. Reflecting this, the desirability of the Text Icon feature pattern (Feature Pattern III) increased considerably when drivers viewed video 2. This result suggests that drivers may require a concrete demonstration of system features before they fully appreciate the associated benefits. At the same time, desirability of several feature patterns decreased after seeing video 2. The type of data collected from these experiments matches that collected during the TravTek System Demonstration Project in Orlando, and so it supports future analyses revealing how actual driving experience compares to relatively crude simulations of the same ATIS. The immediate results of experiments 1 and 1B revealed strong effects of different types of ATIS simulations. Experiment 1 showed that subjective perception of simulation FIDELITY influenced several important user acceptance variables. These effects were fully delineated in the model–based approach adopted in experiments 1 and 1B. This same explanation may hold for the diminishing effect of video 2 on the Map Simplification feature pattern (Feature Pattern V). Alternatively, the desirability of the Map Simplification feature pattern (Feature Pattern V) may have been reduced by a growing appreciation of a fuller set of features with video 2.

In addition to the effect of the two videotapes, subjective ratings of the videotapes played an important role in understanding driver acceptance. Subjective ratings of FIDELITY and ATTENT devoted to the experiment were used to gauge how well the "simulation" of the TravTek system, created by the videotapes, conveyed the "feel" of the actual system to the subjects. Although FIDELITY and ATTENT are not listed in table 44, they played important roles because FIDELITY strongly influenced ATTENT and both together influenced SYSTRUST and subsequent desirability of feature patterns. Of note, FIDELITY and ATTENT were not seen to influence driver understanding of the TravTek system (or later the CityGuide system), as posited earlier (figure 9). This may have resulted because enhanced FIDELITY can focus driver ATTENT on the more global "feel" of a system than on the specific technical details (thus enhancing SYSTRUST, but reducing detailed technical understanding). This phenomenon is not uncommon in simulator research where driver acceptance is the appropriate focus and knowledge of system functionality is of secondary importance.

In summary, the model presented in figure 56 was successful in explaining driver acceptance of the TravTek system features. This model addresses different user dimensions (e.g., SYSTRUST) than either the Mackie–Wylie MIAT model discussed earlier in chapter 1 or alternatives such as the Technology Acceptance Model (TAM). This experiment shows that these other variables are important factors and that this simpler model provides a basis for assessing and controlling them in future research and design activities.

CityGuide System Feature Desirability, Driver Attitudes, and System Understanding

Experiment 1B examined the factors affecting driver acceptance using the CityGuide system. The CityGuide experiment was successful in determining the feature patterns drivers would like to have (analogous to the results for TravTek). The four feature patterns that emerged were the: (1) Recreational Information feature pattern (Feature Pattern I); (2) Routing Assistance feature pattern (Feature Pattern II); (3) Accommodation Related feature pattern (Feature Pattern III); (4) Restaurant and Other Coordination feature pattern (Feature Pattern IV).

Table 47 summarizes the statistical relationships among these feature patterns and the variables that were hypothesized to influence the feature patterns (figure 57). This table also considers independent variables of AGE, GENDER, and DRIVER TYPE (commercial drivers vs. private drivers).

Table 47. Summary of the CityGuide system multiple-correlation results

As with the TravTek system, SYSTRUSTC significantly influenced the desirability of feature patterns. Specifically, SYSTRUSTC affected the Accommodation Related Information feature pattern (Feature Pattern III) and the Restaurant and Other Coordination feature pattern (Feature Pattern IV). These two positive relationships are consistent with the three positive TravTek system relationships seen earlier. SELFCONC also had a positive relationship with the Recreational Information Pattern feature pattern (Feature Pattern I). This, similar to the positive relationship identified for the TravTek system Basic Map feature pattern (Feature Pattern I) appeared to mildly contradict other research on automation (Lee & Moray, 1994; Experiment 2 this report) that has shown a negative relationship between SELFCONC and the use of automation. However, the apparent paradox of these differences in the influences of SELFCONC may be the result of perceptions of the value of the patterns for promoting driver self–reliance. Recreational Information (Feature Pattern I) may be perceived as directly supporting self–reliance and requiring the associated high levels of SELFCONC since they reduce the need for asking other guidance. For the other feature patterns, the reverse is true.

Finally, the positive relationships between UNDRSTDC and two feature patterns also makes sense. Increasing UNDRSTDC corresponds to increasing desirability of Recreational Information feature pattern and Routing Assistance feature pattern (Feature Patterns I and II). These results and the others point out the similarity of results for the CityGuide and the TravTek systems.

Driver Characteristics and Feature Pattern Desirability for the CityGuide System

Reflecting the same kind of result seen in the TravTek system analysis, younger drivers again were generally more comfortable than older drivers with the CityGuide system. Within the younger group, the private and commercial drivers were generally equivalent in their ratings. (The one exception was with regard to specific ratings of the "ease of learning" where the commercial drivers were more like older than younger private drivers). The CityGuide system overall was judged by the younger drivers to be easier to learn, use, and to have more value than by older drivers (table 48). Younger drivers overall also gave higher ratings to the map display features, text instructions, and the overall system (analogous to TravTek feature results).

Table 48. Age differences in the CityGuide system feature acceptance.

In the context of the model of driver acceptance, figure 57 and table 47, the strongest effect across the feature patterns was exerted by the DRIVER TYPE (commercial driver vs. private driver). As would be expected, the desirability of Recreational Information (Feature Pattern I) was less for commercial drivers than for private drivers. Likewise, the commercial drivers valued Routing Assistance (Feature Pattern II) less compared to private drivers. Also, not inconsistent with expectations, Routing Assistance (Feature Pattern II) was somewhat more appreciated by commercial drivers than private drivers. The DRIVER TYPE effects were generally in keeping with expectations based on the differing concerns of commercial drivers and private drivers.

AGE also had negative effects on the desirability of both Routing Assistance and Accommodation Related Information (Feature Pattern II and III). With regard to the first of these, this could be because older drivers are less able to process the full array of routing assistance provided by the CityGuide system. Supporting this interpretation, older drivers particularly appreciated Map Simplification (Feature Pattern V) for the TravTek system. This first result would also be consistent with other results on information processing in the aging driver (Barfield et al., 1993). The second result was also not surprising given the expectation that older drivers might have less need for such information due to greater experience and lifestyle differences. AGE effects were generally in keeping with expectations, based on concerns that differed from those of younger drivers.

Influence of Demonstration Fidelity of the CityGuide System

Using the CityGuide system provided a second opportunity to examine the influence of demonstration fidelity (FIDELITYC) that complimented our first study of the TravTek system (experiment 1). Hence, this study also addressed the same fundamental concerns that guided experiment 1, (1) identification of feature patterns that influence driver acceptance; (2) development of empirical methods to study driver acceptance. Contrasting with the two videotape presentations used in the TravTek system study, the CityGuide system was a physically operational system that was demonstrated to the drivers. Of particular concern was the impact that this difference would have on driver acceptance, particularly with regard to the subjective ratings, e.g., FIDELITYC and ATTENTC. Specific results are discussed, followed by a discussion of the broader implications of demonstration FIDELITYC, in the context of the CityGuide system and the TravTek system results.

As with experiment 1, subjective ratings of the "simulation" played an important role in understanding how the feature patterns influenced driver acceptance. In experiment 1B, the "simulation" consisted of a computer–based demonstration of the CityGuide system compared to the videotapes of the TravTek system presented in experiment 1. Ratings of the degree of FIDELITYC were important because they influence how much attention the simulation commands. In addition, FIDELITYC and ATTENTC both influence system trust (SYSTRUSTC) and the subsequent desirability of various feature patterns. Similar to the results of experiment 1, the ratings of FIDELITYC and ATTENTC did not influence driver understanding (UNDRSTDC) of the CityGuide system feature patterns. These results also support the view that the influence of enhanced FIDELITYC may be to focus driver attention on the more global "feel" of a system (vs. on technical details). This CityGuide system experiment consequently served to support the general finding of experiment 1 that the acceptance model (figure 57) provides a basis for future research and design activities.

Broader Implications of Experiments 1 and 1B

Experiments 1 and 1B are part of an integrated series of studies directed at exploring both questions of users' acceptance and evaluation methodology. Taken together, the results of experiments 1 and 1B also begin to provide a basis for the broader considerations of user acceptance and evaluation methodology. This final section begins the consideration of the integrated results of these experiments with regard to the general questions of user acceptance and evaluation methodology.

User Acceptance

The TravTek system and CityGuide system results together indicate some broad user acceptance trends in the three areas delineated below. The first broad user acceptance finding regarding AGE found that younger drivers were generally more comfortable with the TravTek system and the CityGuide system technology. They found the TravTek system and the CityGuide system easier to learn and having more value than did older drivers. With regard to several specific patterns of the TravTek system and the CityGuide system feature patterns, AGE was also often found to have negative effects on the desirability. In part, this may be because older drivers have more experience and a different lifestyle than younger drivers (as suggested for aspects of the CityGuide system). More often this difference may be because older drivers tend to be less able to process information than younger drivers. Supporting this interpretation were 1) the finding that older drivers particularly appreciated the Map Simplification feature pattern (Feature Pattern V) for the TravTek system and 2) other general results on information processing in the aging driver (Barfield et al., 1993).

A second broad user acceptance finding was the generally positive relationships between UNDRSTD1, UNDRSTD2, and UNDRSTDC variables and the desirability of feature patterns (six positive results across the TravTek system and the CityGuide system). These results show that the better features are understood, the more they are appreciated and desired. Even the one significant exception to this finding, the negative relationship between UNDRSTD2 and Map Simplification (Feature Pattern V) supports the view as greater appreciation of features would be expected to tend to reduce the desirability of a feature pattern which simplifies them. The general result supports the view that better understanding of system capabilities will tend to enhance desirability.

The third and final broad user acceptance finding concerns FIDELITY, ATTENT and SYSTRUST for both the TravTek system and the CityGuide system, FIDELITY was found to strongly influence ATTENT and both together were found to tend to positively influence SYSTRUST. SYSTRUST was subsequently positively associated with desirability of five different feature patterns, across results for the CityGuide system and the TravTek system. FIDELITY and ATTENT were not seen to influence UNDRSTDC of the CityGuide system for the same reasons as were posited earlier and addressed in the experiment 1 portion of this discussion.

Each of these three broad findings reflect a growing understanding of user acceptance that will be useful in future system development activities. The next section touches on several of these during its consideration of the evaluation methodology used during the course of the experiments 1 and 1B.

Evaluation Methodology

A general question of developing methodologies to evaluate driver acceptance concerns how well data collected in laboratory conditions, using simulations of ATIS, will generalize to actual systems. Experiments 1 and 1B provide a basis to examine effects of different types of ATIS simulations on the evaluation of driver acceptance. These experiments used quite crude simulations of ATIS (videotapes and a demonstration on a computer–based system). Neither of these "simulations" supported interaction with the system and neither placed subjects in an actual driving situation.

The model–based experimental method used across experiments 1 and 1B proved highly productive both practically and theoretically. Practically, experiments 1 and 1B were successful in revealing both distinct patterns of respective TravTek and CityGuide features that drivers would like to have, and driver characteristics and other factors that influenced their acceptance. These results, among other uses, provide a basis for the grouping of features in respective initial and enhanced Advanced Traveler Information Systems. In turn, the correlational analyses revealed factors that could influence acceptance and point toward methods for practically increasing such acceptance. For example, the positive influence of perceived FIDELITY on user acceptance and its subsequent positive impact on the acceptance of both TravTek and CityGuide feature patterns suggests its systematic examination to variously (1) increase the FIDELITY of the presentations of systems being introduced to the public and (2) control for its impact in studies of user acceptance. These are among the most obvious examples pointing out the practical utility of the model–based approach employed in experiments 1 and 1B.

The model–based methodology also proved productive with regard to advancing our theoretical understanding of user acceptance in two broad respects. First, as noted above, the structural model presented in figure 39 proved a successful basis for explaining the driver acceptance of the TravTek and CityGuide systems (confirming much of its structure). [In itself, this result would recommend continuing to use such a theory in future evaluations of user–acceptance.] Second, as seen earlier in the individual TravTek and CityGuide system results, there were some surprises. System understanding(s), for example, were not influenced by ATTENT in either experiments 1 or 1B. This may have been because the influence of FIDELITY on ATTENT with respect to the more "global" aspects of the way a system works, and not on detailed verbal understanding (i.e., like the results of a flight simulator that teaches the skill of flying but not ground–school details). Of course, together with the other suggested modifications to the structural–model, this posited attentional relationship is an area for future theoretical consideration. Model–based methodology similar to that used herein is consequently expected to continue to be productive in future investigations.

FHWA-RD-96-143