U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

Report
This report is an archived publication and may contain dated technical, contact, and link information
Publication Number: FHWA-RD-96-177
Date: October 1997

Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies

 

3. RESULTS

Our analysis of the rating data is divided into four parts. First, we consider the criteria in table 3 one at a time. Second, we use a linear model to combine ratings on several criteria. Third, we perform a sensitivity analysis to determine the effects of imperfect weightings in the linear model. Fourth, we validate the combination model used.

 

SINGLE CRITERIA

MULTIPLE CRITERIA

SENSITIVITY ANALYSIS

VALIDATION

 

SINGLE CRITERIA

This section contains several tables with the highest 20 studies/issues ranked on one criterion only. Tables 5 through 9 contain the ITS America criteria rankings. Tables 10 through 13 contain the other criteria rankings. In the tables below, tied items are given the same rank rather than average ranks. This procedure will have no effect on the sensitivity analyses and gamma calculations discussed later in this section. Labels from table 2 are shown in brackets, e.g., [H1]. The Methodology ratings are not considered singly because there is little gained by having an optimal method for an unimportant issue.

It is clear from these tables that the Safety criterion is most important from a human factors perspective. Table 14 contains all studies/issues with a mean rating of 4.50 or greater. Thirteen issues are related to the Safety criterion, 6 issues are related to the Guidelines criterion, 3 issues are related to the Older Driver criterion, and two issues are related to the Existing Data criterion. Only two studies/issues were rated above 4.50 on two criteria: A1 – Safety and Existing Data, and E3 – Safety and Older Drivers. No issues were rated 4.50 or greater on three or more criteria.

 

Top

 

MULTIPLE CRITERIA

A linear psychometric model (Dawes & Corrigan, 1974) was used to combine ratings into a single score for each rater. While giving each criterion an equal weighting would be an obvious way to combine data, this would be a poor choice since all criteria are not equally important. The following section on Sensitivity compares the weighting used, based upon the relative importance of the criteria, to the equally–weighted case.

Table 14 was used to generate weights for all but the Methodology items. If all the Methodology items had been considered equally, the resultant score would favor issues amenable to more than a single methodology. Instead, we calculated the single methodology, be it Laboratory, Field, or Survey, that had the highest score for any issue. This Maximum Methodology score was given a weight of 15 percent when combined with the other important criteria in table 14. The weights for these criteria were Safety (53 percent), Guidelines (18 percent), Older Driver (9 percent) and Existing Data (6 percent). Other weightings that maintained this ordering (e.g., Safety most important, etc.) do not produce dramatic differences in final rankings of weighted scores (table 15).

 

Table 5. Top 20 ranked issues for Congestion.

RANK RATING STUDY/ISSUE
1 4.38 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1].
2 4.00 Investigate information requirements associated with driver "way–finding" and destination selection strategies [H3].
3 3.88 Examine the effectiveness of destination approach guidance for CVO in a congested, reduced sight distance, and increased pedestrian traffic environment [G7].
4 3.75 Examine how information can be displayed to dispatchers to support the complex decision–making process associated with allocation of emergency response crews [D20].
4 3.75 Evaluate the effect of information reliability and inaccuracies on driver acceptance and use of ATIS/CVO systems [G1].
4 3.75 Evaluate acceptability of different types of In–Vehicle Routing and Navigation Systems maps, symbols, and icons [G5].
4 3.75 Investigate factors that influence driver's perception of the effectiveness of automatic routing [H2].
8 3.63 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatchers to ensure effective decision making [A8].
8 3.63 Describe the specific information needs/wants of CVO drivers for various situations, such as local versus long distance, urban versus rural, and emergency response versus commercial [B6].
8 3.63 Determine the information content of maps to support different ATIS/CVO functions [H5].
11 3.50 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
11 3.50 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS [C4].
11 3.50 Since commercial vehicles have limited maneuverability, identify the most effective type and timing of information to present to CVO drivers [C8].
11 3.50 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events [F2].
11 3.50 Identify measures comprehensive enough to allow metacomparisons with existing research [K2].
16 3.38 Identify information drivers need and want from in–vehicle road sign (ISIS) and warning systems (IVSAWS) [B2].
16 3.38 Identify how ATIS information prioritization can enhance driving performance and response to ATIS information [C3].
16 3.38 Identify how display characteristics, such as modality, influence driver comprehension of advisory system information [D3].
16 3.38 Examine the effect of display form (e.g., text versus graphic) on driver decision making and problem solving during route planning and selection [D7].
20 3.25 Observe actual driving behavior to reveal the need to integrate functions in ways that were not identified in the analytic approach. These observations should not be bound by current ATIS technology [A3].

 

Table 6. Top 20 ranked issues for Safety.

RANK RATING STUDY/ISSUE
1 5.00 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].
1 5.00 Examine how attention to different types of ATIS information influences the primary task of driving [F1].
3 4.88 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
3 4.88 Examine the workload implications of requiring drivers to transform and enter information into the system [A5].
3 4.88 Assess how the fatigue that plagues CVO drivers (e.g., 8 to 12 h shift) might interact with complex in–vehicle systems to degrade driver performance [D16].
3 4.88 Examine the limits of visual and cognitive attention concerned with receiving information from ATIS when driving [E3].
7 4.75 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS [C4].
7 4.75 Identify potential for overload of specialized CVO drivers such as emergency vehicle operators [E13].
9 4.63 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
9 4.63 Examine how in–vehicle road sign information (e.g., ISIS) affects workload, especially under nighttime, poor weather, and other reduced visibility conditions [E14].
11 4.50 Examine the effect on driver performance of integrating non–ATIS/CVO equipment (radar/laser detectors, laptop computers, cellular phones) with ATIS equipment [A7].
11 4.50 Examine how the timing of ISIS and IVSAWS information, with respect to the location of the incident, influences driver reaction to the information [C6].
11 4.50 Identify the performance implications of inconsistent display formats across ATIS subsystems [D1].
11 4.50 Evaluate the types of information suitable for a HUD display [D9].
11 4.50 Evaluate the effectiveness of multimodality displays, such as voice in combination with text [D12].
11 4.50 Identify the allowable functions of ATIS/CVO systems under conditions of driver impairment [E7].
11 4.50 Identify how estimates of real–time driver workload can be used to avoid overload by moderating information from ATIS [E9].
11 4.50 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events [F2].
19 4.38 Identify features that will benefit/require standardization across many types of ATIS systems and functions [D2].
19 4.38 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels [D4].

 

Table 7. Top 20 ranked issues for Mobility.

RANK RATING STUDY/ISSUE
1 3.63 Investigate information requirements associated with driver "way–finding" and destination selection strategies [H3].
2 3.50 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1].
2 3.50 With mass market penetration, identify methods to evaluate and predict consequences of ATIS/CVO system use [K3].
4 3.25 Identify how ATIS information prioritization can enhance driving performance and response to ATIS information [C3].
4 3.25 Examine whether availability of information and functions should depend on individual differences such as age, gender, and experience [E6].
4 3.25 Determine the information content of maps to support different ATIS/CVO functions [H5].
4 3.25 Identify measures comprehensive enough to allow metacomparisons with existing research [K2].
8 3.13 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].
8 3.13 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
8 3.13 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
8 3.13 Since commercial vehicles have limited maneuverability, identify the most effective type and timing of information to present to CVO drivers [C8].
8 3.13 Examine how information can be displayed to dispatchers to support the complex decision-making process associated with allocation of emergency response crews [D20].
8 3.13 Identify ATIS functions a driver might need or want to use during the various driving activities, such as driving, parking, and stopping [E5].
8 3.13 Evaluate how driver acceptance may decline when the driver is forced to interact with multiple subsystems, particularly when this interaction is exacerbated by demanding road conditions [G3].
8 3.13 Evaluate acceptability of different types of In–Vehicle Routing and Navigation Systems maps, symbols, and icons [G5].
16 3.00 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatchers to ensure effective decision making [A8].
16 3.00 Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc.) [B1].
16 3.00 Identify what types of ATIS information should be available upon request [B5].
16 3.00 Identify how estimates of real–time driver workload can be used to avoid overload by moderating information from ATIS [E9].
16 3.00 Examine how in-vehicle road sign information (e.g., ISIS) affects workload, especially under nighttime, poor weather, and other reduced visibility conditions [E14].

 

Table 8. Top 20 ranked issues for Environment.

RANK RATING STUDY/ISSUE
1 3.50 With mass market penetration, identify methods to evaluate and predict consequences of ATIS/CVO system use [K3].
2 3.38 Identify measures comprehensive enough to allow metacomparisons with existing research [K2].
3 3.13 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1].
4 3.00 Examine how information can be displayed to dispatchers to support the complex decision–making process associated with allocation of emergency response crews [D20].
4 3.00 Investigate information requirements associated with driver "way–finding" and destination selection strategies [H3].
6 2.88 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatchers to ensure effective decision making [A8].
7 2.75 Since commercial vehicles have limited maneuverability, identify the most effective type and timing of information to present to CVO drivers [C8].
7 2.75 Determine how learning about ITS systems and increased experience with them over time affects acceptance of ATIS/CVO [G9].
7 2.75 Investigate factors that influence driver's perception of the effectiveness of automatic routing [H2].
10 2.63 Describe the specific information needs/wants of CVO drivers for various situations, such as local versus long distance, urban versus rural, and emergency response versus commercial [B6].
10 2.63 Identify how and when information related to weather conditions and availability of specialized fuel services should be provided to CVO drivers [C7].
10 2.63 Examine how to display information such as "weight-in-motion" to promote regulatory compliance, especially if the information is different than expected [D19].
10 2.63 Examine how to display CVO-specific highway safety information (e.g., bridge clearance, width, and restrictions) with minimum interference in attention to the roadway [F5].
10 2.63 Examine the effectiveness of destination approach guidance for CVO in a congested, reduced sight distance, and increased pedestrian traffic environment [G7].
15 2.50 Observe actual driving behavior to reveal the need to integrate functions in ways that were not identified in the analytic approach. These observations should not be bound by current ATIS technology [A3].
15 2.50 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
15 2.50 Identify how ATIS information prioritization can enhance driving performance and response to ATIS information [C3].
15 2.50 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS [C4].
15 2.50 Examine how the timing of ISIS and IVSAWS information, with respect to the location of the incident, influences driver reaction to the information [C6].
15 2.50 Examine the effect of display form (e.g., text versus graphic) on driver decision making and problem solving during route planning and selection [D7].

 

Table 9. Top 20 ranked issues for Economic.

RANK RATING STUDY/ISSUE
1 4.00 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatchers to ensure effective decision making [A8].
1 4.00 Examine the cost/benefit trade-off associated with training CVO drivers to accommodate more ATIS information [I5].
3 3.88 Examine how to display information such as "weight-in-motion" to promote regulatory compliance, especially if the information is different than expected [D19].
4 3.75 Describe the specific information needs/wants of CVO drivers for various situations, such as local versus long distance, urban versus rural, and emergency response versus commercial [B6].
4 3.75 Since commercial vehicles have limited maneuverability, identify the most effective type and timing of information to present to CVO drivers [C8].
4 3.75 Examine the requirements to support the complex onboard data management requirements that commercial vehicle drivers experience [E12].
4 3.75 Identify measures comprehensive enough to allow metacomparisons with existing research [K2].
8 3.63 Examine the effectiveness of destination approach guidance for CVO in a congested, reduced sight distance, and increased pedestrian traffic environment [G7].
8 3.63 With mass market penetration, identify methods to evaluate and predict consequences of ATIS/CVO system use [K3].
10 3.50 Assess how the fatigue that plagues CVO drivers (e.g., 8 to 12 h shift) might interact with complex in–vehicle systems to degrade driver performance [D16].
10 3.50 Examine how the accuracy of information pertaining to availability and current deployment of resources affects dispatcher acceptance and interaction with the system [G10].
12 3.38 Identify how priorities specific to CVO information compare to other ATIS information [C1].
12 3.38 Identify how and when information related to weather conditions and availability of specialized fuel and services should be provided to CVO drivers [C7].
12 3.38 Examine how information can be displayed to dispatchers to support the complex decision–making process associated with allocation of emergency response crews [D20].
12 3.38 Examine how to display CVO–specific highway safety information (e.g., bridge clearance, width, and restrictions) with minimum interference in attention to the roadway [F5].
12 3.38 Determine system characteristics that eliminate or minimize training needs [I1].
17 3.25 Investigate the effect of dynamic route scheduling on the perceived quality of worklife and corresponding CVO driver acceptance [G6].
17 3.25 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1].
17 3.25 Investigate information requirements associated with driver "way–finding" and destination selection strategies [H3].
20 3.13 Identify features that will benefit/require standardization across many types of ATIS systems and functions [D2].

 

Table 10. Top 20 ranked issues for Existing Data.

RANK RATING STUDY/ISSUE
1 4.50 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].
1 4.50 Identify how ATIS information flow might be managed to capitalize on the dynamic nature of driver workload [E8].
3 4.38 Investigate how to display multiple ISIS and IVSAWS messages so that drivers can identify relevant information and react appropriately [C5].
3 4.38 Investigate how to structure design guidelines to help designers address human factors issues in ATIS designs which support the needs of the driver [J1].
3 4.38 Investigate the design process to identify designer needs for guideline content and format [J3].
6 4.25 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
6 4.25 Identify how estimates of real-time driver workload can be used to avoid overload by moderating information from ATIS [E9].
6 4.25 Examine the requirements to support the complex onboard data management requirements that commercial vehicle drivers experience [E12].
6 4.25 Examine how in–vehicle road sign information (e.g., ISIS) affects workload, especially under nighttime, poor weather, and other reduced visibility conditions [E14].
6 4.25 Investigate alternate guideline presentation formats (e.g., expert systems, electronic data base, standard data base) so as to be compatible with designer needs [J2].
11 4.13 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatchers to ensure effective decision making [A8].
11 4.13 Identify information drivers need and want from in–vehicle road sign (ISIS) and warning systems (IVSAWS) [B2].
11 4.13 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
11 4.13 Identify the allowable functions of ATIS/CVO systems under conditions of driver impairment [E7].
11 4.13 Examine how attention to different types of ATIS information influences the primary task of driving [F1].
11 4.13 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events [F2].
11 4.13 Determine whether drivers' reliance upon navigational cues outside the vehicle influences their observation of external hazards, compared to when they rely on navigational cues presented by an ATIS [F3].
11 4.13 Evaluate driver acceptance of "lock–out" designs that only allow the driver access to functions under certain circumstances [G3].
11 4.13 Examine the effectiveness of destination approach guidance for CVO in a congested, reduced sight distance, and increased pedestrian traffic environment [G7].
11 4.13 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1].

 

Table 11. Top 20 ranked issues for Guidelines.

RANK RATING STUDY/ISSUE
1 5.00 Investigate how to structure design guidelines to help designers address human factors issues in ATIS designs which support the needs of the driver [J1].
2 4.88 Investigate alternate guideline presentation formats (e.g., expert systems, electronic data base, standard data base) so as to be compatible with designer needs [J2].
2 4.88 Investigate the design process to identify designer needs for guideline content and format [J3].
4 4.63 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1].
5 4.50 Investigate how to display multiple ISIS and IVSAWS messages so that drivers can identify relevant information and react appropriately [C5].
5 4.50 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels [D4].
7 4.38 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
7 4.38 Examine the effect of display form (e.g., text versus graphic) on driver decision making and problem solving during route planning and selection [D7].
9 4.25 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].
9 4.25 Identify information drivers need and want from in–vehicle road sign (ISIS) and warning systems (IVSAWS) [B2].
9 4.25 Describe the specific information needs/wants of CVO drivers for various situations, such as local versus long distance, urban versus rural, and emergency response versus commercial [B6].
9 4.25 Identify features that will benefit/require standardization across many types of ATIS systems and functions [D2].
9 4.25 Evaluate the types of information suitable for a HUD display [D9].
9 4.25 Identify the relationship between icon characteristics and information types that maximize icon effectiveness and salience [D13].
9 4.25 Evaluate how different types of information, displayed using a HUD, affect cognitive attention devoted to the roadway [E2].
9 4.25 Examine the limits of visual and cognitive attention concerned with receiving information from ATIS when driving [E3].
9 4.25 Examine the requirements to support the complex onboard data management requirements that commercial vehicle drivers experience [E12].
9 4.25 Evaluate acceptability of different types of In–Vehicle Routing and Navigation Systems maps, symbols, and icons [G5].
19 4.13 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
19 4.13 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatchers to ensure effective decision making [A8].

 

Table 12. Top 20 ranked issues for Older Drivers.

RANK RATING STUDY/ISSUE
1 4.63 Examine whether availability of information and functions should depend on individual differences such as age, gender, and experience [E6].
2 4.50 Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc.) [B1].
2 4.50 Examine the limits of visual and cognitive attention concerned with receiving information from ATIS when driving [E3].
4 4.38 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].
4 4.38 Examine how attention to different types of ATIS information influences the primary task of driving [F1].
6 4.25 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
6 4.25 Identify how message length and wording for voice–based interfaces affect driving performance and message comprehension [D14].
8 4.13 Examine the workload implications of requiring drivers to transform and enter information into the system [A5].
8 4.13 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS [C4].
8 4.13 Evaluate the effectiveness of multimodality displays, such as voice in combination with text [D12].
8 4.13 Examine how in–vehicle road sign information (e.g., ISIS) affects workload, especially under nighttime, poor weather, and other reduced visibility conditions [E14].
8 4.13 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events [F2].
13 4.00 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
13 4.00 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels [D4].
13 4.00 Determine what factors influence synthesized voice message intelligibility [D15].
13 4.00 Evaluate how different types of information, displayed using a HUD, affect cognitive attention devoted to the roadway [E2].
17 3.88 Examine the physical demands placed on the driver by the need to transition from one ATIS function to another [A2].
17 3.88 Identify how display characteristics, such as modality, influence driver comprehension of advisory system information [D3].
17 3.88 Examine the effect of display form (e.g., text versus graphic) on driver decision making and problem solving during route planning and selection [D7].
17 3.88 Identify a subset of environmental factors that interacts with ATIS to influence driver workload levels [E1].

 

Table 13. Top 20 ranked issues for Younger Drivers.

RANK RATING STUDY/ISSUE
1 4.00 Examine whether availability of information and functions should depend on individual differences such as age, gender, and experience [E6].
1 4.00 Examine how attention to different types of ATIS information influences the primary task of driving [F1].
3 3.88 Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc [B1].
3 3.88 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events [F2].
5 3.75 Identify the allowable functions of ATIS/CVO systems under conditions of driver impairment [E7].
6 3.63 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS [C4].
7 3.50 Examine the limits of visual and cognitive attention concerned with receiving information from ATIS when driving [E3].
7 3.50 Determine whether drivers' reliance upon navigational cues outside the vehicle influence their observation of external hazards, compared o when they rely on navigational cues presented by and ATIS [F3].
9 3.38 Observe actual driving behavior to reveal the need to integrate functions in ways that were not identified in the analytic approach. These observations should not be bound by current ATIS technology [A3].
9 3.38 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4].
9 3.38 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3].
9 3.38 Identify factors that may influence drivers to defer to the "Expert System" and fail to recognize a hazard when one is present [F4].
9 3.38 Identify what training techniques could be incorporated into the operation of the different subsystems [I3].
14 3.25 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].
14 3.25 Examine the workload implications of requiring drivers to transform and enter information into the system [A5].
14 3.25 Identify what types of ATIS information should be available upon request [B4].
14 3.25 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels [D4].
14 3.25 Examine the effect of display form (e.g., text versus graphic) on driver decision making and problem solving during route planning and selection [D7].
14 3.25 Identify a subset of environmental factors that interacts with ATIS to influence driver workload levels [E1].
14 3.25 Evaluate the effect of information reliability and inaccuracies on driver acceptance and use of ATIS/CVO systems [G1].

 

Table 14. Studies/Issues rated 4.50 or greater on single dimensions.

RATING STUDY/ISSUE CRITERIA

5.00


4.50

Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another [A1].

Safety


Existing Data

5.00 Examine how attention to different types of ATIS information influences the primary task of driving [F1]. Safety
4.88 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system [A4]. Safety
4.88 Examine the workload implications of requiring drivers to transform and enter information into the system [A5]. Safety
4.88 Assess how the fatigue that plagues CVO drivers (e.g., 8– to 12–h shift) might interact with complex in–vehicle systems to degrade driver performance [D16]. Safety

4.88


4.50

Examine the limits of visual and cognitive attention concerned with receiving information from ATIS when driving [E3].

Safety


Older Drivers

4.75 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS [C4]. Safety
4.75 Identify potential for overload of specialized CVO drivers such as emergency vehicle operators [E13]. Safety
4.63 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior [B3]. Safety
4.63 Examine how in–vehicle road sign information (e.g., ISIS) affects workload, especially under nighttime, poor weather, and other reduced visibility conditions [E14]. Safety
4.50 Examine the effect on driver performance of integrating non–ATIS/CVO equipment (radar/laser detectors, laptop computers, cellular phones) with ATIS equipment [A7]. Safety
4.50 Examine how the timing of ISIS and IVSAWS information, with respect to the location of the incident, influences driver reaction to the information [C6]. Safety
4.50 Identify the performance implications of inconsistent display formats across ATIS subsystems [D1]. Safety
4.50 Evaluate the types of information suitable for a HUD display [D9]. Safety
4.50 Evaluate the effectiveness of multimodality displays, such as voice in combination with text [D12]. Safety
4.50 Identify the allowable functions of ATIS/CVO systems under conditions of driver impairment [E7]. Safety
4.50 Identify how estimates of real–time driver workload can be used to avoid overload by moderating information from ATIS [E9]. Safety
4.50 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events [F2]. Safety
4.50 Identify how ATIS information flow might be managed to capitalize on the dynamic nature of driver workload [E8]. Existing Data
5.00 Investigate how to structure design guidelines to help designers address human factors issues in ATIS designs which support the needs of the driver [J1]. Guidelines
4.88 Investigate alternate guideline presentation formats (e.g., expert systems, electronic data base, standard data base) so as to be compatible with designer needs [J2]. Guidelines
4.88 Investigate the design process to identify designer needs for guideline content and format [J3]. Guidelines
4.63 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability [H1]. Guidelines
4.50 Investigate how to display multiple ISIS and IVSAWS messages so that drivers can identify relevant information and react appropriately [C5]. Guidelines
4.50 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels [D4]. Guidelines
4.63 Examine whether availability of information and functions should depend on individual differences such as age, gender, and experience [E6]. Older Drivers
4.50 Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc.) [B1]. Older Drivers

 

Table 15 shows the final weighted rankings for all studies/issues. Ratings varied from 4.34 for the most important study/issue [A1] to 2.96 for the least important issue [G5]. The last column of table 15 shows the Methodology with the maximum score for each issue. In general the Laboratory Methodology was most preferred (55 issues), followed by Survey Methodology (23 issues) and On–Road Field Study (16 issues).

Table 15. Final weighted rankings.

RANK STUDY/ISSUE LABEL RATING METHOD
1 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another. A1 4.34 Laboratory
2 Examine the workload implications of requiring drivers to transform and enter information into the system. A5 4.31 Laboratory
3 Examine how attention to different types of ATIS information influences the primary task of driving. F1 4.29 Laboratory
4 Identify features that will benefit/require standardization across many types of ATIS systems and functions. D2 4.25 Laboratory/Survey
5 Identify specific concerns regarding how display formats and modality impact CVO driver workload. D17 4.21 Laboratory
6 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhance the potential for an improper response to ISIS/IVSAWS. C4 4.21 Laboratory
7 Identify how the dynamic characteristics of driver workload interact with the form of the ATIS interface. E11 4.20 Laboratory
8 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior. B3 4.19 On–road
9 Evaluate the effectiveness of multimodality displays, such as voice in combination with text. D12 4.16 Laboratory
10 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels. D4 4.14 Laboratory
11 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system. A4 4.13 Laboratory
12 Investigate how to display multiple ISIS and IVSAWS messages so that drivers can identify relevant information and react appropriately. C5 4.13 Laboratory
13 Examine how display design might aid the dynamic allocation of driver visual and cognitive resources. D21 4.12 Laboratory
14 Identify the compensatory actions drivers take to moderate their workload, and how ATIS/CVO systems might affect those actions. E10 4.11 On–road
15 Examine how the timing of ISIS and IVSAWS information, with respect to the location of the incident, influences driver reaction to the information. C6 4.10 Laboratory
16 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events. F2 4.08 Laboratory
17 Identify how estimates of real–time driver workload can be used to avoid overload by moderating information from ATIS. E9 4.05 Laboratory
18 Examine how in–vehicle road sign information (e.g., ISIS) affects workload especially under nighttime, poor weather, and other reduced visibility conditions. E14 4.04 On–road
19 Identify information drivers need and want from in–vehicle road sign (ISIS) and warning systems (IVSAWS). B2 4.03 Survey
20 Identify which functions to "lock out" during in–transit and provide only during zero–speed and pre–drive conditions. E4 4.02 Survey
21 Identify how ATIS information flow might be managed to capitalize on the dynamic nature of driver workload. E8 4.00 Laboratory
22 Determine how interface form and modality influences driver interpretation of ISIS and IVSAWS features. D10 4.00 Laboratory
23 Evaluate how input device characteristics might need to vary across subsystems. D5 3.99 Laboratory
24 Generate rules to pair types of information with display modality, for individual displays and for combinations of displays. D11 3.98 Laboratory
25 Identify the display design characteristics required to support ATIS/CVO systems in large, noisy, and vibration prone commercial vehicles. D18 3.97 On–road
26 Identify the performance implications of inconsistent display formats across ATIS subsystems. D1 3.96 Laboratory
27 Assess how the fatigue that plagues CVO drivers (e.g., 8– to 12–h shift) might interact with complex in–vehicle systems to degrade driver performance. D16 3.96 Laboratory
28 Identify the allowable functions of ATIS/CVO systems under conditions of driver impairment. E7 3.96 Laboratory
29 Identify ATIS functions a driver might need or want to use during the various driving activities, such as driving, parking, and stopping. E5 3.94 Survey
30 Examine the effect on driver performance of integrating non–ATIS/CVO equipment (radar/laser detectors, laptop computers, cellular phones) with ATIS equipment. A7 3.94 Laboratory
31 Determine whether drivers' reliance upon navigational cues outside the vehicle influence their observation of external hazards, compared to when they rely on navigational cues presented by an ATIS. F3 3.93 On–road
32 Identify a subset of environmental factors that interacts with ATIS to influence driver workload levels. E1 3.93 On–road
33 Evaluate the effect of information reliability and inaccuracies on driver acceptance and use of ATIS/CVO systems. G1 3.92 Laboratory
34 Identify what types of ATIS information should be available upon request. B5 3.92 Survey
35 Examine the requirements to support the complex onboard data management requirements that commercial vehicle drivers experience. E12 3.91 Survey
36 Evaluate how different types of information, displayed using a HUD, affect cognitive attention devoted to the roadway. E2 3.87 Laboratory
37 Identify how ATIS information prioritization can enhance driving performance and response to ATIS information. C3 3.87 Laboratory
38 Examine how information can be displayed to dispatchers to support the complex decision–making process associated with allocation of emergency response crews. D20 3.86 Survey
39 Describe the specific information needs/wants of CVO drivers for various situations, such as local versus long distance, urban versus rural, and emergency response versus commercial. B6 3.85 Survey
40 Identify what types of ATIS information should be displayed to drivers automatically. B4 3.85 On–road
41 Determine what factors influence synthesized voice message intelligibility. D15 3.85 Laboratory
42 Examine how to display CVO–specific highway safety information (e.g., bridge clearance, width, and restrictions) with minimum interference in attention to the roadway. F5 3.83 Laboratory
43 Identify factors that may influence drivers to defer to the "Expert System" and fail to recognize a hazard when one is present. F4 3.80 Laboratory/On–road
44 Identify how display characteristics, such as modality, influence driver comprehension of advisory system information. D3 3.78 Laboratory
45 Examine whether availability of information and functions should depend on individual differences such as age, gender, and experience. E6 3.77 Laboratory
46 Investigate the effect of dynamic route scheduling on the perceived quality of worklife and corresponding driver acceptance. G6 3.76 On–road
47 Investigate how to structure design guidelines to help designers address human factors issues in ATIS designs which support the needs of the driver. J1 3.76 Survey
48 Investigate factors that influence acceptance of non–verbal and verbal alerts and messages, such as repeat cycle frequency. G4 3.74 Laboratory
49 Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc.). B1 3.74 Survey
50 Identify how message length and wording for voice–based interfaces affect driving performance and message comprehension. D14 3.74 Laboratory
51 Observe actual driving behavior to reveal the need to integrate functions in ways that were not identified in the analytic approach. These observations should not be bound by current ATIS technology. A3 3.72 On–road
52 Examine the physical demands placed on the driver by the need to transition from one ATIS function to another. A2 3.71 Laboratory
53 Evaluate the types of information suitable for a HUD display. D9 3.71 Laboratory
54 Examine the limits of visual and cognitive attention concerned with receiving information from ATIS when driving. E3 3.71 Laboratory
55 Evaluate driver acceptance of "lock–out" designs that only allow the driver access to functions under certain circumstances. G2 3.71 Laboratory
56 Since commercial vehicles have limited maneuverability, identify the most effective type and timing of information to present to CVO drivers. C8 3.70 Survey
57 Determine system characteristics that eliminate or minimize training needs. I1 3.69 Laboratory
58 Determine the information content of maps to support different ATIS/CVO functions. H5 3.67 Laboratory/Survey
59 Evaluate how driver acceptance may decline when the driver is forced to interact with multiple subsystems, particularly when this interaction is exacerbated by demanding road conditions. G3 3.66 Laboratory
60 Evaluate driver perception and control characteristics such as hand–finger coordination and touch accuracy that might influence ATIS design. D6 3.64 Laboratory
61 Identify the relationship between icon characteristics and information types that maximize icon effectiveness and salience. D13 3.64 Laboratory
62 Examine the effectiveness of destination approach guidance for CVO in a congested, reduced sight distance, and increased pedestrian traffic environment. G7 3.63 On–road
63 Identify how and when information related to weather conditions and availability of specialized fuel and services should be provided to CVO drivers. C7 3.61 Survey
64 Investigate how ATIS interface designs and parameters of routing algorithms (e.g., optimize for safety, time, distance) affect route acceptability. H1 3.59 Laboratory
65 Examine how the accuracy of information pertaining to availability and current deployment of resources affects dispatcher acceptance and interaction with the system. G10 3.59 On–road
66 Identify potential for overload of specialized CVO drivers such as emergency vehicle operators. E13 3.58 Survey
67 Investigate how best to support driver performance when the ATIS fails due to unreliable GPS signals or other anomalous circumstances. C2 3.58 Laboratory
68 Investigate alternate guideline presentation formats (e.g., expert systems, electronic data base, standard data base) so as to be compatible with designer needs. J2 3.57 Survey
69 Examine how the interface form (e.g., text versus graphic, touch screen versus steering wheel controls) should change from pre–drive to drive and park modes. D8 3.55 Laboratory
70 Examine how the reliability and priority of regulatory information affect CVO driver workload. C9 3.53 Laboratory
71 Examine how to support sharing information (e.g., status, location, availability of resources) among dispatches to ensure effective decision making. A8 3.51 On–road
72 Generate multivariate constructs to measure driver capacity and workload. K1 3.51 Laboratory
73 Investigate information requirements associated with driver "way–finding" and destination selection strategies. H3 3.49 Laboratory
74 Identify how priorities specific to CVO information compare to other ATIS information. C1 3.48 Survey
75 Examine the driver acceptance implications of requiring drivers to transform and enter information into the system. A6 3.47 Laboratory
76 Identify what training techniques could be incorporated into the operation of the different subsystems. I3 3.47 Laboratory
77 Examine how to display information such as "weight–in–motion" to promote regulatory compliance, especially if the information is different than expected. D19 3.47 On–road
78 Investigate the design process to identify designer needs for guideline content and format. J3 3.46 Survey
79 Examine the cost/benefit trade–off associated with training CVO drivers to accommodate more ATIS information. I5 3.44 Laboratory
80 Examine the effect of display form (e.g., text versus graphic) on driver decision making and problem solving during route planning and selection. D7 3.41 Laboratory
81 Evaluate the effect of destination–focusing navigational strategies on driver stress and driver acceptance. H4 3.40 On–road
82 Develop methods to link driver performance to more global measures of system performance, such as specified by the ITS goals. K4 3.38 Survey
83 Investigate factors that influence driver's perception of the effectiveness of automatic routing. H2 3.35 Laboratory
84 Examine how training might enhance driver acceptance of routing suggestions. I2 3.30 On–road
85 Identify measures comprehensive enough to allow metacomparisons with existing research. K2 3.30 Survey
86 Determine whether CVO drivers will accept a "lock–out" design which limits access to all or selected functions while moving. G8 3.27 Survey
87 With mass market penetration, identify methods to evaluate and predict consequences of ATIS/CVO system use. K3 3.19 Survey
88 Examine how best to develop videotape–based training to illustrate ATIS functions. I4 3.10 Laboratory
89 Determine how learning about ITS systems and increased experience with them over time affects acceptance of ATIS/CVO. G9 3.10 Survey
90 Develop a set of consistent subjective measures across experiments so that rating scales are common across experiments. K5 3.02 Survey
91 Evaluate acceptability of different types of In–Vehicle Routing and Navigation Systems maps, symbols, and icons. G5 2.96 Laboratory

 

Top

 

SENSITIVITY ANALYSIS

Figure 2 shows the effect of increasing the weight for the Safety criterion with the set of nine criteria (Methodology excluded). Goodman's/Kruskal's gamma (Reynolds, 1977) is a measure of association, that ranges from +1.0 to -1.0, suitable for ordinal data. The baseline for calculating gamma has all nine criteria weighted equally. Figure 2 shows how gamma decreases when the weighting of the Safety criterion is increased. When Safety has a weight of one (not shown in figure 2) gamma = 1.0. As the weight for Safety increases, gamma decreases slightly until it asymptotes at .46 when Safety is the only criterion (e.g., all eight other criteria assigned a weight of zero). Thus, there is only a modest change in rankings for the linear model when the importance of the Safety criterion is varied. There is no effect on either the sensitivity analysis nor the gamma calculation associated with the method of calculating ranks shown in tables 5 through 13.

A graph of Goodman's/Kruskal's Gamma as a function of Safety weighting.

Figure 2. Goodman's/Kruskal's Gamma as a function of Safety weighting.

Figure 3 shows the asymptotic value of Goodman's/Kruskal's gamma for all nine criteria. The Economic criterion changes the most, implying that from a human factors perspective it is a less valuable dimension. The Existing Data, Guidelines and Older Drivers criteria offer the same sensitivity as the Safety criterion. Thus, the sensitivity analysis confirms that the linear psychometric model is robust so that changes in individual weights used to produce table 15 will not greatly alter the rankings of table 15.

A graph of Goodman's/Kruskal's Gamma for each criterion.

Figure 3. Goodman's/Kruskal's Gamma for each criterion.

 

Top

 

VALIDATION

Table 16 shows the number of studies/issues deleted by the raters for Lists A, B, and C.

List A (appendix B) used the rankings in table 15. List B used the same weights as table 16, but applied only to the data of the individual rater. List C (appendix B) was a stratified random sample (appendix B) with four studies/issues from each quartile. If the rating methodology used in this study is valid, raters should have rejected more items from List C (appendix B). This was indeed the case, F(2,14) = 15.1, p < .001. List C (appendix B) differs from both Lists A and B, ts(14) > 7.28, ps < .001. Lists A and B do not differ, t(14) = 1.62, p > .05.

 

Table 16. Number of studies/issues deleted by raters by list.

RATER LIST A LIST B LIST C
N. M. 1 1 3
T. D. 2 1 5
B. K. 1 0 6
J. L. 1 3 4
W. W. 5 6 7
M. M. 1 4 5
F. L. 1 0 2
T. T. 0 1 2
MEAN 1.50 2.00 4.25

Table 17 shows the eight studies/issues added to List A (appendix B) by the raters. Each rater was allowed to add only one issue. Table 18 shows issues deleted by two raters. No issue was deleted by more than two raters. In general, issues were deleted because raters thought studies would not be practical given time and resource limits of this project.

 

Table 17. Studies/Issues added to List A by raters.

LABEL STUDY/ISSUE
B1 Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc).
B6 Describe the specific information needs/wants of CVO drivers for various situations, such as local versus long distance, urban versus rural, and emergency response versus commercial.
D16 Assess how the fatigue that plagues CVO drivers (e.g., 8– to 12–h shift) might interact with complex in–vehicle systems to degrade driver performance.
D17 Identify specific concerns regarding how display formats and modality impact CVO driver workload.
D20 Examine how information can be displayed to dispatchers to support the complex decision–making process associated with allocation of emergency response crews.
E2 Evaluate how different types of information, displayed using a HUD, affect cognitive attention devoted to the roadway.
E11 Identify how the dynamic characteristics of driver workload interact with the form of the ATIS interface.
G1 Evaluate the effect of information reliability and inaccuracies on driver acceptance and use of ATIS/CVO systems.

 

Table 18. Studies/Issues deleted from List A by two raters.

LABEL STUDY/ISSUE
A5 Examine the workload implications of requiring drivers to transform and enter information into the system.
E11 Identify how the dynamic characteristics of driver workload interact with the form of the ATIS interface.
E14 Examine how in-vehicle road sign information (e.g., ISIS) affects workload especially under nighttime, poor weather, and other reduced visibility conditions.
F1 Examine how attention to different types of ATIS information influences the primary task of driving.

 

Top

 

FHWA-RD-96-177

 

Previous | Table of Contents | Next

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101