Skip to contentUnited States Department of Transportation - Federal Highway Administration FHWA Home
Research Home
Report
This report is an archived publication and may contain dated technical, contact, and link information
Publication Number: FHWA-RD-96-177
Date: October 1997

Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies

 

APPENDIX B: STUDY/ISSUE RATING FORMS

 

Our analysis of the rating data is divided into four parts. First, we consider the criteria in table 3 one at a time. Second, we use a linear model to combine ratings on several criteria. Third, we perform a sensitivity analysis to determine the effects of imperfect weightings in the linear model. Fourth, we validate the combination model used.

 

INSTRUCTIONS FOR RATING ATIS/CVO STUDIES/ISSUES

STUDIES/ISSUES

CRITERIA

STUDIES/ISSUES RATING FORM

VALIDATION RATING FORMS

 

INSTRUCTIONS FOR RATING ATIS/CVO STUDIES/ISSUES

Previous tasks of this project have identified a broad range of concerns which human factors research does not address. For example, a technical basis for ATIS/CVO (Advanced Traveler Information System/Commercial Vehicle Operations) is not covered within human factors guidelines. To develop a technical basis for human factors guidelines, the Battelle team will conduct a series of laboratory, survey, and on-road studies. However, the number of potential issues that could be studied far exceeds the resources to conduct those studies. Therefore, to maximize the benefit of the studies, the issues require prioritization. Issue priority depends upon a weighted average of several criteria.

To support this prioritization process we ask you to use the attached sheets to rate all studies/issues according to all the criteria. The left side of the rating sheet begins with a one-sentence description of the research issue/study. Immediately to the right is a series of criteria associated with the ITS America objectives and other criteria associated with how well the studies would support guideline development. Please rate each of these nine criteria using the rating scale shown in the upper left corner of the rating sheet. The larger the value, the more closely studies/issues meet the criteria. For example, a study to examine the effects of a route guidance system on freeway congestion might receive a "5" under the "Congestion" criteria. A study that goes beyond all existing data would receive a "5" for "Existing data." Please enter a rating for all studies/issues.

Criteria for rating the feasibility of the study associated with each issue lie on the right hand side of the rating sheet. These criteria address the cost, time, practicality, generality, and suitability associated with potential laboratory, field, and survey studies. Please rate all criteria for the different types of studies (laboratory, field and survey studies). Scales for these ratings lie at the far right of the rating sheet. The cost rating scale applies to the "Cost" criteria for the laboratory, field, and survey methodologies. The larger the number, on a 3-point scale, the more expensive the study. The time rating scale applies to the "Time" criteria for laboratory, field, and survey methodologies. Larger values correspond to longer experiments, as measured in months to complete. The practicality, generality, and suitability criteria are rated with a scale on the lower right of the rating sheet. The more practical, general, or suitable the study and methodology the larger the rating. For example, laboratory studies will never be able to replicate the complete driving experience and so will not be able to address many issues. In these situations "Suitability" under "Laboratory Methodology" would receive a rating of "1." Please enter a rating for all criteria, even if a one or more methodologies appear completely unsuitable.

The issues shown on the rating sheets are a composite of lists and summaries provided by several scientists. Many of whom, alas, neglected to supply the requested cross-reference to project working papers. As such, the reference to specific tasks may be ambiguous. However, we would like you to note the task that might contain the issue. Even better, if you know a page number please include it as well. We do not expect anyone to fill in references for all issues, but we would like benefit from the broad perspective of the group by identifying where these issues originated. Thus, where possible, please insert the initial of the form from where the issue came in the far right column. For example, insert "E298" to signify that the issue originated from page 298 of the Task E working paper.

Thank you very much for your patience with this task; we appreciate your help in rating all criteria for all studies/issues. Your ratings will provide a valuable resource in determining the priority of the ATIS human factors studies that will follow.

The following pages describe the general categories of studies/issues and the criteria used to judge them. The descriptions of issue/study categories augment the single sentence explanations on the rating sheets. Following the descriptions of issue/study categories each criterion is described in more detail.

 

Top

 

STUDIES/ISSUES

The issues fall into 11 general categories to make the diverse set of issues more comprehensible. The range of issues includes the need to identify the factors that govern how drivers adapt to ATIS, the need to understand how best to present human factors guidelines to designers so that human factors concerns are addressed in future ATIS, and the need to develop sound experimental methodologies to study ATIS. The 11 general categories are defined below.

Coordination of Multiple ATIS Functions

ATIS will, in all likelihood, include a wide variety of functions that range from route planning, services/attractions direction and route guidance to automatic aid requesting and vehicle condition monitoring. The human factors issues in this category address the need to understand the cognitive and physical demands associated with transitions between these functions.

Driver Function and Information Requirements

Because potential ATIS applications have only recently emerged, the precise driver information requirements are only vaguely understood. Issues in this category address the need to understand how information requirements differ between types of drivers (commercial/private, experienced/novice) and types of environments (rural/urban, familiar/unfamiliar). These issues also include questions concerning what information should appear automatically and what information drivers should request manually.

Reliability, Timing, and Priority of Information

ATIS can provide drivers with a large amount of information not currently available to drivers, and it is unclear how drivers will adapt their behavior to this information. Issues in this category address how to present and prioritize information so that drivers safely perceive and respond. Also, the inherent difficulties in obtaining consistently reliable information concerning traffic and road conditions suggest that it may be important to understand how drivers react to inaccurate information.

Interface Form and Modality

ATIS development can draw upon a wide variety of display and input device technology; however, a clear understanding of how to match this technology to driver requirements has not been developed. Interface form refers to the physical characteristics and configuration of the interface, and does not describe the information content. For example, interface form differentiates between touch screens and keypads, and between analog and digital displays. Issues associated with interface form and modality include the effect of different display devices on warning message comprehension and perceived criticality, and the need to identify appropriate display devices for the different types of information provided by an ATIS.

Time Sharing, Attention, and Workload

Introducing ATIS devices to the driver has the potential to provide the driver with an overwhelming amount of information that could escalate workload above acceptable levels. At the same time, in-vehicle systems may reduce workload by providing advance warning of dangerous situations. Issues in this section focus on identifying when drivers should use various ATIS functions, and how interaction with ATIS can be structured to minimize detrimental effects and maximize beneficial effects on driving performance.

Effect of ATIS on Driving Performance

Because ATIS devices may increase driver workload and draw driver attention away from the roadway, they have the potential to adversely affect driver performance. One potential effect of these systems is their tendency to draw driver attention into the vehicle and away from the roadway. This may lead to a failure to detect potentially important external events. Over reliance on ATIS hazard warnings may also lead drivers to blindly accept potentially erroneous warning information. This could lead, for example, to excessive speeds on icy roadways.

Driver Acceptance

Because the success of nearly all ATIS functions depends on driver acceptance and willing cooperation, it is critically important to develop a better understanding of how system designers can avoid jeopardizing driver acceptance of these systems. A general issue in this category involves how "lock out" designs that limit the range of function available to the driver while in motion influence driver acceptance. Other general issues in this category address driver tolerance for inaccurate information and driver preference for various interface characteristics (e.g., types of map symbols and repetition frequency of verbal warnings).

Navigation and Route Selection Strategies

A large component of ATIS involves aiding drivers trip planning, routing selection, and navigation, so understanding how drivers will adapt to these features is an important consideration in developing new systems. For instance, if an ATIS provides a non-intuitive route a driver might not appreciate the time savings that it generates, and so may not see any advantage in using the system. Similarly, the utility of automatic routing systems depends on the drivers' abilities to specify useful routing parameters and algorithms. Routing parameters and algorithms refer to the mathematical operations that guide the system's choice of route. Potential routing parameters include fastest route, shortest route, safest route, fewest number of turns. These parameters may vary dramatically with driver characteristics, such as age, familiarity with the area, current goals (e.g., sight seeing, commuting journey).

Training and Education

Since it is unlikely that drivers will choose to engage in much training, systems should be developed to minimize training requirements. To address remaining training requirements, it may be possible to include help functions and incorporate training directly into system operation. Research needs to establish exactly how these techniques might enhance driver operation of ATIS/CVO systems. Research has shown a strong relationship between understanding of system functions and increased acceptance, so training may be an effective way to promote greater acceptance for system capabilities.

Design and Presentation of Human Factors Design Guidelines

The end product of this project is a set of design guidelines to ensure human factors considerations are addressed in new systems. To be successful, these guidelines must be more than a list of human limits, they must be tailored to the needs of the designers. To develop guidelines that designers will actually use requires a deep understanding of the design process. This understanding can help identify an appropriate structure (e.g., relationships between information in the guideline compilation) and format (e.g., computer-based hypertext, traditional paper–based, or an expert system).

Research Strategies and Methods

Executing and interpreting human factors studies requires a commitment to rigorous research strategies and methods. This is particularly true for a comprehensive program that must combine results into a unified product. Measures of driver workload provide a good example of how a robust experimental method, based on a multivariate construct of workload, could avoid potential confusion of apparently conflicting results. Developing comprehensive measures is another example. Broad, comprehensive measures of driver acceptance and workload could provide a link between studies in this project and research elsewhere.

 

Top

 

CRITERIA

To identify the criticality of each issue, we request a rating on each of several criteria. These criteria include: 1) the role of the issue in achieving ITS goals, 2) how likely a study will address unresolved issues related to developing ATIS human factors guidelines, and 3) the feasibility of the associated experimental methodology.

The ITS America Strategic Plan (Intelligent Vehicle-Highway Society of America, 1992, Task B working paper) identified five major goals which ITS should support. Because ATIS design guidelines should help designers create systems that achieve these goals the criticality of any issue depends on how closely it relates to each ITS goal. The following short definitions describe the five ITS America goals.

Congestion

Decreased traffic congestion has been identified as the primary goal of ITS. Major benefits include better use of roadway capacity by shifting traffic from congested roadways to routes with excess capacity. Benefits to both private and commercial vehicle operators are seen as directly related to reduced travel time. Decreased traffic congestion may result from a variety of ATIS functions including those that provide travelers with information regarding alternative modes of transportation, such as bus, rail, air, and ride-sharing. Issues directly related to reducing congestion will be more critical.

Safety

ITS technology is seen as providing an opportunity to improve safety by reducing crashes, contrasted with the traditional approach of increasing "crash worthiness." The basic strategies identified for improving safety center around avoiding areas of congestion, being warned of hazards, reducing levels of congestion (that are associated with a higher incidence of accidents). Another possible positive aspect of ATIS would be the reduction in frequency of drivers simultaneously holding a map while driving a vehicle. ATIS may also jeopardize safety. Specifically, concerns include erroneously directing drivers down one-way streets the wrong direction, reducing the time spent by drivers monitoring the roadway, and reducing reaction time to unanticipated hazards due to high levels of mental workload. Issues closely related to driver safety will be more critical.

Mobility

The objective of increased and higher quality mobility is used to refer to a broad range of associated performance requirements that address the traveler's well being, comfort, enjoyment, and access to travel. Well being, comfort, and enjoyment requirements range from reducing the general level of stress while driving to increasing access to scenic and recreation areas. Access to travel requirements includes both improved automobile access, as well as improved access to alternative modes of travel. Finally, this objective is commonly referred to when noting the requirement to increase the mobility of the elderly, disabled, and economically disadvantaged segments of the population. Critical issues will directly relate to enhanced mobility.

Environment

The objective of environmental quality includes improved energy efficiency and other benefits such as reduced noise pollution, reduced travel, and shifts in the mode of travel. Improved environmental quality and energy efficiency will be accomplished by decreasing traffic congestion, diverting travelers from single-occupancy vehicles, accommodating smoother more evenly distributed traffic flow, and reducing travel time demand management based on road pricing. Issues closely related to furthering the environment should be rated highly.

Economic

The objective of improved economic productivity has consequences for both commercial and private drivers. From the institutional perspective, this objective can be achieved by reducing total institutional expenditures for the transportation infrastructure. From the individual and CVO perspective, improved economic productivity relates to specific gains by individuals and commercial operators. A closely-related objective is improved energy consumption, which translates into cost savings for all components of the economy. Issues closely related to furthering the economic improvement goals of ITS should be rated highly.

Beyond the relationship to the ITS goals, the criticality of issues depends on several other criteria that address how well studies will support a technical basis for human factors guideline development. These criteria include how far a study extends beyond existing data, potential to support guideline development, relevance to issues concerning older drivers and younger, inexperienced drivers.

Existing Data

Although human factors research from other applications may help resolve many issues associated with ATIS development, placing advanced ATIS technology in cars and trucks involves a number of unique issues. The most critical issues on the list are those for which no previous research exists and so studies that resolve questions that no existing data address should receive high ratings.

Guidelines

The purpose of this project is to develop human factors design guidelines for ATIS, not to simply examine how driver behavior may change as a result of these systems. Therefore, studies/issues that directly address potential design alternatives and would support designers' data base of human limits and abilities should receive high ratings.

Older Drivers

Older drivers present a particular challenge to ATIS design. They have special needs and limits that if addressed will help ensure a flexible system that can be adjusted to meet the special needs of a variety of populations. Because of the need to devote special attention to the needs of older drivers, studies particularly well suited should receive high ratings.

Younger Drivers

Younger drivers have highest accident rates and so special attention should be paid to enhancing their safety through ATIS designs.

Besides the issues related to the development of ATIS design guidelines, the criticality of the issues depends on the feasibility of the studies they imply. The feasibility of each study is assessed considering potential choices of methodologies. The same criteria apply for laboratory experiments, field experiments, or surveys. These criteria include the cost and length of the experiment, the practicality, the generality of the results, and the suitability of the methodology for the particular study.

Cost

To complete this rating, the rating scale corresponds to rough estimates of how much the study will cost using a particular methodology. For instance, a simple laboratory study may cost $25,000 and should receive a low rating of "1." A complicated on-road study may cost much more and may deserve a rating of "3." The rough estimates should include staff time and materials associated with data collection, analysis, and report preparation.

Time

This rating reflects the anticipated duration of the experiment with larger ratings corresponding to greater time requirements. As with cost estimates, this rating should be based on a rough estimate of the resource requirements of each study, using a particular methodology. For instance, a laboratory study to examine a simple issue may require less than 3 months, and a large survey study may take more than a year. The time estimate should span the entire duration of the experiment, from data collection to analysis and report preparation.

Practicality

This rating reflects the anticipated likelihood of success of the study. A practical study, given the constraints of the methodology and resources, is likely to resolve the issue in question. A simple issue, easily resolved with the given methodology, should receive a high rating of practicality.

Generality

Some issues address specific problems associated with single ATIS functions, while others address general human limits. General human limits may have more general implications for the design of many ATIS functions. A rating of high generality would result from a methodology that combines with an issue to provide information that applies to a wide variety of circumstances. For instance, a study that identifies a method for predicting driver workload could have benefits for a wide number of applications and should receive a "5."

Suitability

Suitability refers to how well a particular methodology applies to resolving the issue. Laboratory, field and survey studies all have limits and benefits and their suitability depends on how well they match the particular issue. For example, laboratory studies will never completely replicate the driving experience and so may not be suitable to address many issues. When the capabilities of the methodology match the requirement of the issue, suitability should receive a rating of "5."

 

Top

 

STUDIES/ISSUES RATING FORM

Studies/Issues Rating Form section 1

Studies/Issues Rating Form section 2

Studies/Issues Rating Form section 3

Studies/Issues Rating Form section 4

Studies/Issues Rating Form section 5

Studies/Issues Rating Form section 6

Studies/Issues Rating Form section 7

Studies/Issues Rating Form section 8

Studies/Issues Rating Form section 9

Studies/Issues Rating Form section 10

Studies/Issues Rating Form section 11

Studies/Issues Rating Form section 12

Studies/Issues Rating Form section 13

Studies/Issues Rating Form section 14

Studies/Issues Rating Form section 15

Studies/Issues Rating Form section 16

Studies/Issues Rating Form section 17

Studies/Issues Rating Form section 18

Studies/Issues Rating Form section 19

Studies/Issues Rating Form section 20

 

Top

 

VALIDATION RATING FORMS

Table 20. Validation rating forms--List A.

A1 Examine the cognitive demands placed on the driver by the need to transition from one ATIS function to another.
A4 Identify how complex interactions among ATIS functions might affect driver understanding and response to the system.
A5 Examine the workload implications of requiring drivers to transform and enter information into the system.
B2 Identify information drivers need and want from in-vehicle road sign (ISIS) and warning systems (IVSAWS).
B3 Identify how the information drivers need and want from road sign (ISIS) and warning systems (IVSAWS) might influence behavior.
C4 Examine how information reliability (e.g., false alarms) influences driver adaptation and enhances the potential for an improper response to ISIS/IVSAWS.
C5 Investigate how to display multiple ISIS and IVSAWS messages so that drivers can identify relevant information and react appropriately.
C6 Examine how the timing of ISIS and IVSAWS information, with respect to the location of the incident, influences driver reaction to the information.
D2 Identify features that will benefit/require standardization across many types of ATIS systems and functions.
D12 Evaluate the effectiveness of multimodality displays, such as voice in combination with text.
D21 Examine how display design might aid the dynamic allocation of driver visual and cognitive resources.
E11 Identify how the dynamic characteristics of driver workload interact with the form of the ATIS interface.
E14 Examine how in-vehicle road sign information (e.g., ISIS) affects workload, especially under nighttime, poor weather, and other reduced visibility conditions.
F1 Examine how attention to different types of ATIS information influences the primary task of driving.
F2 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events.
D20 Examine how information can be displayed to dispatchers to support the complex decision–making process associated with allocation of emergency response crews.
D21 Examine how display design might aid the dynamic allocation of driver visual and cognitive resources.
E2 Evaluate how different types of information, displayed using a HUD, affect cognitive attention devoted to the roadway.
E10 Identify the compensatory actions drivers take to moderate their workload, and how ATIS/CVO systems might affect those actions.
F2 Examine how route guidance systems might adversely influence driver detection and recognition of unusual roadway events.
J1 Investigate how to structure design guidelines to help designers address human factors issues in ATIS designs which support the needs of the driver.

DELETE

ADD (Maximum of 1 entry):

 

Table 21. Validation rating forms--List C.

B1 "Identify how specific information needs vary as a function of driver characteristics (e.g., age, gender, etc.)."
B5 Identify what types of ATIS information should be available upon request.
C1 Identify how priorities specific to CVO information compare to other ATIS information.
C5 Investigate how to display multiple ISIS and IVSAWS messages so that drivers can identify relevant information and react appropriately.
C9 Examine how the reliability and priority of regulatory information affect CVO driver workload.
D4 Examine the performance differences associated with focusing all ISIS and IVSAWS information through either single or multiple display channels.
D7 "Examine the effect of display form (e.g., text vs. graphic) on driver decision making and problem solving during route planning and selection."
D13 Identify the relationship between icon characteristics and information types that maximize icon effectiveness and salience.
D18 "Identify the display design characteristics required to support ATIS/CVO systems in large, noisy, and vibration prone commercial vehicles."
D20 Examine how information can be displayed to dispatchers to support the complex decision-making process associated with allocation of emergency response crews.
E8 Identify how ATIS information flow might be managed to capitalize on the dynamic nature of driver workload.
E9 Identify how estimates of real-time driver workload can be used to avoid overload by moderating information from ATIS.
E12 Examine the requirements to support the complex onboard data management requirements that commercial vehicle drivers experience.
G2 "Evaluate driver acceptance of "lock–out" designs that only allow driver to access functions under certain circumstances."
G10 Examine how the accuracy of information pertaining to availability and current deployment of resources affects dispatcher acceptance and interaction with the system.
K4 Develop a set of consistent subjective measures across experiments so that rating scales are common across experiments.

DELETE

ADD (Maximum of 1 entry):

 

Top

 

FHWA-RD-96-177

 

Previous | Table of Contents | Next

ResearchFHWA
FHWA
United States Department of Transportation - Federal Highway Administration