U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
 |
| This report is an archived publication and may contain dated technical, contact, and link information |
|
Publication Number: FHWA-RD-96-177
Date: October 1997 |
Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Definition and Prioritization of Research Studies
APPENDIX B: STUDY/ISSUE RATING FORMS
Our analysis of the rating data is divided into four parts. First, we consider the criteria in table 3 one at a time. Second, we use a linear model to combine ratings on several criteria. Third, we perform a sensitivity analysis to determine the effects of imperfect weightings in the linear model. Fourth, we validate the combination model used.
INSTRUCTIONS FOR RATING ATIS/CVO STUDIES/ISSUESSTUDIES/ISSUESCRITERIASTUDIES/ISSUES RATING FORMVALIDATION RATING FORMS
INSTRUCTIONS FOR RATING ATIS/CVO STUDIES/ISSUESPrevious tasks of this project have identified a broad range of concerns which human factors research does not address. For example, a technical basis for ATIS/CVO (Advanced Traveler Information System/Commercial Vehicle Operations) is not covered within human factors guidelines. To develop a technical basis for human factors guidelines, the Battelle team will conduct a series of laboratory, survey, and on-road studies. However, the number of potential issues that could be studied far exceeds the resources to conduct those studies. Therefore, to maximize the benefit of the studies, the issues require prioritization. Issue priority depends upon a weighted average of several criteria. To support this prioritization process we ask you to use the attached sheets to rate all studies/issues according to all the criteria. The left side of the rating sheet begins with a one-sentence description of the research issue/study. Immediately to the right is a series of criteria associated with the ITS America objectives and other criteria associated with how well the studies would support guideline development. Please rate each of these nine criteria using the rating scale shown in the upper left corner of the rating sheet. The larger the value, the more closely studies/issues meet the criteria. For example, a study to examine the effects of a route guidance system on freeway congestion might receive a "5" under the "Congestion" criteria. A study that goes beyond all existing data would receive a "5" for "Existing data." Please enter a rating for all studies/issues. Criteria for rating the feasibility of the study associated with each issue lie on the right hand side of the rating sheet. These criteria address the cost, time, practicality, generality, and suitability associated with potential laboratory, field, and survey studies. Please rate all criteria for the different types of studies (laboratory, field and survey studies). Scales for these ratings lie at the far right of the rating sheet. The cost rating scale applies to the "Cost" criteria for the laboratory, field, and survey methodologies. The larger the number, on a 3-point scale, the more expensive the study. The time rating scale applies to the "Time" criteria for laboratory, field, and survey methodologies. Larger values correspond to longer experiments, as measured in months to complete. The practicality, generality, and suitability criteria are rated with a scale on the lower right of the rating sheet. The more practical, general, or suitable the study and methodology the larger the rating. For example, laboratory studies will never be able to replicate the complete driving experience and so will not be able to address many issues. In these situations "Suitability" under "Laboratory Methodology" would receive a rating of "1." Please enter a rating for all criteria, even if a one or more methodologies appear completely unsuitable. The issues shown on the rating sheets are a composite of lists and summaries provided by several scientists. Many of whom, alas, neglected to supply the requested cross-reference to project working papers. As such, the reference to specific tasks may be ambiguous. However, we would like you to note the task that might contain the issue. Even better, if you know a page number please include it as well. We do not expect anyone to fill in references for all issues, but we would like benefit from the broad perspective of the group by identifying where these issues originated. Thus, where possible, please insert the initial of the form from where the issue came in the far right column. For example, insert "E298" to signify that the issue originated from page 298 of the Task E working paper. Thank you very much for your patience with this task; we appreciate your help in rating all criteria for all studies/issues. Your ratings will provide a valuable resource in determining the priority of the ATIS human factors studies that will follow. The following pages describe the general categories of studies/issues and the criteria used to judge them. The descriptions of issue/study categories augment the single sentence explanations on the rating sheets. Following the descriptions of issue/study categories each criterion is described in more detail.
STUDIES/ISSUESThe issues fall into 11 general categories to make the diverse set of issues more comprehensible. The range of issues includes the need to identify the factors that govern how drivers adapt to ATIS, the need to understand how best to present human factors guidelines to designers so that human factors concerns are addressed in future ATIS, and the need to develop sound experimental methodologies to study ATIS. The 11 general categories are defined below. Coordination of Multiple ATIS Functions ATIS will, in all likelihood, include a wide variety of functions that range from route planning, services/attractions direction and route guidance to automatic aid requesting and vehicle condition monitoring. The human factors issues in this category address the need to understand the cognitive and physical demands associated with transitions between these functions. Driver Function and Information Requirements Because potential ATIS applications have only recently emerged, the precise driver information requirements are only vaguely understood. Issues in this category address the need to understand how information requirements differ between types of drivers (commercial/private, experienced/novice) and types of environments (rural/urban, familiar/unfamiliar). These issues also include questions concerning what information should appear automatically and what information drivers should request manually. Reliability, Timing, and Priority of Information ATIS can provide drivers with a large amount of information not currently available to drivers, and it is unclear how drivers will adapt their behavior to this information. Issues in this category address how to present and prioritize information so that drivers safely perceive and respond. Also, the inherent difficulties in obtaining consistently reliable information concerning traffic and road conditions suggest that it may be important to understand how drivers react to inaccurate information. Interface Form and Modality ATIS development can draw upon a wide variety of display and input device technology; however, a clear understanding of how to match this technology to driver requirements has not been developed. Interface form refers to the physical characteristics and configuration of the interface, and does not describe the information content. For example, interface form differentiates between touch screens and keypads, and between analog and digital displays. Issues associated with interface form and modality include the effect of different display devices on warning message comprehension and perceived criticality, and the need to identify appropriate display devices for the different types of information provided by an ATIS. Time Sharing, Attention, and Workload Introducing ATIS devices to the driver has the potential to provide the driver with an overwhelming amount of information that could escalate workload above acceptable levels. At the same time, in-vehicle systems may reduce workload by providing advance warning of dangerous situations. Issues in this section focus on identifying when drivers should use various ATIS functions, and how interaction with ATIS can be structured to minimize detrimental effects and maximize beneficial effects on driving performance. Effect of ATIS on Driving Performance Because ATIS devices may increase driver workload and draw driver attention away from the roadway, they have the potential to adversely affect driver performance. One potential effect of these systems is their tendency to draw driver attention into the vehicle and away from the roadway. This may lead to a failure to detect potentially important external events. Over reliance on ATIS hazard warnings may also lead drivers to blindly accept potentially erroneous warning information. This could lead, for example, to excessive speeds on icy roadways. Driver Acceptance Because the success of nearly all ATIS functions depends on driver acceptance and willing cooperation, it is critically important to develop a better understanding of how system designers can avoid jeopardizing driver acceptance of these systems. A general issue in this category involves how "lock out" designs that limit the range of function available to the driver while in motion influence driver acceptance. Other general issues in this category address driver tolerance for inaccurate information and driver preference for various interface characteristics (e.g., types of map symbols and repetition frequency of verbal warnings). Navigation and Route Selection Strategies A large component of ATIS involves aiding drivers trip planning, routing selection, and navigation, so understanding how drivers will adapt to these features is an important consideration in developing new systems. For instance, if an ATIS provides a non-intuitive route a driver might not appreciate the time savings that it generates, and so may not see any advantage in using the system. Similarly, the utility of automatic routing systems depends on the drivers' abilities to specify useful routing parameters and algorithms. Routing parameters and algorithms refer to the mathematical operations that guide the system's choice of route. Potential routing parameters include fastest route, shortest route, safest route, fewest number of turns. These parameters may vary dramatically with driver characteristics, such as age, familiarity with the area, current goals (e.g., sight seeing, commuting journey). Training and Education Since it is unlikely that drivers will choose to engage in much training, systems should be developed to minimize training requirements. To address remaining training requirements, it may be possible to include help functions and incorporate training directly into system operation. Research needs to establish exactly how these techniques might enhance driver operation of ATIS/CVO systems. Research has shown a strong relationship between understanding of system functions and increased acceptance, so training may be an effective way to promote greater acceptance for system capabilities. Design and Presentation of Human Factors Design Guidelines The end product of this project is a set of design guidelines to ensure human factors considerations are addressed in new systems. To be successful, these guidelines must be more than a list of human limits, they must be tailored to the needs of the designers. To develop guidelines that designers will actually use requires a deep understanding of the design process. This understanding can help identify an appropriate structure (e.g., relationships between information in the guideline compilation) and format (e.g., computer-based hypertext, traditional paper–based, or an expert system). Research Strategies and Methods Executing and interpreting human factors studies requires a commitment to rigorous research strategies and methods. This is particularly true for a comprehensive program that must combine results into a unified product. Measures of driver workload provide a good example of how a robust experimental method, based on a multivariate construct of workload, could avoid potential confusion of apparently conflicting results. Developing comprehensive measures is another example. Broad, comprehensive measures of driver acceptance and workload could provide a link between studies in this project and research elsewhere.
CRITERIATo identify the criticality of each issue, we request a rating on each of several criteria. These criteria include: 1) the role of the issue in achieving ITS goals, 2) how likely a study will address unresolved issues related to developing ATIS human factors guidelines, and 3) the feasibility of the associated experimental methodology. The ITS America Strategic Plan (Intelligent Vehicle-Highway Society of America, 1992, Task B working paper) identified five major goals which ITS should support. Because ATIS design guidelines should help designers create systems that achieve these goals the criticality of any issue depends on how closely it relates to each ITS goal. The following short definitions describe the five ITS America goals. Congestion Decreased traffic congestion has been identified as the primary goal of ITS. Major benefits include better use of roadway capacity by shifting traffic from congested roadways to routes with excess capacity. Benefits to both private and commercial vehicle operators are seen as directly related to reduced travel time. Decreased traffic congestion may result from a variety of ATIS functions including those that provide travelers with information regarding alternative modes of transportation, such as bus, rail, air, and ride-sharing. Issues directly related to reducing congestion will be more critical. Safety ITS technology is seen as providing an opportunity to improve safety by reducing crashes, contrasted with the traditional approach of increasing "crash worthiness." The basic strategies identified for improving safety center around avoiding areas of congestion, being warned of hazards, reducing levels of congestion (that are associated with a higher incidence of accidents). Another possible positive aspect of ATIS would be the reduction in frequency of drivers simultaneously holding a map while driving a vehicle. ATIS may also jeopardize safety. Specifically, concerns include erroneously directing drivers down one-way streets the wrong direction, reducing the time spent by drivers monitoring the roadway, and reducing reaction time to unanticipated hazards due to high levels of mental workload. Issues closely related to driver safety will be more critical. Mobility The objective of increased and higher quality mobility is used to refer to a broad range of associated performance requirements that address the traveler's well being, comfort, enjoyment, and access to travel. Well being, comfort, and enjoyment requirements range from reducing the general level of stress while driving to increasing access to scenic and recreation areas. Access to travel requirements includes both improved automobile access, as well as improved access to alternative modes of travel. Finally, this objective is commonly referred to when noting the requirement to increase the mobility of the elderly, disabled, and economically disadvantaged segments of the population. Critical issues will directly relate to enhanced mobility. Environment The objective of environmental quality includes improved energy efficiency and other benefits such as reduced noise pollution, reduced travel, and shifts in the mode of travel. Improved environmental quality and energy efficiency will be accomplished by decreasing traffic congestion, diverting travelers from single-occupancy vehicles, accommodating smoother more evenly distributed traffic flow, and reducing travel time demand management based on road pricing. Issues closely related to furthering the environment should be rated highly. Economic The objective of improved economic productivity has consequences for both commercial and private drivers. From the institutional perspective, this objective can be achieved by reducing total institutional expenditures for the transportation infrastructure. From the individual and CVO perspective, improved economic productivity relates to specific gains by individuals and commercial operators. A closely-related objective is improved energy consumption, which translates into cost savings for all components of the economy. Issues closely related to furthering the economic improvement goals of ITS should be rated highly. Beyond the relationship to the ITS goals, the criticality of issues depends on several other criteria that address how well studies will support a technical basis for human factors guideline development. These criteria include how far a study extends beyond existing data, potential to support guideline development, relevance to issues concerning older drivers and younger, inexperienced drivers. Existing Data Although human factors research from other applications may help resolve many issues associated with ATIS development, placing advanced ATIS technology in cars and trucks involves a number of unique issues. The most critical issues on the list are those for which no previous research exists and so studies that resolve questions that no existing data address should receive high ratings. Guidelines The purpose of this project is to develop human factors design guidelines for ATIS, not to simply examine how driver behavior may change as a result of these systems. Therefore, studies/issues that directly address potential design alternatives and would support designers' data base of human limits and abilities should receive high ratings. Older Drivers Older drivers present a particular challenge to ATIS design. They have special needs and limits that if addressed will help ensure a flexible system that can be adjusted to meet the special needs of a variety of populations. Because of the need to devote special attention to the needs of older drivers, studies particularly well suited should receive high ratings. Younger Drivers Younger drivers have highest accident rates and so special attention should be paid to enhancing their safety through ATIS designs. Besides the issues related to the development of ATIS design guidelines, the criticality of the issues depends on the feasibility of the studies they imply. The feasibility of each study is assessed considering potential choices of methodologies. The same criteria apply for laboratory experiments, field experiments, or surveys. These criteria include the cost and length of the experiment, the practicality, the generality of the results, and the suitability of the methodology for the particular study. Cost To complete this rating, the rating scale corresponds to rough estimates of how much the study will cost using a particular methodology. For instance, a simple laboratory study may cost $25,000 and should receive a low rating of "1." A complicated on-road study may cost much more and may deserve a rating of "3." The rough estimates should include staff time and materials associated with data collection, analysis, and report preparation. Time This rating reflects the anticipated duration of the experiment with larger ratings corresponding to greater time requirements. As with cost estimates, this rating should be based on a rough estimate of the resource requirements of each study, using a particular methodology. For instance, a laboratory study to examine a simple issue may require less than 3 months, and a large survey study may take more than a year. The time estimate should span the entire duration of the experiment, from data collection to analysis and report preparation. Practicality This rating reflects the anticipated likelihood of success of the study. A practical study, given the constraints of the methodology and resources, is likely to resolve the issue in question. A simple issue, easily resolved with the given methodology, should receive a high rating of practicality. Generality Some issues address specific problems associated with single ATIS functions, while others address general human limits. General human limits may have more general implications for the design of many ATIS functions. A rating of high generality would result from a methodology that combines with an issue to provide information that applies to a wide variety of circumstances. For instance, a study that identifies a method for predicting driver workload could have benefits for a wide number of applications and should receive a "5." Suitability Suitability refers to how well a particular methodology applies to resolving the issue. Laboratory, field and survey studies all have limits and benefits and their suitability depends on how well they match the particular issue. For example, laboratory studies will never completely replicate the driving experience and so may not be suitable to address many issues. When the capabilities of the methodology match the requirement of the issue, suitability should receive a rating of "5."
STUDIES/ISSUES RATING FORM
VALIDATION RATING FORMSTable 20. Validation rating forms--List A.
DELETE ADD (Maximum of 1 entry):
Table 21. Validation rating forms--List C.
DELETE ADD (Maximum of 1 entry):
FHWA-RD-96-177
|
