• Federal Highway Administration >
• Publications >
• Research Publications >
• 95197 >
• Sec9.Cfm >
• Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Comparable Systems Analysis

Development of Human Factors Guidelines for Advanced Traveler Information Systems and Commercial Vehicle Operations: Comparable Systems Analysis

CHAPTER 9. THE SIKORSKY COGNITIVE DECISION–AIDING SYSTEM

GENERAL SYSTEM DESCRIPTION AND OBJECTIVES

USER INTERFACE

DESIGN GUIDELINES USED

LESSONS LEARNED

GENERAL SYSTEM DESCRIPTION AND OBJECTIVES

The Army's Aviation Advanced Technology Directorate (AATD) sponsors the Rotorcraft Pilot's Associate (RPA) program, which is intended to promote the application of advanced technology to enhance man–machine performance in Army aviation. Their approach has been to emphasize cognitive decision aiding (CDA) across a variety of cockpit tasks and subsystems. One aspect of that program is to develop prototype systems for aiding the pilot in navigation and piloting tasks (flying, navigating, and communicating) during extreme conditions such as nap–of–the–earth (NOE) flight at night in adverse weather. This Day–Night Adverse Weather Pilotage System (D/NAPS) program is a subset of RPA and is intended to enhance mission effectiveness through innovative integration of advanced sensors, computing technologies, and controls and displays.

Sikorsky Aircraft and Texas Instruments were one of two teams of contractors selected by AATD to develop and demonstrate a Cognitive Decision–Aiding System (CDAS) for the D/NAPS program. The CDAS was selected as one of the non–ground transportation systems in the ATIS Comparable Systems Analysis because it shares certain features with future intelligent navigation systems in ground transportation. The CDAS objectives were to aid the pilot in determining appropriate navigation and route selection decisions, to combine conventional and Artificial Intelligence (AI) processing to reason about data received from multiple sources (including the pilot), to use expert systems to formulate recommendations or decision augmentation, and to perform within operational time constraints.

The Sikorsky CDAS focused on mission replanning. After a pilot has established a route, perhaps with the aid of a mission planning system, and has begun to fly the mission, he or she may encounter unanticipated threats or other factors that override the planned route. Replanning is a difficult task for the pilot while fully engaged in flight tasks. Thus, the CDAS provides an expert system to process information about alternate routes while the pilot continues to focus on flying the aircraft at or below treetop level (Casper, 1993). NOE flight involves maintaining minimal clearance over terrain to limit exposure to threats. It requires nearly constant out–the–window attention to maintain clearance over ground and vegetation. Also, the pilot must keep hands on the flight controls for continual adjustment of the flight path. This "eyes out – hands on" requirement, plus communications, navigation, and other cockpit tasks, induce high–workload conditions that make navigation replanning particularly difficult for the hypothetical single–crew situation (Casper, Smith, Smith, and Hubanks, 1991).

Similar difficulties in navigation replanning may be faced by a driver in heavy traffic who has missed a turn and cannot safely consult a paper map to investigate alternate routes. The CDAS also may generate an alternate route when a mission change is promulgated by higher command. Similarly, a Commercial Vehicle Operator (CVO) may receive a priority change in destination from a dispatcher while en route and engaged in heavy traffic.

One objective of the Sikorsky effort was to achieve an intelligent Pilot–Vehicle Interface (PVI). The concept is to combine technological advances in automation and artificial intelligence with human factors engineering to fully integrate the pilot, the cockpit, and the aircraft (Casper et al., 1991). Key elements of the intelligent PVI are seen as pilot command assessment, pilot capability assessment, pilot intent assessment, and cockpit display management. The CDAS included a set of six cooperating expert systems that provided a single pilot with assistance in threat avoidance, navigation, and system failures.

The CDAS demonstration was accomplished on a full–mission simulator, comparing a baseline configuration, representing the RAH–66 Comanche, and the combination of the Comanche and CDAS. The CDAS calculated a route "cost" estimate based on terrain, threat location and lethality, fuel, time, distance, weather, and other information. When triggered by various events, the mission replanner computed alternative routes and presented the least–cost recommended route on the digital map. The recommended alternate route is presented to the pilot by a distinct color on the digital map.

Several CDAS evaluations were conducted in simulation, the results of which are not yet published in the open literature (Casper, 1993). The ATIS functional characteristics that apply to the Sikorsky system are shown in table 8.

Top

USER INTERFACE

General Description

The pilot interface to the CDAS was implemented through four displays--the digital map, the Helmet-Mounted Display (HMD) symbology, the Right Multi-Purpose Display (RMPD), and by synthesized speech output. Pilot controls included bezel switches around the digital map screen and a cursor control on the handle of the collective (flight control).

Visual Information Display

Symbology overlayed on the digital map was the primary display of route information. The display included waypoint locations and a path indication between the waypoints. When the CDAS recommended a new route, it was displayed in red to distinguish it from the original route, which was displayed in yellow. The portion of the route already flown was displayed in blue.

Table 8. Comparison of Sikorsky functions with those from ATIS/CVO systems.

In addition, waypoints were shown as symbology on the HMD. They appeared as earth–referenced signs, similar to a stop sign, but with a circular rather than octagonal top, with the waypoint number and distance depicted on the sign. The round portion of the sign for the upcoming waypoint was rendered a constant 2.58 degrees visual angle and the height of the sign pole was proportional to the aircraft altitude. This meant that the sign at the waypoint was always easy to see and appeared to be at the same altitude as the ownship. When a new waypoint was out of the field of view of the HMD by less than 30 degrees, an indication (> or <) was given showing the direction to the next waypoint. If the waypoint was greater than 30 degrees outside of the HMD field of view, the indication appeared as double arrows.

The alternate route information computed by the intelligent software and displayed on the digital map is directly relevant to ATIS/CVO systems. Also, the earth–referenced symbology shown in the HMD has potential application for future ATIS systems capable of displaying symbology on the vehicle windshield as a head–up display (HUD). One major difference, however, is that in the aircraft application, the field of view is wider and the user can slew it by natural head movements. In a ground vehicle, earth–referenced symbology on the windshield would be limited to the windshield field of view. Some ground–vehicle applications could be envisioned, however, where the driver wears a helmet, thus providing an extra safety margin, plus a head–oriented display medium.

When a new route was calculated by the CDAS and presented to the pilot, this fact was indicated by an icon on the RMPD. The icon represented three joined waypoints enclosed in a box. The box was depicted in inverse video when CDAS was calculating a new route. It changed to normal video when the new route was computed and presented.

Adjacent to the new–route icon was an "explanation" icon that gave an indication of the reason for calculating a new route. Examples include Fuel, Off Course, Threat, and Message (referring to receipt of a digital message that redirected the mission or warned of bad weather).

The digital map also had the capability to display other aids to location and navigation, such as grid lines (2 km), three levels of map scale, and coordinates (equivalent to latitude/longitude).

Auditory Information Display

When the system began to calculate a new route, in addition to displaying an icon, a synthesized voice message also was given, such as, "Threat; Planning," followed by "New Plan Ready." Time to compute a new route was around 5 seconds. Speech output also was given to note deviations in either time or course from the planned route.

Other speech output was associated with the baseline Comanche cockpit and included threat information, aircraft health and status information, and cautions and warnings, such as, "Warning, Engine Failure." In the simulator, the speech was produced by a Dec Talk speech synthesis system. Examples of speech output information are location and type of threat and, if lethality appears high, it suggests avoidance maneuvers.

User Input (Controls)

The pilot did not initiate a request for navigation assistance. The pilot did have normal control over the digital map (independent of the CDAS), including zoom in/out and slew (X,Y reposition of map). Other relevant controls included a toggle that either centered the map on the ownship symbol, or froze the map and allowed the ownship symbol to move over the frozen map.

The pilot activated an "Accept" button to acknowledge and accept a new route recommended by the CDAS.

Communications Systems

Communication tasks were included as part of the mission in the CDAS simulation demonstration, but were not an integral part of the decision–aiding system.

Cognitive Demands

The objective of the CDAS was to aid pilots in navigation and route replanning under adverse conditions when cognitive task loading is extreme (Casper et al., 1991). Flying at NOE levels at night and in adverse weather is assumed to be a worst–case scenario for pilot workload. It is under such extreme cases of cognitive demand that expert systems for decision aiding can be most beneficial. Under such circumstances, pilots are in a moment–to–moment survival situation and simply do not have spare resources to evaluate multiple sources of information and thoroughly consider alternative navigation solutions. Because this level of demand was difficult to achieve in the simulator, an extra task was given to the pilots, namely, an "authentication" task, essentially requiring mental arithmetic and verbal report.

System Temporal Requirements

The CDAS demonstration required that the system operate in real time with respect to the pilot's tasks and the simulated mission.

Top

DESIGN GUIDELINES USED

Human Factors Design Guidelines

Human factors guidelines, such as MIL-STD 1472D and Boff and Lincoln (1988), were used in the development of the baseline Comanche aircraft that provided the basis for the simulator. However, for the CDAS itself, no human factors guidelines were used directly. A small group of human factors experts played a role in the development and demonstration of the system and were able to make some inputs to the design of the system. For this demonstration, the emphasis was placed on AI software, expert system architectures, and real–time performance, rather than optimizing the pilot interface to the system.

Top

LESSONS LEARNED

[SK 01] COLOR–CODED ROUTES SUCCESSFULLY DEPICTED RECOMMENDED PATH ON DIGITAL MAP

• The use of color symbology overlays on the digital map was successful in allowing the user to distinguish among: (1) planned route already traveled, (2) planned route to go, and (3) new, recommended alternative route.

• Recommended routes on a digital map are easy to perceive when they are color–coded.

[SK 02] SPEECH OUTPUT SHOULD BE USED FOR EXCEPTIONS, NOT FOR CONFIRMATION OF ROUTE COMPLIANCE

• Pilots did not like a "chatty" speech output system. They wanted to hear from the speech output system only when they deviated from the planned route (in time or course) by a non–trivial amount. This example falls into the "display by exception" concept.

• The use of speech output to confirm conformance with planned route should be avoided. Consider the user a negative–feedback servomechanism and provide route compliance information only when an error needs to be corrected.

[SK 03] THE TIME AT WHICH EXPERT SYSTEMS GENERATE NAVIGATION ADVICE CAN BE OF INTEREST TO THE USER

• The "freshness" of a new route was a concern of CDAS users, particularly if more recent incidents or events may have reduced the validity of the advice.

• Intelligent navigation systems should include information as to the time at which a new route is recommended.

[SK 04] USERS OF NAVIGATION EXPERT SYSTEMS MAY WANT DIAGNOSTIC INFORMATION

• Users of CDAS sometimes wanted information about why a new route was being recommended. They were not satisfied with command information about rerouting, and preferred diagnostic information supporting the recommendation. The expert system should be capable of providing an explanation to the user for a recommended action, if the user requests it.

• At a minimum, such diagnostic information should be available to the user upon request.

[SK 05] DANGEROUS AREAS CAN BE COLOR–CODED ON A DIGITAL MAP

• Current altitude information was made relevant by showing all terrain on the digital map in a different color if it was equal to or greater than current altitude.

• Although there is no direct analog to ground transportation systems, it might be possible to use dynamic color–coding to distinguish potentially dangerous routes, locations, weather, traction (ice), etc.

[SK 06] PILOTS PREFERRED TO FLY IN THE HEADING–UP ORIENTATION ON THE DIGITAL MAP

• Given the option of operating in heading–up or north–up modes, virtually all of the pilots chose to fly in the heading–up mode.

[SK 07] MESSAGE SEMANTICS AND SYNTACTICS IMPORTANT IN SYSTEM DESIGN

• Ambiguities existed in some common usage of terms such as the meaning of "Current Waypoint" and "Next Waypoint."

• Care must be given to semantic and syntactic conventions in the use of language.

[SK 08] NAVIGATION EXPERT SYSTEMS SHOULD ADVISE, BUT NOT ACTIVELY CONTROL WITHOUT USER CONSENT

• Based on input from the pilot SME's, the CDAS was never allowed to execute an action without first announcing the proposed action to the user and receiving a "go–ahead" confirmation. This rule applied to all actions that could affect any current activity of the user, including display mode changes and map scale changes.

• Intelligent software navigation aids should provide information, but not take action without active concurrence of the user (e.g., the expert system should not autonomously slew the map away from the ownship position).

[SK 09] ACKNOWLEDGEMENT OF ACTIONS TAKEN BY AN EXPERT SYSTEM PROVIDE SYSTEM STATUS INFORMATION TO THE USER

• Interaction with the navigation expert system followed the principle of positive communication, e.g., transmit, acknowledge, and confirm. F or example, the expert system might recommend slewing the map to show the location of an important incident (transmit) if the pilot concurs (acknowledge), then the system executes the action and confirms successful execution (confirm).

• Clear communication of system status should be given to the user whenever an expert system performs an action that could affect the user. Expert system actions should not only be withheld until approved, but confirmation should be communicated to the user afterwards.

[SK 10] USER–EXPERT SYSTEM INTERFACE DESIGN MUST BE EXPLAINED TO THE USER

• The responsibilities of the navigation expert system and the user must be assigned and understood. The pilots using CDAS were able to perform better when there was no confusion about which tasks were to be assumed by the CDAS, under what circumstances, and how they were to be performed.

• Effective use of a navigation expert system requires the user to have an accurate mental model of the "behavior" of the expert system. This can be achieved through training and consistent functioning of the user–system interface design.

[SK 11] TIMELY ASSISTANCE FROM THE EXPERT SYSTEM CAN BE ACHIEVED IF THE SYSTEM "ANTICIPATES" THE USER

• Experience with CDAS indicated that timely system performance was achieved by the expert system "anticipating" what navigation information the pilot might require.

• User acceptance is reduced when time–critical navigation information is not readily available.

[SK 12] INTELLIGENT INTERFACES CAN MAKE MULTIPLE DISPLAY MODE CHANGES TO ACHIEVE A USER'S REQUEST FOR INFORMATION

• If the pilot initiates a menu option such as "Show on Map," which refers to a location not currently within the bounds of the map scale, the system could "decide" whether to change the map scale to display both the ownship and the identified location on the same screen, or display a line drawn from the ownship to the identified location.

• Intelligent display control can be responsive to user requests for information, rather than forcing the user to manually implement a series of inputs.

[SK 13] KNOWLEDGE ACQUISITION IS THE KEY TO EFFECTIVE EXPERT SYSTEM DEVELOPMENT

• The successful development of the CDAS was attributed in part to the effort placed into knowledge acquisition early in the design process.

• Knowledge acquisition (sometimes called "knowledge engineering") for the development of expert systems requires thorough, time–consuming analysis that must include detailed knowledge of all sources of information, including sensor capabilities and limitations, user strategies, and system performance objectives.

[SK 14] A USER–VEHICLE INTERFACE FOCUS CONTRIBUTED TO A SUCCESSFUL DECISION–AIDING SYSTEM

• The pilot–vehicle interface, or, more generally, the user–vehicle interface, should be a major focus for the development and application of cognitive decision–aiding systems.

[SK 15] HMD's AND HUD's SHOULD BE USED ONLY FOR CRITICAL INFORMATION

• Navigation and vehicle control information that is non–critical in nature should not be presented on the helmet–mounted display (HMD) unless called for or accepted by the pilot. This same concept is believed to apply for head–up displays (HUD) in ground vehicles.

• Navigation and vehicle control information presented in HMD or HUD formats should not interfere or distract the user from the visual scene of the terrain and environment.

[SK 16] NON–INTRUSIVE CURSOR CONTROL FOR IN–VEHICLE NAVIGATION DISPLAYS PROMOTES CONTINUOUS VEHICLE CONTROL

• Cursor control achieved by a switch on the collective (left–hand flight control lever) enabled pilots to interact with the digital map without taking their hands off of the flight controls. The analog for ATIS/CVO systems would be to provide cursor control on the steering wheel.

• Placing the cursor control function on a primary vehicle control surface (e.g., stick, cyclic, collective, or steering wheel) reduces the user's time–sharing workload.

[SK 17] SME's PROVIDE INVALUABLE INFORMATION FOR THE DESIGN OF INTELLIGENT SYSTEMS

• Subject matter experts (SME's) (Army pilots) were involved at several stages of the CDAS program, particularly during the knowledge acquisition stage of expert system development. Early prototypes produced on SuperCard or HyperCard were used to extract information and opinions from SME's.

• Involvement of SME's and prototyping techniques contributed to successful design and development of the system.

[SK 18] SIMULATION WAS A VALUABLE DEVELOPMENT TOOL

• The CDAS was evaluated in a full–mission simulation context. Difficulties and challenges in simulation, such as performance measurement, also occur in field studies. The simulator allows control over salient variables and repetition of events that cannot be achieved under field–test conditions.

• High–fidelity, operator–in–the–loop simulation is recommended as a test and evaluation medium for advanced technology.

[SK 19] SYSTEM PERFORMANCE MEASUREMENT ISSUES ARE IMPORTANT

• Empirical approaches to the assessment and evaluation of intelligent systems are challenging because of the tradeoff between "realism" and experimental control. Long scenarios and realistic, complex environments equate to variability, which, in turn, requires large sample sizes to achieve statistical reliability. Short, constrained (unrealistic) test scenarios limit the generalizability of results.

• The methodological challenges inherent in evaluating intelligent man–machine interfaces, including intelligent ATIS/CVO systems, should not be underestimated.

[SK 20] INTELLIGENT DECISION AIDING IS FEASIBLE

• The CDAS program demonstrated the potential for the application of computational methods and expert systems to provide intelligent decision aiding for navigation tasks.

• Real–time cognitive decision aiding for navigation is feasible with today's (1993) technology.

Top

FHWA-RD-95-197