U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590

Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-13-054    Date:  November 2013
Publication Number: FHWA-HRT-13-054
Date: November 2013


The Exploratory Advanced Research Program

A Primer for Agent-Based Simulation and Modeling in Transportation Applications

CHAPTER 2: Agent-Based Methods in Human Decisionmaking

Human Decision-Behavior Modeling Framework

According to Lee et al.,(25) human decision behaviors have been studied in various disciplines, such as AI, psychology, cognitive science, and decision science. Lee et al.(25) classified these models theoretically into three major categories: an economics-based approach, a psychology-based approach, and a synthetic engineering-based approach. Each approach exhibits strengths and limitations. For example, models of the economics-based approach have a solid theoretical foundation, based mainly on the fundamental assumption that the decisionmakers are rational.(26-29) One major limitation, however, is the lack of capability to capture the nature of the human cognition process. To overcome this limitation, researchers proposed models using a psychology-based approach.(30-33) Although these models account for human cognition, they generally examine human behaviors under simplified and well-controlled laboratory conditions. The synthetic engineering-based models, which supplement economics- and psychology-based models, engage a range of engineering methodologies and technologies to assist in reverse-engineering and representing human behaviors in complex and realistic environments.(34-40) Human decisionmaking models in this category consist of engineering techniques used to implement submodules; however, given the possible interactions between submodules, the complexity of such comprehensive models makes it difficult to validate them against real human decisions. Lee et al.(25) proposed a novel, comprehensive model of human decisionmaking behavior, effectively integrating engineering-, psychology-, and economics-based models.

According to Lee et al.,(25) three popular synthetic engineering-based models are Soar, Adaptive Control of Thought-Rational (ACT-R), and BDI. Soar and ACT-R have their theoretical bases in the unified theories of cognition-an effort to integrate various disciplines to capture a single human cognition.(36) Thus, Soar and ACT-R use concepts that correspond to the real mechanisms of those brain activities involved in information processing, including activities of reasoning, planning, problem-solving, and learning. In contrast, the core concepts of the BDI paradigm, originally proposed in folk psychology, allow the use of a programming language to describe human reasoning and actions in everyday life.(41) The BDI paradigm has been successfully applied in many medium-to-large-scale software systems, including an air-traffic management system.(42)

Models of Learning

Lee and Son(43) provided a comprehensive review of various models of human learning behaviors, part of which is summarized in this section. Researchers have conducted extensive research on applying various machine learning algorithms and models, such as statistics, neural networks, and control theory, to mimic human learning behavior. For example, statisticians have introduced Bayesian models as a way to understand how a human being deals with uncertainties. Although many researchers have studied various Bayesian belief network (BBN) methods, such as Bayesian methods,(44,45) quasi-Bayesian methods,(46,47) and non-Bayesian methods,(48,49) a major obstacle to practical implementation of a BBN is the difficulty in constructing an accurate model, especially when the training data is limited. To tackle this problem, Niculescu et al.(50) introduced a framework for incorporating general parameter constraints into estimators for the parameters of a BBN. In a similar vein, Djan-Sampson and Sahin(51) used Scatter Search as a heuristic for identifying the best structure of a BBN. In spite of the above efforts, identifying a BBN structure is still a difficult task compared with other learning techniques. In addition, there is a gap between the BBN learning model and actual human learning, as most of the existing models focus more on finding the best solution (optimal behavior).

Lee and Son(43) described another attempt to develop a human-like learning machine, known as reinforcement learning (RL), which was adopted initially in the domain of animal learning psychology that concerned learning by trial and error. Later, in the 1980s, RL was implemented in some of the earliest work in the field of AI. In this field, RL was used in cognitive models that simulate human performance during problem-solving and skill acquisition.(52-55) RL was also used in the human error-processing system.(56) As such, the RL technique was demonstrated to be successful in mimicking human behavior in some sorts of simple problems, especially when prior knowledge is limited. Although BBN training is a non-deterministic polynomial-time hard (NP-hard) problem, training in RL could also be performed relatively easily based on the recursive mathematical formula. A major drawback of RL, however, is its difficulty in complex problems, because the states (which can be exhaustive for complex problems) and actions need to be clearly defined beforehand. As a result, if the environmental factors change, they need to be redefined accordingly. In addition, as compared with the BBN method, the RL method is more limited in its ability to take into account more extensive prior knowledge.

Models of Interactions

Kim et al.(57) discussed various models of human interactions, some of which are summarized in this section. Thomas(58) defined five types of human interactions for conflict management: collaboration, compromise, competition, accommodation, and avoidance. Given a pair of interacting participants, each one selects an interaction type based on his or her goal and willingness to help the other party. Collaboration entails achieving the mutual goal of both participants, in which they are willing to share their knowledge to satisfy their interests. In contrast, competition emphasizes each party's own interest, in which no information is exchanged, and the interests of the other party are predicted based on a win-lose paradigm. Collaboration and competition constitute major concerns in individual and collective behaviors, and these two types of interactions have attracted much attention from researchers. In accordance, several agent-based modeling methods have been applied to address these two issues.(59) For example, Danielson(60) employed an evolutionary game as a theoretical framework to model the collaboration between agents. In a similar vein, Macy and Flache(61) explored collaboration between agents in mixed-motive games by using an RL approach. With regard to competition, Read(62) employed differential equations to represent competition interactions, whereas Pan et al.(63) employed decision rules to represent competitive behaviors between agents.

In a compromise interaction, both parties tended to exchange part of their resources to maximize each party's outcome. Zlotkin and Rosenschein(64) introduced a unified negotiation protocol to illustrate the negotiation process in multiagent systems. When one or both participants are either assertive or cooperative, the interaction type may become accommodation or avoidance. In an accommodation interaction, although the participants may have different personal goals, the more unassertive one sacrifices his or her interests for the other's interests. In this case, the more unassertive one acts as an information sender while the other one is an information receiver. In the case when the conflicting issue is trivial, or both parties believe that the costs outweigh the benefits of conflict resolution, they will choose to avoid interaction, which will resolve the conflict situation. Although there is limited research in modeling accommodation and avoidance interactions among the agents, Kim et al.(57) addressed all five interaction types under the extended BDI framework.(25)


Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101