Trends of Transportation Simulation and Modeling Based on A Selection of Exploratory Advanced Research Projects - Workshop Summary Report, August 2011

The Exploratory Advanced Research Program

PDF files can be viewed with the Acrobat® Reader®

FOREWORD

Our nation continues to look for new and innovative strategies that can improve the transportation system we use daily. Consequently, effective tools such as computer simulation and models are needed to assess the validity of various proposed transportation solutions.

On August 22, 2011, the Federal Highway Administration's (FHWA) Office of Operations Research and Development (R&D), in partnership with the Exploratory Advanced Research Program, hosted a workshop as part of an ongoing effort to examine advancement in simulation and modeling. The workshop successfully brought together experts from multidisciplinary fields to share their knowledge and experience in applying various modeling and simulation methodologies to study topics including driver and traveler behavior, intersection control, and autonomous vehicles. Summaries of the five simulation studies presented in the workshop as well as recommendations provided by participants are reflected in this document.

Through the distribution of this workshop summary report, it is our hope that the information and thoughts shared at the meeting will help shape the FHWA's further investment in modeling and simulation research through open solicitation, interagency agreements, or other appropriate mechanisms.

Joseph I. Peters
Director, Office of Operations R&D
Federal Highway Administration

Technical Report Documentation Page

Executive Summary

The Federal Highway Administration's Exploratory Advanced Research (EAR) Program Workshop, "Trends of Transportation Simulation and Modeling Based on a Selection of Exploratory Advanced Research Projects," took place August 2011, at the Turner-Fairbank Highway Research Center in McLean, VA.

The workshop brought together five research projects examining new approaches and applications for modeling and simulation, specifically focused on two critical areas: human behavior and Connected Vehicle systems. The projects looked at a range of technological advances, including near-future advances and long-term breakthroughs. Some explored how to apply simulation to help understand a specific transportation problem; others examined simulation itself and looked for modeling breakthroughs.

Approximately 50 people listened and commented on the following presentations:

• Driver Behavior in Traffic—a study using naturalistic driving data to create realistic agent models of both car and truck drivers.
• Intersection Control for Autonomous Vehicles—a study examining the feasibility of autonomous intersections and vehicles using a computer simulation developed by the project team.
• Advanced Traffic Control Signal Algorithms—a study developing algorithms, both vehicle-to-infrastructure and vehicle-based, to improve operational efficiency at intersections.
• Agent-Based Approach for Integrated Driver and Traveler Behavior Modeling: Theory, Methodology, and Applications to Transportation Systems Management and Investment Planning—a study considering both driver and traveler modeling, where the traveler is the entity that makes the mode, departure time, and route decisions.
• VASTO—Evolutionary Agent System for Transportation Outlook—a study using agent-based modeling methodology to examine how the transportation system evolves in response to the actions of drivers, commercial vehicles, traffic management centers, and policy agents.

For the last part of the day, participants representing government, academia, and industry, were divided into three groups and asked to consider overall trends of transportation simulation and modeling, and specific simulation and modeling needs. Themes from this discussion included the need for data, the interdisciplinary aspects of this work (in particular, human factors and driver behavior), the need to integrate micro- and macro-level issues, and the possibility of new applications for microsimulation, such as safety.

Table of Contents

INTRODUCTION

PART ONE: PROJECT PRESENTATIONS

1. Driver Behavior in Traffic
2. Intersection Control for Autonomous Vehicles
3. Advanced Traffic Control Signal Algorithms
4. Agent-Based Approach for Integrated Driver and Traveler Behavior Modeling: Theory, Methodology, and Applications to Transportation Systems Management and Investment Planning
5. VASTO - Evolutionary Agent System for Transportation Outlook

PART TWO: GROUP DISCUSSIONS

1. Overall Trends of Transportation Simulation and Modeling
2. Specific Simulation and Modeling Needs

APPENDIX A: AGENDA

APPENDIX B: KEY CONTACT INFORMATION

APPENDIX C: ACRONYMS

REFERENCES

Introduction

The Federal Highway Administration's (FHWA) Office of Operations Research and Development (R&D), in partnership with the Exploratory Advanced Research (EAR) Program, hosted the workshop "Trends of Transportation Simulation and Modeling Based on a Selection of Exploratory Advanced Research Projects" on August 22, 2011, at the Turner-Fairbank Highway Research Center (TFHRC) in McLean, VA.

As technological advancements continue to be applied to vehicles and to the operation and management of the transportation network, there is now a need to precisely model behavior of drivers and address the increase in Connected Vehicle and autonomous systems. Additionally, when implementing new technology, it is critical to have valid methods to test the safety and efficiency of the technology, something that is difficult to accomplish with the available tools.

The workshop brought together five research projects examining new approaches and applications for modeling and simulation, specifically focused on two critical areas: human behavior and Connected Vehicle systems. The projects looked at a range of technological advances, including near-future advances and long-term breakthroughs. Some explored how to apply simulation to help understand a specific transportation problem; others examined simulation itself and looked for modeling breakthroughs.

The workshop began with a welcome from Michael Trentacoste, FHWA's Associate Administrator for Research, Development, and Technology (RD&T) and TFHRC Director. Trentacoste reminded participants of the importance of the EAR Program, citing the need to invest in high-risk, long-term research to drive innovation and bridge the gap between research and deployment.

Joseph Peters, Director of TFHRC's Office of Operations R&D, described the context of the EAR Program within the Office of Operations R&D. Peters cited the surface transportation legislation that requires research for simulation and modeling, developing predictive capabilities, and evaluating research projects and policy outcomes. He provided an overview of recent Federal contributions to the body of research and development, recalled the desired outcomes and fiscal year 2010 FHWA strategic initiatives, and described simulation and modeling connections to other program areas such as the Connected Vehicles Program, Small Business Innovative Research, and Transportation Pooled Fund Studies.

David Kuehn, EAR Program Manager, presented the purpose and operations of the EAR Program. The EAR Program focuses on the long-term needs of the larger highway community and aims to address persistent research gaps. It looks for original ideas that are going to advance the science and engineering of highway transportation and aims to accelerate the capacity of the industry to think about tough problems, and through innovative research and collaboration, attempt to solve them. Kuehn reiterated Trentacoste's observation that the EAR Program focuses on high-risk, high-payoff research, and described the rigorous selection process for EAR Program-funded projects. Kuehn ended by asking participants to consider, during the workshop, possible synergies among research projects and ways to communicate research to practitioners.

David Yang of FHWA wrapped up the opening remarks by reviewing the agenda, providing brief overviews of the day's five presentations, and requesting that participants consider three topics during the course of the day:

• Feedback on the EAR Program projects.
• Overall trends in transportation simulation and modeling.
• Specific simulation and modeling needs.

Part One of this report summarizes the following presentations:

• Driver Behavior in Traffic.
• Intersection Control for Autonomous Vehicles.
• Advanced Traffic Control Signal Algorithms.
• Agent-Based Approach for Integrated Driver and Traveler Behavior Modeling: Theory, Methodology, and Applications to Transportation Systems Management and Investment Planning.
• VASTO—Evolutionary Agent System for Transportation Outlook.

Part Two of the report summarizes the group breakout discussions addressing overall trends of transportation simulation and modeling, and specific simulation and modeling needs.

Part One

Project Presentations

1.Driver Behavior in Traffic

Overview

Existing traffic analysis and management tools do not model drivers' ability to recognize their environment and respond to it differently depending on the situation. The literature on characterizing driver behavior is very limited. What research exists is typically limited to specific locations (i.e., by collecting data on specific intersections or freeway sections) and is very limited in scope. The majority of traffic modeling and parameter calibration research has assumed somewhat similar driving conditions and behavioral sets for the entire driver population. This approach does not capture or predict the effects of situational factors on individual drivers' actions.

The goal of this research is to characterize driver behavior under naturalistic driving experiences with respect to critical parameters related to freeway driving. Naturalistic driving data were analyzed and used as the basis for 20 simulation agents that encapsulate individual drivers' decisions in response to varying traffic situations. The developed agents were designed to learn individual drivers' actions for any given traffic state, actions that were retrieved from the Virginia Tech Transportation Institute's (VTTI) database of naturalistic driving data. These characteristic driving rules of the agents were coded in a VISSIM simulation environment (a microscopic, multimodal traffic flow simulation model) to test and study their effects on agents' behavior in different conditions and scenarios. This research is expected to provide the simulation and modeling industry with methods for developing more accurate and more sensitive traffic models. It could also enable future research to develop new traffic simulation models that accurately model driver behavior during incidents and other complex traffic situations.

Naturalistic Driving Data

Naturalistic data collection is the collection of driver behavior and performance data in a real-world environment. It has allowed researchers to examine what happens in the final seconds before crash, near-crash, and safety-critical events for which researchers would otherwise depend on eyewitness accounts and police reports. Data regarding vehicle speed, acceleration, range, range rate, headway, time-to-collision (TTC), brake pedal input, and qualitative data such as pre-incident maneuvers can be used to describe driver behavior. Other data such as roadway type, number of lanes, traffic density, time of day, and weather can be used to describe the driving environment. Tying these data together allows researchers to understand the exact conditions that exist during crash events. Naturalistic driving data collection is a powerful approach used by researchers to understand crash and near-crash causation. As opposed to traditional epidemiological and experimental or empirical approaches, this in situ process uses drivers who operate vehicles that have been equipped with specialized sensors as well as processing and recording equipment.

Naturalistic data collection methods require a sophisticated network of sensors, processing, and recording systems. The Data Acquisition System provides a diverse collection of both on-road driving and driver (participant, nondriving) data, including measures such as driver input and performance (e.g., lane position, headway, etc.), four camera video views, and driver activity data (figure 1). This information may be supplemented by subjective data, such as questionnaire data. A tremendous amount of data is acquired when carrying out these naturalistic studies.

Figure 1. Naturalistic driving data collection

Due to the tremendous amount of information collected during these procedures, naturalistic data collection methods require significant post-collection processing. Typically, the first step in the data reduction process is to identify events of interest, including crashes, near-crashes, and crash-relevant conflicts. To find events of interest, VTTI developed a Data Analysis and Reduction Tool that scans the dataset using user-defined threshold algorithms to identify notable actions (e.g., hard braking, quick steering maneuvers, short TTC, and lane deviations (including median encroachments)). All detected events are analyzed based on "instant replays" of video data and associated dynamic data recordings of the events. This analysis captures both the observable causal sequences leading to events as well as the conditions and correlates of event occurrence.

Three datasets were used for this effort:

• An 8-truck database using 100 commercial motor vehicle (CMV) drivers from four fleet companies, yielding 1.2 million km (735,000 mi) (14,500 hours) of driving data.
• A 34-truck database using 103 CMV drivers from three fleet companies, yielding approximately 3.7 million km (2.3 million mi) of driving data.
• Data from 100 cars, with approximately 3.2 million km (2 million mi) recorded. This data collected included 15 police-reported crashes, 67 non-police-reported crashes, and 761 near-crashes.

Nine truck drivers and eleven car drivers were selected for analysis and agent implementation based on the type of crashes and near-crashes they experienced. The drivers with the most crashes and near-crashes that were forward conflicts were selected.

Comparison with Existing Models

The Wiedemann car-following model was originally formulated in 1974 by Rainer Wiedemann.¹ This model is used extensively in VISSIM. The simulation program chosen for the final implementation of trained agents as a result of this project was VISSIM, making the Wiedemann model a good illustrative example for this effort. The exact shape and formula used in the model were updated using the naturalistic data.

Reconstruction of the Wiedemann model was a two-step process: adapting the Wiedemann model according to the naturalistic data, and adding two new thresholds, passing and hook-following (this occurs after a faster lead vehicle merges in front of the subject vehicle. The subject vehicle driver then decides to "hook" on to the lead vehicle, accelerating to match the lead vehicle's speed), that aim to represent recurrent phenomena that were found in the naturalistic data.

These new threshold equations were evaluated and calibrated using a genetic algorithm framework. The results found that inclusion of a hook-following threshold adds value to the model by providing the ability to include a significant natural driving behavior. The addition of the passing threshold provides the model with the ability to easily transition from car-following to lane-changing behavior. This threshold also provides a way to force car-following behavior when a lane change is not possible. The addition of these new thresholds and the driver-specific equations improve upon the Wiedemann model's ability to represent real-world driving behavior.

Researchers then attempted to further improve on the Wiedemann model by replacing its acceleration equations in the following regimes: approaching, closely approaching, acceleration following, and deceleration following. The Gazis-Herman-Rothery (GHR) model² was given a different set of calibration parameters for each of those regimes. The reaction time, T, was used as a calibration parameter to obtain a measure of the attentiveness of the different drivers. The combination of the Wiedemann model and the GHR model presents advantages when calibrating to the car-following behaviors of individual drivers. The hybrid Wiedemann-GHR model calibrated to four individual drivers' resulted in 5 to 43 percent less errors than the Wiedemann model alone (figure 2).

Figure 2. Comparison of car-following models.

Researchers additionally selected, calibrated, and compared several other car-following models. Commonly used car-following models that were chosen for comparison included GHR, Wiedemann, Fritzsche, Gipps, intelligent driver model, and the velocity difference model (VDIFF). Each model was calibrated by the use of a genetic algorithm. A genetic algorithm was used because of its ability to adequately and accurately find the optimal solution when multiple parameters are present, as in some of the models. Comparison results found that the VDIFF and the Wiedemann models both performed adequately across all the drivers tested. The behaviors of drivers during safety-critical events differ from their normal behavior, but current models do not have the capacity to address the complex dynamics that occur during a safety-critical event.

Agent Development

The team developed agents that encapsulated individual driving behaviors of 11 car drivers and 9 truck drivers. They also developed two "mega" agents that encapsulated the behaviors of all car and truck drivers, respectively. Figure 3 illustrates the learning process for the agents.

Figure 3. Agent learning process.

The basic training for car-following behavior was handled via back-propagation artificial neural network (BP-ANN). BP-ANN is applied to associate the nonlinear relationship between input states and actions and output states and actions. In this case, BP-ANN network was used to determine driver longitudinal actions in normal car-following driving conditions and to estimate state transition based on current traffic state and an agent's longitudinal action. Basic training was done by extracting car-following situations from the naturalistic driving database. The research team used both 10 Hz and 1 Hz data from one car-following episode, the same dataset that was used for the GHR car-following model calibration. The training performance for both the 10 Hz and 1 Hz datasets demonstrated stronger performance by the agent than by the GHR model, as did the model validation using a different dataset.

The research team also trained agents using neuro-fuzzy actor-critic reinforcement learning (NFACRL). Traffic state variables (e.g., speed) and driver actions (e.g., acceleration) are continuous, and conventional reinforced-learning methods cannot solve high-dimensional, continuous-state problems when agent actions are not drawn from a discrete set. NFACRL is able to translate high traffic state input variables into discrete fuzzy sets and generate continuous action using a weighted average of discrete actions. A basic cross-validation was performed by applying the training for one agent (driver) to another driver's situation (figure 4 and figure 5).

Figure 4. Acceleration of mega-agent.

Figure 5. Yaw angle of mega-agent.

Simulation and Analysis

The team performed a discriminant analysis to distinguish normal car following from near-crash behaviors. Discriminant scores were based on the acceleration of the subject vehicle, vehicle speed, lane offset, yaw angle, range, and range rate. There appeared to be a clear difference in discriminant scores, in many cases at about 6 seconds prior to the collision.

These agents were then implemented in VISSIM, and compared with the existing VISSIM car-following model. Evaluation metrics included:

• Minimum following distance (m).
• Maximum deceleration (m/s²).
• Average acceleration (m/s²).
• Average of absolute acceleration (m/s²).
• Average speed (km/h).

The Wiedemann car-following model did not produce any conflicts when the TTC threshold was set to 1.5 seconds. The agent-based models produced conflicts, with most of the agents occasionally entering the "emergency" regime, where the agent's speed is greater than that of the lead vehicle, but the following distance is small. The agent's deceleration rates were realistic and resulted in several collision and run-off-the-road incidents.

Conclusions and Recommendations

Recommendations for future study include the following:

• Assessing the frequency at which certain agent behaviors occur in the actual driving population.
• Extending the NFACRL framework to simulate other traffic behavior such as lane changing and merging.
• Examining the adaptability of agents in real-time, i.e., reprogramming them during the simulation.
• Examining human factor issues related to warning individual human drivers about a change in their driving behavior that might lead to a safety-critical event.

Discussion

One question raised during the discussion was whether methods other than reinforcement learning would provide a closer match to naturalistic data. Reinforcement learning steers toward what an agent "ought" to do to maximize some long-term reward. The idea is that the agent's performance will become "better" over time. In this case, it rewards actions that are close to naturalistic actions to provide a good match between what the agents do and what actual drivers do. Since in this case the goal is to match existing driver behavior, one audience member suggested that learning by demonstration might be a better approach.

Three larger issues were raised, but not resolved, during the discussion. The first is the broad question of the possible applications for this research. A number of potential benefits were stated in the conclusions, above, but they have yet to be realized.

The second is gaining an understanding of why drivers do what they do. What happens in terms of driver perception, risk-taking, and cognitive challenges? How do drivers change their driving as prevailing traffic densities change? Improving our understanding of the cognitive aspects of car following and other driver behaviors could lead to some breakthroughs in both the transferability and scalability of models. Naturalistic data are very helpful, but have some limitations. They do not directly reveal what drivers are thinking, and do not have all of the information on the traffic situation. For example, the naturalistic driving data might not have information on parallel lanes.

The third large issue is the bridge between the micro and macro levels. The agents model microscopic behavior (e.g., how do I follow the vehicle in front?). Models, however, need to be calibrated to macroscopic effects (e.g., what is the relationship between overall speed and traffic volume?).

Additional Resources

For additional information, contact Montasir Abbas at Virginia Tech, 540-231-9002 (email: abbas@vt.edu), or view the following papers:

"A Revised Reinforcement Learning Algorithm to Model Complicated Vehicle Continuous Actions in Traffic," Proceedings of the 14th International IEEE Annual Conference on Intelligent Transportation Systems, Chong, L., Abbas, M., Higgs, B., Medina, A., and Yang, C. Y. D., Washington, DC, USA, 2011 (http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6083005).

"Agent-based Evaluation of Driver Heterogeneous Behavior during Safety Critical Events," Proceedings of the 14th International IEEE Annual Conference on Intelligent Transportation Systems, Abbas, M., Chong, L., Higgs, B., Medina, A., and Yang, C. Y. D., Washington, DC, USA, 2011 (http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6083089).

"Identification of Warning Signs in Car Driving Behavior Before Safety-Critical Events," Proceedings of the 14th International IEEE Annual Conference on Intelligent Transportation Systems, Abbas, M., Higgs, B., Medina, A., and Yang, C. Y. D., Washington, DC, USA, 2011 (http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6083093).

2. Intersection Control for Autonomous Vehicles

Introduction

Enhancements to road-based infrastructure may improve the potential for adoption of fully autonomous vehicles. Advances in intersection controls developed as part of this research seek to allow autonomous vehicles to travel safely through intersections, to share the road with human drivers, and to reduce fuel consumption and emissions while improving network throughput.

Autonomous Intersection Management

Originally developed by Dresner and Stone in 2004,³ autonomous intersection management (AIM) allows vehicles approaching an intersection to communicate with an automated reservation manager. The vehicle sends information regarding estimated time of arrival, velocity, size, acceleration capability, and arrival and departure lanes through the intersection. The reservation manager checks to see if there is a clear path through the intersection at the space and time requested by the approaching vehicle. The intersection is logically divided into a small grid, and grid squares are reserved for individual vehicles (figure 6). A nominal reservation policy is first-come-first-served (FCFS), but other policies are possible (a batch policy is described below). The reservation manager grants or denies the request. (In the future it might suggest an alternative slot to the requesting vehicle.) Cars whose requests are denied are required to submit another request and potentially slow down or stop until a request can be granted.

Figure 6. Vehicle path through intersection.

The throughput of the intersection may be fine-tuned by increasing or decreasing the size of the reserved grid buffer reserved for each vehicle as it travels.

As an early test of the AIM concept, researchers monitored the interactions between a single real autonomous vehicle and many simulated vehicles, all of whom scheduled reservations through an intersection manager.

Performance of AIM FCFS Compared With Signalized Intersections

AIM results in significantly higher utilization of an intersection than a typical signal cycle allows. The average per-vehicle delay of a signalized intersection is 15 times greater than that of an AIM intersection operating under an FCFS policy, assuming all cars are autonomous vehicles (figure 7).

Figure 7. Intersection delay.

Fully autonomous vehicles will not displace manually controlled vehicles immediately, and there will likely be a transition period lasting years or decades. To facilitate transition, the AIM system may be integrated with signalized intersections by reserving lanes for manually controlled vehicles at all times and spaces during the green signal. Autonomous vehicles will be allowed on paths that do not conflict with these routes. Autonomous vehicles will also be allowed to utilize other parts of the intersection, as long as they do not cross into reserved grid spaces.

The performance benefits of AIM improve as the ratio of autonomous vehicles to human-driven vehicles increases. Sizeable performance gains are achieved only when over 90 percent of vehicles are autonomous (figure 8).

Figure 8. Impact of partial deployment.

Unbalanced Intersections

A significant performance issue with the FCFS policy was found in intersections where traffic along one axis is much heavier than traffic in another axis. During periods of high traffic density in these intersections, vehicles along the dominant axis take all the reservations, and "starve" access to reservations by vehicles on the subordinate axis. Vehicles along the subordinate axis are stopped at the intersection, and over time, congestion builds in the subordinate axis.

Researchers created a policy alternative to FCFS, the batch policy, to address starvation in unbalanced intersections. The batch policy does not accept or reject requests immediately, but rather stores them in a queue whose cost function increases as wait time increases. The intersection manager then processes several requests in the target batch at the same time, allowing batches of vehicles in the subordinate axis to gain reservation priority over single vehicles in the dominant axis.

For unbalanced intersections, performance of the batch policy is similar to that of FCFS in low-congestion conditions. The batch policy is superior to that of the FCFS policy at traffic levels of over 1000 vehicles per lane per hour.

Safety Measures

If all autonomous vehicles follow the protocol and no mechanical failures or environmental conditions interfere, the reservation manager guarantees no collisions will occur. If and when mechanical failures and environmental conditions disrupt the ability of autonomous vehicles to comply with their reservations, the potential for accidents exists. These accidents would be similar to accidents that occur when manually controlled cars run red lights.

To minimize collisions, when a crash occurs, the intersection manager will send a "STOP" message to all vehicles entering an intersection. This will prevent most collisions, but not all. The protocol is also designed so that lost packets will not cause collisions. If packets are lost, particular vehicles may be prevented from entering the intersection until the correct data packets have been received.

Research Summary and Future Directions

Other current research components include integration of vehicle-to-vehicle communications, assigning priority passage for emergency vehicles, policy switching (for example, switching among FCFS, batch, and other potential policies depending on overall traffic conditions), and integrating multiple intersection managers to dynamically assign routes and optimize system travel time through multiple intersections.

Researchers will release the source code of the AIM4 simulator to the public as open-source free software. Prior to the release, they plan to improve user interface of the simulator and create documentation including an installation guide, a user guide, and application programming interface documentation.

Next steps of the research include investigating the following:

• Dynamic route planning by connecting multiple intersection reservation managers.
• Dynamic lane reversal to implement more aggressive contraflow strategies.
• Micro-tolling via very fine-grained level auctioning or dynamic road and intersection pricing.
• Sufficient conditions for so-called "liveness," in which all vehicles can eventually leave an automated road network.
• Vehicle control planning, by which precise control of autonomous vehicles can further increase intersection throughput.

Discussion

The audience discussion after the presentation centered on human versus machine performance. Specifically, is the computer better than the human driver? Some argued "yes," given the efficiency improvements; others argued "no," given the mean-time-between-failure rate for human drivers (50,000 vehicle hours per injury crash and 2 million vehicle hours per fatal crash). The question was raised: will a machine be able to improve on human driver results in real-world conditions?

It was also noted that a batch policy is similar to a traffic signal in that it batches the flow of vehicles.

Additional Resources

For additional information, contact Peter Stone at University of Texas, 512-471-9796 (email: pstone@cs.utexas.edu), or visit:

Project Web site: www.cs.utexas.edu/~aim/.

Project videos: www.cs.utexas.edu/~aim/video/fcfs-insanity.mov and www.cs.utexas.edu/~aim/papers/MixedReality.wmv.

Project paper: "A Multiagent Approach to Autonomous Intersection Management," Dresner, K., Stone, P., University of Texas at Austin, Texas, March, 2008 (http://www.cs.utexas.edu/~pstone/Papers/bib2html/b2hd-JAIR08-dresner.html).

3. Advanced Traffic Control Signal Algorithms

Introduction

This ongoing project is primarily focused on an area of vehicle-infrastructure integration, namely the use of connected-vehicle probe data to develop traffic signal control algorithms. Project objectives include developing performance measures and estimation methods for evaluating connected vehicle-based signal control systems, developing and validating new signal control strategies enabled by connected-vehicle probe data, and limited experimental validation.

Applications

Some applications include the following:

• Perimeter control gating on a congested network, to prevent gridlock.
• Improved intersection safety and efficiency, through on-demand all-red extension, management of platoons on a corridor to reduce arrivals on the yellow interval, and enhanced green wave progression.
• Adaptive priority for certain vehicles, particularly under low-volume traffic conditions.
• Eco-driving on signalized corridors.

Excessive density of vehicles in a highway network can lead to gridlock, where intersections are blocked by stopped vehicles, and no one can move. One means of addressing gridlock is through perimeter gating, where traffic signal controls are used to delay vehicles that wish to enter a region to prevent excess vehicle density from building up.

Safety applications include an on-demand all-red extension, where if a vehicle is detected to be about to enter the intersection just after the signal turns red, the red signal is extended on the cross street. A second safety application is the management of platoons to reduce arrivals at an intersection on the yellow interval. A VISSIM simulation was performed on a six-intersection corridor, with yellow phase arrivals being reduced by 49 percent, and corridor efficiency (as measured by delay, number of stops, and progression ratio) remaining essentially the same.

Adaptive signal priority could include preemption for emergency vehicles, priority for transit vehicles, and treatments for other individual vehicles under low traffic conditions. Such treatments might include green extensions, use of red signals to discourage speeding, and advisory messages when a driver is likely to have to stop at a downstream intersection.

The fourth major application is eco-driving, which is the adjustment of speed and acceleration to minimize fuel consumption. Real-time information on signal status (infrastructure to vehicle) can be used to minimize stops and starts along the corridor. Preliminary results indicate a 15-percent fuel savings.

Market Penetration

The team explored the impacts of market penetration rates for the cooperative vehicle technology, by examining the travel delay, stopping percentage, speed by roadway segment, and queue length as a function of market penetration. None of the probe algorithms and sampling strategies did well when sampling through a single signal cycle at low market penetrations. As market penetration increased, there were no clear break points, rather a more or less linear relationship between market penetration and benefit measures, depending on which strategy has been adopted.

A challenge in this work is communications bandwidth; can the dedicated short-range communications channel accommodate the data broadcast by large numbers of vehicles when high market penetration is achieved?

Discussion

During the discussion two questions were raised about eco-driving. First, what is the impact of eco-driving on speed and travel time? The second concerns the impact on following vehicles. If the lead vehicle is following an eco-driving profile, does that force the following vehicle to follow a more eco-driving profile?

Additional Resources

For additional information, contact Alex Skabardonis at University of California, Berkeley, 510-642-9166 (email: dromeas@berkeley.edu), or visit:

"Estimation of Arterial Measures of Effectiveness with Connected Vehicle Data," Argote, J., E. Christofa, Y. Xuan, and A. Skabardonis, 91st TRB Annual Meeting, Washington DC, January 2012 (www.ocf.berkeley.edu/~xuanyg/Argote_Christofa_Xuan_Skabardonis_2012_TRB.pdf).

"Dynamic Lane Grouping at Isolated Intersections: Problem Formulation and Performance Analysis," Zhang, L and G. Wu, 91st TRB Annual Meeting, Washington DC, January 2012 (http://amonline.trb.org/1snvve/1snvve/1).

4. Agent-Based Approach for Integrated Driver and Traveler Behavior Modeling: Theory, Methodology, and Applications to Transportation Systems Management and Investment Planning

Overview

This project began in April 2011, and a progress report was presented. The focus is on modeling agents as both travelers and drivers, where the travel decisions include mode, departure time, pre-trip route, and en-route diversion choices. In agent-based modeling, the possible research scope can have an extremely broad range. The most micro-level decisions involve individual driving behavior (e.g., how close do I follow the car in front?) made in response to traffic conditions. At the other extreme are long-term decisions such as location choices and vehicle ownership decisions (e.g., where do I live and how many cars do I have?). This project strives to balance two objectives: (1) developing new theories and methods related to agent-based modeling in transportation (ABM-T) that can potentially bring breakthrough applications in the transportation system, (2) demonstrating the immediate value of ABM-T in transportation operations and planning decision support.

This research is focusing on the decision areas presented in table 1:

Table 1. Research areas.

© Transportation Systems Research Lab at University of Maryland

Challenges

The major modeling task is to develop an agent-based traveler and driver behavior model with descriptive behavior theory (i.e., how agents actually make decisions), rule-based methods, and high-quality behavior process data. The model will describe the interaction between agents (persons) and the environment (land use and transportation systems). As such, it is not an optimization model.

The major data task is to design and implement innovative data collection methods in support of this agent-based model. Challenges include:

• Data collection—Approaches under consideration include interactive laboratory experiments, a driving simulator, surveys and GPS-based vehicle or person tracking.
• Agent behavior representation and estimation—The approach used considers agent learning and derives behavioral rules from empirical data.
• Agent decisions for the Agent-Based Modeling and Simulation (ABMS)—A single agent-based model will model selected agent driving and travel decisions.
• Implementation of the ABMS.

The Search, Information, Learning and Knowledge descriptive behavior theory was presented. It includes the following elements:

• If the subjective search gain exceeds the perceived search cost, then search for a better alternative (e.g., route, departure time); otherwise, repeat the previous behavior.
• Information on travel time, travel cost, etc. comes from experience and other sources.
• Learning is based on information and is used to update knowledge.
• Knowledge is the cognitive map and subjective beliefs of the person.

Gathering Data

A useful source of empirical data is Maryland's Intercounty Connector, which opened in late 2011. A number of surveys will be conducted, including stated preference surveys and a GPS-based survey of actual user behavior responses. One preliminary finding concerns the willingness of travelers to search for alternative departure times (figure 9). For example, if the perceived cost to search is equivalent to 20 percent of the current utility level, then 70 percent of travelers will be willing to search for alternatives; however, if perceived cost reaches 60 percent of the current utility level, then only 20 percent of the travelers will search. Once travelers decide to search, subsequent search rules and decision rules, both of which are also empirically derived, determine how they will behave at the micro-level in the next agent-based simulation iteration. The emergence of important macro-level patterns, such as congestion, project impact, and long-term system performance, from these micro-level behavior and interactions, as simulated in the agent-based model, can produce a variety of useful information in support of transportation systems operations and investment planning decisionmaking.

Figure 9. Percentage of travelers who will search for alternative departure times as a function of search cost.

Applications

A prototype agent-based model developed from this research project is being used to examine peak spreading and the evolution of congestion patterns due to worsening system performance during peak periods. Other applications within the scope of the project include major investment analysis, congestion mitigation via dynamic pricing, land development impacts, and multimodal corridor traffic operations. Some of the applications are supported with matching funds from the Maryland State Highway Administration.

Discussion

It was briefly noted that a learned black box model can be converted to more visible decision rules. The relationship between driver and traveler behavior was also noted, where driver behavior influences the performance of the network, which in turn influences traveler behavior.

5. VASTO—Evolutionary Agent System for Transportation Outlook

Overview

Researchers for the project are exploring the integration of agent-based modeling (ABM) with traffic simulation. It works across a wide variety of time scales, from the very short to the very long, including policy agents. The types of agents considered in the Evolutionary Agent System for Transportation Outlook (VASTO) model include commuters, commercial travelers, traffic management agencies, and policy agents. The policy agents deal with urban planning decisions that influence the infrastructure supply (figure 10).

Figure 10. VASTO model framework.

Three topics were presented: policy agent modeling, agent-based simulation, and geographic information system (GIS)–based visualization. This project has recently started, so the presentations consisted of broad overviews of the issues within each topic.

Policy Agent Modeling

A number of organizational structures were briefly presented, including a classic (hierarchical), modern (several divisions), and an adhocracy (organic) organizational structure, where the organization is represented as a network of interacting specialists. A number of decisionmaking models were also presented, ranging from the traditional "rational" approach to various approaches that assume imperfect information and that decisionmakers might be willing to choose an option that is satisfactory but not necessarily optimal.

Agent-Based Simulation

Young-Jun Son spoke on agent-based simulation highlights. He presented an expanded belief-desire-intention (BDI)⁴ framework (figure 11) where the agent may have varying levels of confidence in his decisions. He then presented examples for evacuation planning, where the time to evacuate was influenced by the type of traveler (commuter versus tourist) and the presence of either leaders or police officers.

Figure 11. Overview of an extended belief-desire-intention framework developed by Lee, Son, and Jin.⁴

GIS-Based Visualization

Jyh-Ming Lien spoke on GIS-based visualization. He noted that VASTO will generate a huge amount of raw, dynamic data with multiple dimensions. Data mining techniques are needed to make sense of it. One such technique is trajectory clustering. It is also desirable to show the evolution of traffic data over time.

Part Two

Group Discussions

For the afternoon session of the workshop, attendees were divided into three small groups to discuss two key topics: overall trends of transportation simulation and modeling, and specific simulation and modeling needs. Following extensive group discussion involving multidisciplinary experts from government, academia, and industry, the attendees reconvened, summarized their findings, and made recommendations. This section presents the overall findings of the breakout groups.

Overall Trends of Transportation Simulation and Modeling

Complexity

During group discussion, one of the key trends identified by workshop participants was that research for transportation simulation and modeling is becoming increasingly more complex. The state of the art of simulation and modeling research is moving further away from the modeling performed by transportation practitioners. In many cases practitioners continue to use four-step models, despite the existence of newer and more complex activity- and agent-based models (ABMs).

The discussion raised the issue that for those practitioners using more complex models, researchers may not be focusing on the types of problems transportation practitioners are facing. One example put forward was modeling unplanned route changes, which when considered in aggregate, may have profound impacts on a transportation system.

Data Collection

It was also noted that data issues continue to plague modelers—although advances in model complexity require increased data collection efforts, more data and more fine-grained data mean more robust methods for cleansing and validating are necessary. Participants raised several key issues, including that datasets themselves may be bad, government agencies may not have the resources to appropriately validate data, and data collected or generated by companies or individual researchers need to be validated and distributed by independent third parties.

Participants were also concerned that datasets are often used inappropriately because they either may not contain metadata, users may not always refer properly to metadata, or end users may unknowingly misuse the metadata. Additionally, some participants shared concerns about the marginal costs of additional data collection, particularly in situations in which additional data collection may not yield more accurate results.

The group discussed concerns about appropriately matching datasets with problem statements, and noted that perhaps agent-based models may not be appropriate or necessary for making long-term policy decisions. It was also noted that policy questions are becoming more nuanced, and that ABMs may be appropriate for situations such as land use or signage policy decisions, which are highly dependent on end-user behavior.

Applications

Several other trends were identified during the workshop group discussion. Participants reported the positive trend of closer relationships among university researchers and State agencies, noting that in some cases these relationships may ease the transition from research to application.

Other applications for modeling were also highlighted, including utilizing normative and descriptive modeling to inform policy and operations at both the micro and regional levels, crowd sourcing, open source modeling, cloud computing, learning-based systems, connected vehicles, and research partnerships.

Finally, the group identified some additional modeling trends that are currently in practice. It was initially suggested that there is a substantial institutional inertia, with a sunken investment in current models and no standardization of modeling practice; however, some current trends identified during the workshop included models that offer researchers a broader view beyond transportation, for example land use. Another trend in practice was identified as the use of off-the-shelf tools for microsimulation and visualization, particularly crowd sourcing, and increasingly more reliance on Google^™ tools.

Specific Simulation and Modeling Needs

Decisionmaking

Group discussion continued with participants identifying several areas of interest with specific simulation and modeling needs. Data collection issues were highlighted as an area of interest, specifically what is being collected and how. Participants also expressed a desire to understand why people make various decisions—something that is not always readily apparent from the outcome of a decision. The theory of how people learn was also discussed, and it was suggested that it may prove helpful to track the evolution of a person's decisions over several months. One possible method for gathering such data could be the use of smart phones, which may provide a rich source of data; however, it was noted that a user's privacy needs to be considered when planning data collection. Finally, participants discussed whether data can ever be "good" enough to earn the trust of end users of models based on it.

Scale

Participants continued to discuss the issue of scale and getting from extreme micro-scale modeling to the larger effects. It was noted that ABMs are at an extreme micro-scale and have many dimensions, which makes calibration particularly difficult. Participants also expressed a desire to identify the fundamental components and rules that matter the most for large-scale impacts of ABMs. They also highlighted a need to work among multiple levels of resolution in both space and time.

Disciplines

Discussion progressed to the need to pull various disciplines together, particularly traffic and human factors. It was noted that the work in ABM has revealed that various disciplines within transportation (in particular human factors and traffic modeling) may benefit from a multidisciplinary framework and the provision of guidelines for ABM validation.

Traffic and Safety Applications

Several potential applications were suggested during group discussion. These included real-time decisions and how modeling and simulation can work to support operations. It was suggested that there is an opportunity for continuously adapting models to make real-time predictions of traffic conditions. Participants also discussed insights that may be gained from studying agents in relation to the beneficial or harmful behaviors of real drivers. Finally, safety applications were discussed as a significant opportunity for modeling and simulation. It was noted that existing models generally assume safe behavior; however, as modeling of driver behavior becomes more realistic, the behavior of agents will sometimes lead to crashes. It was suggested that the use of such models could be used to gain insights on improving road safety.

Implementation

Participants recognized the need to further inspire, educate, and train practitioners on newer modeling and simulation methods. The group noted that most American practitioners still use aggregate, four-step transportation models, and that these models should be retired in favor of more advanced activity- and agent-based approaches. A suggestion to improve implementation was to place less emphasis on the four-step model in university classrooms. Another suggestion put forward was to split the responsibilities of transportation practitioners in much the same way as weather modelers and weather forecasters. Transportation modelers would still be responsible for complex methods and creating model outputs, while transportation forecasters would focus more on interpreting and applying those outputs in practice.

Data Management

Participants agreed on the need for improved data management to better ensure good data and proper use. One suggestion for this was to create an independent clearinghouse that would be responsible for collecting datasets from researchers, validating those datasets, verifying that metadata is in place, and communicating the existence and proper use of data. Similar work has been done along these lines with national and State GIS data clearinghouses.

Data Quality

Participants acknowledged that vehicle control researchers are particularly interested in greater quantities of crash or near-crash event data; however, the expense of collecting these data was also noted, given the relatively low frequency of these events compared with vehicle-miles traveled.

Participants also expressed a need for an industry standard benchmark dataset. It was suggested that this dataset could be used to objectively compare the pros and cons of proposed traffic control algorithms and reduce study bias toward independent training and testing datasets. Such a benchmark dataset would be similar to tools used in other simulation and modeling industries such as electronics and fire dynamics.

Environmental Needs

Group discussion also covered the interactions of transportation with land use, economics, jobs, energy use, air quality, and emissions. It was recognized that environmental impacts are particularly significant because of Environmental Protection Agency requirements and constraints. Participants confirmed a need to provide better integration of models among different levels and modes and between planning, operations, mobility, and safety. They also noted a need to predict trends that affect long-term investments and their effects on the transportation system.

Research Priorities

With so many sources of data and many scales in time and space, participants suggested that modeling could follow a dynamic data-driven application system paradigm, where the model itself can gather the most appropriate data (see www.dddas.org for more information). Another research area put forward was user behavior and the provision of incentives to move toward system optimization.

Participants summarized research priorities as benchmarks, case studies, and test beds for certifying new tools. Other priorities include a need to balance the effort among data collection, model development, model application, model reusability, and information sharing. It was also suggested that there is a need to consider new funding mechanisms for research.

Appendices

Appendix A—Agenda

Trends of Transportation Simulation and Modeling Based on a Selection of Exploratory Advanced Research Projects
Turner-Fairbank Highway Research Center, McLean, VA, August 22, 2011

Appendix B—Key Contact Information

Technical Information

• C. Y. David Yang
• Office of Safety R&D
• Turner-Fairbank Highway Research Center
• Federal Highway Administration
• 202-493-3284
• david.yang@dot.gov

EAR Program Information

• David Kuehn
• Exploratory Advanced Research Program
• Turner-Fairbank Highway Research Center
• Federal Highway Administration
• 202-493-3414
• david.kuehn@dot.gov

For more information visit www.fhwa.dot.gov/advancedresearch/

Appendix C—Acronyms

References

1. “Simulation des Strassenverkehrsflusses,” Wiedemann, R., University Karlsruhe, Germany, 1974.
2. “Nonlinear follow-the-leader models of traffic flow,” in Operations Research, Vol. 9, Gazis, D. C., R. Herman and W. R. Richard, 1961.
3. Multiagent Traffic Management: A Reservation-Based Intersection Control Mechanism,” in The Third International Joint Conference on Autonomous Agents and Multiagent Systems, Kurt Dresner and Peter Stone, University of Texas, 2004.
4. “Integrated Human Decision Making and Planning Model under Extended Belief-Desire-Intention Framework,” Seungho Lee, Young-Jun Son, Judy Jin, in ACM Transactions on Modeling and Computer Simulation, University of Arizona, 2010.