• Federal Highway Administration >
• Publications >
• Research Publications >
• 08051 >
• 01.Cfm >
• Surrogate Safety Assessment Model and Validation: Final Report

Surrogate Safety Assessment Model and Validation: Final Report

PDF files can be viewed with the Acrobat® Reader®

Chapter 1. Introduction

Motor vehicle crashes are the leading cause of death in the United States for people between the ages of 3- and 33-years old.⁽¹⁾ Crashes are complex events, often resulting from multiple contributing factors. Human behavior, the roadway environment, and vehicle failures are factors found to contribute in approximately 94 percent, 34 percent, and 12 percent of crashes, respectively.⁽²⁾ Transportation agencies focused on the safety of their respective roadways generally use statistical analysis of historical crash records as the primary yardstick to measure the safety of intersections, interchanges, and other traffic facilities. As it becomes evident that a specific location is experiencing an unusually high frequency of crashes, this location is subjected to investigation and possible remediation. Unfortunately, this process of remediating roadways presents the considerable drawback of actually realizing an excess of crashes. Thus, motivated by the need to assess and manage the safety of traffic facilities more effectively, this report presents new research on the use of surrogate safety measures—that is, measures of safety not based on a series of actual crashes.

This report develops and evaluates a method for the surrogate safety assessment of traffic facilities that has been codified into a software utility referred to as the Surrogate Safety Assessment Model (SSAM). The following sections briefly present additional background information, a synopsis of the technique, and a summary of the chapters in this report.

BACKGROUND

Throughout the United States, there are over 6 million police-reported motor vehicle crashes each year, resulting in 43,000 deaths, 2.7 million injuries, and $230 billion in economic losses.⁽³⁾ However, at one specific location, it may take years of infrequent and sporadically occurring crashes (and injuries) to reveal the need for remediation of the roadway layout or traffic control strategy. This section briefly reviews current crash analysis techniques to establish a basic perspective of their capabilities and limitations.

Safety Prediction Models

A safety prediction model is most commonly designed to estimate the expected number of crashes per year for a given traffic facility, based on traffic factors (e.g., average daily traffic), geometric layout, and traffic control features. The reported accuracy of representative crash prediction models—specifically looking at intersections—can be fairly capable at times but is also fairly variable. For example, in a study of 205 rural California and Michigan intersections of various types (e.g., three-legged, four-legged, stop-controlled, and signalized), Vogt found that the correlation between predicted crash rates and actual crash rates—expressed in terms of coefficient of determination (R-squared (R²))—ranged between 0.31 and 0.51 depending on the intersection type, averaging about 0.41 across all intersections.⁽⁴⁾ Thus, there remains a considerable degree of unexplained variation in the prediction of a “normal” rate of crashes for a given facility.

However, it is not specifically the accuracy of estimated “normal” crash rates that presents the greatest challenge. Rather, it is most perplexing that the nature of motor vehicle crashes—being so infrequent and exhibiting such variable yearly crash counts—is such that it may take years of crash data to reasonably narrow down the actual underlying crash rate of a location.

Statistical Challenges

The statistical challenges posed by the nature of motor vehicle crashes can be appreciated by considering the examples in the next few paragraphs.

Yearly crash counts, similarly to hourly traffic volume, can be approximated fairly well by a Poisson distribution, which exhibits variance (σ²) as high as the mean (μ).[1] Thus, the standard deviation (σ) is equal to the root of the mean (i.e., σ = μ^½). With a mean greater than 6, the Poisson distribution can be approximated by a normal distribution, and, in a normal distribution, approximately 95 percent of all outcomes fall within two standard deviations of the mean. Applying these assumptions to intersections with a mean rate of 100 crashes in a given timeframe, it could be said that most of these intersections (about 95 percent of them) would have crash counts during this timeframe that would fall within two standard deviations of the mean. With one standard deviation being 10 crashes, most crash counts would fall within 20 crashes of the mean (i.e., within the range of 80 to 120 crashes). Thus, in the timeframe necessary to generate 100 crashes, one could expect that all but 5 percent of intersections to exhibit crash counts within plus or minus 20 percent of the mean (100).

Crash rates are generally discussed as yearly rates, and intersections generally have far fewer than 100 crashes per year. In the aforementioned study by Vogt, a group of 49 intersections exhibited a mean crash rate of approximately 20 crashes per year. Thus, it would take about 5 years, on average, for each of these intersections to accumulate 100 crashes. However, Vogt’s study was based on 3 years of crash data for each intersection, which would result in an average of 60 crashes per intersection. Assuming, for the sake of illustration, that all 49 intersections had an identical mean rate of 20 crashes per year, then most of these intersections would have 3-year crash counts between 45 and 75 (i.e., within 25 percent of the mean). Vogt also studied a group of 84 unsignalized intersections with a mean of about 4 crashes per year. Intersections in this group would take 25 years to accumulate enough crashes—100 crashes—such that most intersections (even with identical crash rates) would be within 20 percent of the mean rate. However, with only a 3-year crash history, resulting in a mean of 12 crashes per intersection, it could only be said (assuming all intersections had an identical crash rate of 4 crashes per year) that most intersections would have crash counts between 5 and 19 (within 58 percent of the mean). Thus, the infrequent and variable nature of crashes presents a significant challenge in accurately pinpointing an underlying crash rate. The problem is increasingly difficult for areas with lower (more infrequent) crash rates, such as rural intersections.

Collecting 25 years of crash data is clearly impractical, and it is unlikely that underlying traffic conditions will remain static for even a few years. Thus, for lack of being able to pinpoint crash rates of intersections quickly and accurately, an agency might select the intersections with the most crashes for remediation. Consider, for example, that an agency was managing a set of 84 unsignalized intersections that all exhibited Poisson-distributed crashes at a rate of 4.0 crashes per year. Having collected three years of data, it could be expected that 10 percent of these signals (about 8 signals) would have accumulated 16 or more crashes, exhibiting a crash rate of 5.3 crashes per year. This agency could then contract a consultant to sprinkle a few drops of “safety water” on these intersections. After three more years, with no relevant change to the intersections and their underlying crash rate, one would expect these 8 signals to then exhibit a mean crash rate of 4.0 crashes per year. Thus, the consultant could update his sales brochure to claim a reduction in crashes of 33 percent on average, including perhaps one signal that did not respond to the treatment. This artificial before-and-after benefit is well known as regression-to-the-mean bias. Hauer and Persaud have shown a significant bias can exist even with 6 years of data, and they recommend using Empirical Bayes (EB) techniques to correct for this bias.⁽⁵⁾ However, correcting for the bias does not recover the resources spent (in this example on “safety water”) to treat intersections that were not as unsafe as they seemed to be with only 3 years of crash data.

Surrogate Safety Measures

The notion of surrogate safety measures—that is, measures other than actual crash frequency—is of interest to address the following needs:

• There is a need for the capability to assess the safety of traffic facilities without waiting for a statistically significant “abnormal” or “relatively greater” number of crashes to actually occur.
• There is a need for the capability to assess the safety of experimental roadway designs and/or operational strategies before they are actually built or employed in the field.

Several surrogate safety measures have been proposed in the literature and have been reviewed by Gettman and Head in a preceding report, FHWA-RD-03-050, Surrogate Safety Measures from Traffic Simulation.⁽⁶⁾ The following list provides several examples:

• Vehicle delay or travel time.
• Approach speed.
• Percentage of stopped vehicles.
• Queue lengths.
• Stop-bar encroachments.
• Red-light violations.
• Percentage of left turns.
• Speed distribution.
• Deceleration distribution.

The most prevalent literature in surrogate measures is related to the traffic conflicts technique, which is focused on observing traffic conflicts.

Traffic Conflicts

A conflict is defined as an observable situation in which two or more road users approach each other in time and space to such an extent that there is risk of collision if their movements remain unchanged.⁽⁷⁾

The traffic conflicts technique utilizes field observers to identify conflict events at intersections by watching for strong braking and evasive maneuvers.⁽⁸⁾ The method has a long history of development, formally beginning with studies at General Motors (GM) Research Laboratories in the late 1960s.⁽⁹⁾ The method has been shown to have some correlation to crashes. There is, however, still some debate regarding the connection between conflict measures and crash predictions. The main criticism of the technique is that the subjectivity of field observers induces additional uncertainty into the collection of accurate data on conflicts.

Nonetheless, conflict studies are still used to rank locations with respect to safety and a corresponding need for construction upgrades. There is general consensus that higher rates of traffic conflicts can indicate lower levels of safety for a particular facility. Aside from using total conflict counts, conflict events can also be categorized based on the type of driving maneuver (crossing, rear-end, and lane-change events) and by several measures of severity of the event.

In this study, it was found that the ratio of traffic conflicts to actual crashes was approximately 20,000 to 1. Thus, traffic conflicts occur with adequate frequency to overcome the statistical challenges posed by infrequent crashes. Also, since adequate conflict data can be collected in a relatively short time, conflict analysis is not subject to the problem of changing underlying conditions (e.g., traffic volumes and pavement conditions) that affect long-term crash records.

Surrogate Safety Assessment Model

To analyze new and innovative traffic facility designs, microscopic traffic simulation models are often used to predict the performance of those facilities before they are deployed in the real world. Simulation tools are extremely valuable in assessing the relative performance of one design versus another. In terms of measures of safety, existing simulation systems provide no guidance to analysts. This project is intended to evaluate the ability of simulation models to output meaningful measures of safety based on the occurrence of conflicts during the simulation. Relative differences in the frequency and severity of conflicts recorded from distinct traffic facility designs for the same underlying traffic demand would indicate that one facility design was safer than another. Thus, the purpose of this project is to test this hypothesis and provide guidance to the traffic engineering community in this regard.

The high-level scope of this study is twofold:

• To develop a software application to automate the task of conflict analysis.[2]
• To conduct validation testing to gauge the efficacy of this approach.

The use of surrogate safety assessment methods is grounded in the discussion of the methodology and recommendations outlined in report FHWA-RD-03-050. That study recommended the combination of traffic simulation and automated traffic conflict analysis, which in this project has been realized by the development a software utility referred to as SSAM. The software development effort included the following tasks:

• To determine data requirements and develop a “universal” data format—trajectory file format—that could be easily supported by all major simulation vendors and also be used for real-world datasets.
• To develop an algorithmic approach while concurrently defining data requirements to efficiently identify and classify conflict events and compute a series of desired surrogate safety measures for such events.
• To provide basic visualization and statistical features to facilitate analysis and report generation.
• To team with simulation vendors to add SSAM support to their software via support of the trajectory file format.

SSAM is compatible with the following traffic simulation software from four vendors who participated in the project:

• AIMSUN.
• Paramics.
• TEXAS.
• VISSIM.

The validation of the SSAM includes three distinct efforts:

• Theoretical validation.
• Field validation.
• Sensitivity analysis.

Each of these efforts are documented in a corresponding chapter and summarized in the next section.

Report Organization

The chapters of the report are summarized as follows:

• Chapter 1 has provided an introduction to SSAM and provided motivational background information.
• Chapter 2 provides an overview of SSAM, including a brief description of how the vehicle trajectory data are processed to identify conflict events. Chapter 2 includes a listing and definitions of various surrogate safety measures calculated by the software. A user manual for the software is also available separately from FHWA.
• Chapter 3 presents “theoretical” validation, which consists of relative safety comparisons of 11 design pairs. These comparison tests included assessment of basic roadway improvements with well-established incremental safety benefits, such as intersections with and without exclusive turn lanes. Tests were also performed for facilities without established safety assessments, including a comparison of three- and four-phase interchange control strategies, and a comparison of a traditional interchange to a double-roundabout.
• Chapter 4 discusses a field validation study based on 83 intersections in British Columbia, Canada. These intersections were modeled and simulated in VISSIM and then assessed with SSAM. Actual crash records were obtained from the Insurance Corporation of British Columbia (ICBC), and several validation tests were performed to compare simulation-based conflict data with the real-world crash records.
• Chapter 5 presents a sensitivity analysis (of the choice of simulation software) that parallels the field validation study on a smaller scale, comparing the results of SSAM assessment conducted with each of the four simulation systems (AIMSUN, Paramics, TEXAS, and VISSIM) on 5 of the 83 intersections from field validation study.
• Chapter 6 summarizes the overall effort under this project, recounting the salient findings and providing recommendations for future work.

Previous | Table of Contents | Next