• Federal Highway Administration >
• Publications >
• Research Publications >
• 08051 >
• 06.Cfm >
• Surrogate Safety Assessment Model and Validation: Final Report

Surrogate Safety Assessment Model and Validation: Final Report

PDF files can be viewed with the Acrobat® Reader®

Chapter 6. Conclusion

Summary

This project evaluated a method of safety assessment utilizing a traffic conflicts analysis technique applied to simulation models of intersections, interchanges, and roundabouts. The high-level scope of project was two-fold:

• Develop a software application to automate the task of traffic conflicts analysis.
• Conduct validation testing to gauge the efficacy of the assessment method.

Model Development

The safety assessment approach in this project is grounded in the discussion of surrogate safety assessment methodology and recommendations outlined in report FHWA-RD-3-050: Surrogate Safety Measures from Traffic Simulation.⁽⁶⁾ The current project fleshed out the algorithmic proposals outlined in that preceding project, and the method was codified into a software utility, referred to as the Surrogate Safety Assessment Model (SSAM). SSAM identifies conflict events by processing detailed vehicle trajectory data, which can be exported from the following traffic simulation software of four corresponding vendors who collaborated on the project:

• AIMSUN
• Paramics
• TEXAS
• VISSIM

Conflicts are identified when the trajectories of two vehicles (headings and velocities) indicate an imminent collision with a TTC of less than 1.5 seconds. Although the evaluation focus of this project was oriented toward intersections, interchanges, and roundabouts, the SSAM algorithms can identify conflicts on any type of roadway where two vehicles travel in close proximity (e.g., a section of freeway). The conflict events are classified by maneuver type (path-crossing, rear-end, and lane-change events), and SSAM computes corresponding surrogate safety measures (TTC, PET) and hypothetical collision severity measures (Delta-V). It was hypothesized that the measures such as the relative frequency of conflicts of two traffic facilities may be used to distinguish the relatively frequency of crashes, and thus, the relative safety of the two traffic facilities.

Model Validation

Validation of SSAM was a three-part undertaking, consisting of the following:

• Theoretical validation.
• Field validation.
• Sensitivity analysis.

These three validation activities are summarized along with their corresponding findings in the next section.

FINDINGS

Theoretical Validation

The theoretical validation effort assessed the use of SSAM to discern the relative safety of a pairs of intersection/interchange design alternatives in a series of eleven case studies as follows:

• Signalized, four-leg intersection with permitted left turn versus protected left turn.
• Signalized, four-leg intersection with and without left-turn bay.
• Signalized, four-leg intersection with and without right-turn bay.
• Signalized, four-leg intersection with leading left turns versus lagging left turns.
• Signalized, four-leg intersection versus a pair of offset T-intersections.
• Diamond interchange with three-phase timing versus four-phase timing.
• SPUI versus diamond interchange.
• Signalized, four-leg intersection with left turns versus no left turns with median U-turn bays.
• Signalized, four-leg intersection versus roundabout.
• Signalized, three-leg, T-intersection versus roundabout with three legs.
• Diamond interchange versus double roundabout.

It was found that under equivalent traffic conditions (e.g., traffic volumes and turning percentages), for both intersection design alternatives, SSAM could discern statistically significant differences in the total number of conflicts, the number of conflicts by type (i.e., crossing, lane-change, or rear-end events), and conflict severity indicators (e.g., average TTC, PET, Delta-V values). However, in most cases the comparison of the two alternatives did not reveal a clearly preferable design but rather a trade-off of surrogate safety measures. It was typical, for example, that one design exhibited a higher frequency of conflicts, but those conflicts exhibited lower severity ratings than the alternative design. This type of assessment outcome hinders unequivocal decision-making about which design is the safer of the two.

These results clearly point to the need for future research to develop a “conflict index” or “safety index”. This might be accomplished by computing appropriate weightings of observed conflicts of different types, frequencies and severities, and aggregating results observed from a distribution of daily traffic conditions to form a composite safety assessment of a traffic facility. This would facilitate safety assessment efforts, alleviating analysts from the need to undertake their own series of complex calculations and judgments.

Field Validation

The field validation effort was concerned with the direct accuracy of surrogate safety assessment, as opposed to the relative safety assessment of the theoretical validation. A set of 83 field sites were selected-all four-leg, urban, signalized intersections-and were modeled in VISSIM, simulated, and assessed with SSAM. The conflict analysis results of these intersections were compared to actual crash histories (based on corresponding insurance claims records), using five statistical tests. This effort also provided an opportunity for benchmark comparison of surrogate safety estimates versus traditional crash prediction models based on ADT volumes.

It was found that the simulation-based intersection conflicts data provided by SSAM were significantly correlated with the crash data collected in the field, with the exception in particular of conflicts during path-crossing maneuvers, which were under-represented in the simulation. The relationship between total conflicts and total crashes exhibited a correlation (R-squared) value of 0.41, which is consistent with the typical performance reported in several studies using traditional crash prediction models on urban, signalized intersections. However, it was notable that in this study, the traditional (volume-based) crash prediction models were better correlated with the crash data than the surrogate measures in all test cases. For example, ADT-based crash prediction models exhibited a correlation (R-squared) value of 0.68 with actual crash frequencies.

It is well-established that as traffic volume increases, so does the occurrence of crashes and conflicts. Thus, some correlation of conflicts frequencies and crash frequencies is to be expected. This effort did find a significant correlation between simulated conflicts and actual crashes; however, a good correlation between intersections with abnormally high conflicts and abnormally high crashes was not found. This finding does not suggest that such a relationship can be definitively rejected, as tests conducted to that end proved somewhat unsuitable to the task. Thus, while the SSAM approach shows significant potential, the validation results did not reach a definitive conclusion.

Sensitivity Analysis

The sensitivity analysis effort complemented the field validation study, which was limited solely to intersection modeling with the VISSIM simulation. The sensitivity analysis reassessed 5 intersections-of the 83 considered in the field validation-with each of 4 simulation systems: AIMSUN, Paramics, TEXAS, and VISSIM. A series of comparisons were employed to characterize the sensitivity and/or bias of the surrogate safety measures, as they differed when obtained from each of the four simulations.

It was found that a fairly wide range of results can be obtained from applying different simulation models to the same traffic facility designs. In general, intersections modeled in VISSIM exhibited the fewest total conflicts, and intersections modeled in TEXAS had the highest conflict frequency-approximately 10 times higher than VISSIM. Conflict totals from AIMSUN and Paramics fell between these extremes. The abnormally high number of conflicts in TEXAS seems to stem (somewhat paradoxically) from the explicit inclusion of active conflict avoidance in the driver behavior model of TEXAS, whereas other simulations employ more reactive driver behavior modeling. An example of reactive behavior manifested in the form of particularly extreme braking/deceleration events observed in the AIMSUN and Paramics simulations.

In all of the simulation systems, rear-end conflict events made up the bulk of the total conflicts at all evaluated TTC thresholds (0.5s, 1.0s, and 1.5s). This bias persisted even after eliminating low-speed events from the analysis (i.e., events occurring at speeds less than 16.1 km/h (10 mi/h)). There were no major differences in the average TTC values across the models, although AIMSUN and Paramics did exhibit higher average deceleration rates (DR) and lower PET, consistent with their relatively reactive driver behavior modeling. In general, the traffic performance measures such as throughput and delay were variable but vaguely comparable from all systems under light traffic; however, the differences in the default driving behaviors and modeling assumptions pronounced differences in results at higher congestion levels. Also, SSAM identified questionable scenarios in all simulations where vehicles were driving directly through one another (i.e., crashes or conflicts with a TTC of 0).

Recommendations

The SSAM approach demonstrated significant correlations with actual crash data, consistent with the range of correlations reported in several studies with traditional (primarily volume-based) crash prediction models; although, in direct comparison, volume-based prediction models provided better correlation to field data (crash records) than simulated conflicts. [8] SSAM also demonstrated a capability to distinguish safety differences between different intersection design features under the same traffic volumes, though differences were often a trade-off, improving one measure with degrading another. Additionally, SSAM is applicable to the analysis of traffic facilities that have not yet been constructed and traffic control policies not yet enacted in the field. Thus, the SSAM approach exhibits promise, while at the same time the validation results are not definitive.

SSAM and corresponding documentation is available to the public at no cost and can be obtained from the FHWA. [9] It may well serve as a useful assessment tool in capable hands where analysts are cognizant of the underlying limitations discussed in this report.

There were indeed a number of limitations, which motivate the recommendation of certain directions in future research:

• Improve driver behavior modeling in simulations.
• Develop a composite “safety index”.
• Study the underlying nature of conflicts in real-world data.
• Collect adequate vehicle trajectory data sets from the real world.
• Investigate conflict classification criteria.

Improve Driver Behavior Modeling in Simulations

As discussed in the report, SSAM analysis was often confounded by occurrences of unintended crashes in the model that could not be removed completely without modeling techniques that resulted in unrealistic driver behaviors and unrealistic facility performance (e.g., reduced approach capacity and throughput). Notably, ongoing simulation enhancements have already been reported to the project team which may enhance the quality of analysis results possible with SSAM in the coming months and years:

• VISSIM has added the notion of “conflict areas” to version 4.3, which incorporates the notion of explicit conflict avoidance logic as an alternative/adjunct to priority rules.
• TEXAS has undergone revisions that have substantially reduced unintended crashes in benchmark test models.

Develop a Composite “Safety Index”

It is evident from the validation effort that modification of the intersection design or traffic control policy may lead to a trade-off in surrogate safety measures where, for example, there is a significant increase in rear-end conflicts but a significant decrease in crossing conflicts, or, as another example, a decrease in total conflicts but an increase in severity measures. Thus, there is a need to research the development of a composite index scheme that could factor in the multitude of often contradictory surrogate safety indicators. This could facilitate easier and more accurate safety comparisons and decision-making.

Study the Underlying Nature of Conflicts in Real-World Data

The field validation and sensitivity analysis efforts both exhibited a distribution of conflicts by type and severity that lean more heavily toward less dangerous events than the distribution of events found in actual crash records. For example, the conflict-to-crash ratio is higher for rear-end conflicts than it is for lane-change conflicts and crossing conflicts. However, in digesting this finding, it is difficult to discern whether this discrepancy is due to flaws in the underlying simulations models or the conflict identification scheme or if the conflict occurrence in the field does indeed differ from crash occurrence. Thus, there is a need to study real-world data to quantify the shortcomings of simulated conflict data versus real-world conflict data and to learn any potential shortcomings of the current analysis method. Manual (human observer) field studies are not capable of collecting the rich surrogate safety information that SSAM can glean from detailed vehicle trajectory data. Thus, using SSAM to study real-world vehicle trajectory data and conflict phenomena could yield new understanding. Additionally, efforts to develop an appropriate composite would benefit from real-world data for calibration.

Collect Adequate Vehicle Trajectory Datasets from the Real World

Aside from studying real-world data, that data must actually be collected. Manual (human observer) studies are not capable of recording detailed vehicle trajectory data. Efforts to collect data from video image processing are improving, though additional research and development effort is warranted. This is an ambitious task unto its own and thus is listed as separate research direction.

Investigate Conflict Classification Criteria

This study classified vehicle conflicts as one of three types: rear end, lane changing, or crossing. The classification logic was initially based only the angle of two converging vehicles and then was revised for more accurate capture of rear-end and lane-changing events, utilizing knowledge of underlying lanes and links where possible. However, link and lane information is often not applicable (which is the subject of a protracted conversation not included here). For example, in potentially applying SSAM to process real-world data, an underlying link/lane model might not be available. As a concrete example, perhaps the most conspicuous case where the “lines” of classification are blurred is where a vehicle entering a roundabout conflicts with a vehicle within the traffic circle. Supposing the vehicle traveling within the traffic circle crashes into the rear of the entering vehicle-can it be said clearly where/how a lane-change event is differentiated from a rear-end event in this case? Is there a precise angle at which the entering vehicle should be classified as crossing rather than lane changing? This topic warrants further investigation into appropriate angles and additional criteria/logic for classification. It would also be useful to document the underlying value and motivation of classifying conflicts. Perhaps more conflict types or subtypes should be considered, and perhaps a conflict should be allowed to have multiple classifications with either binary or partial memberships in those classes. Aside from classification by movement types, perhaps there are also useful classifications by severity type that (like movement type classification) are properly identified only with multidimensional considerations rather than imposing threshold ranges on a single measure. Such investigation should include consideration of field data and provide guidance on effective and useful classification.

Resources

As mentioned previously, SSAM and corresponding documentation is available to the public at no cost and can be obtained from the FHWA. As more analysts gain experience with the technique and analyze additional types of traffic facilities, additional directions for development and research will likely be identified. It is recommended that FHWA continue to collect experiences of analysts and form a committee of experts to discuss issues of surrogate measures in further determining the needs for future research.

Previous | Table of Contents | Next