• Federal Highway Administration >
• Publications >
• Research Publications >
• 08051 >
• 03a.Cfm >
• Surrogate Safety Assessment Model and Validation: Final Report

Surrogate Safety Assessment Model and Validation: Final Report

PDF files can be viewed with the Acrobat® Reader®

Chapter 3. Theoretical Validation

The validation effort for SSAM consists of a theoretical validation, field validation, and sensitivity analysis. This chapter presents the theoretical validation effort.

Purpose

The main purpose of theoretical validation of SSAM is to determine if the surrogate measures computed with the SSAM approach can discriminate between intersection designs in a simulation model. The secondary purpose of the theoretical validation effort is to identify any correlation between the surrogate measures produced by the SSAM approach and existing crash prediction models available from the literature.

Methodology

The hypothesis for the utility of surrogate measures of safety is that they will discriminate between two design alternatives implemented in a simulation. This involves the following steps:

• Model several intersection designs in a traffic simulation system.
• Run the simulation for various traffic scenarios and collect trajectory data.
• Process the trajectory data with SSAM to identify conflict events and derive surrogate measures of safety.
• Statistically compare the results from each design to identify statistical significant differences.

In addition, this effort also includes analysis of the following:

• Identify of the sensitivity of surrogate measures to simulation input variables (e.g., volumes).
• Identify the sensitivity of the results to severity thresholds for TTC

Implement Alternative Intersection Designs

As discussed in chapter 1, the resulting frequency and severity distributions of the conflict events that occur in the simulation are hypothesized to represent the surrogate measures of the safety of a particular intersection design. To evaluate the viability of using these measures for assessing safety, alternative intersection designs have been implemented in microscopic simulation systems and the corresponding output surrogate measures of safety for each conflict event or aggregation of the conflict events among alternative designs have been compared.

The intersection designs studied include many of the intersection types that are used in the real world. For each set (or pair) of alternative designs, traffic conditions (e.g. volumes for each approach, vehicle class, speed limit, driver's aggressive distributions, gap acceptance threshold, etc.) have been configured identically in order to make the alternatives comparable. Where alternative traffic flow scenarios were investigated, with a range of volumes and/or turning probabilities, the same conditions were applied to both intersection designs in the design-pair to maintain a reasonable basis for comparison. To ensure statistically representative measures for comparison, each situation was replicated 10 times for each design alternative.

Measures of Discrimination Between Designs

After running the simulation for each design, the corresponding surrogate measures were collected with SSAM, and statistical distributions of various aggregations were compared by the following:

• Total number of conflict events.
• Number of conflicts of a particular type.
• Mean and variance of surrogate measures of safety (TTC, PET, etc.).

The analysis of design alternatives has been conducted in a comparative manner because the essential information is more likely found in the differences between the results for two scenarios rather than from the absolute results for a particular scenario.

Data from One Simulation Run

After each simulation run, the vehicle trajectory data were processed by SSAM to compute the surrogate measures. For each conflict event identified by SSAM, the following has been recorded:

• Conflict type.
• Starting and ending points.
• Values of surrogate measures of safety.

An example of the data collected is shown in table 1. More data are collected on each event than is shown in the table below. Refer to the SSAM user manual for detail of all measures collected by SSAM.

SSAM classifies each conflict event as one of three conflict types: crossing, lane-change, or rear-end. Conflict type classification is based on the ConflictAngle, as defined in chapter 2. During the theoretical validation study, the conflict type was classified as a rear-end conflict if ||ConflictAngle|| < 2 °, a crossing conflict if ||ConflictAngle|| > 45 °, or a lane-changing conflict if 2 ° ≤ ||ConflictAngle|| ≤ 45 °. However, it is important to note that the classifications logic of SSAM changed subsequent to the theoretical validation in this chapter to achieve more accurate classification. (The revised logic appears in its entirety in the definition of ConflictType in chapter 2.) Revising the classification logic allowed recognition that many of the lane-change conflicts in a particular AIMSUN round-about model were actually events between pairs of vehicles on the same link and in the same lane. These events were clearly rear-end events, but due to the curvature of the roadway, the difference in vehicle headings (i.e., the conflict angle) exceeded the 2 ° threshold for a rear-end event. The revised logic improved classification, though there are still "gray area" cases (e.g., a vehicle entering into a roundabout collides with a vehicle within the roundabout) where classification of an event as crossing, lane-changing, or rear-end is arguably a subjective judgment. Indeed, it could be argued that some conflicts are simultaneously of two or three types (e.g., lane-change and rear-end).

Table 1. Conflict Events Data for Each Replication.

Time/ Index	TTC	PET	Max S	DeltaS	DR	Max D	Max DeltaV	Conflict type1
								I	II	III
1	0.2	0.5	29	9.3	1	1	7.6
2	0.1	0.5	44	31.6	-0.6	-1.45	20.5
3	1.4	4.5	27	5.7	-21.2	-21.2	25
...	0.9	3	15.1	15.1	-11.2	-16.3	13
Total	Total conflict events: 177							55	65	57

Table 1 is an example of the data available for each conflict event. Aggregated values or summary measures have also been collected, such as the total number of conflict events with TTC values in different severity ranges (e.g. 0 < TTC ≤ 0.5, 0 < TTC ≤ 1.0). This down-selection of the data is done by using the Filter function of SSAM. An example aggregation by conflict type is shown in table 2.

Table 2. Mean Safety Measures for Each Conflict Type.

Conflict Type	# of Conflict Events
I	55	1.2	1.1	-4.5	20.5	17.3
II	65	1.5	1.3	-1.68	19.5	16
III	57	1.33	1.12	-5.73	22.6	21.3

Note: The bar over the variable indicates the average value of that surrogate safety measure over all the conflict events of each conflict type.

Statistical Results from Multiple Replications

Each intersection design is simulated with multiple replications, each using different random number seeds, and statistical distributions of the results were collected and analyzed. For the intersection design alternatives, a sample size of 10 replications was used throughout this study.

Comparison of Alternative Designs

For each set (pair) of alternative designs, the output measures have been compared statistically to identify the significance of the difference between the designs. An example of this comparison is shown in table 3. The student's t-test was used to compare each type of surrogate safety measures and the frequency of conflicts for alternative designs. The t-test calculates the probability of the difference of the two means. In this test, the null hypothesis (H0) indicates that the difference between the means of two samples is 0. Based on the difference level of the two sample variances, t-ratios and degree of freedom are calculated in different ways. Whether or not the sample variances are significantly different is verified by using the F-test before the t-test is performed. When the average number of events in a conflict type category and/or total conflicts is less than 0.5 (meaning that out of the 10 replications, an event occurs approximately every other simulation run), the data are marked as N/A, and no test outcome is recorded.

Table 3 and table 4 are examples of the statistical analyses that were performed in the theoretical validation study.

Comparison to Predicted Crash Frequency

In addition to the comparison analysis for each set (pair) of the alternative designs, the theoretical validation study also compared the relative values of surrogate measures of safety to predictions of safety from regression-based models of crash prediction developed and calibrated by others.

Regression models were used to calculate the expected crash frequency for each simulated scenario. Lognormal regression models have been applied in this study. Specific models are used for each of the following for classes of intersections:

• Urban, four-leg, signalized/stop-controlled intersection.
• Urban, three-leg, signalized/stop-controlled intersection (T-intersection).
• Diamond interchange.
• Roundabout.

Many of the models presented in this chapter use the term accident instead of the term crash. Crash is the preferred term used in this document; however, these terms may be considered interchangeable. The term accident is retained at times due to the historical use of variables or acronyms, such as AMF, which stands for accident modification factor.

Accident prediction models for urban, four-leg, signalized intersection are established by Harwood and Council:⁽¹³⁾

Figure 17. Equation. Accident Prediction Model for an Urban, Four-Leg, Signalized Intersection.

Where:

A is the predicted number of total intersection-related accidents per year.

is the accident modification factor for the presence of left- turn lane:

0.82 for one major-road approach.

0.67 for both major-road approaches.

is the accident modification factor for the presence of right-turn lane:

0.975 for a right-turn lane on one major-road approach.

0.95 for right-turn lanes on both major-road approaches.

is the average daily traffic (ADT) volume (veh/day) on the major road.

is the ADT volume (veh/day) on the minor road.

Accident prediction for urban, four-leg, stop-controlled intersection are also given by Harwood and Council:⁽¹³⁾

Figure 18. Equation. Accident Prediction Model for an Urban, Four-Leg, Stop-Controlled Intersection.

Where:

Accident prediction models for urban, three-leg, signalized intersection (T-intersection) are given by Bared and Kaiser:⁽¹⁴⁾

Figure 19. Equation. Accident Prediction Model for a Three-Leg, Signalized Intersection.

Where:

A is the predicted number of total intersection-related accidents per year.

is the entering ADT on the main road.

is the entering ADT on the crossroad.

An accident model for urban, three-leg, stop-controlled intersection (T-intersection) is provided by Harwood and Council:⁽¹³⁾

Figure 20. Equation. Accident Prediction Model for an Urban, Three-Leg, Stop-Controlled Intersection.

Where:

A is the predicted number of total intersection-related accidents per year

is the accident modification factor for the presence of left-turn lane on major road: 0.78 for one major-road approach.

is the accident modification factor for the presence of right-turn lane: 0.95 for a right-turn lane on one major-road approach.

is the accident modification factor for the sight restrictions: 1.05 if sight distance is limited in one quadrant of the intersection. 1.10 if sight distance is limited in two quadrants of the intersection.

1.15 if sight distance is limited in three quadrants of the intersection.

1.20 if sight distance is limited in four quadrants of the intersection.

is 0.53, accident modification factor for the conversion from minor road to all-way stop-control.

SKEW is the intersection skew angle (degrees), expressed as the absolute value of the difference between 90 ° and the actual intersection angle.

is the ADT volume (veh/day) on the major road.

is the ADT volume (veh/day) on the minor road.

An accident prediction at a diamond interchange is given by Wolshon:⁽¹⁵⁾

Figure 21. Equation. Accident Prediction Model for a Diamond Interchange.

Where:

A is the predicted number of intersection related accidents at the cross-road of a diamond interchange.

is the ADT volume (veh/day) on the cross-road.

is the ADT volume (veh/day) on the off-ramps.

Roundabout

Crash prediction models have been developed for four-leg, signalized intersections in the United States, as discussed previously. However, no crash prediction models exist for U.S. roundabouts and driver behavior. Given the relatively recent introduction of roundabouts to the United States and driver unfamiliarity with them, crash prediction models from other countries have been used.

Crash models relating crash frequency to roundabout characteristics are available from the United Kingdom. The British crash prediction equations for each type of crash are listed in figure 22 through figure 26. Note that these equations are only valid for roundabouts with four legs. However, the use of these models for relative comparisons may still be reasonable.⁽¹⁶⁾

1. Entry-Circulating:

Figure 22. Equation. Entry-Circulating Roundabout Accident Prediction Model.

Where:

A are personal injury accidents (including fatalities) per year per roundabout approach.

is entering flow (1,000s of vehicles/day).

is circulating flow (1,000s of vehicles/day).

is entry curvature (= 1/).

is entry path radius for the shortest vehicle path (m).

e is entry width (m).

v is approach width (m).

R is ratio of inscribed circle diameter/central island diameter.

is proportion of motorcycles (percent, %).

is the angle to next leg measured centerline to centerline (degrees, °).

2.   Approaching:

Figure 23. Equation. Accident Prediction Model for Roundabout Approaches.

Where:

A are personal injury crashes (including fatalities) per year at roundabout approach or leg.

is entering flow (1,000s of vehicles/day).

is entry curvature =1/.

is entry path radius for the shortest vehicle path (m).

e is entry width (m).

3.   Single vehicle: [5]

Figure 24. Equation. Single-Vehicle Accident Model for Roundabouts.

Where:

A are personal injury crashes (including fatalities) per year at roundabout approach or leg.

is entering flow (1,000s of vehicles/day).

is entry curvature =1/.

is entry path radius for the shortest vehicle path (m).

V is approach width (m).

is approach curvature = 1/.

is approach radius (m). Defined as the radius of a curve between 50m (164 ft) and 500 m (1,640 ft) of the yield line.

4.   Other (vehicle):

Figure 25. Equation. Other Vehicle Accident Prediction Model for Roundabouts.

Where:

A are personal injury crashes (including fatalities) per year per roundabout approach.

is entering flow (1,000s of vehicles/day).

is circulating flow (1,000s of vehicles/day).

is proportion motorcycles (percent, %).

5. Pedestrian:

Figure 26. Equation. Pedestrian Accident Prediction Model for Roundabouts.

Where:

A are personal injury crashes (including fatalities) per year at roundabout approach or leg.

is the product (+ ).

is entering flow (1,000s of vehicles/day).

is exiting flow (1,000s of vehicles/day).

is pedestrian crossing flow (1,000s of pedestrians/day).

Since the current method only defines conflict events for pairs of vehicles, crash types 3, 4, and 5 (single, other, and pedestrian, respectively) have been ignored in using the prediction models for roundabouts.

Comparison of Intersection Rankings by Conflict and Crash Frequencies

Another important indicator that would validate SSAM would be a correlation of surrogate measures with predicted crash frequencies. Such a comparison has been performed for each comparison scenario in the theoretical validation study. To do this, first the simulation for each intersection design was run with different traffic volumes (low, medium, high annual average daily traffic (AADT)) and the corresponding conflicts (total conflicts and total number of conflicts of each event type) were analyzed. The results were then ranked from highest to lowest. Summary measures with the same values were assigned equal rank.

For each design scenario, the predicted number of crashes using an existing crash prediction model was also calculated . This prediction is repeated for each level of traffic volume (i.e., AADT). A rank of the number of crashes was then established and compared to the ranking of number of conflicts of each type. Table 5 gives an example of the data needed for the correlation calculation. Table 6 shows an example of the paired rank data.

The Spearman rank correlation coefficient was then computed to determine the level of agreement between each pair of rankings. The Spearman rank correlation coefficient is defined by

Figure 27. Equation. Spearman Rank Correlation Coefficient.

Where:

d is difference between ranks.

N is number of paired ranks.

Then the resulting correlation coefficient is compared with the critical coefficient value with the appropriate sample size and the significance level. If the absolute value of the coefficient is greater than the critical value, then it can be concluded that there is a rank order relationship between these samples. If the R_s value is -1, then there is a perfect negative correlation between the two sets of data. If the R_s value is 1, then there is a perfect positive correlation between the two sets of data. Table 8 provides a numeric example of this. In this example, we would find that the conflict data have a positive, but weak, correlation with the predicted crash frequency.

Note: M = average value of the measure.

Issues with Validation Metrics

Reconciliation of ADT with Hourly Volumes

In crash prediction models, traffic volumes are in the unit of ADT while traffic volumes used in all of the simulation systems are in the unit of vehicles per hour. To ensure the consistency of the comparison, converting rules need to be applied to reconcile these two terms. By using K factors, we have converted ADT to vehicles per hour and vice versa as shown in figure 28:⁽¹⁷⁾

Figure 28. Equation. Using K-Factors to Scale Hourly Volume to Daily Volume.

Where:

ADT is the average daily traffic volume.

HV is the hourly volume.

K is the conversion factor.

The K value should vary with different area types. For the general purpose of this study, values from the Highway Capacity Manual (2000) were used as shown in table 7:⁽¹⁷⁾

Where:

• Urbanized areas are those designated by the U.S. Census Bureau.
• Urban areas are places with a population of at least 5,000 not already included in an urbanized area.
• Transitioning areas are the areas outside of, or urbanized areas expected to be included in, an urbanized area within 20 years.
• Rural areas are whatever is not urbanized, urban, or transitioning.

Overlapping Vehicles in TRJ Output ("Crashes")

In each of the simulation models, some situations result in "virtual" crashes. These are situations where the logic in the simulation model does not accurately and completely represent the physical possibility of a particular maneuver.

This does not happen frequently relative to the total number of traffic maneuvers being performed in a simulation; however, because the data are being analyzed at an extremely "nanoscopic" scale, SSAM identifies these modeling inaccuracies as conflicts with TTC = 0 ("crashes"). In this report, all crashes have been removed before the statistical calculations are performed. In some cases during the analysis of the theoretical validation data, it was observed that including the virtual-crashes in the analysis results in a different statistical determination. As many crashes as possible have been removed by appropriate modeling of the design case. For all the models tested, it is imperative that the analyst implement the design appropriately.

Previous | Table of Contents | Next