• Federal Highway Administration >
• Publications >
• Research Publications >
• 08051 >
• 05a.Cfm >
• Surrogate Safety Assessment Model and Validation: Final Report

Surrogate Safety Assessment Model and Validation: Final Report

PDF files can be viewed with the Acrobat® Reader®

Chapter 5. Sensitivity Analysis

PURPOSE

The sensitivity analysis serves to supplement the field validation effort. The main purpose of the sensitivity analysis is to capture the differences between using one simulation model versus another and provide guidance to the safety analysis community.

METHODOLOGY

SSAM was tested across the VISSIM, AIMSUN, PARAMICS, and TEXAS microsimulation platforms by implementing five U.S. intersections from the Hamilton Associates' database in all four platforms. The conflict summary results from each simulation for the five U.S. intersections was tabulated and analyzed with and without accounting for crashes (e.g., crashes resulting from lane changing). The performance of SSAM was compared on the various conflict measures output by SSAM (conflict rate, TTC, PET, etc.). Various threshold values were used (e.g., TTC < 1.5 s, TTC < 1.0 s, etc.) to evaluate the differences in output from each microsimulation.

The same five intersections of representative types were implemented in all four of the microsimulation platforms (VISSIM, AIMSUN, PARAMICS, and TEXAS) used in the study. This involved the following steps:

• Implement the same field intersection designs across the four microsimulation platforms.
• Run the simulation with multiple replications and collect trajectory data and measures of effectiveness (MOEs).
• Run the SSAM tool to derive surrogate measures of safety.
• Compare the results from different microsimulation platform for the same simulated case.

Implement Intersection Designs in all four of Microsimulation Platforms

SSAM was tested across VISSIM, AIMSUN, PARAMICS, and TEXAS. The sensitivity analysis was performed by implementing five U.S. intersections from the Hamilton Associates' database in all these four platforms.

For each simulated case, the four simulation models are modeled to keep the geometry data, traffic volume, signal timing, and some important network parameters the same. Other parameters such as driving-behavior parameters remain at default values in each simulation model to avoid unrealistic maneuvers. As indicated in the theoretical analysis report, in most scenarios, evaluated crashes cannot be completely eliminated from the simulation without modifying driver-behavior parameters or design geometry beyond reasonable limits. The number of crashes and low-speed events constituted a large percentage of the total conflict events that occurred in most of the scenarios tested. Moderate adjustment of simulation parameters was made during the sensitivity analysis for each microsimulation platform to reduce such crashes. Multiple replications were implemented for each simulated case in each simulation model to minimize random factors during comparison effort.

Obtain Measures for Each Case

Safety Measures for Each Case

Safety measures recorded from SSAM were tabulated for each simulated case, including total conflicts, conflict number for each conflict type, TTC, PET, etc. Each type of measure was compared across VISSIM, AIMSUN, PARAMICS, and TEXAS.

Conflicts Layout Display for Each Case

Another important task within sensitivity analysis was to compare the conflicts layout across the four simulation platforms for each simulated case.

MOEs for Each Case

Network MOEs for each simulated case were recorded and compared across the four simulation platforms (e.g., travel time and average delay).

SENSITIVITY ANALYSIS RESULTS

Five U.S. intersections from the Hamilton Associates' database were executed in the sensitivity analysis. In terms of traffic volumes and turning counts, each intersection had AM peak-hour data and PM peak-hour data. Two out of the five intersections also had mid-day data.

1. Briarcliff Rd & North Druid Hills Rd, Dekalb County, Atlanta, GA.
2. Roswell Road & Abernathy Road , Fulton County, Atlanta, GA.
3. Lafayette Ave & Fulton Street, Grand Rapids, MI.
4. Ryan Ave & Davison Ave, Detroit, MI.
5. Howe Ave & Fair Oaks Boulevard, Sacramento, CA.

Intersection Description

Intersection 1: Briarcliff Rd & North Druid Hills Rd, Dekalb County, Atlanta, GA

This was a skewed intersection with two approaches having right-turn bays. All four approaches had independent left-turn bays, ranging from 32.9 m (108 ft) to 111.3 m (365 ft). The intersection was under semiactuated signal control and had three timing plans according to three peak-hour periods-AM peak, mid peak, and PM peak. Traffic volumes were around 4,300-4,800 vehicles per hour. The speed limit specified for all approaches was 56.35 km/ h (35 mi/h), and the 3-year annual collision frequency for this intersection was about 27.7. Figure 134 shows a Google^TM map view of this intersection. Figure 135 through 138 show the intersection layout modeled within each simulation platform.

Figure 135. Screen Capture. VISSIM Model of Briarcliff Rd & North Druid Hills Rd.

Figure 136. Screen Capture. AIMSUN Model of Briarcliff Rd & North Druid Hills Rd.

Figure 137. Screen Capture. PARAMICS Model of Briarcliff Rd & North Druid Hills Rd.

Figure 138. Screen Capture. TEXAS Model of Briarcliff Rd & North Druid Hills Rd.

Intersection 2: Roswell Road & Abernathy Road, Fulton County, Atlanta, GA

This was a skewed intersection with all approaches having right-turn bays and left-turn bays. The intersection was under semiactuated signal control and had three timing plans according to three peak-hour periodsAM peak, mid peak, and PM peak. Traffic volumes were around 5,2005,700 vehicles per hour. The speed limit specified for all approaches is 56.35 km/h (35 mi/h), and the 3-year annual collision frequency for this intersection is about 92. Figure 139 shows a Google^TM map view of this intersection. Figure 140 through figure 143 show the intersection layout modeled within each simulation platform.

Figure 139. Photo. Google ^TM Map View of Roswell Road & Abernathy Road.

Figure 140. Screen Capture. VISSIM Model of Roswell Road & Abernathy Road.

Figure 141. Screen Capture. AIMSUN Model of Roswell Road & Abernathy Road.

Figure 142. Screen Capture. PARAMICS Model of Roswell Road & Abernathy Road.

Figure 143. Screen Capture. TEXAS Model of Roswell Road & Abernathy Road.

Intersection 3: Lafayette Ave & Fulton Street, Grand Rapids, MI

This intersection was under semiactuated signal control and had two timing plans according to two peak-hour periodsAM peak and PM peak. Only S-N bound had independent left-turn bays, and there were no right-turn bays for any approach. Traffic volumes were around 1,9002,400 vehicles per hour. The speed limit specified for all approaches was 48.3 km/h (30 mi/h), and the 4-year annual collision frequency for this intersection is about 33. Figure 144 shows a Google^TM map view of this intersection. Figure 145 through figure 148 show the intersection layout modeled within each simulation platform.

Figure 144. Photo. Google ^TM Map View of Lafayette Ave & Fulton Street.

Figure 145. Screen Capture. VISSIM Model of Lafayette Ave & Fulton Street.

Figure 146. Screen Capture. AIMSUN Model of Lafayette Ave & Fulton Street.

Figure 147. Screen Capture. PARAMICS Model of Lafayette Ave & Fulton Street.

Figure 148. Screen Capture. TEXAS Model of Lafayette Ave & Fulton Street.

Intersection 4: Ryan Ave & Davison Ave, Detroit, MI

This intersection was under fixed-time signal control and had two timing plans according to two peak-hour periodsAM peak and PM peak. Only E-W bound had independent left-turn bays, and there were no right-turn bays for any approach. Traffic volumes were around 2,6003,000 vehicles per hour. The speed limit specified for N-S approaches was 40.25 km/h (25 mi/h) while the E-W approaches had a 48.36-km/h (30-mi/hr) speed limit. The 2-year annual collision frequency for this intersection was about 20.5. Figure 149 shows a Google^TM map view of this intersection. Figure 150 through figure 153 show the intersection layout modeled within each simulation platform.

Figure 149. Photo. Google ^TM Map View of Ryan Ave & Davison Ave.

Figure 150. Screen Capture. VISSIM Model of Ryan Ave & Davison Ave.

Figure 151. Screen Capture. AIMSUN Model of Ryan Ave & Davison Ave.

Figure 152. Screen Capture. PARAMICS Model of Ryan Ave & Davison Ave.

Figure 153. Screen Capture. TEXAS Model of Ryan Ave & Davison Ave.

Intersection 5: Howe Ave & Fair Oaks Boulevard, Sacramento, CA

This was a skewed intersection with independent left-turn bays and right-turn bays for all approaches. The intersection was under semiactuated signal control and had two timing plans according to two peak-hour periodsAM peak and PM peak. Traffic volumes were around 6,5008,800 vehicles per hour. The speed limit specified for N-S approaches was 64.4 km/h (40 mi/h) while the E-W approaches had a 72.45-km/h (45-mi/h) speed limit. The 3-year annual collision frequency for this intersection was about 28.3. Figure 154 shows a Google^TM map view of this intersection. Figure 155 through figure 158 show the intersection layout modeled within each simulation platform.

Figure 154. Photo. Google ^TM Map View of Howe Ave & Fair Oaks Boulevard.

Figure 155. Screen Capture. VISSIM Model of Howe Ave & Fair Oaks Boulevard.

Figure 156. Screen Capture. AIMSUN Model of Howe Ave & Fair Oaks Boulevard.

Figure 157. Screen Capture. PARAMICS Model of Howe Ave & Fair Oaks Boulevard.

Figure 158. Screen Capture. TEXAS Model of Howe Ave & Fair Oaks Boulevard.

Data Analysis and Comparison Results

Number of Conflicts (Rate)

The number of conflicts was extracted from SSAM for each case. The conflicts rate is expressed as a ratio between number of conflicts and traffic volume. With modeling limitations from each simulation platform, total conflicts were filtered out by SSAM to obtain more accurate data, resulting in three set of conflicts: total conflicts, denoted as C-N1; conflicts without abnormal deceleration (max braking rate less than -9.15 m/sec² (-30 ft/sec²)), denoted as C-N2; and non-crash (TTC > 0) and non-low-speed (max speed = 16.1 km/h (10 mi/h)) conflicts without abnormal deceleration, denoted as C-N3. Conflicts within each TTC category were also tabulated and investigated.

Note that a maximum deceleration rate of 9.15 m/sec² (30 ft/sec²) is considered very extreme. To put this in perspective, consider the results of the recent testing of several fleet vehicles by the Michigan State Police where it was found that among all vehicles tested (including motorcycles), the maximum deceleration rate obtained in was just under 9.15 m/sec² (30 ft/sec²), with emergency (panic) braking conditions using vehicles equipped with anti-lock braking systems (ABS).⁽³³⁾

Intersection 1: Briarcliff Rd & North Druid Hills Rd, Dekalb County, Atlanta, GA:

The number of conflicts/conflict rates under AM, mid, and PM peak-hour conditions was tabulated and is shown in table 93 through table 104. Column charts were created to compare the conflicts number generated from the four simulation platforms, as seen in figure 159 through figure 182.

As shown in the following tables and figures, for all simulation models, most of the conflicts were rear-end conflicts, and crossing conflicts were the least conflicts among all conflicts types. Crossing conflicts had a smaller TTC value while rear-end conflicts had a larger TTC value. The differences among the four simulation models were the following:

• VISSIM had the least conflict number for all categories. TEXAS had the most conflict number for all categories. TEXAS had more than 10 times the conflicts of VISSIM.
• The ascending ranking order in most cases was VISSIM, AIMSUN, PARAMICS, TEXAS. However, the order for C-N3 was VISSIM, PARAMICS, AIMSUN, TEXAS, implying AIMSUN had a higher percentage of low-speed conflicts or crashes than PARAMICS.
• VISSIM and TEXAS had no abnormal deceleration maneuvers while AIMSUN and PARAMICS had 20 to 30 percent conflicts with abnormal deceleration. Most of the abnormal deceleration maneuvers were from rear-end conflicts. More than half of the conflicts with abnormal deceleration had TTC = 1.0.
• Less than 40 percent of the conflicts from VISSIM and AIMSUN were low-speed or crash events; PARAMICS and TEXAS had a higher percentage of low-speed or crash events (60 to 75 percent). Most of the low-speed conflicts were rear-end conflicts.
• More than half of the conflicts in all simulation platforms were less severe conflicts (conflicts with TTC = 0.5), while in PARAMICS, the percentage of severe conflicts was the highest among all of the simulation platforms (based on comparison of C-N2).

Figure 159. Graph. Conflicts Number C-N2 Comparison for Intersection 1 at AM Peak.

Figure 160. Graph. Conflicts Number C-N3 Comparison for Intersection 1 at AM Peak.

Figure 161. Graph. Conflicts Number C-N2 Comparison for Intersection 1 at Mid Peak.

Figure 162. Graph. Conflicts Number C-N3 Comparison for Intersection 1 at Mid Peak.

Figure 163. Graph. Conflicts Number C-N2 Comparison for Intersection 1 at PM Peak.

Figure 164. Graph. Conflicts Number C-N3 Comparison for Intersection 1 at PM Peak.

Intersection 2: Roswell Road & Abernathy Road, Fulton County, Atlanta, GA:

The number of conflicts/conflict rates under AM, mid, and PM peak-hour conditions was tabulated and is shown in table 96 to table 98. Column charts were created to compare the conflicts number generated from the four simulation platforms, as seen in figure 165 through figure 170.

As shown in the following tables and figures, for all simulation models, most of the conflicts were rear-end conflicts, and crossing conflicts were the least conflicts among all conflicts types. Crossing conflicts had a smaller TTC value while rear-end conflicts had a larger TTC value. Most of the low-speed conflicts were rear-end conflicts. The differences among the four simulation models were the following:

• VISSIM had the least conflict number for all categories. TEXAS had the most conflict number for all categories. TEXAS had 10 times more conflicts than VISSIM.
• The ascending ranking order in most cases was VISSIM, PARAMICS, AIMSUN, TEXAS. However, for the TTC < 0.5 category, the order was VISSIM, PARAMICS, AIMSUN, TEXAS, implying that PARAMICS had a higher percentage of severe conflicts than AIMSUN.
• VISSIM and TEXAS had almost no abnormal deceleration maneuvers, while AIMSUN and PARAMICS had 10 to 30 percent conflicts with abnormal deceleration. Most of the abnormal deceleration maneuvers were from rear-end conflicts. Conflicts with abnormal deceleration were almost evenly distributed in the three TTC categories for AIMSUN. While in PARAMICS, more than half of the conflicts with abnormal deceleration had a TTC = 1.0.
• Most of the conflicts in TEXAS and AIMSUN were less severe conflicts (conflicts with TTC = 0.5), while in PARAMICS and VISSIM most conflicts were severe conflicts (conflicts with TTC < 0.5) (based on comparison of C-N2).
• The percentage of lane-change conflicts in TEXAS was the smallest.

Figure 165. Graph. Conflicts Number C-N2 Comparison for Intersection 2 at AM Peak.

Figure 166. Graph. Conflicts Number C-N3 Comparison for Intersection 2 at AM Peak.

Figure 167. Graph. Conflicts Number C-N2 Comparison for Intersection 2 at Mid Peak.

Figure 168. Graph. Conflicts Number C-N3 Comparison for Intersection 2 at Mid Peak.

Figure 169. Graph. Conflicts Number C-N2 Comparison for Intersection 2 at PM Peak.

Figure 170. Graph. Conflicts Number C-N3 Comparison for Intersection 2 at PM Peak.

Intersection 3: Lafayette Ave & Fulton Street, Grand Rapids, MI:

The number of conflicts/conflict rates under AM and PM peak-hour conditions was tabulated and is shown in table 99 and table 100. Column charts were created to compare the conflicts number generated from the four simulation platforms, as shown in figure 171 through figure 174.

As shown in the following tables and figures, for all simulation models, most of the conflicts were rear-end conflicts, and crossing conflicts were the least conflicts among all conflicts types. Crossing conflicts had a smaller TTC value while rear-end conflicts had a larger TTC value. The differences among the four simulation models were the following:

• VISSIM had the least conflict number for all categories.
• The ascending ranking order in most cases was VISSIM, PARAMICS, TEXAS, AIMSUN. However, for the TTC < 0.5 category, the order was VISSIM, AIMSUN, TEXAS, PARAMICS, implying that PARAMICS had a higher percentage of severe conflicts than AIMSUN.
• VISSIM and TEXAS had almost no abnormal deceleration maneuvers, while AIMSUN and PARAMICS had 9 to 15 percent conflicts with abnormal deceleration. Most of the abnormal deceleration maneuvers were from rear-end conflicts.
• Less than 15 percent of conflicts from VISSIM were low-speed or crash events; AIMSUN, PARAMICS, and TEXAS had a higher percentage of low-speed or crash events (25 to 70 percent). Most of the low-speed conflicts were rear-end conflicts.
• Most of the conflicts in VISSIM and AIMSUN were less severe conflicts (conflicts with TTC = 0.5), while in PARAMICS and TEXAS, the percentage of severe conflicts (conflicts with TTC < 0.5) was higher than in the other two simulation platforms (based on comparison of C-N2).
• The percentage of lane change conflicts in TEXAS was the smallest.

Table 99. Conflicts Number Under AM Peak-Hour Condition for Intersection 3.

Figure 171. Graph. Conflicts Number C-N2 Comparison for Intersection 3 at AM Peak.

Figure 172. Graph. Conflicts Number C-N3 Comparison for Intersection 3 at AM Peak.

Figure 173. Graph. Conflicts Number C-N2 Comparison for Intersection 3 at PM Peak.

Figure 174. Graph. Conflicts Number C-N3 Comparison for Intersection 3 at PM Peak.

Intersection 4: Ryan Ave & Davison Ave, Detroit, MI:

 The number of conflicts/conflict rates under AM and PM peak-hour conditions was tabulated and is shown in table 101 and table 102. Column charts were created to compare the conflicts number generated from the four simulation platforms, as seen in figure 175 to figure 178.

As shown in the following tables and figures, for all simulation models, most of the conflicts were rear-end conflicts, and crossing conflicts were the least conflicts among all conflicts types. Crossing conflicts had a smaller TTC value while rear-end conflicts had a larger TTC value. The differences among the four simulation models were the following:

• TEXAS had the most conflict number for most of the categories.
• PARAMICS had a higher percentage of severe conflicts than AIMSUN.
• VISSIM and TEXAS had almost no abnormal deceleration maneuvers, while AIMSUN and PARAMICS had 10 percent conflicts with abnormal deceleration, and most of the abnormal deceleration maneuvers were from rear-end conflicts.
• PARAMICS and TEXAS had a higher percentage of low-speed or crash events (over 50 percent). Most of the low-speed conflicts were rear-end conflicts.
• Most of the conflicts in VISSIM, AIMSUN, and TEXAS were less severe conflicts (conflicts with TTC = 0.5), while in PARAMICS, most conflicts were severe conflicts (conflicts with TTC < 0.5), based on comparison of C-N2.
• The percentage of lane change conflicts in TEXAS was the smallest.

* C-N1: total conflict analysis by SSAM.
C-N2: Conflicts with MaxD = -9.15 m/sec² (-30 ft/sec²).
C-N3: Conflicts with MaxD greater than and equal to -9.15 m/sec² (-30 ft/sec²) TTC > 0 and MaxS = 16.1 km/h (10 mi/h).

Figure 175. Graph. Conflicts Number C-N2 Comparison for Intersection 4
at AM Peak.

Figure 176. Graph. Conflicts Number C-N3 Comparison for Intersection 4
at AM Peak.

Table 102. Conflicts Number Under PM Peak-Hour Condition for Intersection 4.

* C-N1: total conflict analysis by SSAM.
C-N2: Conflicts with MaxD = -9.15 m/sec² (-30 ft/sec²).
C-N3: Conflicts with MaxD greater than and equal to -9.15 m/sec² (-30 ft/sec²) TTC > 0 and MaxS = 16.1 km/h (10 mi/h).

Figure 177. Graph. Conflicts Number C-N2 Comparison for Intersection 4
at PM Peak.

Figure 178. Graph. Conflicts Number C-N3 Comparison for Intersection 4
at PM Peak.

Intersection 5: Howe Ave & Fair Oaks Boulevard, Sacramento, CA:

The number of conflicts/conflict rates under AM and PM peak-hour conditions was tabulated and are shown in table 103 and table 104. Column charts were created to compare the conflicts number generated from the four simulation platforms, as seen in figure 179 through figure 182.

As shown in the following tables and figures, for all simulation models, most of the conflicts were rear-end conflicts, and crossing conflicts were the least conflicts among all conflicts types. Crossing conflicts had a smaller TTC value while rear-end conflicts had a larger TTC value. The differences among the four simulation models were the following:

• VISSIM had the least conflict number for all categories. TEXAS had the most conflict number for all categories except for lane change category.
• The ascending ranking order in most cases was VISSIM, PARAMICS, AIMSUN, TEXAS.
• VISSIM and TEXAS had almost no abnormal deceleration maneuvers, while AIMSUN and PARAMICS had 30 to 40 percent conflicts with abnormal deceleration).
• Most of the abnormal deceleration maneuvers were from rear-end conflicts. Conflicts with abnormal deceleration were almost evenly distributed in the three TTC categories for AIMSUN. While in PARAMICS, more than half of the conflicts with abnormal deceleration have TTC = 1.0.
• Most of the conflicts were less severe conflicts (conflicts with TTC = 0.5), and PARAMICS had a higher percentage of severe conflicts (conflicts with TTC < 0.5) than others, based on comparison of C-N2.
• The percentage of lane change conflicts in TEXAS was the smallest.

Table 103. Conflicts Number Under AM Peak-Hour Condition for Intersection 5.

* C-N1: total conflict analysis by SSAM.
C-N2: Conflicts with MaxD = -9.15 m/sec² (-30 ft/sec²).
C-N3: Conflicts with MaxD greater than and equal to -9.15 m/sec² (-30 ft/sec²) TTC > 0 and MaxS = 16.1 km/h (10 mi/h).

Figure 179. Graph. Conflicts Number C-N2 Comparison for Intersection 5
at AM Peak.

Figure 180. Graph. Conflicts Number C-N3 Comparison for Intersection 5
at AM Peak.

Table 104. Conflicts Number Under PM Peak-Hour Condition for Intersection 5.

Figure 181. Graph. Conflicts Number C-N2 Comparison for Intersection 5 at PM Peak.

Figure 182. Graph. Conflicts Number C-N3 Comparison for Intersection 5
at PM Peak.

In general, most of the conflicts were rear-end conflicts, and Crossing conflicts were the least. Crossing conflicts had a smaller TTC value while rear-end conflicts had a larger TTC value. Differences among VISSIM, AIMSUN, PARAMICS, and TEXAS were as follows:

• For all cases, VISSIM had the least conflict number across all categories.
• For most of the cases, TEXAS had the most conflict number across all categories. When traffic volume was increasing, the ratio between conflicts number from VISSIM and conflicts number from TEXAS also increased.
• The ascending ranking order in most cases was VISSIM, AIMSUN, PARAMICS, TEXAS. When there was less traffic, the ranking order was VISSIM, PARAMICS, TEXAS, AIMSUN.
• AIMSUN had a higher percentage of low-speed conflicts or crashes than PARAMICS, while PARAMICS had a higher percentage of severe conflicts than AIMSUN.
• VISSIM and TEXAS had no abnormal deceleration maneuvers, while AIMSUN and PARAMICS had 10 to 40 percent conflicts with abnormal deceleration. Most of the abnormal deceleration maneuvers were from rear-end conflicts. More than half of the conflicts with abnormal deceleration had TTC = 1.0.
• Conflicts with abnormal deceleration were almost evenly distributed in the three TTC categories for AIMSUN. In PARAMICS, more than half of the conflicts with abnormal deceleration had TTC = 1.0.
• VISSIM and AIMSUN had a smaller percentage of low-speed or crash events than PARAMICS and TEXAS. PARAMICS and TEXAS had over 50 percent of total conflicts as low-speed conflicts or crashes. Most of the low-speed conflicts were rear-end conflicts.
• Most conflicts in VISSIM, AIMSUN, and TEXAS were less severe conflicts (conflicts with TTC = 0.5), while in PARAMICS, most conflicts were severe conflicts (conflicts with TTC < 0.5), based on comparison of C-N2.
• The percentage of lane change conflicts in TEXAS was the smallest.

Previous | Table of Contents | Next