Safety Evaluation of Intersection Conflict Warning Systems

Chapter 3. Study Design

The study design involved a sample size analysis and prescription of needed data elements. The sample size analysis assessed the size of sample required to statistically detect an expected change in safety and also determined what changes in safety could be detected with available sample sizes.

Sample Size Estimation Overview

When planning a before–after safety evaluation study, it is vital to ensure that enough data are included such that the expected change in safety can be statistically detected. Even though in the planning stage, the expected change in safety is unknown, it is still possible to make a rough estimate of how many sites would be required based on the best available information about the expected change in safety. Alternatively, one could estimate, for the number of available sites, the change in safety that could be statistically detected. For a detailed explanation of sample size considerations, as well as estimation methods, see chapter 9 of Hauer.⁽¹³⁾ The sample size analysis presented here is limited to two cases: (1) how large a sample would be required to statistically detect an expected change in safety, and (2) what changes in safety could be detected with available sample sizes.

For case 1, it was assumed that a conventional before–after study with comparison group design would be used because available sample size estimation methods were based on this assumption. The sample size estimates from this method would be conservative in that the EB methodology would likely require fewer sites. To facilitate the analysis, it was also assumed that the number of comparison sites was equal to the number of installation sites and the duration of the before and after periods were equal, which, again, was a conservative assumption.

Table 2 provides the crash rate assumptions. The locations of interest for the ICWS strategy were four-legged, stop-controlled intersections. Intersection crash rates differ substantially depending on a number of factors (e.g., traffic control, traffic volume, geometric configuration, and area type). Therefore, the intersection crash rates assumed for these computations represented the before data for installation sites in North Carolina, Missouri, and Minnesota. Rates A and B were calculated as the weighted average crash rate for two-lane and multilane major routes, respectively.

Table 3 provides estimates of the required number of before and after period intersection-years for statistical significance at both a 90- and 95-percent confidence level for crash rates A and B. The minimum sample indicates the level for which a study seemed worthwhile; that is, it was feasible to detect with the level of confidence the largest effect that could reasonably be expected based on what was currently known about the ICWS strategy. These sample size calculations were based on specific assumptions regarding the number of crashes per intersection and years of available data. Rate A (from table 2) was used for two-lane at two-lane intersections, and rate B (from table 2) was used for four-lane at two-lane intersections. Site-years are the number of sites where the strategy was implemented multiplied by the number of years of data before or after implementation. For example, if a strategy was implemented at nine sites and data were available for 3 years since implementation, then there would be a total of 27 site-years of after period data available for the study.

¹Assumes equal number of site-years for ICWS installation and comparison sites
and equal length of before and after periods.
Boldface indicates the sample size values recommended in this study.

The sample size values recommended in this study are highlighted in bold in table 3. These were recommended based on the likeliness of obtaining the estimated sample size as well as the anticipated effects of the ICWS strategy. As noted, the sample size estimates provided were conservative in that the state-of-the-art EB methodology proposed for the evaluations would require fewer sites than the less robust conventional before–after study with a comparison group that had to be assumed for the calculations. Estimates could be predicted with greater confidence or a smaller reduction in crashes would be detectable if there were more site-years of data available in the after period. The same holds true if the actual data used for the analysis had a higher crash rate for the before period than was assumed.

Case 2 considers the data collected for both the before and after periods. The total site-years of data available for two-lane major roadways was 360 for the before period and 255 for the after period. The total site-years of data available for multilane major roadways was 126 for the before period and 100 for the after period. The statistical accuracy attainable for a given sample size is described by the standard deviations of the estimated percent change in safety. From this, P‑values were estimated for various sample sizes and expected changes in safety for a given crash history. A set of such calculations is shown in table 4 and table 5. The calculations are based on the methodology in Hauer.⁽¹³⁾

For the available data, the minimum percentage change in crash frequency that could be statistically detected at 90- and 95-percent significance levels were estimated using the same crash rates in table 2. The results indicate that the data should be able to detect the anticipated crash reduction effects highlighted in table 3 (i.e., 20-percent reductions for all crash types except for rear-end and nighttime crashes for both two-lane and multilane roadways), if such an effect were present. Using these results, a decision was made to proceed with the evaluation using the data available at the time.

¹Results are to nearest 5-percent interval, and the crash rate assumption is based on actual crash rate for the before period.

¹Results are to nearest 5-percent interval, and the crash rate assumption is based on actual crash rate for the before period.