• Federal Highway Administration >
• Publications >
• Research Publications >
• 11024 >
• 004.Cfm >
• Safety Evaluation of the Safety Edge Treatment

Safety Evaluation of the Safety Edge Treatment

Chapter 4. Analysis Results for Safety Evaluation

This chapter presents the safety evaluation approach, the development of SPFs, and the safety evaluation results. The safety evaluation results include the findings of a before-period compatibility study, a before-after evaluation using the EB technique, a cross-sectional analysis, and an analysis of shifts in crash severity.

4.1 Evaluation Approach

Two statistical approaches were used to evaluate the safety effectiveness of the safety edge treatment: (1) a before-after comparison of the effect of pavement resurfacing with and without the safety edge treatment using the EB technique and (2) a cross-sectional comparison of the effect of pavement resurfacing with and without the safety edge treatment based on after-period data only. These two evaluation approaches were applied concurrently to provide alternative statistical approaches to the key issues being addressed. The following discussion describes these evaluations, including issues related to the specific nature of the safety edge treatment.

A key objective of the evaluation was to determine the safety effectiveness of the safety edge treatment while avoiding the potential confounding effects of regression to the mean and the safety effect of pavement resurfacing. Regression to the mean is a characteristic of repeated measures data in which observations move toward the mean value over time. That is, if an observation in a year is unusually high, then the observation in the following year will nearly always be lower (and vice versa), returning to the mean. This phenomenon often leads to an overestimation or underestimation of safety for some sites. Thus, the effect of the treatment is likely to be partially confounded with the expected decrease or increase in crash experience from regression to the mean. Regression to the mean can only be accounted for with knowledge of the "normal" or expected value of before-period crash experience at the treated sites. The EB technique has the advantage of compensating for regression to the mean. The cross-sectional approach does not explicitly compensate for regression to the mean. This concern is lessened by the availability of 3 years of crash data for the period after resurfacing.

The second potential confounding effect is the safety effect of pavement resurfacing since it is always used in conjunction with the safety edge treatment. Previous research has indicated that pavement resurfacing by itself may have an effect on safety, increasing crashes because of increased speeds. This effect was found in one study to be statistically significant but was found to persist for only 12-30 months after resurfacing.⁽⁴⁾ However, a more recent, larger study in National Cooperative Highway Research Program Project 17-9(2) found inconsistent results; increases in crash frequency with resurfacing were found in some States, but decreases in crash frequency with resurfacing were found in others.⁽⁵⁾ Therefore, the safety effects of the pavement resurfacing and installation of the safety edge treatment will be confounded, at least for some time, following resurfacing.

The study design was developed to address the safety effect of resurfacing and the safety edge treatment as well as the confounding effect of resurfacing. First, the study period after resurfacing was selected to be 3 years. This is sufficiently long as to extend beyond the duration of any short-term resurfacing effect. Annual interim evaluations to monitor time trends were conducted to address this issue. Thus, the results for safety effectiveness of the safety edge treatment in the first- and second-year interim reports may be confounded by the safety effect of pavement resurfacing, but it is expected that this confounding effect is lessened in the final results. Second, resurfaced sites both with and without the safety edge treatment were considered. The ratio of safety between resurfaced sites with and without the safety edge treatment (i.e., the treatment and comparison sites) may represent an effect of the safety edge treatment as long as the sites can be assumed comparable in other respects.

The first evaluation approach is an observational before-after comparison using the EB technique, as formulated by Hauer.^(6,7) The specific version of the EB technique used in this evaluation was developed for the FHWA SafetyAnalyst software tools.⁽⁸⁾ The primary objective of the before-after evaluation is to compare the observed number of crashes after the treatment is implemented to the expectednumber of crashes in the after period had the countermeasure not been implemented. This provides an estimate of the overall safety effectiveness of the countermeasure expressed as a percent change in the crash frequency.

When performing before-after evaluations using the EB technique, it is typical to collect data at sites where countermeasures were implemented (i.e., treatment sites) and at sites similar to the treatment sites with respect to area type (rural/urban), geometric design, and traffic volumes, but where no countermeasures were installed. Data from this comparison group of sites are used to create SPFs, which are then used with the observed crash counts at the treated sites in the before period to estimate the number of crashes that would have occurred at the treated sites in the after period if no improvement had been made. These SPFs are discussed in section 4.2.

The comparability before resurfacing of the two types of sites (treatment and comparison sites) is critical to interpreting the difference of the two estimated before-after effects as an effect of the safety edge treatment. For example, if one of the site types had a higher mean in the before period and both site types had the same mean in the after period, then the effectiveness of one treatment may be presumed greater than the other treatment. The comparability of sites was established through analysis of the before-period crash data. These analyses are discussed in section 4.3.1.

The EB before-after evaluation produced separate estimates of the effectiveness of resurfacing with the safety edge (treatment sites) and resurfacing only (comparison sites) for each target crash type in each State. From each pair of estimated percent changes in safety (treatment and comparison), the effect of the safety edge alone was estimated as the ratio between the two measures of effectiveness. For every combination of site characteristics under consideration, the mean and standard error of the percent change in target crash frequency and its statistical significance are presented in section 4.3.2.

It was anticipated that the effectiveness measure for the safety edge treatment would be relatively small since it was expected that the safety edge treatment would affect only certain crash types and would have the greatest impact on two-lane highways with no paved shoulders. Most such sites have relatively low traffic volume and therefore are not expected to have a high frequency of run-off-the-road and drop-off-related crashes.

The EB-based before-after comparison technique is theoretically the strongest approach to evaluations of this type. However, because of the confounding of the pavement resurfacing effect and the safety edge treatment effect, it cannot be assured that this approach correctly identifies the treatment effectiveness. Therefore, an alternative cross-sectional comparison was also conducted.

A cross-sectional evaluation of the after data at the treated sites was conducted to directly compare the crash data between the two types of treatment-resurfacing with the safety edge treatment and resurfacing without the safety edge treatment. Assuming that all roadway factors except resurfacing are held constant, one could hypothesize that the differences in either after-period crash frequencies or crash severity distributions between treatment and comparison sites are due to the provision of the safety edge treatment. This comparison was made with a cross-sectional approach using data for the period after resurfacing while accounting for the effects of AADT.

The cross-sectional comparison of crash data for the period after resurfacing was conducted using negative binomial regression models to compare the crash frequencies for the period after resurfacing for the sites with the safety edge treatment to those of the sites resurfaced without the safety edge treatment. Site type (i.e., treatment versus comparison) was the main factor of interest in the analysis. The effect of AADT was accounted for in this approach by quantifying the relationship between AADT and specific target crash types. When significant, the effect of lane width was also accounted for in the model. The safety edge treatment effect and its standard error were then calculated for each target crash type. The treatment effect was converted to a percent change in crash frequency for ease in interpreting the results. The results of the cross-sectional analysis are presented in section 4.4.3.

In addition to evaluating mean crash frequencies, a comparison of the before-after data by crash severity level was performed to determine shifts in the crash severity distribution. These comparisons were accomplished by calculating a confidence interval for the average difference in proportions across all sites at a preselected significance level of 10 percent. However, a non-parametric statistical test, the Wilcoxon signed-rank test, was also applied as the differences in proportions may not follow a normal distribution. Results from this analysis are presented in section 4.4.4.⁽⁹⁾

4.2 Safety Performance Functions

This section documents the SPFs and calibration factors developed for use in the before-after EB evaluation of the safety effectiveness of the safety edge treatment. SPFs are regression relationships between target crash frequencies and traffic volumes that can be used to predict the long-term crash frequency for a site. SPFs are used in the before-after EB evaluation to estimate what the safety performance of a treated site would be in the after period if the treatment had not been implemented.

Negative binomial regression models were developed using data from the reference group of untreated sites for use in three categories of target crashes (all crash types combined, run-off-road crashes, and drop-off-related crashes) and two severity levels (total crashes and fatal and injury crashes). Thus, a total of six dependent variables were considered. Traffic volume and lane width were the only independent variables considered in the SPFs. Separate models were developed for Georgia and Indiana for each of the three classifications, as follows:

• Rural multilane highways with paved shoulders with widths of 4 ft or less.
• Rural two-lane highways with paved shoulders with widths of 4 ft or less.
• Rural two-lane highways with no paved shoulders (i.e., unpaved shoulders only).

Regression models were not developed for New York due to the limited number of treated sites.

All regression models were developed to predict target crash frequencies per mile per year as a function of traffic volume and, in some cases, lane width in the functional forms shown in equation 1 and equation 2.

(1)

(2)

Where:

N = predicted number of target crashes per mile per year

AADT = average daily traffic volume (vehicles per day) for the roadway segment

LW = lane width for the roadway segment (ft)

a, b, c = regression coefficients

The AADT in the regression models was statistically significant in all cases. The lane width term was included in the regression model only when it was statistically significant.

Two generalized linear modeling techniques were used to fit the data. The first method used a repeated measures correlation structure to model yearly crash counts for a site. In this method, the covariance structure, assuming compound symmetry, is estimated before final regression parameter estimates are determined by general estimating equations. Consequently, model convergence for this method is dependent on the covariance estimates as well as parameter estimates. When the model failed to converge for the covariance estimates, an alternative method was considered. In this method, yearly crash counts for a site were totaled and annual daily traffic (ADT) values were averaged to create one summary record for a site. Regression parameter estimates were then directly estimated by maximum likelihood without an additional covariance structure being estimated.

Both methods produced an estimate of the overdispersion parameter, the estimate for which the variance exceeds the mean. Overdispersion occurs in traffic data when a number of sites being modeled have zero accident counts, which creates variation in the data. When the estimate for dispersion was very small or even slightly negative, the model was refit assuming a constant value. Both methods were accomplished with the GENMOD procedure of SAS^®.⁽³⁾

Statistically significant models were not found for all dependent variables for some road type/ shoulder type combinations. In these three cases, the intercept coefficient of the total crashes or fatal and injury crashes model was adjusted by the proportion of the applicable dependent variable to produce the final model. The model coefficients with their standard errors are presented in table 10 for Georgia and in table 11 for Indiana. All AADT coefficients shown are significant at the 10 percent significance level or better. Lane width coefficients shown are significant at the 20 percent significance level or better. Total crash and fatal and injury crash SPFs are illustrated in figure 3 for Georgia and in figure 4 for Indiana.

Table 10. SPFs for Georgia sites.

Note: Blank cells indicate lane width coefficient was not significant.

Table 11. SPFs for Indiana sites.

Note: Blank cells indicate lane width coefficient was not significant.

Figure 3. Graph. Comparison of Georgia SPFs by crash severity and roadway and shoulder type.

Figure 4. Graph. Comparison of Indiana SPFs by crash severity and roadway and shoulder type.

As noted earlier, the proportion of run-off-road and drop-off-related crashes (developed from reference sites) was sometimes needed to adjust total or fatal and injury SPFs for prediction of those crash types. Table 12 presents these proportions estimated from the reference site data.

Table 12. Run-off-road and drop-off-related crash frequencies as a proportion of total crashes.

FI = Fatal and injury crashes.

PDO = Property-damage-only crashes.

Additionally, yearly calibration factors were developed from the SPFs to provide a better yearly prediction in the methodology. These factors are needed because the SPFs are developed as an average of all years. The yearly calibration factor is determined as the ratio of the sum of observed crashes for all sites for a specific roadway type/shoulder type combination to the sum of the predicted crashes for the same sites using the AADT and crash count values for that year. These factors are presented in table 13 for Georgia and in table 14 for Indiana.

Table 13. Georgia SPF calibration factors.

FI = Fatal and injury crashes.

PDO = Property-damage-only crashes.

Table 14. Indiana SPF calibration factors.

FI = Fatal and injury crashes.

PDO = Property-damage-only crashes.

4.3 Safety Evaluations

As previously discussed, four types of safety evaluations were performed as part of this study: (1) a safety comparison of treatment and comparison sites in the period before resurfacing; (2) an EB before-after evaluation; (3) a cross-sectional analysis; and (4) an analysis of shifts in the severity distribution from before to after resurfacing. The findings of these evaluations are presented in this section.

4.3.1 Safety Comparison of Treatment and Comparison Sites in the Period Before Resurfacing

An evaluation was conducted to compare the safety performance of treatment and comparison sites before resurfacing for specific States and roadway type/shoulder type combinations. This evaluation is critical to the interpretation of the safety differences between the treatment and comparison sites as an effect of the safety edge treatment. If the safety performance of the two types of sites differed in the period before resurfacing, the comparison of treatment and comparison sites in the period after resurfacing may be influenced.

Initial comparisons were made by examination of scatter plots of crashes and traffic volumes (crashes per mile per year versus lnAADT). Ideal plots would contain no discernable differences between treatment and comparison sites nor any extreme points. Separation of the data points between the two groups may indicate a potential concern in the subsequent analyses. Furthermore, if one group had systematically higher crash frequencies in the period before resurfacing, then the analysis for the period after resurfacing might need to account for this difference. Finally, large variation in crash frequencies for the same AADT values could inhibit crash analysis of the treatment and comparison groups. Inspection of these plots with data from year 3 (see appendix C) showed an improvement in the plots from year 1 and year 2.

Yearly total crash and target crash distributions were also presented in box plots to review data consistency from year to year. Ideal plots would have approximately the same distribution for crashes each year within a given site type and between site types. Additionally, potential concerns for the crash analysis-specifically, a regression to the mean or resurfacing effect- may be identified if the period after resurfacing is included.

Since crash frequencies are known to experience random variation around the mean or regression to the mean, the average over several years for the period before resurfacing should be compared to the average of several years for the period after resurfacing. Therefore, if the after period data are within the range of yearly crash means but numerically higher than the before period average, then safety analyses might show an increase in crash frequency due to the treatment (provided AADT growth was minimal). Conversely, if the after implementation year data are lower than the before period average, then the treatment effect would be a decrease in crash frequency. Examination of these graphs indicated that the after period data were almost always higher than the average of the before years but within the range of variation in yearly crash totals for both types of treated sites.

The apparent increase in crashes was examined to determine if it could be attributed to resurfacing. A resurfacing effect occurs when the reference sites remain the same or decrease in crashes while the treatment and comparison sites both increase. This effect was observed in nearly all of the plots.

One additional potential problem was found in this analysis. One treatment site on a two-lane highway with paved shoulders in Georgia site doubled in crash frequency from the before to the after period. Subsequent investigation found that this site was reconstructed during the second year after resurfacing, and therefore, it was excluded from the safety analysis presented in this report.

Formal crash frequency comparisons of means between the treatment and comparison sites for the period before resurfacing were conducted for each State/roadway type/shoulder type combination and target crash type. Two types of comparisons were made, a comparison of EB-adjusted expected crash frequencies and a comparison of observed crash frequencies. Both comparisons were performed using PROC GENMOD, a generalized linear model procedure available in SAS^®, assuming a negative binomial crash distribution.⁽³⁾ This procedure uses predictive modeling to test the means between the two treatment groups for statistical significance.

The results of these analyses are presented in table 15 and table 16. For the EB-adjusted crash analysis, results are provided only for those roadway type/shoulder type combinations for which SPFs could be developed. However, all target crash types were considered as they can be estimated by the EB technique. Regression coefficients with their standard errors are shown in the tables for each independent variable, including AADT and the treatment versus comparison site effect. The significance and p-value for each effect are also presented. Blank rows in the tables represent models that did not converge.

Table 15. Evaluation of treatment versus comparison site effect for the period before resurfacing using EB-adjusted crash frequencies.

See notes at end of table.

Table 15. Evaluation of treatment versus comparison site effect for the period before resurfacing using EB-adjusted crash frequencies-Continued.

¹ At the 0.20 level.

TOT = total crashes (all severity levels combined).

Fl = fatal and injury crashes.

PDO = property-damage-only crashes.

ror = run-off-road crashes.

do = drop-off-related crashes.

Note: Blank cells represent models that did not converge.

Table 16. Evaluation of treatment versus comparison site effect for the period before resurfacing using observed crash frequencies.

See notes at end of table.

Table 16. Evaluation of treatment versus comparison site effect for the period before resurfacing using observed crash frequencies-Continued.

¹ At the 0.20 significance level.

TOT = total crashes (all severity levels combined).

Fl = fatal and injury crashes.

PDO = property-damage-only crashes.

ror = run-off-road crashes.

do = drop-off-related crashes.

Note: Blank cells indicate models that did not converge.

Results from the analysis of EB-adjusted crash frequencies in table 15 show that there tended to be significant differences between treatment and comparison site crash frequencies for Georgia sites with unpaved shoulders in the period before resurfacing. Comparison sites that had unpaved shoulders had lower crash rates than treatment sites. There is also evidence of differences in drop-off-related and run-off-road crashes for Georgia paved shoulder locations. Similarly, Indiana unpaved shoulder locations differed for drop-off-related crashes. These locations had treatment sites with lower crash rates.

Results from the analysis of observed crash frequencies somewhat confirmed the results of the EB-adjusted crashes. However, there tended to be fewer significant results and poorer fit of the models in general. This was to be expected because EB-adjusted crashes are smoothed by the SPF model predictions, causing smaller differences and less variation and leading to more significant results. Differences between treatment and comparison sites were confirmed for Georgia unpaved shoulder locations and drop-off-related crashes for paved shoulder locations. Additionally, New York locations, which were not tested by EB-adjusted crashes, showed differences for run-off-road crashes. All other significant differences were associated with poor models.

It was also desirable to confirm the existence of a cause-and-effect chain leading from the frequency and height of pavement-edge drop-offs to the likelihood of crashes. The drop-off height analysis reported in chapter 3 indicated that two-lane highway sites with unpaved shoulders and the multilane highway sites in Georgia did not have significant differences in the proportion of high drop-offs and therefore should have non-significant differences in crash frequency in the period before resurfacing. This expectation was not entirely supported by crash analysis results. However, for cases in which there were significant differences, these differences were in the same direction indicated in the drop-off analysis. That is, if drop-offs were more prevalent, then the sites had more crashes. Similarly, two-lane highway sites with paved shoulders in Georgia had comparison sites with a significantly higher probability of high drop-offs, and the crash analysis showed the comparison sites had more crashes, although the result was not significant.

Results for Indiana sites on two-lane highways with paved shoulders were consistent with the analysis of drop-off measurements, but the results for Indiana sites on two-lane highways with unpaved shoulders were not consistent with the analysis of drop-off measurements.

Overall, the treatment and comparison sites showed similar crash frequencies for paved shoulder sites in the period before resurfacing. By contrast, there were some statistically significant differences in crash frequencies between treatment and comparison sites for unpaved shoulders during the period before resurfacing. It should be noted that only 2 years of crash data were available for the period before resurfacing in Indiana, in comparison to 6 years for the period before resurfacing in Georgia. Thus, the variability of the Indiana crash frequencies was expected to be higher. In most cases (with the single exception previously noted), the differences in crash frequencies between treatment and comparison sites were similar to the differences in proportions of extreme drop-off heights for the period before resurfacing.

4.3.2 Before-After Evaluation Using the EB Technique

An observational before-after evaluation was conducted using the EB technique to estimate the safety effectiveness of the safety edge treatment. Separate before-after evaluations were conducted for resurfacing projects with the safety edge (treatment sites) and resurfacing projects without the safety edge (comparison sites). The ratio of these results was used to estimate the effect of the safety edge treatment.

All crash severity levels for total crashes, run-off-road crashes, and drop-off-related crashes were evaluated. The study period before resurfacing for these evaluations was the 4-year period from 2001 to 2004. The study period after resurfacing was the 3-year period from 2006 to 2008. The entire year in which resurfacing was performed, 2005, was excluded from the evaluation. The rationale for excluding crashes during the construction year is that it takes time for drivers to adjust to new driving conditions, and so the transition period is not necessarily representative of the long-term safety performance of the site. All of the crash data used in the evaluation were for complete calendar years so that there would be no opportunity for seasonal biases to affect the results.

The EB procedure was programmed and executed in SAS^®.⁽³⁾ Effectiveness estimates and their precision estimates, along with their statistical significance, are presented for specific crash types in table 17 through table 25.

Table 17. Before-after EB evaluation results for total crashes.