The Application of an Improved Accident Analysis Method for Highway Safety Evaluations
The evaluation of safety effects for various treatments has long been a subject of controversy within the transportation safety research community. Safety evaluations mostly rely on assessment of accident experience over before-and-after periods. One of the common pitfalls in the assessment methodologies is the failure to account for regression-to-the-mean (r-t-m) bias. The sampling bias due to the r-t-m phenomenon may seriously affect conclusions drawn in safety treatment evaluation studies. Safety treatment sites are generally selected because they have a high accident rate or accident count. If a site has an unusually high number of accidents occurring before the treatment, accident occurrence at that same site the following period would, in all probability, be lower even without any intervention at that site. This is the phenomenon known as r-t-m. Therefore, a simple before-and-after comparison for sites where the treatment is selected based on the accident experience is likely to result in an overestimation of the treatment's effect.
In a recent Federal Highway Administration (FHWA) study, a new method, titled Empirical Bayes Estimation of Safety and Transportation (EBEST), was developed for providing a better estimate of the expected accident experience for a treated site, adjusted for any r-t-m bias.(1) With this method, a microcomputer program was developed to allow easy application of the analysis technique. The methodology was developed and initially tested using simulation and hypothetical examples. This study was undertaken to apply the EBEST methodology to actual data from the Highway Safety Information System (HSIS). The installation of traffic signal controls at previously unsignalized intersections was selected as the treatment to be evaluated.
State Data Bases Used
This study employed data from only one HSIS State. Minnesota was selected because of the availability of sufficient safety treatment data, including the start and completion dates of roadway improvement and signal installation projects as well as a videodisc photolog system. The videodisc system allows users to access images of the State-maintained roadway network, collect additional information about study locations, and verify existing HSIS data.
Analysis Methods
A critical requirement of the EBEST methodology is the use of data from a reference group. The reference group is a sample of sites that are generally similar to the treatment sites with respect to roadway and traffic characteristics. The reference group and treatment group should represent the population of potential treatment sites. A recent enhancement made to the EBEST method includes a regression model to control for factors that may differ among treatment and reference sites.(2) The EBEST method also incorporates measures of exposure (traffic volume, section length, etc.) and can account for changes that occur over time. In many cases, the reference group can serve as a comparison group to account for potential time effects.
In addition to the EBEST method, the traditional before-and-after method with comparison group (i.e., the "classical" method) was also applied in this study. The purpose of applying both methods was to compare the results of a method that corrects for r-t-m bias to a method that does not. In this study, the reference group also served as the comparison group to adjust for time effects. Statistically, the reference group was found to be an appropriate comparison group.
The treatment group consisted of 13 intersections where new traffic signal controls were installed. Accident, traffic, and intersection configuration information for each treatment site were extracted from HSIS. The reference group was defined to include intersections that were comparable to the treatment group with respect to specific criteria (e.g., daily entering traffic,
number of approach legs, intersection configuration, etc.). After applying these criteria, 79 sites were selected for the reference group. Depending upon the size of the treatment group, the reference group should be two to five times larger than the size of the treatment group. All study sites (treatment and reference sites) were examined using Minnesota's videodisc system to verify the site information and locations.
Because most traffic engineers in the United States determine the need for traffic signal control based on the warrants in the Manual on Uniform Traffic Control Devices (MUTCD), a crude signal warrant analysis was attempted to determine how many of the reference group sites were candidates for signal installation. However, because detailed volume data were not available, average hourly approach volume were estimated and compared to the minimum vehicle volume warrant and the interruption of continuous traffic warrant. Relying on several assumptions, it was found that 38 percent of the reference sites met either of the volume warrants and 14 percent met the accident experience warrant. These results illustrate the practical difficulties encountered in selecting a reference group that represents the population of potential treatment sites. After further examination of the characteristics of the treatment and reference groups, it was concluded
that the reference group was acceptable for this analysis.
All accidents that were reported between 1985 and 1990 that occurred within 76.2 m (250 ft) of the treatment and reference site intersections on both major and minor cross road approaches were retrieved from HSIS for the analysis. Accidents that occurred during the construction period were excluded from the analysis. Reference sites were matched with treatment sites and corresponding before-and-after time periods for the treatment site were used in the analysis. Consequently, the before-and-after treatment periods varied from 21 months to 31 months.
Results
Figure 1 shows that the treatment sites had a higher accident experience than the reference sites in the before period. This is an indication of a sampling bias with potential for r-t-m.
Figure 1. Before accident rate distribution for all sites.
Table 1 shows that for total accidents, the treatment effect is 30 percent (30 percent reduction in total accidents after new traffic signals were installed) with the classical method and 25 percent with the EBEST method. The 95 percent confidence intervals are -42 percent, -15 percent and -38 percent, -10 percent for the classical and EBEST methods, respectively. In other words, based on the classical method, there is a 95 percent probability that the true treatment effect is between a 15 percent and 42 percent reduction in total accidents. Based on the EBEST method, there is a 95 percent probability that the true effect is between a 10 percent and 38 percent reduction. For injury accidents, the treatment effect is 38 percent with the classical method and 23 percent with the EBEST method. The 95 percent confidence intervals are -56 percent, -13 percent and -43 percent, +5 percent for the classical and EBEST methods, respectively.
| Table 1. Results from EBEST method and classical method. |
| |
|
EBEST |
Classical |
| Total Accidents |
Treatment Effect |
-25% |
-30% |
| |
95% Interval |
-38%, -10% |
-42%, -15% |
| Injury Accidents |
Treatment Effect |
-23% |
-38% |
| |
95% Interval |
-43%, +5% |
-56%, -13% |
The study results show a significant difference between the EBEST method and the classical method in the evaluation of injury accidents. It appears that the classical method has overestimated the treatment effect. This overestimation can be attributed to the lack of
adjustment for r-t-m bias.
Study Implications
This was the first study to apply the EBEST method to actual data to evaluate the safety effect of a treatment. Reference sites were identified using the roadway data commonly found in State data bases and were verified with videodisc photo-logs. A crude traffic signal warrant examination was conducted to determine how well the reference sites represented the population of potential treatment sites.
The findings of the signal warrant examination raise questions about the methods for selecting reference sites. To many traffic engineers, an intersection in the reference group that does not satisfy at least one of the MUTCD warrants is not a site that could potentially be "treated" with traffic signal control. Consequently, the question remains: What truly constitutes an appropriate reference group for the treatment being analyzed? This issue is being addressed in a current research study in which the EBEST method is being applied to several data sets. This research study will include several before-and-after studies, such as the one reported here. It will also examine identification and ranking of high-frequency accident locations using the EBEST method.
The strengths and limitations of the EBEST method will be further identified and questions
concerning the selection of the reference group will be studied.
For More Information
This research was conducted by Jun Wang, a former graduate research fellow at the Turner-Fairbank Highway Research Center (TFHRC), under the supervision of Michael Griffith of FHWA. The study was sponsored by the graduate research fellowship program of FHWA. For more information, contact Jeffrey F. Paniati, HSIS Program Manager, HSR-30, (703) 285-2568.
References
1 Pendleton, O.J. Application of New Accident Analysis Methodologies: Volume I,
General Methodology, Publication No. FHWA-RD-90-091, Washington, DC, 1991.
2 Christiansen, C.L. and Morris, C.N., Empirical Bayes Analysis of Rates Using a
Hierarchical Regression Model, Technical Report No. 83, Center for Statistical Sciences, University of Texas at Austin, 1990.
|
