U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
 |
| This report is an archived publication and may contain dated technical, contact, and link information |
|
Publication Number: FHWA-RD-02-089
Date: July 2002 |
|||||||||
Safety Effectiveness of Intersection Left- and Right-Turn LanesPDF Version (1.48 MB)
PDF files can be viewed with the Acrobat® Reader® 5. EVALUATION PLANThis section of the report presents the evaluation plan for determining the safety effectiveness of intersection improvement projects involving left- and right-turn lanes. The discussion includes the target accident types and locations for the improvement projects to be evaluated and the accident severity levels considered. An overview and comparison of three alternative statistical approaches to before-after evaluation are presented together with a detailed discussion of each of those three approaches. All three alternative statistical approaches were used in the evaluation and the results obtained from each approach are presented in section 6. Target Accident Types and LocationsAs part of the evaluation, data were obtained for each accident that occurred at or near each study intersection in specified time periods both before and after the projects were evaluated. A decision was made concerning which accidents were the "target accidents" to which the evaluation should be applied. Target accidents included all accidents that occurred at or near each intersection during the study period. In addition, it was also considered desirable to limit the evaluation to those accidents of collision types or collision locations that were likely to be affected by the improvements being evaluated. If the accident data for both the before and after periods included accidents of types that could not conceivably be affected by the improvement, then this "noise" would have introduced unnecessary variability into the accident counts that may have prevented the researchers' ability to observe the effect of the improvement. Thus, while the effect of the improvement projects on total intersection accidents should be considered, it also is desirable to consider specific subsets of total intersection accidents as the target accidents for the evaluation. On the other hand, the effects of some improvement types may be so pervasive that nearly every intersection accident may be affected. Clearly, the appropriate target accidents depend on the nature of the improvement being evaluated. Section 3 of this report highlights four key types of intersection improvement projects to be considered in the before-after evaluation:
These project types vary in the portions of the intersection and the accident types they potentially affect. The effects of right-turn lanes appear to be the simplest and, therefore, are addressed first. Projects involving left-turn lanes, both left- and right-turn lanes, and extended turn lanes will then be addressed. Projects Involving Addition of Right-Turn LanesThe safety effects of adding a right-turn lane on an intersection approach are expected to be limited to only certain types of intersection accidents. For example, it might be supposed that installation of a right-turn lane on a particular approach would affect primarily the following accident types:
However, because other collision types could potentially be affected by installation of a right-turn lane, a particular portion of the intersection can be designated as the "target area" rather than designating particular accident types as target accidents. Figure 1 illustrates the portion of the intersection area that would be expected to be affected by installation of a right-turn lane on a major-road approach. Only accidents occurring within the target area would be expected to be affected by the addition of the right-turn lane. Since only a limited portion of the intersection area is potentially affected by installation of a right-turn lane, it may also be possible to use the accidents that occur in the portions of the intersection outside the target area as a comparison group in the analysis. This idea is explored further below. Figure 2 illustrates an intersection at which right-turn lanes have been added on both major-road approaches. Here there are separate target areas for the two improvements that touch but do not overlap. Thus, it appears to be possible to evaluate each added right-turn lane separately, and accidents outside the target areas (i.e., on the minor-road approaches and on the departing roadways) could still be considered as part of a comparison group, unaffected by the improvement. A review of the descriptors of individual accidents available in data from the participating states established that there are no data to explicitly identify accidents within the target areas shown in figures 1 and 2 (essentially accidents that occur on a particular intersection approach). It is possible, however, to identify:
Figure 1. Target Area for Evaluation of an Intersection with a Right-Turn Lane Added on One Approach.
Figure 2. Target Area for Evaluation of an Intersection with Right-Turn lanes Added on Two Approaches.
The preceding discussion leads to definitions of three accident categories that can be used to evaluate added right-turn lanes. These are:
Projects Involving Addition of Left-Turn LanesThe safety effects of adding a left-turn lane on an intersection approach are also limited to only a portion of the intersection area, but that portion is larger for an added left-turn lane than for an added right-turn lane. Figure 3 illustrates the target area affected by addition of a left-turn lane on one intersection approach. An added left-turn lane affects a primary target area, where collisions may occur between a vehicle turning left and same-direction or opposing-direction vehicles. In addition, collisions could also occur in secondary target areas on the other approaches. Furthermore, at a signalized intersection, accidents on all of the approaches may be indirectly affected if installation of a left-turn lane results in changes in the signal phasing or timing. Figure 3 illustrates that the primary and secondary target areas for installation of a left-turn lane can include nearly the entire intersection. If left-turn lanes were installed on opposing approaches at the same intersection, the coverage of the intersection area would be even greater. As in the evaluation of right-turn lanes, the available accident data could not be used to identify explicitly whether any particular collision occurred within the primary or secondary target areas shown in Figure 3. However, the following accident categories can be evaluated:
Figure 3. Target Area for Evaluation of an Intersection with a Left-Turn Lane Added on One Approach.
Projects Involving Addition of Both Left- and Right-Turn LanesWhere both left- and right-turn lanes are added at a single intersection approach, the area of the intersection affected by the project is the combined extent of the shaded areas shown in figures 1 and 3. For such projects, total intersection accidents and intersection approach accidents can be defined as they are for the left- and right-turn lane project types discussed above. However, if both a left- and right-turn lane are installed on the same intersection approach, the definition of project-related accidents must combine the separate definitions for left- and right-turn lane projects given above. Projects Involving Extension of the Length of Existing Turn LanesThe target area for projects in which the length of an existing turn lane is extended could potentially be smaller than the target areas for right- and left-turn lanes shown in Figures 1 and 3, respectively. Accidents susceptible to correction by extending a turn lane should probably include only those accidents that occur in the through travel lane in the area upstream of the location where the turn lane begins in the condition before improvement. However, there is no way to identify such accidents explicitly in the available accident data. Therefore, projects involving extension of the length of existing left-turn, right-turn, or left- and right-turn lanes have been evaluated with the same target accident types defined above for addition of those specific types of turn lanes. Accident Classification by Approach and MovementThe intersection improvement projects being evaluated, by definition, would be expected to directly affect some parts of the intersection and not others. For example, installation of a left-turn lane on one intersection approach would certainly directly affect accidents involving vehicles using that approach, but only indirectly would affect intersection accidents that did not involve that approach. Therefore, researchers developed a method for classifying intersection accidents by the approach and actual or intended turning movement. Each accident-involved vehicle was classified by its initial direction of travel and intended movement (i.e., by the same data that are normally used to construct a collision diagram). The categories used were as follows:
Thus, at four-leg intersections, there are 12 approach/movement combinations. Major-road approach 1 is the northbound or eastbound approach, while major-road approach 2 is the southbound or westbound approach. Similarly, minor-road approach 1 is the northbound or eastbound approach, while minor-road approach 2 is the southbound or westbound approach. At three-leg intersections, 6 of the 12 approach/movement combinations do not exist. In the case of three-leg intersections, the one minor-road approach is designated as minor-road approach 1; approach/movement combinations 31, 41, 42, 43, and either 12 and 23 or 13 and 22 do not exist at a three-leg intersection. Each two-vehicle accident was classified by the appropriate pair of approach/movement codes for the involved vehicle. For example, a collision between a northbound major-road vehicle turning left and a southbound through vehicle was classified as a 13/21 collision. Accidents involving only a single vehicle were classified by the approach/movement code for that vehicle. Accidents involving more than two vehicles were classified by the approach/movement codes of the first two involved vehicles (whichever vehicles were designated as Vehicle 1 and Vehicle 2 in the accident record). Accidents were classified by involved approaches using the approach/movement classes shown above. For example, accidents involving the northbound major-road approach to a particular intersection were identified as all accidents for which the approach/movement code for one of the involved vehicles was 11, 12, or 13. Multiple-vehicle accidents involving vehicles from different approaches were counted as approach-related for each of those approaches. The same concept was used to identify accidents related to particular improvement projects. For example, if the improvement project at a particular intersection involved installing left-turn lanes on both the northbound and southbound major-road approaches, then all accidents for which the approach/movement code of one of the involved vehicles was either 13 or 23 would be considered to be project-related accidents. For sites at which left-turn lanes were added, approximately 11 percent of total intersection accidents before project construction were of accident types related to the project. The comparable percentages of project-related accidents before project construction were 18 percent for installation of right-turn lanes, 28 percent for installation of both left- and right-turn lanes, and 7 percent for extension of the length of existing turn lanes. This accident classification system was applied to both improved and comparison sites so that when approach-related or project-related accidents were evaluated for an improved site, accidents with comparable approach/movement combinations could be selected for the comparison site. Accident Severity LevelsTraffic accidents are generally classified into three severity levels—fatal, injury, and property-damage-only—based on the severity of the most serious injury suffered by any party to the accident. Injury accidents are often subdivided further based on the severity of the injury. Property-damage-only accidents are often subdivided further by the amount of property damage sustained in the accident or by whether one or more vehicles are towed from the accident scene. Accident severity levels are an important consideration in planning a safety evaluation, because the completeness of accident reporting varies by severity level. All accidents involving a fatality or a personal injury are required to be reported to police authorities. Fatal accidents are nearly always reported to police authorities and, therefore, become part of accident databases. Injury accidents are less completely reported. Estimates of the reporting of accidents involving personal injuries vary from 50 to 90 percent and are undoubtedly dependent on the injury severity. The threshold amount at which property-damage accidents are required to be reported to police authorities varies from jurisdiction to jurisdiction. Even those property-damage-only accidents that meet the reporting threshold, and are thus required to be reported, are, in fact, reported in less than 50 percent of cases. Reporting is presumed to be higher for the most severe property-damage-only accidents in which at least one vehicle is towed from the scene; police authorities are usually aware of cases when a tow truck is summoned. However, not all jurisdictions identify in their accident records whether accident-involved vehicles have been towed from the scene. Unfortunately, most of the eight states that contributed data to this study did not have a code in their data indicating that one or more involved vehicles were towed from the scene. Therefore, data on property-damage-only accidents involving a towaway were not used as a separately category in the study because they could not be consistently identified. Because of the concerns about incomplete accident reporting, some evaluations have focused exclusively on fatal and injury accidents, excluding property-damage-only accidents because of their low reporting percentage. However, some intersection design improvements are implemented to mitigate patterns of minor accidents that often involve property-damage only. Thus, the evaluation did not focus exclusively on fatal and injury accidents. The accident severity levels considered in the study were:
Evaluation ApproachesThis section of the report discusses and compares three alternative approaches to before-after evaluation that were utilized in the research. These are:
An overview of each approach is presented below, followed by a discussion of other analysis considerations. A more detailed discussion of the three evaluation approaches is then presented. This later discussion includes a conceptual overview, the statistical analysis approach, and the strengths and weaknesses of the approaches. The three alternative evaluation approaches combine evaluation concepts recommended in two sources. These are: a recent FHWA report by Griffin and Flowers entitled A Discussion of Six Procedures for Evaluating Highway Safety Projects;(1) and the recently published book by Hauer, entitled Observational Before-After Studies in Road Safety.(2) These sources share some of the same concepts but use different terminology and notation. To make an appropriate comparison of these concepts, we have introduced a common terminology and notation drawing liberally (but not exclusively) on the terminology and notation used by Hauer.(2) Before-After Evaluation with Yoked ComparisonsThe first of the three analysis approaches is the before-after evaluation with yoked comparisons, or the YC approach. This is a traditional approach to the evaluation of traffic accident countermeasures and involves one-to-one matching between intersections that have been improved by the addition of left- or right-turn lanes and similar intersections that have not been improved. The purpose of the matched or yoked comparison sites is to account for the effects of time trends. This approach has been recommended by Griffin and Flowers.(1) The one-to-one matching of treatment and comparison intersections requires selection of intersections that are similar in key characteristics such as area type (rural/urban), traffic control (signalized/unsignalized), number of legs, and traffic volume. In the YC approach, it is assumed that the change in accidents from before to after the improvement at each treatment site, had the site been left unimproved, would have been in the same proportion as at the matching comparison site. Under this assumption, the accident frequency at each treatment site in the before period would be multiplied by the ratio of "after-to-before" accidents at the comparison site to predict what would have been the expected number of accidents in the after period at the treated site without the improvement. The specific YC approach that has been employed is one of the designs recommended by Griffin and Flowers.(1) The YC approach is described in more detail later in this section of the report. Before-After Evaluation with a Comparison GroupThe second of the three evaluation approaches that have been utilized in the research is before-after evaluation with a comparison group, which will be termed the CG approach. This is a variation of the YC approach to the evaluation of traffic accident countermeasures and is intended to estimate the safety effectiveness of an improvement, or combination of improvements, while controlling for time-trend effects. This is achieved by careful selection of a suitable comparison group of intersections to match the improved intersections, so that the above-mentioned effects will be manifested equally in the treatment and the comparison groups. The before-after approach with comparison groups differs from the stronger before-after approach with randomized control groups in that the choice of whether or not to improve an intersection was already made by the participating highway agency prior to the study and is therefore not within the control of the research team. In the CG approach, it is assumed that the change in accidents from before to after at the improved intersections, had they been left unimproved, would have been in the same proportion as in the comparison group. Under this assumption, the "before" accident frequency would be multiplied by the ratio of the after-to-before accidents in the comparison group to predict what would have been the expected number of accidents in the "after" period without the improvement. Similar procedures can be used to adjust for differences in traffic volumes (e.g., exposure) at the improved intersections between the before and after periods. The proper choice of comparison intersections with similar characteristics to those of the improved intersections is, therefore, very important to a valid before-after evaluation. Intersections in the treatment (improved) and comparison groups are matched on their geometric features, traffic control features, and traffic volumes, but not necessarily on their accident experience. Before-period accident frequencies for the treatment and comparison groups do not necessarily need to be similar since the assumption is made in this design that if the improvements were not undertaken, then the change in accidents from before to after conditions for the improved sites would be similar to that for the comparison sites. The specific CG approach used in the study is a variation of that recommended by Hauer.(2) This approach incorporates a multivariate formula to adjust for the differences in traffic volume between the before and after periods and between treatment and comparison sites. While Hauer develops the CG approach as far as possible within its conceptual limitations, it should be noted that Hauer considers the Empirical Bayes approach, discussed below, to be superior to the CG approach. The CG approach is discussed in more detail later in this section of the report. Before-After Evaluation with the Empirical Bayes ApproachAn alternative analysis approach used in the study is the Empirical Bayes (EB) method. The distinctive features of the EB method are threefold. First, since there is potential for selection bias in the choice of improvement sites, the EB method attempts to account for that bias, which neither the YC nor the CG approach can. Second, the EB method attempts to account explicitly for changes from "before" to "after" in causal factors such as traffic volume. This is particularly important for intersections, since the expected number of accidents at an intersection is a nonlinear combination of the various conflicting flows, and it is often inappropriate to use a simple accident rate to account for the influence of changes in traffic volume.(2,20) Third, in the CG approach, it is common to use only two to three years of "before" accident data for fear that older accident counts are no longer relevant; the EB method can correctly exploit the information in older accident counts, which is particularly important for intersection types that experience only a limited number of accidents per year. The EB method requires richer data and more effort in analysis. What is referred to above as a comparison group is referred to in the EB method as a sample from a reference population or reference group. For the reference group, data are required not only about accidents, but also about traffic flow (and perhaps other variables). Using these, one then estimates suitable multivariate models linking accident frequency and causal variables. The result of this modeling then accounts for both selection bias and changes in causal variables. The EB approach has recently been implemented in an evaluation of the safety effects of resurfacing projects by Hauer, Terry, and Griffith(64) and has been used by Persaud et al.(53) to evaluate the safety effects of converting conventional intersections to roundabouts. The specific EB method used in this study is based on the recently published book by Hauer, entitled Observational Before-After Studies in Road Safety.(2) This method is described in more detail later in this section of the report. Choice Between Alternative Analysis ApproachesThe three alternative analysis approaches described above have the same goal but use different methods. In each case, there is a comparison or reference group to provide a means for estimating the accident experience that would have been observed in the after period at the treated sites if no treatment had been made. The YC approach does this by assuming that accidents at each treated site would change between the before and after periods as the accidents did at a similar comparison site. The CG approach replaces the one matched comparison site with a group of similar sites. The EB approach relies on a regression relationship from a group of similar sites (called a reference group rather than a comparison group) to estimate the accident experience in the after period at a treated site if no treatment had been made. Each of the three approaches described above are valid alternative approaches to the before-after evaluation of intersection design improvements. The EB approach appears to have advantages over the YC and CG methods in its ability to address the effect of regression to the mean, but the EB approach also requires more complete data and greater analysis effort. The CG approach is generally considered to be preferable to the YC approach, because the CG approach relies on multiple sites in a comparison group, while the YC approach relies on a single comparison site. This research has used all three approaches rather than making an a priori choice between them. An assessment of the relative performance of the three approaches is presented later in the report. Dependent VariablesThe dependent variable in any statistical analysis is the variable whose value is to be determined or predicted. For all analyses in this study, the single most important dependent variable to assess safety effectiveness as a result of an improvement is the accident frequency at the selected intersections. Yearly accident frequencies, with a minimum of three years (and preferably five years) of "before" data and as much "after" data as possible have been obtained and analyzed. The option of analyzing yearly accident rates (number of accidents per million entering vehicles) is less desirable because accident rates presume a linear relationship between accidents and traffic volume, while most previous studies have shown such relationships to be nonlinear. Data on all accidents occurring at the intersections during study periods before and after construction of each improvement project have been obtained from the participating states. Accident severity levels that have been used as the dependent variable are:
The eight specific safety measures considered in the evaluation are:
Independent VariablesThe independent variables in an accident study are those variables whose effects on accidents are to be determined or controlled in the analysis. The primary independent variable is the implementation of the improvement project whose effectiveness is to be determined. Independent variables have been used in several ways in the study:
The primary independent variables included in the study are those geometric, traffic control, and traffic volume variables obtained from the participating state highway agencies and in field visits to the study sites. Before and After Study PeriodsAccident data for each site were obtained from the participating state highway agencies for specific time periods. These periods are presented in table 19. Thirteen years of accident data were available for one State, 12 years were available for three States, 11 years were available for one State, 10 years were available for two States, and 9 years were available for one State. As shown in tables 10 and 15, the projects evaluated were constructed on various dates during the period for which these accident data were obtained. The study periods before and after improvement of each site were selected as follows:
Both the before and after study periods were composed of complete calendar years. Partial years were not used because they are subject to seasonal effects which could bias the evaluation. The entire calendar year during which the improvement project was constructed was omitted from the evaluation. This approach avoided the use of partial years of accident data. In addition, because projects in many parts of the country are completed during the summer construction season, exclusion of the entire construction year provides a buffer period of several months between the end of construction and the beginning of the "after" study period. This buffer period provides an opportunity for drivers to become familiar with the improved intersection before the assessment of the project's effectiveness begins. For the improved or treatment sites as a whole, the "before" study periods ranged from 1 to 10 years in duration, with a mean duration of 6.7 years. The "after" study periods also ranged from 1 to 10 years in duration, with a mean duration of 3.9 years. Before-After Evaluation with Yoked ComparisonsThe first of the three alternative evaluation approaches that will be presented is the before-after evaluation with yoked comparisons, referred to as Design 4 by Griffin and Flowers.(1) Of the six evaluation designs presented by Griffin and Flowers, this is the most appropriate for evaluation of intersection design improvements. Conceptual OverviewThe before-after evaluation with yoked comparisons involves a one-to-one matching between treatment and comparison sites. Thus, for each improved or treatment site, a comparison site with similar features was identified. Each selected comparison site was similar to the corresponding treatment site with respect to area type (rural/urban), intersection type (three-leg or four-leg), traffic control (signalized/two-way STOP), geometric design, and traffic volumes. The matched treated and comparison sites were always located in the same state and, usually (but not necessarily) in the same geographic region of the state. The comparison site had to have undergone no geometric design or traffic control improvements (beyond routine maintenance) during the periods for which data were available before and after improvements of the corresponding treatment site. Thus, for any project type of interest, there were n pairs of treatment and comparison sites for consideration in the evaluation. The term "yoked comparison" used by Griffin and Flowers refers to the one-to-one matching between the treatment and comparison sites.(1) Accident data were obtained for periods as long in duration as possible before and after the improvement at each treated site and for the same time periods at the matched comparison site. Griffin and Flowers assume that the durations of the before and after periods are identical at any given treated site, although they may vary from one treated site to another. Despite this assumption, there is no particular reason that the duration of the before and after periods need to be identical, because the adjustment to account for the difference between, for example, a three-year "before" period and a two-year "after" period is obvious. The key assumption in the YC approach is that the change in accidents between the before and after periods at any comparison sites is representative of the change in accidents that would have occurred at the corresponding treatment site had the improvement at that site not been made. Thus, it is postulated that, if the implementation of the improvement project at any treatment site was beneficial to safety, it resulted in the number of crashes at the treatment site falling more rapidly, or rising less rapidly, than accidents at the comparison site. The YC approach, as formulated by Griffin and Flowers,(1) does not include a mechanism to account for changes in traffic volume between the before and after periods at the treatment and control sites. Since traffic volume for the before and after periods are available for each site, we have modified the YC approach to adjust for the effects of traffic volume, assuming that those effects are linear (i.e., proportional to the changes in volume). Table 22 illustrates the accident data that was gathered to employ the before-after design with yoked comparisons for any given type of project. In the table, the values of K, L, M, and N are counts of the number of accidents observed during periods before and after the treatments. The values of p, q; and E; are statistics derived from these data. The analyses employed to derive these measures are described below.
Statistical AnalysisFor any pair of treatment and comparison sites (designated by subscript i), the expected number of accidents at the treated site in the "after" period, had no improvement been made (pi), is best estimated as:
The best estimate of the expected number of accidents after the treatment (li) is the observed accident frequency. In other words:
The expected number of accidents without the treatment, pi, is then compared to the observed number of accidents, li or Li, to assess the accident reduction effectiveness of the project at that site. The accident reduction effectiveness of the project can then be assessed as the ratio of what the accident experience was with the treatment to what it would have been without the treatment:
or, equivalently:
When qi < 1, the accident frequency has decreased, and the treatment appears to be effective; when qi > 1, accident frequency has increased, and the treatment appears to be harmful to safety. The treatment effectiveness can also be expressed as the percentage change in the expected accident frequency, E, estimated as 100 (q - 1). A negative value of E represents a reduction in accident frequency. If the before and after periods differ in duration, or if traffic volumes have changed between the before and after periods, the proportional changes need to be incorporated in Equations (3) through (5). (More sophisticated methods of accounting for the traffic volume effects will be discussed in conjunction with the CG and EB approaches.) The first step in the analysis is simply to plot the pairs of observed and expected accident frequencies for the "after" period (known as Li and pi, respectively), as illustrated in Figure 4. If all of the data points were to fall on the diagonal line in the figure, then one would conclude that the treatment had no effect. Points that fall below the diagonal line suggest that the treatment was beneficial, while points that fall above the diagonal line suggest that the treatment was harmful. Equations (3) through (6) address the estimated treatment effectiveness at a single site. An overall estimate of the treatment effectiveness can be derived from the effectiveness estimates for the individual sites using a weighted average. The weight, wi, for each site represents the reciprocal of the squared standard error of the log odds ratio, Ri, generated from the data for that site, or:
The squared standard error for Ri is calculated as:
Figure 4. Plot of Observed vs. Expected Accident Frequencies. from which the weight, wi, is simply calculated as:
A weighted average (mean) log odds ratio across all n pairs of sites can be determined as:
By exponentiating Equation (10), an overall average (mean) odds ratio, or project effectiveness, can be obtained for the n sites as:
Thus, the overall mean percentage accident reduction effectiveness of a treatment can be estimated as:
The next step in the analysis is to assess whether the estimated effectiveness, qmean, is statistically significantly different from one, or whether the mean percentage accident reduction effectiveness is statistically significantly different from zero. Since Rmean is asymptotically normally distributed, a z-test is used to test for significance, as follows. The standard error of Rmean is computed as:
A standard normal z-score can then be obtained as:
If z falls within the interval from -1.96 to +1.96, there is no apparent treatment effect at the 95 percent confidence level. If z falls outside the interval from -1.96 to +1.96, there is a statistically significant treatment effect (beneficial if z is negative, harmful if z is positive). The approach described above is also used to estimate a 95 percent confidence interval for the estimated treatment effect, qmean. First, the upper and lower 95 percent confidence limits around Rmean would be estimated as:
Next, the upper and lower 95 percent confidence limits for the treatment effect expressed as a weighted average log odds ratio would be determined by exponentiating Equations (15) and (16) as:
Finally, substituting qmean (upper) and qmean (lower) for qmean in Equation (12) provides a 95-percent confidence interval for the estimated treatment effect expressed as a percentage accident reduction:
Thus, it can be said that the estimated percentage effect of the treatment is, on the average, Emean, with 95 percent confidence that it is in the range from Emean (lower) to Emean (upper). For example, it might be concluded that the estimated effectiveness of a particular intersection improvement project type in reducing accidents is 25 percent, on the average, with 95-percent confidence that it is in the range from 9 percent to 38 percent. Using Equations (11), (12), and (13), the standard error of Emean is computed as:
To complete the analysis and estimation of the mean measure of effectiveness, the homogeneity of the individual estimated treatment effects, Ri, is tested. The plot of the pairs of observed versus expected accidents in figure 4 provides a view of the scatter of the data around the diagonal. A chi-square (c2) analysis that partitions the total chi-square value into a chi-square for treatment and a chi-square for homogeneity will be performed to determine whether the scatter of the data points about the overall estimate of treatment effectiveness is within expectations. The calculations for the c2 analysis are shown in table 23.
The probability, or significance, levels pT and pH, associated with the treatment and homogeneity effects, respectively, provide a measure of statistical significance of these two sources of variability in the data. pT provides the same test for significance as did Equation (13). If pH is less than 0.05 (i.e., a significance level of 5 percent), then Griffin and Flowers(1) recommend a conclusion that the data are not homogeneous across pairs of sites and that some other factors besides treatment are affecting the analysis. In such a case, Griffin and Flowers maintain that the data should not be combined into a single measure of effectiveness of the treatment. By contrast, Hauer(2) maintains that variations in treatment effectiveness between sites are to be expected and that otherwise valid effectiveness measures should not necessarily be excluded on the basis of lack of homogeneity. In this research, the test for homogeneity recommended by Griffin and Flowers was performed and the results were documented in appendix C of this report, but no effectiveness measures were excluded due to lack of homogeneity. The statistical analysis described above has been programmed in the commercially available SAS software package and performed for a variety of intersection and treatment types as described in section 6 of this report. Adjustment for Traffic Volume and Study Period DurationThe equations presented above are applicable if both the treatment and comparison sites have the same traffic volumes, both before and after the project, and if the duration of the before and after periods for both the treated and comparison sites are equal. In most cases, these assumptions are not appropriate. Typically, the traffic volumes of the treated and matched comparison sites differ and, for both sites, the traffic volumes typically change from the before to the after period; traffic volume growth over time is most common, but in some cases traffic volumes may actually decline from before to after the improvement project. Furthermore, it is not uncommon for the durations of the before and after periods to differ. An adjustment factor for traffic volume in the yoked comparison analysis can be computed as:
where: Adj1 = Traffic volume adjustment factor. ADTBT = Traffic volume (veh/day) for treated site in the before period. ADTAT = Traffic volume (veh/day) for treated site in the after period. ADTBC = Traffic volume (veh/day) for comparison site in the before period. ADTAC = Traffic volume (veh/day) for comparison site in the after period. An adjustment factor for duration of study periods in the yoked comparison analysis can be computed as:
where: Adj2 = Duration adjustment factor. YEARSBT = Duration of before period for treated site (years). YEARSAT = Duration of after period for treated site (years). YEARSBC = Duration of before period for comparison site (years). YEARSAC = Duration of after period for comparison site (years). Normally, YEARSBT and YEARSBC are equal, as are YEARSAT and YEARSAC, so that ADj2 is usually equal to 1.0. To apply these adjustments in the YC analysis, Equation (6) must be recast as:
The remainder of the analysis proceeds as described above. Strengths and Weaknesses of This Evaluation ApproachThe strength of the YC approach is its simplicity and its conceptual appeal to engineers. Since there is one and only one matching comparison site for each treatment site, care can be taken in assuring that the treatment and comparison sites are similar in a number of engineering factors such as traffic volume, geometric design, and traffic control. Also, the data needs for this approach are readily apparent and known in advance. The YC approach has three major weaknesses, however. First, by relying on only one comparison site for each treated site, the YC approach relies on very limited data in estimating the values of i, the accident experience that would have been observed at particular treatment sites if no treatment had been made. If the treatment and comparison sites are well matched, the values of Mi and Ni will be similar in magnitude to Ki and Li (i.e., often quite small) and will be highly variable. The YC approach does not utilize data from other similar treatment sites to increase the magnitudes of Mi and Ni and, therefore, the reliability of the estimates that can be made with them. Thus, the YC approach is likely to produce accident reduction effectiveness estimates with relatively wide confidence limits. Second, the YC approach cannot deal with the well-known phenomenon of "regression to the mean." If the treated sites have been selected for improvement because of high short-term accident experience in the before period, then simple probability theory suggests that accident experience is likely to be lower in the after period even if no improvement is made. Thus, the effect of the treatment is likely to be partially confounded with the expected decrease in accident experience from regression to the mean. Regression to the mean can only be accounted for with knowledge of the "normal" or expected value of before-period accident experience at the treated sites and the YC approach cannot supply such information. Third, the yoked comparison approach has difficulty dealing with accident frequencies with values equal to zero. Specifically, if either Ki or Ni is zero in Equation (6) or Equation (24), then the effectiveness of the treatment is undefined. This problem is usually resolved by substituting 0.5 for zero as the value of Ki or Ni in these equations, but the existence of this problem represents a conceptual weakness of the YC approach. Despite these weaknesses, the YC approach has been used in this research because one objective of the study is to assess the performance of alternative analysis approaches. In particular, the YC approach was useful because the matched comparison sites required by the YC approach also served as the foundation for a comparison group for the CG approach and a reference group for the EB approach. Before-After Evaluation with a Comparison GroupThe second of the three alternative evaluation approaches used in the study is before-after evaluation with a comparison group. This approach has been formulated based on recommendations by Hauer,(2) with variations to handle the adjustments for traffic volume, study period duration, and state-to-state differences inherent in this study. The CG approach overcomes one weakness of the YC approach—the limitation to a single comparison site for each treatment site. For each treatment site, the CG approach replaces the yoked or matched comparison site with a group of comparison sites so that the accident experience of the entire comparison group is used in estimating what the accident experience of each treatment site would have been had the improvement not been made. It should be noted that, although Hauer has developed the CG approach as far as its limitations permit, he considers the EB approach, presented below, to be conceptually superior to the CG approach. Conceptual OverviewIn the CG approach, the idea of one-to-one matching of the treatment and comparison sites is discarded and the available comparison sites, taken as a whole, are considered as a comparison group. Indeed, the comparison group should preferably include more sites than the treatment group. The purpose of the comparison group is still to estimate the change in accidents that would have occurred at the treated sites if the treatment had not been made. The CG approach has been implemented as follows. For any particular project type, such as those discussed in section 3 of this report, a certain number of intersections at which improvements of that particular type have been made were available for evaluation. These intersections will be referred to as the treatment group. Researchers identified a second group of intersections at which no geometric design or traffic control changes (other than routine maintenance) were made during the time periods for which data are available for the treated intersections. This second group of intersections constitutes the comparison group. Accident and traffic volume data were generally available for the same time periods for both the comparison group and the treatment group. The comparison sites would normally be similar to the treatment group in geometric design and traffic control features, although Hauer does not consider close physical similarity between the treatment and comparison sites to be critical. Instead, Hauer maintains that close agreement between the treatment and comparison groups in the monthly or yearly time series of accident frequencies during the period before improvement of the treated sites is more important.(2) In other words, Hauer considers that it is not vital that the comparison sites look like the treatment sites, but it is vital that the comparison sites have accident histories similar to the accident histories for treatment sites for the period before improvement of those sites. Such similarity in accident experience during the period before improvement increases confidence (but cannot assure) that the comparison group will behave as the treatment group would have behaved had the improvement not been made. Hauer suggests a statistical technique to assess analytically the appropriateness of any particular comparison group for the corresponding treatment group.(2) Hauer's approach of matching on the basis of safety rather than physical characteristics was not fully implemented in this study, because researchers had the matched comparison sites that were used in the yoked comparison analysis available for use as a comparison group. The matching comparison sites had been selected to be physically similar to the individual treatment sites, so the matched comparison sites, as a group, were physically similar to the treatment sites as a group. The treatment and comparison sites were also similar in traffic volume levels. Additional reference sites with similar physical characteristics and traffic volumes were added to increase the size of the comparison group. The key features that distinguish the CG approach from the YC approach are as follows:
The CG approach leads to a very similar formulation of the data for the evaluation to that used in the YC approach. Table 24 is analogous to table 22 and differs only in that the columns formerly headed "Comparison sites" are now headed "Comparison group." In each row of table 24, Mi and Ni are based on an entire comparison group and not just on one matching site. However, the values of Mi and Ni in table 24 may vary even among the sites that use the same comparison group if the time periods on which Ki and Li are based vary. Furthermore, the data for the individual comparison sites that are combined to determine Mi and Ni must first be adjusted for differences in traffic volume between the treatment and comparison sites. An adjustment for state-to-state differences in accident frequency is also needed where a specific comparison state is located in a different state than the treatment site.
Figure 5. Typical Regression Relationship for Predicting Intersection Accident Frequency as a Function of Entering Traffic Volume
The statistical analysis methodology used to determine the values of p, q, and E in the CG approach is explained below. Statistical AnalysisThe statistical analysis methodology for the CG approach must be explained in terms of both the observed accident counts and their expected values. As shown in Table 25, the observed accident counts that correspond to the row and column headings shown in the table will be referred to as K, L, M, and N, and their expected values, which are unknown, will be referred to by the Greek letters k, l, m, and n.
The comparison ratio, rC, for the comparison group is defined as the ratio of the expected accident count during the after period to the expected accident count in the before period. In other words:
The CG approach assumes that the expected number of accidents for the treatment group in the after period, had no treatment been implemented, can be predicted as:
Implicit in Equation (25) is the assumption that the corresponding ratio for the treatment group, had no treatment been implemented:
is equal to rC. In other words:
The ratio rC/rT is also known as the odds ratio, wi. The customary effectiveness of a treatment at a given site is the same as that derived for the YC approach in Equations (3) through (6). However, for the CG approach, the subscript i in these equations represents the appropriate data for a particular treatment site. Each treatment site has a corresponding comparison group, and the individual sites in that comparison group will be represented here by the subscript j. The CG analysis proceeds in a manner similar to the YC analysis except that for any given treated site, Mi and Ni in Equation (6) must be determined as the sum of the adjusted accident frequencies for the individual comparison sites within the comparison group. Adjustments to accident frequencies for the individual comparison sites are needed when (1) the treatment and comparison sites have different traffic volume levels, (2) the study periods for the treatment and comparison sites have different durations, or (3) the treatment and comparison sites are located in different states, and the safety performance of the particular intersection type in question differs between those states. Because of the need to make these adjustments, the computation of the comparison group analysis is more complex than suggested by Equations (6) through (11). Specifically, a set of adjustments were first made to the data for the comparison group sites that correspond to each treatment site. Then, the comparison group sites corresponding to each treatment site were combined to determine pooled accident frequencies for the comparison group as a whole. Finally, an adjustment was made to the combined treatment and comparison site data in determining the treatment site odds ratio. In the comparison group analysis, the traffic volume adjustment was made using a regression relationship between accident frequency and traffic volume, rather than assuming that the adjustment should be proportional to traffic volumes as shown in Equation (22). These regression relationships were developed with negative binomial regression and involved separate coefficients for major- and minor-road traffic volumes, as described in the next section. These regression relationships also involved a coefficient to represent the differences in accident frequency between states. For each individual comparison site, its accident frequency was adjusted to be comparable to its equivalent value under the same conditions (traffic volume, duration of study period, and state) as the treatment site. The adjustment factor for the comparison site in the before period is:
where: AdjB = Adjustment factor for comparison site accident frequency in the before period. f(x,y,z) = Predicted accident frequency as a function of major-road traffic volume (x), minor-road traffic volume (y), and state (z) from a negative binomial regression relationship (see discussion later in this section). MajADTBT = Major-road traffic volume (veh/day) in the before period at the treatment site. MajADTBC = Major-road traffic volume (veh/day) in the before period at the comparison site. MinADTBT = Minor-road traffic volume (veh/day) in the before period at the treatment site. MinADTBC = Minor-road traffic volume (veh/day) in the before period at the comparison site. YEARSBT = Duration of before period for treatment site (years). YEARSBC = Duration of before period for comparison site (years). Similarly, the adjustment factor for the comparison site in the after period is:
where: AdjA = Adjustment factor for comparison site accident frequency in the after period. MajADTAT = Major-road traffic volume (veh/day) in the after period at the treatment site. MajADTAC = Major-road traffic volume (veh/day) in the after period at the comparison site. MinADTAT = Minor-road traffic volume (veh/day) in the after period at the treatment site. MinADTAC = Minor-road traffic volume (veh/day) in the after period at the comparison site. YEARSAT = Duration of after period for treatment site (years). YEARSAC = Duration of after period for comparison site (years). With these adjustment factors, the adjusted accident frequencies for an individual comparison site can be determined as:
and
Then, the values of the adjusted before period accident frequencies, MADJj, are summed over all of the comparison sites corresponding to a specific treatment site, i, to calculate the total adjusted before-period comparison group accident frequency, Mi. Similarly, the values of the adjusted after-period accident frequencies, NADJj, are summed over all comparison sites to calculate the total adjusted after-period comparison group accident frequency, Ni. In combining the treatment site and comparison group accident frequencies, a final adjustment must be made to the after-period accident frequency for the treatment site. This adjustment also uses the negative binomial regression relationships. The adjustment is determined as:
where: Adj3 = Adjustment factor applied to after-period accident frequency [analogous to Adj1 in Equation (22)]. This adjustment is applied by modifying the value of the observed after-period accident frequency for the treatment site, Li, as follows:
The odds ratio is then determined as in Equation (6), and the analysis proceeds as in the YC approach. Appropriateness of Comparison GroupsHauer (2) states explicitly that the foundation of the CG method rests on the assumption (or the hope) that Equation (28) is correct. While Hauer discusses the importance of agreement in key safety measures between the treatment and comparison groups, no specific statistical methodology for assessing the level of agreement between the treatment and comparison groups is presented. Therefore, such a methodology has been developed for use in the current study. Confirming the degree of agreement between the group of treatment or improved sites and the comparison group is an important aspect in the before-after analysis. The comparison sites were selected because they were similar in physical characteristics and traffic volumes to the treatment sites, but there was no a priori assurance that the treatment and comparison sites were similar in safety performance in the time period before improvement of the treatment sites. A statistical approach to providing this assurance was developed and implemented using groups of treatment and matched comparison sites over the entire period before improvement of the treatment sites. For each combination of area type, traffic control, and project type, a time series of total intersection accidents at the treatment sites was compared to the time series of accidents at the matching comparison sites. An example pair of treatment and comparison time series is illustrated in Figure 6. Each of the two time series shown in the figure represents the series of average accident frequencies per site per year over the entire before period. Each time series is based on the same number of intersections, and the number of intersections included decreases from year to year as more and more treatment sites reach the year during which they were improved. The evaluation approaches used in this report do not require that the treatment and comparison time series shown in figure 6 coincide. However, if the treatment and comparison groups are well matched, the average annual accident frequency for the comparison group should rise when the average for the treatment group rises, and fall when the average for the treatment groups falls. A perfect match in accident trends between the treatment and comparison groups would exhibit a pair of two jagged but parallel lines in plots like figure 6. Figure 6. Comparison of Accident Experience for Treatment and Comparison Groups for Rural Unsignalized Intersections at Which Left-Turn Lanes Were Added. The objective of the statistical assessment of the degree of agreement between treatment and control groups is to test whether the corresponding time series, like those shown in the figure, are parallel. A basic two-way analysis of repeated measures approach with interaction was used to test whether the two time series (treatment and comparison) of yearly accidents deviate significantly from parallelism. The two main factors used in the analysis of variance are the type of site (i.e., treatment vs. comparison) and the year (i.e., the sequence of calendar years as shown in the figure). The various sites were nested within their respective type of site. The repeated measures nature of the design refers to the yearly measurements (observed accident counts) made on the same sites. In addition, a first-order autoregressive covariance structure was assumed for the accidents to reflect the property of correlations being larger for nearby times than for far-apart times. In this approach, the evaluation of the interaction between the two factors, type of site and year, provides a test for parallelism. PROC MIXED of SAS was used to perform the analyses of variance. The 10-percent significance level was used to assess the results of these analyses of variance. This approach was implemented for selected treatment and comparison groups for two safety performance measures—total intersection accidents and fatal and injury intersection accidents. The results of 20 analyses are shown in table 26. This table presents results for all treatment and comparison groups that included more than 20 site-years of data. For each safety performance measure, site type, and improvement project type, the table shows the number of site-years in the individual treatment or comparison groups and an indication of whether the two time series can be considered to be parallel at the 90 percent confidence level. For the cases shown in the table, there were no significant effects for lack of parallelism. Based on these results, it was concluded that the groups of treatment and comparison sites are comparable in their safety performance.
Negative Binomial Regression Relationships for Traffic Volume and State AdjustmentsThe adjustments for traffic volume and state effects presented above (AdjA, AdjB, and Adj3) are based on negative binomial regression relationships for predicting accident frequency as a function of major-road traffic volume, minor-road traffic volume, and state. Because the relationship between accident frequency and traffic volume is generally nonlinear (as illustrated in figure 5), the regression relationship can provide a more accurate adjustment for the effect of traffic volume than the proportional adjustment used in the YC approach [see Equation (22)]. State-to-state differences were included in the negative binomial regression models where they were found to be statistically significant. Regression relationships were developed for complected as many combinations of the following intersection characteristics as possible using the comparison and reference site data:
A variety of dependent variables (accident frequency measures) were used in modeling, including:
These relationships were developed with negative binomial regression because (1) negative binomial regression is well suited to deal with accident frequencies which are frequently zero or very low numbers and (2) negative binomial regression provides an overdispersion parameter that makes it useful in the EB analysis as well as the CG analysis. The model development process and the specific models developed are presented in Appendix B of this report. Strengths and Weaknesses of This Evaluation ApproachA strength of the CG approach, in contrast to the YC approach, is that it relies on a group of similar sites, rather than a single site, to determine the values of Mi and Ni. The increased size of the accident sample in the comparison group should decrease the variance of the accident data and, thus, shrink the confidence limits for the accident reduction effectiveness. On the other hand, some unwanted variability in accidents may be introduced by the inevitable diversity of the sites that make up the comparison group. Even if the comparison group as a whole resembles the treatment group as a whole, some of the comparison sites are bound to be quite different in physical characteristics (and in accident experience) from any given treatment site. Like the YC approach, the CG approach cannot determine the treatment effectiveness when Ki or Ni is equal to zero. The same approximation used in the YC approach (setting zero values equal to 0.5) is used in the CG approach. An important weakness of the CG approach is that, like the YC approach, it cannot address the bias created by regression to the mean. This weakness will be addressed by the EB approach. Before-After Evaluation with the Empirical Bayes ApproachThe third of the three alternative evaluation approaches that was used in the research is before-after evaluation with the Empirical Bayes (EB) approach. This approach was formulated by Hauer(2) and is the only known approach to before-after evaluation that directly addresses regression to the mean. Conceptual OverviewIn the EB approach, the comparison group discussed in the CG approach is replaced with a reference group that is used to model the relationship between accident frequency and fundamental intersection descriptors such as traffic volume. These models are then used together with the observed accident counts at the treated sites in the before period to estimate the number of accidents that would have occurred at the treated sites in the after period if no improvement had been made. To accomplish this, the reference group should consist of intersections that are similar to the treated intersections before they were treated (i.e., intersections with left- and right-turn lanes), or intersections whose safety performance is similar to that of such intersections. In this research, the reference groups for the EB approach were drawn from the same sites used as the comparison group for the CG approach, including both matched comparison sites from the YC approach and additional reference sites. The regression relationships used in the EB approach were the same negative binomial regression relationships used for the CG approach and discussed earlier in this section. A more detailed discussion of the development of these regression relationships is presented in appendix B of this report. Separate regression relationships were developed for specific combinations of the following intersection characteristics:
The EB approach leads to another variation of the data layout for the evaluation. Table 27 is analogous to tables 22 and 24 but is adapted to fit the EB approach. A key difference of table 27 from tables 22 and 24 is that the accident experience of the reference group does not appear explicitly. Instead, the reference group is used to develop regression models that are used in estimating the values of p shown in the table. Once the values of p have been determined, the computation of q and E is much as presented previously. The statistical analysis methodology for the EB approach is explained below. Statistical AnalysisThe statistical analysis methodology for the EB approach revolves around the use of the reference group to develop regression relationships between accident experience and key site characteristics such as traffic volumes. Figure 5, which is presented in the discussion of the CG approach of this report, shows a typical regression relationship of this type. The abscissa in figure 5 is a measure of traffic volume such as the total volume per day entering the intersection. Analogous relationships have been developed using both the major- and minor-road average daily traffic volumes (ADTs) as predictor variables; such relationships have been used in the evaluation, but cannot be illustrated in a two-dimensional graph. A key change in the EB approach from the YC approach and the CG approach is in the treatment of the observed accident count in the period before improvement of the treated sites. In both the YC and CG approaches, this observed value is used as the best available estimate of safety at the treated site during the before period. The EB approach recognizes that the expected value of the accident count for a site, as indicated by a regression relationship such as that shown in figure 5, has value as well and may, in some cases, be a much more important piece of information than the observed accident count. In other words, the observed accident count in the before period is simply one observation of a random process and better evaluations will result if, in addition to knowing the observed accident count, the process itself is understood. To illustrate this process, consider the example in figure 7, which utilizes the same regression relationship shown in figure 5. Figure 7 shows the observed accident count for an intersection in the before period as a point above the regression line for the corresponding traffic volume. This indicates that the observed accident count is higher than expected. Such higher-than-expected accident counts are subject to regression to the mean (as are lower-than-expected accident counts). The best estimate of accident experience at this site for the before period, given both the observed accident frequency and that from the regression relationship, is the value corresponding to the x between the observed point and the expected value.
Figure 7. Use of Regression Relationship in the EB Approach. Hauer provides an analytical procedure to estimate the position of the x in figure 7. This analytical procedure involves, in essence, a weighted average of the observed and expected values. This analytical procedure is fundamental to the EB approach. Hauer refers to the observed accident count as K and the expected value of the accident count as E(k). E(k) is an expected value that can be estimated from a regression relationship like that shown in figure 7. As noted above, the regression relationships actually used in the EB analysis are the same negative binomial regressions that were described earlier for use in the CG analysis. These negative binomial regressions predict accidents for a one-year period, so the equation for E(k) also incorporates the duration of the before period:
The x in figure 7, denoted as E{k|K}, is estimated using a weight factor, a, as follows:
The subscript i in Equation (34) indicates that the values apply to a specific treatment site. Hauer demonstrates that to estimate E{k|K}i with maximum precision, a must have the value indicated below:
The value of E{k}i in Equation (35) can be obtained from the regression relationship shown in figure 7. The value of VAR{k}i can be obtained from analysis of the residuals from that regression relationship. It can also be shown that α can be estimated from the overdispersion parameter of the negative binomial regression relationship and the expected before period accident frequency for the treatment site:
where: d = overdispersion parameter of the appropriate negative binomial regression relationship In applying Equation (36), the value of E(k) for a single year was used so that α would not depend on the duration of the before period. The value of E{k|K}i is determined from Equation (36) using the weight, α, determined from Equation (38). The variance of E{k|K}i is then determined as:
As in the YC and CG approaches [see Equation (4)], the best estimate of the expected accident frequency after the treatment, li, is the observed accident frequency after the treatment, Li. Adjustment factors are now needed to account for differences in traffic volume and duration between the before and after study periods. The traffic volume adjustment, rTF, uses the appropriate negative binomial regression relationship:
The adjustment factor for the duration of the study period is:
The next step is to estimate pi, the expected value of the accident count that would have occurred during the after period if the improvement had not been made. This value is estimated as:
where rd is the ratio of the durations of the before and after periods and rtf is the ratio of the expected accident counts for the traffic volume levels (or traffic flow rates) at the intersection during the before and after periods, as illustrated in figure 5. (Note: rtf is not equal to the ratio of the before and after traffic volumes unless the regression relationship being used is linear). The customary estimate of the effectiveness of the treatment is qi, which can be determined as:
The overall effectiveness of a group of treatments at similar sites can be determined by summing and then combining values of li and pi. The overall treatment effectiveness is equal to:
However, the use of q in this form is not recommended because, even if l and p are unbiased estimators of l and p, the ratio l/p is a biased estimator of q. Although this bias is often small, Hauer(2) recommends that removing it is a worthwhile precaution. An approximately unbiased estimator for q is given by:
Investigation during the current research confirmed that the difference between q and q* is very small, usually affecting only the third or fourth significant digit of the treatment effectiveness. Nevertheless, because of the potential bias in q, the value of q* was used as the treatment effectiveness estimate in this research. The variance of l can be determined as:
The variance of p can be determined as:
Finally, the variance of q* can be estimated as:
As indicated in Equation (12), the effectiveness of the treatment can be expressed as a percentage accident reduction in the form:
Typically, Hauer(2) does not estimate a 95 percent confidence interval for q or E. Rather, Hauer reports the percentage accident reduction effectiveness, E, and its standard deviation, also expressed as a percentage. Hauer also suggests that the estimated effect, E, should not be relied upon unless its estimated standard deviation is two to three times smaller than E. In this research, only results where the standard deviation of E was less than half of E were used. This is nearly equivalent to the Z value of 1.96 used in the YC and CG approaches. Hauer has also formulated a more sophisticated EB approach in which the prediction of accident frequencies from the appropriate negative binomial regression relationship to determine the value of E{k}i is done on a year-by-year basis, rather than for the before period as a whole. This more sophisticated EB approach was applied to 32 EB analyses, which constitute all of the EB analyses presented in Section 6 of this report that had sample sizes of 20 improved intersections or more. The differences in effectiveness measures between the EB approach described above and Hauer's more sophisticated approach ranged as high as 6 percent, but was typically less than 2 percent. The mean of the differences in effectiveness measures between the two approaches was 1.3 percent. While Hauer's more sophisiticated approach is desirable on theoretical grounds, and can be applied when year-by-year traffic volume data for the before period are available, the actual change in the results obtained in this study was minimal. Strengths and Weaknesses of This Evaluation ApproachThe major strength of the EB approach is that, among the evaluation approaches presented here, only the EB approach can address the potential bias created by regression to the mean. Neither the YC approach nor the CG approach can do this. In his recent book, Hauer presents a strong theoretical case for the advantages of the EB approach.(2) Another strength of the EB approach is that, since regression modeling makes very efficient use of data, reference groups needed for the EB approach should be smaller than the comparison groups that would be required for the CG approach. This should enable percentage accident reduction for specific projects types to be assessed within the desired precision level in cases where this was not possible with the CG approach. The EB approach eliminates the difficulty with zero values of accident frequency that is inherent in both the YC and CG approaches. In the EB approach, a value of Ki equal to zero can be treated naturally without arbitrarily substituting a value of 0.5. The weighting provided by Equation (36) will result in a non-zero value of E{k|K}i, because E{k}i will be greater than zero even when Ki is not. Execution of the Evaluation PlanThe evaluation plan is presented in this section of the report. The YC, CG, and EB evaluation approaches were applied to the database discussed in sections 3 and 4 of this report. Yoked Comparison EvaluationsA total of 214 YC evaluations were performed for specific combinations of:
The YC approach using one-to-one matching of treatment and comparison sites was executed as described above. The sample sizes for the 214 evaluations performed ranged from 1 to 35 intersections and from 1 to 116 intersection approaches. Statistically significant effectiveness measures were found for 37 of the 214 evaluations performed. Obviously, the evaluations with larger sample sizes are more likely to provide statistically significant results. The results of the YC evaluations are presented in tables C-1 through C-10 in appendix C of this report and are discussed in section 6. The tables in Appendix C show the number of sites included in each evaluation; these are, in some cases, smaller than the total number of sites for that intersection and project type shown in section 4 of this report because of outliers that were omitted from the analysis. Outliers consisted of anomalous sites at which substantial unexplained increases in accident frequency occurred. For example, one site experienced 1 accident in 4 years before the turn lane project and 53 accidents in 6 years after the project. Such large increases in accident frequency suggest problems in accident reporting rather than actual project effects. In all analyses performed in this study, sites were excluded as outliers if the apparent treatment effectiveness was greater than equal to a 100 percent increase in accident frequency (i.e., if the odds ratio, qi, was greater than or equal to 2.0). Sites with less than 5 accidents in the before period were not excluded as outliers, even if qi was greater than or equal to 2.0, because such variations in accident frequency over time are not unusual when the accident frequencies are very small. Comparison Group EvaluationsA total of 150 CG evaluations were performed for specific combinations of:
The CG analysis omitted two safety measures that were considered in the YC analysis. These were:
The CG analysis used the negative binomial regression relationships shown in Appendix B to adjust accident frequencies for differences in traffic volume between the before and after study periods and between the treatment and comparison sites. These negative binomial regression relationships also accounted for state-to-state differences in accident frequency in cases where treatment and comparison sites were located in different states. The state-to-state differences in accident frequencies in this multistate database were, in some cases, substantial (see appendix B). In evaluations for which no satisfactory regression models were available, proportional traffic volume adjustments based on Equation (22) were made. The CG approach, using a comparison group to replace the single comparison site used in the YC analysis, was executed as described above. The sample sizes for the 150 evaluations ranged from 1 to 39 for intersections and from 1 to 155 for intersection approaches. Statistically significant effectiveness measures were found for 47 of the 150 evaluations performed. The resulting CG evaluations are presented in tables C-11 through C-16 in appendix C of this report and are discussed in Section 6. Empirical Bayes EvaluationsA total of 108 EB evaluations were performed for specific combinations of:
The same six safety measures are used as in the CG evaluation, because negative binomial regression relationships, needed for the CG and EB evaluations, could not be developed for the safety measures based on project-related fatal and injury accidents. Several other cases also had to be omitted from the EB analysis because of unsatisfactory models. The EB approach, using the negative binomial regression relationship in place of the comparison sites used in the YC and CG approaches, was executed as described above. The sample sizes for the 108 evaluations ranged from 1 to 39 for intersections and from 2 to 148 for intersection approaches. Statistically significant effectiveness measures were found for 48 of the 108 evaluations performed. The results of the EB evaluations are presented in table C-17 through C-22 in appendix C of this report and are discussed in section 6. |
![Figure 1: Diagram. [Target Area for Evaluation of an Intersection with a Right-Turn Lane Added on One Approach.] This diagram shows a four-leg intersection with an area, labeled as the target area, shaded gray including one major-road approach, the right-turn lane on that approach, and the departing roadway on the minor road.](images/section5Image1.gif)
![Figure 2: Diagram. [Target Area for Evaluation of an Intersection with Right-Turn Lanes Added on Two Approaches.] This drawing shows a four-leg intersection with two areas, both labeled as the target area, shaded gray including both major-road approaches, the right-turn lanes on those approaches, and the departing roadways on the minor road.](images/section5Image2.gif)
![Figure 3: Diagram. [Target Area for Evaluation of an Intersection with a Left-Turn Lane Added on One Approach.] This is a drawing of a four-leg intersection showing two areas shaded in gray. One area, labeled as the primary target area, includes the major-road approach on which the left-turn lane is located, the left-turn lane itself, the departing roadway on the minor road, and the approach roadway used by opposing through vehicles on the major road. The other area, labeled as the secondary target area, includes both minor-road approaches to the intersection.](images/section5Image3.gif)
![Figure 3: Equation. [Name of equation.] Pi hat sub I equals the quantity of K sub I times the quotient of N sub I divided by M sub I](images/section5Image5.gif){kind=small}
(3)![Figure 4: Equation. [Name of equation.] Lambda hat sub I equals L sub I](images/section5Image6.gif){kind=small}
(4)![Figure 5: Equation. [Name of equation.] Theta hat sub I equals the quotient of lambda hat sub I divided by Pi hat sub I, which in turn equals the quotient of L sub I divided by Pi hat sub I](images/section5Image7.gif){kind=small}
(5)![Figure 6: Equation. [Name of equation.] Theta hat sub I equals the quotient of lambda hat sub I divided by Pi hat sub I, which in turn equals the quantity of L sub I times M sub I, that is then divided by the quantity K sub I times N sub I](images/section5Image8.gif){kind=small}
(6)![Figure 7: Equation. [Name of equation.] R sub I equals the natural logarithm of the quotient for the quantity of L sub I times M sub I divided by the quantity K sub I times N sub I, which in turn is equal to the natural logarithm of theta hat sub I](images/section5Image9.gif){kind=small}
(7)![Figure 8: Equation. [Name of equation.] R sub I times SE squared equals the reciprocal of K sub I plus the reciprocal of L sub I plus the reciprocal of M sub I plus the reciprocal of N sub I](images/section5Image10.gif){kind=small}
(8)![Figure 4: Diagram. [Plot of Observed vs. Expected Accident Frequencies.] The figure shows a plot in which the horizontal axis represents expected accidents in the period after improvement with a range from 0 to 70 accidents; the vertical accidents represents observed accidents in the period after improvement with a range from 0 to 70 accidents. A diagonal line crosses the plot from lower left to upper right and points are scattered in approximately equal proportions above and below the diagonal line.](images/section5Image11.gif)
![Figure 9: Equation. [Name of equation.] W sub I equals the reciprocal of R sub I times SE squared](images/section5math9.gif){kind=small}
(9)![Figure 10: Equation. [Name of equation.] R sub mean equals the summation of the quantity W sub I times R sub I divided by the summation of W sub I](images/section5math10.gif){kind=small}
(10)![Figure 11: Equation. [Name of equation.] Theta sub mean equals E raised to the power of R sub mean](images/section5math11.gif){kind=small}
(11)![Figure 12: Equation. [Name of equation.] E sub mean equals 100 times the quantity theta sub mean minus 1](images/section5math12.gif){kind=small}
(12)![Figure 13: Equation. [Name of equation.] R sub mean times SE equals 1 divided by the square root of the summation of W sub I](images/section5math13.gif){kind=small}
(13)![Figure 14: Equation. [Name of equation.] Z equals R sub mean divided R sub mean times SE](images/section5math14.gif){kind=small}
(14)![Figure 15: Equation. [Name of equation.] R sub mean times upper equals the sum of R sub mean plus 1.96 times R sub mean times SE](images/section5math15.gif){kind=small}
(15)![Figure 16: Equation. [Name of equation.] R sub mean times lower equals the difference of R sub mean minus 1.96 times R sub mean times SE](images/section5math16.gif){kind=small}
(16)![Figure 17: Equation. [Name of equation.] Theta sub mean times upper equals E raised to the power of L sub mean times upper](images/section5math17.gif){kind=small}
(17)![Figure 18: Equation. [Name of equation.] Theta sub mean times lower equals E raised to the power of L sub mean times lower](images/section5math18.gif){kind=small}
(18)![Figure 19: Equation. [Name of equation.] E sub mean times upper equals 100 times the quantity theta mean times upper minus 1](images/section5math19.gif){kind=small}
(19)![Figure 20: Equation. [Name of equation.] E sub mean times lower equals 100 times the quantity theta mean times lower minus 1](images/section5math20.gif){kind=small}
(20)![Figure 21: Equation. [Name of equation.] E sub mean times SE equals 100 times theta sub mean divided by the square root of the summation of W sub I](images/section5math21.gif){kind=small}
(21)![Figure 22: Equation. [Name of equation.] The traffic volume adjustment factor equals the quantity ADT sub AT divided by ADT sub BT , which is then divided bythe quantity ADT sub AC divided by ADT sub BC](images/section5math22.gif)
(22)![Figure 23: Equation. [Name of equation.] The duration adjustment factor equals the quotient of YEARS sub AT divided by YEARS sub BT, divided by the quotient of YEARS sub AC divided by YEARS sub BC](images/section5math23.gif)
(23)![Figure 24: Equation. [Name of equation.] Theta hat sub I equals the quantity of L sub I times M sub I divided by the quantity K sub I times N sub I times ADJ sub 1 times ADJ sub 2](images/section5math24.gif){kind=small}
(24)![Figure 5: Graph. [Typical Regression Relationship for Predicting Intersection Accident Frequency as a Function of Entering Traffic Volume.] The figure shows a generic plot in which the horizontal axis is labeled total entering volume (veh/day) and the vertical axis is labeled number of intersection accidents per year. There are no numerical scales. A curve is shown that begins at the origin and generally trends upward from left to right; the slope of the curve decreases as one moves further to the right. One point on the curve is labeled "before" and the height of the curve at that point is A. A point further to the right is labeled "after" and the height of the curve at that point is A plus B. An equation shown in the figure defines R sub TF as equal to the quantity A plus B divided by A.](images/section5Image12.gif)
![Figure 25: Equation. [Name of equation.] T sub C equals nu divided by mu](images/section5math25.gif){kind=small}
(25)![Figure 26: Equation. [Name of equation.] Pi equals R sub C times kappa](images/section5math26.gif){kind=small}
(26)![Figure 27: Equation. [Name of equation.] R sub T equals pi divided by kappa](images/section5math27.gif){kind=small}
(27)![Figure 28: Equation. [Name of equation.] R sub T equals R sub C or, equivalently, R sub C divided by R sub T equals 1](images/section5math28.gif){kind=small}
(28)![Figure 29: Equation. [Name of equation.] ADJ sub B equals function F with arguments MAJADT sub BT, MINADT sub BT, and STATE sub T times YEARS sub BT divided by the quantity function F with arguments MAJADT sub BC, MINADT sub BC, and STATE sub C times YEARS sub BC](images/section5math29.gif){kind=small}
(29)![Figure 30: Equation. [Name of equation.] ADJ sub A equals function F with arguments MAJADT sub AT, MINADT sub AT, and STATE sub T times YEARS sub AT divided by the quantity function F with arguments MAJADT sub AC, MINADT sub AC, and STATE sub C times YEARS sub AC](images/section5math30.gif){kind=small}
(30)![Figure 31: Equation. [Name of equation.] MADJ sub J equals M sub J times ADJ sub B](images/section5math31.gif){kind=small}
(31)![Figure 32: Equation. [Name of equation.] NADJ sub J equals N sub J times ADJ sub A](images/section5math32.gif){kind=small}
(32)![Figure 33: Equation. [Name of equation.] ADJ sub 3 equals function F with arguments MAJADT sub AT, MINADT sub AT, and STATE sub T divided by function F with arguments MAJADT sub BT, MINADT sub BT, and STATE sub T](images/section5math33.gif){kind=small}
(33)![Figure 34: Equation. [Name of equation.] L sub I equals L prime sub I divided by ADJ sub 3](images/section5math34.gif){kind=small}
(34)![Figure 6: Graph. [Comparison of Accident Experience for Treatment and Comparison Groups for Rural Unsignalized Intersections at Which Left-Turn Lanes Were Added.] The figure is a plot with a horizontal axis labeled Year and showing a range of years from 1987 to 1997. The vertical axis is labeled average number of accidents per intersection per year with a scale ranging from 0 to 2.5. There are two lines plotted, one above the other, that go up and down in random fashion, and display similar, but not identical, patterns. The upper line represents the treatment sites and the lower line represents the comparison sites.](images/section5Image13.gif)
![Figure 7: Graph. [Use of Regression Relationship in the EB Approach.] Figure 7 shows the same plot described in Figure 5. The horizontal axis is labeled total entering volume (veh/day) and the vertical axis is labeled number of intersection accidents per year. There are no numerical scales. A curve is shown that begins at the origin and generally trends upward from left to right; the slope of the curve decreases as one moves further to the right. Three points a shown along a vertical line representing a particular value of total entering volume. A point on the curve represents the expected accident count from the regression relationship represented by the curve. A solid dot above the curve represents the observed accident count. An X between the curve and the solid dot represents the expected account count from the EB approach.](images/section5Image14.gif)
![Figure 35: Equation. [Name of equation.] The expected value of kappa equals function F with arguments MAJADT sub BT, MINADT sub BT, and STATE sub T times YEARS sub BT](images/section5math35.gif){kind=small}
(35)![Figure 36: Equation. [Name of equation.] The expected value of kappa given K sub I equals the quantity alpha times the expected value of kappa sub I plus the quantity 1 minus alpha times K sub I](images/section5math36.gif){kind=small}
(36)![Figure 37: Equation. [Name of equation.] Alpha equals 1 divided by the quantity 1 plus the ratio of the variance of kappa sub I to the expected value of kappa sub I](images/section5math37.gif)
(37)![Figure 38: Equation. [Name of equation.] Alpha equals 1 divided by the quantity 1 plus D times the expected value of kappa sub I](images/section5math38.gif){kind=small}
(38)![Figure 39: Equation. [Name of equation.] The variance of the expected value of kappa given K sub I equals the quantity 1 minus alpha times the expected values of kappa given K sub I](images/section5math39.gif){kind=small}
(39)![Figure 40: Equation. [Name of equation.] R sub TF equals function F with arguments MAJADT sub AT, MINADT sub AT, and STATE sub T divided by function F with arguments MAJADT sub BT, MINADT sub BT, and STATE sub T](images/section5math40.gif){kind=small}
(40)![Figure 41: Equation. [Name of equation.] R sub D equals the quotient of YEARS sub AT divided by YEARS sub BT](images/section5math41.gif){kind=small}
(41)![Figure 42: Equation. [Name of equation.] Pi hat sub I equals the expected value of kappa given K sub I times R sub D times R sub TF](images/section5math42.gif){kind=small}
(42)![Figure 43: Equation. [Name of equation.] Theta hat sub I equals lambda hat sub I divided by pi hat sub I](images/section5math43.gif){kind=small}
(43)![Figure 44: Equation. [Name of equation.] Theta hat equals the summation of lambda hat sub I divided by the summation of pi hat sub I, which in turn equals lambda hat divided by pi hat](images/section5math44.gif)
(44)![Figure 45: Equation. [Name of equation.] Theta hat superscript star equals theta hat divided by the quantity 1 plus the ratio of the variance of pi hat to pi hat squared](images/section5math45.gif){kind=small}
(45)![Figure 46: Equation. [Name of equation.] The variance of lambda hat equals the summation of lambda hat sub I](images/section5math46.gif){kind=small}
(46)![Figure 47: Equation. [Name of equation.] The variance of pi hat equals the summation of the variance of the expected value of kappa given K sub I times R sub D squared times R sub TF squared](images/section5math47.gif){kind=small}
(47)![Figure 48: Equation. [Name of equation.] The variance of theta hat superscript star is approximately equal to the ratio of two quantities A and B squared. A equals the ratio of the variance of lambda hat to lambda squared plus the ratio of the variance of pi hat to pi squared. B equals 1 plus the ratio of the variance of pi hat to pi squared](images/section5math48.gif){kind=small}
(48)![Figure 49: Equation. [Name of equation.] E equals 100 times the quantity theta hat star minus 1](images/section5math49.gif){kind=small}
(49)