Enhancing Statistical Methodologies For Highway Safety Research – Impetus From FHWA

APPENDIX C. ASSESSMENT OF ISSUES IN CURRENT HIGHWAY SAFETY IMPROVEMENT EVALUATIONS

This appendix outlines concerns and issues in developing CMFs and SPFs. This list was provided to the technical experts prior to the meeting and any further insights directly gained from the meeting into these issues has been added as underlined text. Opportunities for potentially addressing many of these issues in the process of improving SPF and CMF estimation are also summarized.

Based on the presentations and discussions on the first day of the technical experts meeting, the research team prioritized these issues according to perceived potential for being addressed with statistical tools and processes presented. (Note that this does not imply priority of these issues but instead priority for the second day's discussion based on the tools and processes presented in the first day.) Three priority levels were assigned with level 1 being the highest. A brief summary of each issue is presented next, organized by priority level.

Priority 1

• Low sample means and sample size in SPF development.
• CMFs for rare crash types (e.g., pedestrians, motorcycle-involved).
• Reliability of CMFs inferred from cross-sectional regression models.
• Developing CMFunctions, including the estimation of the variance of a CMF estimated from a CMFunction.
• Use of prior knowledge in SPF or CMF estimations.
• Application of multiple CMFs.

Priority 2

• Isolating effects of individual treatments when treatment combinations are applied.
• Calculate variance of SPF*CMFs.

Priority 3

• CMFs for rare treatments.
• Defining Reference/Comparison group when treatment is universal.
• Assessing potential reference groups for before-after studies.
• Estimating required sample size for EB studies and cross-section studies.

Priority 1 Issue Summary

Low Sample Means and Small Sample Sizes in SPF Development

Data used for road safety research often have low sample means of crashes and/or a small sample size of locations. Research has shown that a low sample mean combined with a small sample size can seriously affect the goodness of fit statistics and the estimation of the overdispersion parameter, no matter which estimator is used within the estimation process. The probability the dispersion parameter becomes unreliably estimated increases significantly as the sample mean and sample size decrease. Are there more appropriate distributions than negative binomial, which may overcome these problems?

CMFs for Rare Crash Types

Some treatments target crash types that are rare in occurrence. For example, crashes between vehicles and pedestrians are typically severe but occur infrequently and spread out over many locations. Current evaluation methods are challenged to find reliable results with low numbers of crashes.

Reliability of CMFs Inferred From Cross-Sectional Regression Models

For some types of treatments there are few instances where a variable of interest is changed, for example, the radius of a horizontal curve. In these cases we rely on cross-sectional regression models to derive a CMF. The reliability of such CMFs is questionable due to omitted variable bias, correlated predictor variables and endogeneity. Tools are needed to deal with these issues.

Developing CMFunctions, Including the Estimation of the Variance of a CMF Estimated from a CMFunction

Techniques for developing CMFunctions are required. CMFunctions are equations relating the expected CMF for a specific site to its characteristics. The variances of the estimated CMFs also need to be estimated.

Use of Prior Knowledge in SPF or CMF Estimations

The development of SPFs and CMFs typically ignores prior knowledge. While full Bayes MCMC modeling has been used to some extent in road safety, prior knowledge is still typically ignored and uninformative priors is the norm. Methods for making use of prior knowledge are required.

The technical experts questioned why uninformative priors are used when previous information does exist.

Application of Multiple CMFs

When multiple CMFs are to be applied, common practice is to multiply the CMFs to estimate the combined effect when multiple countermeasures are implemented at one location. Currently, there is limited research to support the combination of CMFs for this purpose. Although implementing several countermeasures is likely more effective than implementing a single countermeasure, it is unlikely that the full effect of each countermeasure would be realized when implemented concurrently. This is particularly true if the countermeasures target the same crash type (e.g., installing lighting and enhancing pavement markings to address nighttime crashes). Therefore, unless the countermeasures act completely independently and target unique crash types, multiplying several CMFs is likely to overestimate the combined effect. The likelihood of overestimation increases with the number of CMFs that are multiplied.

Priority 2 Issue Summary

Isolating Effects of Individual Treatments When Treatment Combinations Are Applied

Often several treatments are applied simultaneously. For example, widening a shoulder and applying shoulder rumble strips at the same time. Methods for separating the effects of each individual treatment are needed.

Calculate Variance of SPF*CMFs

The current procedure for applying SPFs and CMFs together is to multiply the crash prediction of the SPF by all CMFs to be applied, which may come from various studies. Guidance is needed on how to estimate the variance of this estimate. The variance of the SPF prediction and variance of each CMF estimate should be known.

The technical experts suggested that the SPF*CMF approach is similar to a Cox Proportional Hazards Model and emphasized the statistical inaccuracies of multiplying multiple CMFs together and assuming independence. (See next issue.) The experts identified that the main effects in the HSM method are well established, but the interactions are unknown, which will derail the process. They reaffirmed the fact that the error in the prediction cannot be estimated within the current HSM model.

Priority 3 Issue Summary

CMFs for Rare Treatments

Some treatments are rarely implemented, particularly new and emerging treatments. Current evaluation methods are challenged to find reliable results with low numbers of sites.

Defining Reference/Comparison Group When Treatment Is Universal

Some treatments are universally implemented. For example, a city may implement pedestrian countdown signals at all signalized intersections in the course of a year. When this is the case there is no natural reference/comparison group.

Assessing Potential Reference Groups for EB Before-After Studies

The reference groups selected for EB before-after studies are selected so that they match the treatment sites as close as possible in all factors that may influence crashes. Aside from considering summary statistics of these variables and comparing the crash trends over time there are no tools for assessing and comparing the appropriateness of potential groups.

Estimating Required Sample Size for EB Studies and Cross-Section Studies

Methods for determining the required sample sizes for estimating CMFs from EB before-after studies and cross-sectional regression studies are lacking. For before-after studies we typically assume a comparison-group study for estimating required sample size.