• Federal Highway Administration >
• Publications >
• Research Publications >
• 03037 >
• 02b.Cfm >
• Validation of Accident Models for Intersections

Validation of Accident Models for Intersections

FHWA Contact: John Doremi,
HRDI-10, (202) 493-3052, John.doremi@dot.gov

PDF files can be viewed with the Acrobat® Reader®

2. VALIDATION OF ACCIDENT MODELS (Continuation)

Total Accident Model (TOTACC)

The models were recalibrated with the additional years of accident data. The parameter estimates, their standard errors, and p-values are provided in table 22. All of the variables were estimated with the same sign as in the original model, but the constant term and AADT2 were estimated with larger differences in magnitude than the other parameters. MEDWDTH1 and DRWY1 became insignificant for the recalibration with the additional years of data, and the overdispersion parameter, K, was almost twice as large as for the original model (for a discussion of K, see section 2.3).

Table 23 shows a comparison of GOF measures between the original main models in the Vogt report (1999) applied to the original 1993-95 data and to the 1996-97 data. The Pearson product-moment correlation coefficient was higher for the original years of data than for the additional years of data. The MAD per year was similar, but the MPB per year was larger for the model for the additional years of data. The MSPE per year squared for the additional years was higher than the MSE per year squared for the original years, but the difference was not great.

Table 22. Parameter Estimates for TOTACC Type III Model Using Additional Years of Data

Table 23. Validation Statistics for TOTACC Type III Model Using Additional Years of Data

¹ N/A: not available

Intersection Related Total Accident Model (TOTACCI)

The parameter estimates, their standard errors, and p-values are given in table 24. All of the variables were re-estimated with the same direction of effect as the original model, but the constant term and AADT2 were estimated with larger differences in magnitude. MEDWDTH1 and DRWY1 became statistically insignificant for the additional years data, and the overdispersion parameter was slightly higher than that for the original model.

Table 24. Parameter Estimates for TOTACCI Type III Model Using Additional Years of Data

Table 25 shows a comparison of GOF measures between the original main model in the Vogt report applied the original data and to the 1996-97 data.⁽²⁾

The Pearson product-moment correlation coefficient for the additional years of data was lower than that for the original years. The MAD per year was similar, but the MPB per year was slightly larger for the additional years of data. The MSPE per year squared for the additional years was higher than the MSE per year squared.

Table 25. Validation Statistics for TOTACCI Type III Model Using Additional Years of Data

¹ N/A: not available

Injury Accident Model (INJACC)

There were two variants of the original model for injury accidents. These were validated separately. The original injury accident counts were not obtained. Thus, a comparison of prediction performance measures for the original data and additional years could not be accomplished.

Variant 1

For the recalibration for the additional years of data, the parameter estimates, their standard errors, and p-values are given in table 26. All of the variables were estimated with the same sign as the original model, but that the constant term and AADT2 were estimated with larger differences in magnitude than the other parameters. HAU became insignificant for the additional years of data, and the overdispersion parameter was about half of that for the original model.

Table 26. Parameter Estimates for INJACC Type III Model Using Additional Years of Data: Variant 1

Variant 2

The parameter estimates, their standard errors, and p-values are given in table 28, which reveals that the variables AADT1, AADT2, HAU, and ABSGRD1 were estimated with the same sign as for the original model, but DRWY1 was estimated with an opposite sign, although its estimate was statistically insignificant. HAU and ABSGRD1 were also statistically insignificant for the additional years. AADT2 was estimated with a higher magnitude and level significance than it was for the original model. The overdispersion parameter was much higher than that for the original model.

Table 27. Validation Statistics for INJACC Type III Model Using Additional Years of Data: Variant 1

Table 28. Parameter Estimates for INJACC Type III Model UsingAdditional Years of Data: Variant 2

The Pearson product-moment correlation coefficient (0.38) was lower than that (0.54) for TOTACC. However, the MPB, MAD, and MSPE per year squared were smaller than those for TOTACC.

Table 29. Validation Statistics for INJACC Type III Model Using Additional Years of Data: Variant 2

2.4.4 Model IV

The summary statistics shown in table 30 reveal that the accident frequencies were lower during the additional years. Mean, median, and maximum of the accident frequencies were almost half of those for the 1993-1995 period. Recall that the original 1993-95 INJACC and INJACCI data were not obtained.

Table 30. Accident Summary Statistics of Type IV Sites

¹ Vogt, 1999, (p. 57)

Total Accident Model (TOTACC)

The parameter estimates, their standard errors, and p-values are shown in table 31, which reveals that the constant term and all of the variables were estimated with the original sign, but some parameters had somewhat large differences in the magnitude compared to the original parameter. The overdispersion parameters were similar.

Table 31. Parameter Estimates for TOTACC Type IV Model Using Additional Years of Data

Table 32 shows a comparison of the GOF measures for the original main model in the Vogt report and this model applied to the additional years of data.⁽²⁾ The Pearson product-moment correlation coefficients of the original and additional years of data are similar (0.56 versus 0.58). The MPB per year was slightly larger for the model based on additional years, while the MAD per year was similar. The MSPE per year squared was lower than the MSE per year squared, indicating that the model is performing fairly well on the additional years of data.

Table 32. Validation Statistics for TOTACC Type IV Model Using Additional Years of Data

¹ N/A: not available

Intersection Related Total Accident Model (TOTACCI)

The parameter estimates, their standard errors, and p-values are shown in table 33, which reveals that the parameters were estimated with the same sign as the original model, but that there were some differences in magnitude. In particular, the effect of the log of AADT2 on accident frequency was almost twice as large as for the original model. The overdispersion parameter was slightly lower than that for the original model.

Table 33. Parameter Estimates for TOTACCI Type IV Model Using Additional Years of Data

Table 34 shows a comparison of GOF measures between the original main model in the Vogt report and for this model applied to the 1996-97 data.⁽²⁾ The Pearson product-moment correlation coefficient for the additional years of data was estimated as slightly higher than the original years. The MAD per year was similar, but the MPB per year was somewhat larger for the additional years of data. The MSPE per year squared was again lower than the MSE per year squared of the original model, indicating a good fit to the additional years of data.

Table 34. Validation Statistics for TOTACCI Type IV Model Using Additional Years of Data

¹ N/A: not available

Injury Accident Model (INJACC)

Because the original injury accident counts were not obtained, a comparison of prediction performance measures for the original data and additional years could not be accomplished.

The parameter estimates, their standard errors, and p-values are given in table 35, which reveals that all of the variables were estimated with the same sign as the original model, but that the constant term and AADT1 were estimated with somewhat larger differences in magnitude than the other parameters. The overdispersion parameter was slightly lower than that for the original model.

Table 35. Parameter Estimates for INJACC Type IV Model Using Additional Years of Data

Table 36 shows the GOF measures for the original injury accident model (Variant 1) in the Vogt report applied to the additional years data.⁽²⁾ The Pearson product-moment correlation coefficient (0.48) was lower than that (0.58) for the TOTACC model. However, the MPB, MAD, and MSPE per year squared were smaller than those for TOTACC.

Intersection Related Injury Accident Model (INJACCI)

Since the original injury accident counts were not obtained, a comparison of prediction performance measures for the original data and additional years could not be accomplished.

The parameter estimates, their standard errors, and p-values are given in table 37, which reveals that the constant term and all of the variables were estimated with the same direction of effect as that in the original model. However, the constant term and AADT1 were estimated with somewhat larger differences in magnitude than the other parameters. The overdispersion parameter for the additional years was approximately half that of the original years.

Table 36. Validation Statistics for INJACC Type IV Model Using Additional Years of Data

Table 37. Parameter Estimates for INJACCI Type IV Model Using Additional Years of Data

Table 38 shows the GOF measures for the original intersection related injury accident model (Variant 1) in the Vogt report applied to the additional years data.⁽²⁾ The Pearson product-moment correlation coefficient was the same as that for the TOTACCI model. However, the MPB, MAD, and MSPE per year squared were smaller than those for the TOTACCI model.

Table 38. Validation Statistics for INJACCI Type IV Model Using Additional Years of Data

2.4.5 Model V

The summary statistics shown in table 39 indicate that fewer accidents per year occurred during the additional years (1996-97) than during the original years (1993-95). Again, note that the original years INJACC and INJACCI data were not obtained.

Table 39. Accident Summary Statistics of Type V Sites

¹ Vogt, 1999, (p. 61)

Total Accident Models (TOTACC)

The original report contained a main model and a variant, both of which were validated.

Main Model

The parameter estimates, their standard errors, and p-values are given in table 40, which reveals varying degrees of differences in magnitude and significance for the parameters for additional years data compared to those of the original model. VEICOM was estimated with an opposite sign to that of the original model, but was not statistically significant in the recalibration. The constant term and PKLEFT2 also became insignificant for the additional years of data. The overdispersion parameter was almost twice as large as for the original model.

Table 40. Parameter Estimates for TOTACC Type V Model Using Additional Years of Data: Main Model

Table 41 shows a comparison of GOF measures between the original main models in the Vogt report applied to the original data and the 1996-97 data.⁽²⁾

The Pearson product-moment correlation coefficient for the additional years of data was significantly lower than the original years. The MPB and MAD per year were larger for the additional years of data. The MSPE per year squared for the additional years of accident data was much higher than the MSE per year squared.

Table 41. Validation Statistics for TOTACC Type V Model Using Additional Years of Data: Main Model

¹ N/A: not available

Variant 1

The parameter estimates, their standard errors, and p-values are given in table 42. The constant term, PKLEFT2, VEICOM, PKTRUCK became insignificant for the additional years of data. VEICOM was estimated with an opposite sign to that for the original model, but it was not statistically significant in the recalibration. The overdispersion parameter was almost twice as large as for the original model.

Table 42. Parameter Estimates for TOTACC Type V Model Using Additional Years of Data: Variant 1

Table 43 shows the GOF measures for the original accident model (Variant 1) in the Vogt report and the model applied to the additional years of data.⁽²⁾ The Pearson product-moment correlation coefficient for the additional years of data was significantly lower than for the original years. The MPB and MAD per year were larger for the additional years of data. The MSPE per year squared was also higher than the MSE per year squared.

Table 43. Validation Statistics for TOTACC Type V Model Using Additional Years of Data: Variant 1

¹ N/A: not available

Intersection Related Total Accident Model (TOTACCI)

The main model and one variant were validated. Since the base model in the accident prediction algorithm is identical to Variant 3 of the Vogt model for TOTACCI, the Variant 3 model was also validated.

Main Model

The parameter estimates, their standard errors, and p-values are given in table 44, which again reveals differences in magnitude and significance in the parameter estimates. The constant term, PKLEFT2, and VEICOM became insignificant for the additional years of data, while VEICOM was estimated with an opposite sign to that of the original model, but was not statistically significant in the recalibration. The overdispersion parameter was almost twice as large as for the original model.

Table 45 shows a comparison of GOF measures between the original main model in the Vogt report for the original data and the original model applied to the 1996-97 data.⁽²⁾

The Pearson product-moment correlation coefficient for the additional years of data was significantly lower than for the original years. The MPB and MAD per year were larger for the additional years of data. The MSPE per year squared with the additional years of accident data was also higher than the MSE per year squared.

Table 44. Parameter Estimates for TOTACCI Type V Model Using Additional Years of Data: Main Model

Table 45. Validation Statistics for TOTACCI Type V Model Using Additional Years of Data: Main Model

¹ N/A: not available

Variant 3

The parameter estimates, their standard errors, and p-values are given in table 46, which reveals that most of the variables showed some differences in magnitude and significance for the additional years. VEICOM and DRWY1 were estimated with an opposite sign to those for the original model, but they were not statistically significant for the recalibrated model. AADT2 and PKLEFT2 also became insignificant for the additional years. The overdispersion parameter was almost twice as large as for the original model.

Table 46. Parameter Estimates for TOTACCI Type V Model Using Additional Years of Data: Variant 3

A comparison of GOF measures is given in table 47, which reveals that the Pearson product-moment correlation coefficient for the additional years of data, is significantly lower than that for the original years. The MPB and MAD per year were larger for the models based on additional years of data. The MSPE per year squared with the additional years data was almost twice as high as the MSE per year squared, suggesting a general lack-of-fit to the additional years of data.

Injury Accident Model (INJACC)

The parameter estimates, their standard errors, and p-values are given in table 48, which reveals that the variables AADT1*AADT2 and PKLEFT2 were estimated with a similar degree of magnitude and significance as the original model, but that the other variables showed larger differences in magnitude or significance. VEICOM was estimated with an opposite sign to that for the original years, and PKTRUCK became statistically insignificant, while PRO_LT turned out to be significant in the recalibration. The overdispersion parameter for the additional years was higher than that for the original years.

Table 47. Validation Statistics for TOTACCI Type V Model Using Additional Years of Data: Variant 3

¹ N/A: not available

Table 48. Parameter Estimates for INJACC Type V Model Using Additional Years of Data

Table 49 shows the GOF measures for the original injury accident model (Variant 1) in the Vogt report applied to the additional years of data.⁽²⁾

The Pearson product-moment correlation coefficient was similar to that for the TOTACC model. However, the MPB, MAD, and MSPE per year squared were significantly smaller than those for the TOTACC model.

Table 49. Validation Statistics for INJACC Type V Model Using Additional Years of Data

Intersection Related Total Injury Accident Model (INJACCI)

The parameter estimates, their standard errors, and p-values are given in table 50, which reveals that all of the variables were insignificant for the additional years. VEICOM was estimated with an opposite sign for the recalibration. The overdispersion parameter for the additional years of data was over twice as large as for the original years.

Table 50. Parameter Estimates for INJACCI Type V Model Using Additional Years of Data

Table 51 shows the GOF measures for the original intersection related injury accident model (Variant 1) in the Vogt report applied to the additional years of data.⁽²⁾ The Pearson product-moment correlation coefficient was similar to that for the TOTACCI model. However, the MPB, MAD, and MSPE per year squared were smaller than those for the TOTACCI model.

Table 51. Validation Statistics for INJACCI Type V Model Using Additional Years of Data

2.5 VALIDATION ACTIVITY 2: VALIDATION WITH GEORGIA DATA

For this validation activity, the models were used to predict accidents for the Georgia data that also were used to re-estimate the models. Data from 1996 and 1997 in Georgia were used for accident related variables; Other variables used, such as roadway geometrics and traffic volumes, were based on the 1997 road characteristic files maintained by the Georgia Department of Transportation and on data collected in the field during the summer of 2001. Recall that for Georgia data, two sets of accidents were extracted-those within 0.08 km (0.05 miles) of the intersection and those within 0.06 km (0.04 miles).

2.5.1 Model I

The summary statistics in the original report and for the Georgia data are given in table 52. The summary statistics reveal that Georgia sample had more accidents per year than the original Minnesota data. This difference in underlying safety may be explained by the fact that Georgia sites, for example, had, on average, higher values for the variables related to horizontal curvature, vertical curvature and roadside hazard rating, all of which increase accident risk according to indications from the original model.

Total Accident Model

The model was recalibrated with both sets of the Georgia accident data. The parameter estimates, their standard errors, and p-values are provided in table 53, which reveals differences in the parameter estimates between the two States.

HAZRAT1 was estimated with a similar degree of magnitude and significance as the original model. The constant term, AADT1, AADT2, VCI1, and SPD1 were estimated with the same sign but a larger difference in magnitude. HI1, HAU, and RT (for the 0.04 mile limit) were estimated with opposite signs and large differences in magnitude. The overdispersion parameter, K, was much smaller for the Georgia data.

Table 54 shows a comparison of validation measures between the original data and the Georgia data. The Pearson product-moment correlation coefficient was much higher for the original data as compared to Georgia. The MPB and mean absolute deviations are also higher than for the original Minnesota data. On a per year squared basis the mean squared prediction errors are much higher than the MSE indicating that the model is not performing well on the Georgia data.

Table 52. Summary of Georgia versus Minnesota Data for Type I Sites

¹ N/A: not available

Table 53. Parameter Estimates for Type I Total Accident Model Using Georgia Data

Table 54. Validation Statistics for Type I Total Accident Model Using Georgia Data

¹ N/A: not available

Figure 1 depicts the prediction performance of the original model for individual sites in the Georgia 0.05-mile data. It is quite evident that the original model failed to account for higher accident frequencies in most sites in the Georgia data.

Figure 1. Observed versus Predicted Accident Frequency: Total Accidents Type I

Injury Model

The injury model was recalibrated with both sets of the Georgia accident data. The parameter estimates, their standard errors, and p-values are provided in table 55, which reveals differences in the parameter estimates of the variables between the two States.

HI1, RT MAJ, HAU, and SPD1 (for the 0.05-mile buffer only) were estimated with the opposite sign. Aside from the AADT variables, none of the variables were estimated with satisfactory significance for the Georgia data. Perhaps this should not be surprising given that only two years of accident data were available and injury accidents are relatively few compared to total accidents. The overdispersion parameter, K, was estimated to be approximately one half of that for the original model.

Table 55. Parameter Estimates for Type I lnjury Accident Model Using Georgia Data

The validation measures for the Georgia data is shown in table 56. The Pearson product-moment correlation coefficients were quite low while the MAD was roughly half that for total accidents.

Figure 2 depicts the prediction performance of the original model for individual sites in the Georgia 0.05-mile data. It is quite evident that the original model failed to account for higher accident frequencies in most sites in the Georgia data.

Table 56. Validation Statistics for Type I Injury Accident Model Using Georgia Data

Figure 2. Observed versus Predicted Accident Frequency: Injury Accidents Type I

2.5.2 Model II

The summary statistics in the original report and Georgia data are given provided in table 57. The summary statistics again reveal that the Georgia sample had more accidents per year than the original data.

Table 57. Summary of Georgia versus Minnesota Data for Type II Sites

¹N/A: not available

Total Accident Model

The model was recalibrated with both sets of the Georgia accident data. The parameter estimates, their standard errors, and p-values are provided in table 58, which reveals differences in the parameter estimates of the variables between the two States.

The variables HI1, VCI1, and SPD1 were estimated with opposite signs while HAU was estimated to have no effect on safety for the Georgia data. The constant term and the other variables were estimated with the same sign but with varying differences in magnitude and significance. The overdispersion parameter, K, was estimated to be more than twice as large that of the original model.

Table 58. Parameter Estimates for Type II Total Accident Model Using Georgia Data

Table 59 shows a comparison of validation measures between the original data and the Georgia data. The Pearson product-moment correlation coefficient was much higher for the original data as compared to Georgia. The MPBs and MADs are also higher than for the original Minnesota data. The MSPEs are much higher than the MSE, indicating that the model is not performing well on the Georgia data.

Figure 3 depicts the prediction performance of the original model for individual sites in the Georgia 0.05-mile data. It is quite evident that the original model failed to account for higher accident frequencies in most sites in the Georgia data.

Table 59. Validation Statistics for Type II Total Accident Model Using Georgia Data

¹ N/A: not available

Figure 3. Observed versus Predicted Accident Frequency: Total Accidents Type II

Injury Accident Model

The parameter estimates, their standard errors, and p-values are provided in table 60, which reveals that the variables HI1, VCI1, SPD1, HAZRAT1, and HAU were estimated with the opposite signs. With the exception of the AADT variables none were estimated to be highly significant statistically. Only one Georgia model is shown since, as indicated in table 57, the observed number of accidents at each of the Type II sites was equal for the 0.04- and 0.05-mile buffers. The overdispersion parameter, K, was estimated to be over twice that of the original model.

Table 60. Parameter Estimates for Type II Injury Accident Model Using Georgia Data

The validation measures for the Georgia data are shown in table 61. The Pearson product-moment correlation coefficients were higher than for total accidents but still quite low. The MAD was roughly half that for total accidents.

Figure 4 depicts the prediction performance of the original model for individual sites in the Georgia 0.05-mile data. It is quite evident that the original model failed to account for higher accident frequencies in most sites in the Georgia data.

Table 61. Validation Statistics for Type II Injury Accident Model Using Georgia Data

¹ N/A: not available

Figure 4. Observed vs. Predicted Accident Frequency: Injury Accidents Type II. Graph.

Previous | Table of Contents | Next