• Construction Noise
• Measurement
• Noise Barriers
• Noise Compatible Planning
• Regulation and Guidance
• Resources
• Tire Pavement Noise
• Traffic Noise Model
  • TNM 25th Anniversary
  • Consistency Test Suite
  • Old Versions
    • TNM Version 3.1
    • TNM Version 3.0
    • TNM 2.1
    • TNM 2.0
    • TNM 1.1
    • TNM 1.0b
    • TNM 1.0a
    • TNM 1.0
  • TNM 2.5
  • TNM Version 3.2
• Training

Validation Report

SECTION 7 DIFFERENCES BETWEEN TNM 3.0 PREDICTIONS AND MEASUREMENTS

The differences shown in the previous sections do not indicate which version is performing better with respect to measured data. In order to determine how well a given version performs, comparisons of measured and modeled data are required. Because measured data were not available for the CTS sites, only the sites described in Section 2 are included in these comparisons.

In the following sections, four sets of comparisons to measured data are performed using modeled results from: 1) TNM 2.5 with Average pavement, 2) TNM 3.0 with Average pavement, 3) TNM 2.5 with specific pavements, and 4) TNM 3.0 with specific pavements¹⁰. (Note that since the goal of this section is to evaluate performance and not to identify root causes, TNM 2.6 analyses is omitted.) Although other factors are confounded with pavement type, comparing predicted and measured data modeled with specific pavement type can provide additional insight by accounting for one known deviation between sites as modeled and actual measurements. Assuming that study sites are well represented by the REMELs database, one would expect sites that had PCC roadways to be under-predicted (at least near the source) and sites that had DGAC or OGAC to be over-predicted when Average pavement is used.

Section 7.1 compares the performance of TNM 2.5 and TNM 3.0 relative to measured data for all data analyzed. However, in order to determine if TNM 3.0 is performing better or worse for a specific type of site, a more detailed analysis is required. To this end, the performances of TNM 2.5 and TNM 3.0 relative to measured data were also compared, for sites with acoustically hard or soft ground, for sites with or without a barrier, and receivers at near, medium and far distances, and on a site-by-site basis. The results of these comparisons are summarized in Sections 7.2 through 7.5.

7.1 Comparisons for All Data

Figure 13 depicts the performance of TNM 2.5 and TNM 3.0 relative to measured data for all data analyzed. The formatting is similar to Figure 3, but the levels on the x-axis correspond to measured data instead of a second set of modeled data. The top left pane of Figure 13 shows TNM 2.5 with Average pavement compared to measured data, while the bottom left pane shows TNM 3.0 modeled with Average pavement compared to measured data. The right panes show TNM 2.5 and TNM 3.0 comparisons modeled with specific pavements. As in Figure 3, the colored data points (each site is presented in a different color) represent individual 5-minute model computations; the dashed line shows the first-order linear regression between the two datasets; the dotted lines indicate the 95-percent prediction interval for any new samples; and the solid black line indicates where all results would fall if the model predicted the measured results with perfect agreement. Note that in the upper left hand corner of graph several statistical parameters are presented: the number of samples, the coefficient of determination (r2), the root mean squared error (RMSE), the regression slope and intercept, the regression equation, and the average error. In the lower right-hand corner, a metadata summary is provided covering the number of sites, the presence of a barrier, receiver distances and heights, number of roadway lanes, pavement type, and temperature and wind conditions included in the analysis.

Figure 13, shows that TNM 2.5 (top two graphics) on average over predicts the measured data while TNM 3.0 (bottom two graphics) under predicts the measured data. This trend is consistent over the range of measured sound levels; the offset between the solid black line and dashed regression line is nearly constant. Modeling with specific pavement types does not change the overall picture greatly; the same general trends are visible in both sets of results. These results are very consistent with the comparisons in Section 5, with the main difference being increased variation due the measured data.

In order to better compare and quantitatively describe model performance, the statistical parameters presented in the upper left-hand corner of the graphics are replicated in Table 6 below. When comparing measured to modeled data, the average error statistic provides an indication of the overall average difference between measured and modeled results. Modeled with average pavement, TNM 2.5, on average over predicts these data by 0.53 dB, while TNM 3.0 under predicts these data by 0.53 dB. Modeled with specific pavements, the average error is shifted upward to 0.69 for TNM 2.5 and -0.36 for TNM 3.0; a result of modeling 5 of the 14 sites with PCC pavement. Because PCC typically has much higher sound pressure levels than Average pavement, while DGAC has only slightly lower sound pressure levels than Average pavement for the same traffic, one would expect an upward shift in these results modeled with specific pavements.

The slope for these regressions can indicate if the error changes with a change in the x-axis values (sound level). In all four cases, the slope coefficient is equal to or nearly equal to 1.0, indicating no change in model performance over the range of sound levels.

TNM 3.0 achieves a slightly better r2 values than TNM 2.5; however, the difference is quite small (r2 of 0.88 compared to 0.87 for Average pavement). TNM 3.0 has an RMSE of 2.85 while TNM 2.5 has an RMSE of 3.03. This indicates that the difference between TNM 3.0 predictions and measured data are slightly less random than the difference between TNM 2.5 predictions and measured data.

To further visualize these summary statistics, graphics in Figure 14 depict the 95% confidence intervals (CI) for the estimated slope and average error parameters. These graphics can show if differences between these values are statistically significant. Overlapping lines would indicate that the values are statistically similar, while non-overlapping lines would indicate that these values are not statistically similar. In Figure 15, the slopes of all four regression lines are statistically similar, while the average errors between 2.5 and 3.0 are not.

To determine if TNM 3.0 is performing better or worse for a specific type of site, a more detailed analysis may be required. To this end, the performances of TNM 2.5 and TNM 3.0 relative to measured data were compared for sub-sets of the data including:

• Sites with acoustically hard or soft ground (Section7.2)
• Sites with or without a barrier (Section 7.3)
• Measurements at near, medium and far distances (Section 7.4).

7.2 Comparisons for Sites with Similar Ground Type

Figure 15, Table 7 and Figure 16 show the agreement between predicted and measured results for all cases where acoustically soft ground was the primary ground type between source and receiver. The soft ground data are represented by the black data points; all other data (shown in the graph but not included in the regression) are represented in the background by colored data points. When modeled with Average pavement, TNM 2.5 has good prediction results for all statistics computed and the trend line almost directly coincides with the line representing a one-to-one relationship. The average difference is just 0.23 dB. Results show slightly less agreement when modeled with specific pavements; TNM 2.5 over predicts by 0.56 dB (average error) and the slope of the trend line is now greater than one, influenced by the upward shift in predicted levels for sites modeled with PCC pavement.

TNM 3.0 on average under predicts soft ground by 0.81 dB with Average pavement and by 0.47 dB with specific pavement. Again, the slope of the trend line for specific pavements is influenced by the upward shift in predictions due to sites modeled with PCC pavement.\

Figure 17, Table 8, and Figure 18 show the agreement between predicted and measured results for TNM 2.5 and 3.0 for all cases where hard ground was the primary ground type between source and receiver. As these trend lines are based on only two sites, conclusions drawn from these regressions could be amended in the presence of more data.

TNM 3.0 performs better for these hard ground cases, having smaller average errors and RMSE. On average both versions over predict noise levels, with over-predictions greater at farther distances¹¹. As expected, the over-prediction is slightly less when modeled with specific pavement, which is DGAC for both sites. However, while TNM 2.5 also over-predicts noise levels at near distances, TNM 3.0 has smaller average errors and RMSE at near distances and actually slightly under predicts when modeled with specific pavement. When modeled with specific pavement, the average error for TNM 3.0 is lowest, as the over-prediction at far distances and under-prediction at near distances result in an average of nearly zero. (This does not inherently represent better performance, merely a preferable alignment of errors.)

7.3 Comparisons for Barrier/Open Sites

Figure 19, Table 9, and Figure 20 show the agreement between predicted and measured results for TNM 2.5 and 3.0 for all open sites where no barrier is between source and receiver. On average, TNM 3.0 has nearly perfect agreement when modeled with Average pavement, while there is a slight under-prediction when modeled with specific pavements (most of the sites have DGAC as the specific pavement).

Figure 21, Table 10 and Figure 22 show the agreement between predicted and measured results for TNM 2.5 and 3.0 for all cases where there is a barrier between source and receiver. Both versions of TNM under predict the "with Barrier" cases on average. TNM 2.5 under predicts less than TNM 3.0 (average error -0.44 vs. -1.15). The under-prediction decreases with specific pavement, as most sites in this group had PCC pavements.

7.4 Comparisons for Measurements at Similar Distances

Distance effects are especially difficult to analyze in the aggregate because all possible confounding factors, including hard and soft ground types and barrier presence, tend to show up in each distance category. Therefore, interactions between distance and ground type, barrier presence, and pavement type may be present at all distances. Even so, it is still useful to consider how each model is performing at various distances.

Figure 23, Table 11, and Figure 24 show the agreement between predicted and measured results for TNM 2.5 and 3.0 for distances less than 125 feet between source and receiver. Here TNM 2.5 tends to slightly over predict levels (average error = 0.78 dB) while TNM 3.0 tends to slightly under predict levels (average error = -0.17 dB). TNM 3.0's RMSE is also about 0.5 dB smaller than TNM 2.5. Otherwise, most descriptors are very similar between the two versions for this distance.

Figure 25, Table 12 and Figure 26 show the agreement between predicted and measured results for TNM 2.5 and 3.0 from 125 to 500 feet between source and receiver. TNM 2.5 on average over predicts slightly (0.18 dB) while TNM 3.0 under predicts by about 1.21 dB.

Figure 27, Table 13, and Figure 28 show the agreement between predicted and measured results for TNM 2.5 and 3.0 for distances greater than 500 feet between source and receiver. TNM 2.5 on average with measured results (average error = 0.03) while TNM 3.0 (Average pavement) under predicts by about 0.78 dB. It should be noted that, because the spread of measured data for this last distance range is short, many of the regression statistics (r2, slope, intercept) may be exaggerated.

7.5 Variation by Site

Examining each site on its own is useful to see how well the predictions match for a specific set of conditions (compared to the aggregate groups discussed up to this point). Table 14 presents a summary of the average error statistics and site characteristics for TNM model runs for each site. In general, the trends for these individual sites will mirror trends for the groupings noted in the previous sections. For example, TNM 2.5 tends to over predict measured data while TNM 3.0 tends to under predict measured data; modeling with specific pavements increases overall levels when sites include PCC pavements; Graphics and model summary tables similar to those presented in previous sections are presented in Appendix H: Comparison of Modeled and Measured Results (Not Adjusted for Reference Microphone).

10 Here specific pavement refers to the pavement identified during the site scoping and is either DGAC, OGAC, or PCC.

11 Although other factors play a role, in general lower sound pressure levels indicate data collected far from the roadway and higher sound pressure levels indicate data collected nearer to the roadway. This interpretation is used whenever distance is not explicitly given.