Using Multi-Objective Optimization to Enhance Calibration of Performance Models in the Mechanistic-Empirical Pavement Design Guide

CHAPTER 6. SUMMARY OF FINDINGS AND RECOMMENDATIONS

SUMMARY OF FINDINGS

This report presented the results of an FHWA LTPP data analysis project with the objective of using multi-objective optimization to enhance calibration of the MEPDG performance models. The AASHTO recommended single-objective calibration approach was conducted on the MEPDG rutting models for new pavements using the Florida SPS-1 data and overlaid pavements using the Florida SPS-5 data. In the first alternative scenario for multi-objective calibration, SSE and standard deviation of prediction error were simultaneously minimized to reduce the bias and STE (increase model accuracy and precision) at the same time. In the second scenario, the SSE and STE in predicting permanent deformation of SPS-1 and FDOT APT pavement structures were used as separate objective functions to be minimized simultaneously.

Although there was no fundamental way to prove whether there was a theoretical conflict between the selected objective functions, the shape of the final nondominated front indicated that the selected objective functions conflicted with one another, and therefore, the application of a multi-objective optimization approach was justified. In the first multi-objective calibration scenario, the simultaneous minimization of bias and STE resulted in calibrated models that had higher precision (lower STE) and higher generalization capability (lower difference in bias between calibration and validation data), compared to the single-objective calibration. While this scenario was more successful in calibration of rutting models for overlaid pavements on Florida SPS-5 data, it did not result in desirable accuracy levels for rutting models on new pavements using Florida SPS-1 data. The results of the second multi-objective scenario demonstrated that incorporation of the disparate source of performance data (FDOT APT data) as a separate objective function has significantly improved the prediction accuracy, precision, and generalization capability of the calibrated rutting model on SPS-1 data.

The qualitative comparison of the calibrated models showed that using the multi-objective approach has resulted in predicted rutting distributions that are more similar in flatness (kurtosis) to the measured rutting distributions. However, the same was not true about skewness. The low goodness-of-fit indicator for scatterplots of predicted versus measured rutting in the case of all calibration approaches reveals that the MEPDG rutting models have an inherent lack of precision that might not be addressed with the calibration process. This is perhaps because the variability in pavement materials has not been captured in these models. The final selected calibration factors for rutting in unbound pavement layers (base and subgrade) were more reasonable in the multi-objective approach, compared to insignificant values achieved through single-objective calibration. Once again, this possibility of applying engineering judgment demonstrates the value of the multi-objective calibration in providing a final pool of solutions to choose from.

To combine the quantitative and qualitative success metrics of a performance prediction model, the measured and predicted rutting deterioration trends were examined. While the two-objective calibration on SPS-5 data had significantly improved the prediction of rutting deterioration rates compared to single-objective calibration, the multi-objective calibration results on SPS-1 did not exhibit the same quality. This investigation revealed that simply evaluating the bias and STE is not adequate for a comprehensive evaluation of performance prediction models. Therefore, it is recommended that the comprehensive comparison framework presented in this study be used when selecting suitable performance prediction models.

RECOMMENDATIONS FOR STATE HIGHWAY AGENCIES

Based on the findings of this study, there are several advantages in adopting a multi-objective calibration approach for pavement design within each State or local highway agency:

• The existing trial-and-error approach will be replaced with a systematic framework for calibration of AASHTOWare® Pavement ME Design software.
  
  
• Accuracy and precision of the models can be optimized at the same time.
  
  
• Multiple sources of performance data with disparate data collection protocols (i.e., LTPP, State PMS, APT, etc.) can be incorporated simultaneously, without the need for reconciliation among different data sources.
  
  
• Following the quantitative multi-objective optimization process, engineering judgment and qualitative criteria can be used to select the most reasonable calibration coefficients among the final pool of solutions.

The flowchart in figure 42 demonstrates a potential multi-objective calibration framework that highway agencies can adopt for enhanced performance models.

Source: FHWA.

Figure 42. Flowchart. Framework for implementation of multi-objective calibration.

Using this multi-objective calibration approach results in a final pool of tradeoff solutions. This way, none of the possible sets of calibration factors are eliminated prematurely, and all of the nondominated solutions are included in the final tradeoff front. Exploring the final front might reveal unknown aspects of this calibration problem and result in more reasonable calibration coefficients that could not be identified using single-objective approaches. This study demonstrates the application of engineering judgment and qualitative criteria to select reasonable calibration coefficients from the final pool of solutions that result from the multi-objective optimization. More reasonable calibration factors result in a more justifiable pavement design considering multiple aspects of pavement performance.

RECOMMENDATIONS FOR FURTHER RESEARCH AND VALIDATION

The following are recommendations for future research into enhancing the multi-objective calibration process:

• Other EAs and parameter settings could be tested to identify the most suitable one for this calibration application. Specifically, the stopping criteria need to be evaluated to make sure a good solution has been achieved. In addition, the precision (epsilon value) corresponding to each objective function needs to be examined to avoid additional computational cost that is beyond the required precision.
  
  
• Other multi-objective calibration scenarios could be considered in future research. For example, statistical distribution shape descriptors (skewness and kurtosis, which were used in this study as qualitative comparison criteria) can be used in conjunction with SSE and STE to better match the distribution of measured and calculated rutting data on pavement test sections.
  
  
• Other sources of data such as State PMS data could be incorporated into this multi-objective calibration framework to investigate its feasibility in that regard.
  
  
• The same multi-objective calibration framework can also be applied to other performance prediction models.