Probabilistic Optimization for Profit (Prob.O.Prof) Software

PDF Version (184 KB)

OBJECTIVE

This TechBrief describes probabilistic optimization for profit (Prob.O.Prof 2.0), a Microsoft Excel® spreadsheet-based software program developed to assist both pavement construction contractors and highway agencies. Contractors using Prob.O.Prof 2.0 can set target construction quality levels to optimize project quality, and highway agencies using the software can determine whether the quality assurance (QA) specifications encourage desirable levels of construction quality.

INTRODUCTION

Like all highway construction specifications, statistical QA specifications containing pay adjustment provisions for quality inform contractors which quality levels to target during construction. However, as these specifications are embedded in statistical terms that require an understanding of risks, making the best decision is seldom simple.

Under statistical QA specifications, contractors are provided with a wide range of target quality levels from which to choose, usually for three or more acceptance quality characteristics (AQCs) (i.e., strength, thickness, and smoothness for concrete pavements and asphalt content, density, and smoothness for asphalt pavements). From the contractors' viewpoint, the optimum combination of target quality levels is that which will maximize net pay (i.e., payment from pay equations minus costs) on the project.

The optimum target quality level for any one AQC is not necessarily the highest level of achievable quality. The quality control costs associated with achieving a high-quality level may outweigh any pay increase the agency will award. Therefore, contractors must take into account any maximum allowable upper limits, or caps, that the agency has placed on individual AQC pay factors and on the composite pay equation pay factor.

Additionally, there is no guarantee that the contractor who has targeted and subsequently achieved a high-quality level will be awarded the pay increase corresponding to that level. The acceptance tests used to determine the payment may result in a much lower quality estimate. The greater the number of AQCs, the more complex the assessment the contractor must perform to set target quality levels, making intuition and experience less helpful. Because there are many combinations of potential target quality levels, with each leading to unique pay adjustment and profit probability distributions, the decisionmaking process calls for a computerized probabilistic approach. Currently, most contractors use a deterministic approach that does not consider statistical acceptance risks. In the deterministic approach, it is assumed that the sampled estimated quality level will be the same as the true quality level. This assumption is seldom accurate, and its consequences can be costly.

Prob.O.Prof 2.0 was developed primarily to help highway agencies validate their specifications. By inputting typical AQC quality levels, estimated costs to achieve those quality levels, and certain acceptance plan details, highway agencies can establish the range that is most likely to contain the contractor's chosen combination of target quality levels. If an agency determines that the established range is not satisfactory (i.e., the target quality levels are either too high or too low), it should make adjustments to its acceptance plan system.

USING PROB.O.PROF 2.0

Prob.O.Prof 2.0 is a Microsoft Excel® spreadsheet-based software program that uses Microsoft Visual Basic® macros. To run the program, the user first selects the specific acceptance plan system to be analyzed included in the drop-down menu. Currently, the software includes the American Association of State Highway and Transportation Officials, Arizona, Iowa, and Wisconsin acceptance plan systems for portland cement concrete construction and the Alabama Superpave®, South Carolina Marshall, and Missouri stone matrix asphalt acceptance plan systems for hot-mix asphalt. If a particular agency's system is not included in the software but is similar to one of the systems that is included in the drop-down menu, the user can make minor changes to one or more default settings. However, some acceptance plan systems or revisions may be impossible to analyze through the current version of the software. Prob.O.Prof 2.0 can analyze only the portion of a system that determines pay and not the portion that enables agency verification of test results. As a result, it cannot fully analyze a system that relies on the use of contractor test results in the acceptance decision.

Single Lot Analysis

For each AQC, users enter the design/specified mean and standard deviation, the AQC target values to be analyzed, the relative costs of increased/decreased quality (in percentages), and the number of acceptance samples per lot (n). A screen shot from Prob.O.Prof 2.0 for Portland cement concrete strength is shown in figure 1. Users must also enter project information such as the number of lots and other specific information to control the analyses such as the desired reliability levels and the number of iterations to be performed.

The program uses a Monte Carlo simulation method to generate AQC test results representing samples taken in the field. Each iteration consists of generating the user-entered n acceptance test results for each target AQC, calculating the lot statistics (mean, standard deviation, etc.), and computing the lot payment. The lot payments for the various AQCs are then combined by using the agency pay equation or method. This sampling process is repeated as directed by the user for up to 5,000 iterations.

The simulation generates a probability distribution that is used to obtain the mean net gain, median net gain, and net gains at the chosen reliability levels for each combination of target AQCs for the analyzed lot.

Figure 1. Screen shot. Prob.O.Prof 2.0 input screen for concrete pavement strength.

Multiple Lot (Project) Analysis

For multiple lot construction, the probability distribution of the net gains obtained in the single lot simulation is used to simulate projects (as compared to simulating lots). The contractor's overall pay is generally closer to what it should actually be because positive and negative pay errors on individual lots tend to balance out. As a result, sampling risk is less of a factor in the contractor's strategy. A contractor might be willing to take a risk on a single lot but not on multiple lots.

Displays

The results of the simulation are displayed in Microsoft Excel® worksheets. For each unique combination of target AQCs, the relative cost (as compared with the zero incremental cost alternative) along with the lot payments and net gains or losses at four user-entered reliability levels are displayed in a tabular format for both single and multiple lot analyses. The output sheets also include graphical representations (cumulative probability charts and bar charts) of the simulation distribution for three user-selected AQC combinations as shown in figure 2 and figure 3.

Figure 2. Graph. Comparison of simulation distributions for three target combinations.

Figure 3. Graph. Simulation distribution for target combination 18.

If the user is a contractor, this output information will allow for the selection of an appropriate target AQC combination for the lot or project based on the contractor's risk tolerance. If the user is a highway agency, the agency's cost and reliability inputs will not be as precise as those of a contractor. As a result, the agency must establish a range of potential AQC quality levels that contractors are likely to target. Even if the agency inputs rough estimates based on typical State contractors, which lead to a wider range than input values based on more specific conditions, the established range should still provide useful insight regarding the quality levels the agency is encouraging its contractors to deliver.

SUMMARY

Prob.O.Prof 2.0 is a newly developed software tool that can be used by contractors and agencies to make business decisions regarding pavement quality.

A contractor can use the software during bid preparation to analyze an agency's specifications to form a bid strategy that considers the quality level to be achieved and the resulting net gain at that quality level. During construction operations, the same contractor can use Prob.O.Prof 2.0 to set revised lot-by-lot quality targets, multiple lot quality targets, or a combination of the two based on new information that was not available during bid preparation. Agencies can use the software to develop and/or validate QA specifications and pay adjustment provisions.

The software allows contractors to determine what target quality levels will lead to optimum quality in their specific situation. Additionally, agencies can determine how their quality level specifications encourage their contractors.

General Findings and Recommendations

During the research associated with the development of Prob.O.Prof 2.0, it became evident that many current QA acceptance plan systems are more complicated than they need to be. They are difficult to analyze even through computer simulation and often require analysis-enabling assumptions that are, at best, guesses. In particular, the use of contractor test results in the acceptance decision has complicated operating characteristic curve (statistical acceptance risk) analysis such that it requires one or more guesses regarding the degree to which the agency's test result population differs from that of specific or typical statewide contractors. A contractor first needs to guess the answer in order to receive the correct answer.

The importance of highway agencies and contractors being able to analyze acceptance plan systems cannot be overstressed. Agencies must know the statistical risks associated with the system it has developed and which quality levels the system encourages its contractors to deliver. However, for many agencies, this is not the case, and agencies generally do not understand the associated risks.

Prob.O.Prof 2.0 can improve the situation because it can be applied to some acceptance plan systems that are based on the average, the average absolute deviation, and the percent within limits (PWL) quality measures. All currently popular risk analysis software (e.g., OCPLOT and SPECRISK) were designed for PWL only.

The solution should not be to develop software to address all types of acceptance plan systems. Instead, a more far sighted solution would be for agencies to simplify their acceptance plan systems and make them capable of being analyzed. The simple acceptance plan systems would lead to greater uniformity among State specifications and QA practices.

The use of Prob.O.Prof 2.0 on simple acceptance plan systems capable of being analyzed would bring additional uniformity benefits. Extreme bids and low-quality issues would be minimized because States and their contractors would have the same understanding of specified quality levels.

Prob.O.Prof 2.0 Simulation Findings

Prob.O.Prof 2.0 analyses conducted in this research demonstrated the following:

• When there are more AQCs in an acceptance plan system, any one AQC becomes less important. Therefore, contractors may choose to target relatively low-quality levels for costly AQCs (e.g., thickness). An acceptance system with few AQCs would minimize the potential for such contractor manipulation.

• The composite pay equation has a large influence over a contractor's quality strategy. The optimum target value for a single AQC can be different when the entire multicharacteristic system is analyzed as opposed to when an individual AQC plan is analyzed. It is critical that users perform multicharacteristic system analyses.

• Simulations performed using Prob.O.Prof 2.0 under typical conditions illustrate that the mean net gain can be different from the median net gain. The smaller the sample size, the greater the difference. This encourages contractors to employ risk-taking strategies (as opposed to neutral, expected-value, mean-based strategies), advocating the use of large acceptance sample sizes that yield less variability in estimated quality.

• While estimated default cost values are provided in Prob.O.Prof 2.0, the use of slightly varied cost values can lead to different optimum targets. Contractors who maintain good construction cost and quality databases can have a significant economic advantage when inputting their own known values. Also, agencies that have developed and maintained good construction cost and quality databases can similarly benefit.