Guide on The Consistent Application of Traffic Analysis Tools and Methods

CHAPTER 6. UTILIZING OUTPUTS FROM DIFFERENT TOOLS

This chapter describes how the outputs from different analysis tool types can be combined into an overall presentation of technical analysis results.

PRINCIPLES FOR MANAGING CONSISTENCY IN OUTPUTS

The first principle that the analyst should observe is that any technical answer is the analyst's responsibility. The analyst owns the results, not the tool. The simulation model cannot say the average speed will be 35.4 mi/h. The analyst says so after extensively reviewing the results and comparing them to industry norms. It is the analyst that produces the answers, not the tool.

The second principle is that when confronted with differing results from two different tools, the analyst should place greater confidence in the results produced by the better tool (see chapter 5).

The third principle is that it is the job of the analyst to understand the capabilities and limitations for each tool and to treat results with caution in situations that fall on the fringes of the tool's application range.

Finally, inconsistencies cannot be eliminated; they must be managed and controlled. The natural flaws in any tool that attempts to approximate the real world ensures that no two tools will get precisely the same answer. The job of the analyst is to weigh the differing results and to deliver the best answer. The best answer is not necessarily either model A or model B; it could be something in between. If the manager and the analyst have been successful in ensuring consistency throughout the analysis process, then both models should be pointing to the same overall conclusions, only differing in minor numerical details.

RECONCILING AND UTILIZING OUTPUTS FROM DIFFERENT TOOLS

As described in the previous chapters, when two tools produce performance results for the same situation, if there is a clearly superior tool, then the analyst should rely on the results from the superior tool and neglect the results from the inferior tool.

The most common situation occurs when the analyst is using a demand model to develop forecasts and an HCM-type tool (or optimization tool or simulation model) to produce MOEs. In order to predict demand, the demand model needs to forecast performance, but it uses a very approximate method to predict facility performance. Obviously, the analyst will not report the demand model MOEs related to facility performance. However, since facility performance is an important determinant of demand, it is important that the demand model be working with estimates of facility performance that are similar to those estimated by the HCM-type tool. Steps to achieve this consistency were addressed in the previous chapters.

However, there are situations where the analyst will apply two tools in parallel, because neither tool provides the complete solution to a complex problem. An example is the application of both a simulation model and an HCM-type tool within the same study. This may occur because the analyst wishes to report level-of-service results for complex situations that can best be modeled in a simulation model. The analyst is trying to overcome the difficulty of communicating simulation results by using HCM levels of service, which decisionmakers are more familiar with.

To reconcile the MOEs produced by a simulation model and an HCM type tool, the analyst is faced with the following challenges:

• Differences in temporal periods covered by the two tools (HCM is peak 15-min period within peak hour; simulation models usually cover a much longer period).
• Definitional differences in the MOEs (delay, average speed, queue).
• Measurement differences for the MOEs.

For advice on the sources of inconsistency and methods for reducing them between simulation models and the HCM, the analyst should consult chapters 6, 7, and 24 of the 2010 HCM.⁽⁸⁾ The report for National Cooperative Highway Research Program (NCHRP) project 3-85, Guidance for the Use of Alternative Traffic Analysis Tools in Highway Capacity Analyses, is also a good reference.⁽⁹⁾ Generally, the analyst should be prepared to do significant data processing to convert simulation outputs into something that could be reported as HCM level of service. Even so, the analyst should be prepared to accept that there could still be residual inconsistencies because of fundamental differences in how the two tools model vehicle trajectories in the traffic stream.

EXPLAINING INCONSISTENT RESULTS

Despite the best efforts of the analyst and manager, there could still be apparent and real inconsistencies between tools, between different analysis stages, and among different agencies. In almost all cases, the best approach is to spot these inconsistencies before others do. That way, the manager and analyst can either modify their own analysis or prepare explanations for why the new study is better than the previous study.

There are differing sources of potential inconsistencies and differing ways to deal with them. Three potential sources of inconsistencies are as follows:

• Work the primary agency has performed.
• Work done by another agency in the same study area.
• Observations made by others, including decisionmakers and the public.

Work the Primary Agency Has Performed

It is, of course, best to control and manage any inconsistencies between analysis results and any work previously performed by an agency. A proactive approach to consistency in the scoping phase is the best deterrent to this issue (see chapter 2). However, there may be times where analysis results are inconsistent with prior analyses performed by the agency because the new results are better. This can occur for a number of reasons, including the following:

• Refined analysis tool type employed. The new analysis may have been conducted using a more detailed class of analysis tools, such as microsimulation, while the previous analysis was performed using deterministic methods. Or perhaps the opposite is true if a tool was used that is less sophisticated but more expedient to meet the needs of the current analysis.

• Revised input parameters. The previous analysis may have been conducted using input data that has subsequently changed by the time of the new analysis. Examples could include revised projections of socioeconomic data that serve as inputs to the travel demand model, changes to capital improvement projects expected to be included in baseline conditions, or extended future forecast years for the analysis.

• Updated software tools used within tool class. The new analysis may have been conducted using a more recent version of the analysis software than was employed in the previous analysis, reflecting refinements to the calculation procedures.

The appropriate response to inconsistencies between earlier and later studies performed by an agency is to proactively identify the inconsistencies before they are spotted by others and explain the evolutionary nature of the agency's traffic analysis. As better information and better tools become available, the agency naturally wants to present the best possible information to decisionmakers, stakeholders, and the general public.

Work Done by Another Agency in the Same Study Area

The analyst and manager should make themselves aware of prior and ongoing work by other public agencies and consultants in the area and proactively identify potential inconsistencies between the work of others and their own work. The key is to identify potential inconsistencies before the problem is brought to the agency's attention by others. The manager and the analyst can then determine whether to modify their own analysis or to prepare a rationale for the differences. It is not critical to be consistent if good reasons can be found for differences. Possible reasons include the following:

• Different starting points. The analysis results may vary from another jurisdiction's due to the use of different assumptions at various points in the traffic analysis sequence. This may include the use of different jurisdiction-specific methodologies or different travel demand models for forecasting traffic volumes.

• Different levels of detail depending on study purpose. The analysis may reflect a more detailed approach than was previously conducted, which may be related to the differing purposes of your study compared to one prepared by another agency.

• Reasons listed in previous section. Potentially, the same reasons that may lead to inconsistency with work an agency has conducted may also be valid for inconsistency between one analysis and that of another jurisdiction. These include the usage of revised analysis tool types, the use of revised input parameters, and the use of updated software tools within the same tool class.

Again, the goal is to highlight the differences and point to why the new analysis reflects the correct approach for the current study.

Observations Made by Decisionmakers or General Public

Commonly, someone's personal observations of the traffic characteristics of the study area, be it a decisionmaker or member of the general public, is compared and contrasted with analysis results and used to identify inconsistencies. These inconsistencies may only be perceived rather than actual, as they are based on limited observation.

The best advice is to make field visits to the study area during the time periods that correspond with those of the traffic analysis. This will inform work regarding validation of existing conditions analysis as well as allow confident professional interpretation of field conditions when responding to the observations made by others, boosting credibility along the way.

For any of these cases, anticipate the potential for inconsistency and prepare communication accordingly. This can be accomplished by summarizing and explaining the consistency issues, as well as anticipating questions and answering them in reports or presentations.

CASE STUDY—UTILIZATION OF OUTPUTS FROM DIFFERENT TOOLS

This hypothetical case study is a continuation of the HOV/HOT lanes project case study described previously. The manager is in the process of developing the overall PDAP and has identified the desired MOEs, tools, assumptions, and parameters for the various stages of the project. Now, the manager must identify how the outputs of the various analysis tools will be utilized. Table 7 provides an overview of how the outputs of the selected tools are utilized in each stage.

Table 7. Utilization of tool outputs for HOV/HOT lanes case study.

Notes: The selected system MOEs are VHD, average speed, mode split, and VMT by speed bin. The selected facility MOEs are delay/vehicle, level of service, planning time index, HOV volumes, transit patronage, collision rate during construction, and collision rate after open.

In the project needs and initiation stage, the primary tool is the travel demand model. At this stage, this tool is the only source of reported MOE results. The sketch planning model is used in a supporting role to determine how the demand model inputs should be modified in light of HOT lane considerations. The demand model forecasts for the facility provide the controlling total of SOVs using the facility. The sketch planning model allocates SOVs between the HOT lanes and the mixed-flow lanes.

In the project clearance stage, the demand model is the primary tool for predicting all demands and also for predicting MOEs at the system level. The demand model MOEs for facilities in the vicinity of the proposed project is compared to more precise HCM estimates and reconciled. Once this reconciliation check has been done, the demand model MOEs can be relied upon for the system performance results.

A combination of models are used for facility-level MOEs. For reporting freeway results, the freeway HCM tool (in this case, FREQ) is used. For reporting adjacent surface street results, the street optimization tool (in this case, Synchro) is used. Both tools overlap at the freeway ramps. On-ramp MOEs are taken from the freeway HCM tool. Off-ramp MOEs (which are controlled by the downstream signal) are taken from the street optimization tool.

The targeted simulation results for the HOT lane access points are used to modify the HCM inputs (e.g., capacities and speeds) for the access points. Thus, the simulation MOEs are used to modify the HCM inputs. The HCM freeway analysis tool can then be used to generate the freeway MOEs.

For the PS&E, construction, and operation stages, the same approach is used for utilizing tool outputs as described for the project clearance stage.