Appendix D
Evaluation of Alternative Methodologies
Identification of Alternatives for Refining Benefit-Cost Analysis
Integration of Performance Management and Needs Estimation
Reimagining the C&P Report
Over the past 47 years, the C&P report series has provided an objective assessment of current system conditions and future investment needs. Its target audience includes the U.S. Congress, all levels of government, policy makers and analysts, academia, transportation associations, industry, news media, and the public. It raises public awareness of the physical conditions, operational performance, and financing mechanisms of highways, bridges, and transit systems, and promotes an understanding of the importance of these transportation investments.
The C&P report is a dynamic and evolving product, which has periodically undergone substantial overhauls and improvements. A good example is the introduction of benefit-cost analysis (BCA) to the process for estimating future investment needs through application of the Highway Economic Requirements System (HERS) introduced in the 1995 C&P Report; the Transit Economic Requirements Model (TERM) introduced in the 1997 C&P Report; and the National Bridge Investment Analysis System (NBIAS) introduced in the 2002 C&P Report. These models are presented and described in Appendices A, B, and C, respectively.
As discussed in the Introduction to Part I of this report, MAP-21 (Moving Ahead for Progress in the 21st Century Act) incorporated performance management principles into its requirements. States will set targets for several key performance measures and report on their progress in meeting these targets. This shift toward more performance-driven and outcome-based programs has direct and indirect implications for the C&P report. At the most basic level, the introduction of other performance reporting requirements in MAP-21 might necessitate some content changes to the C&P report, both to take advantage of newly available data and to avoid unnecessary duplication of information presented elsewhere. The shift in the processes that States and metropolitan planning organizations (MPOs) use for planning and performance management also has implications for assessing future transportation investment needs. State and local agencies are adopting more outcome-based approaches to investment decision-making, which has significant implications for the potential impacts of future investment on system performance and how these impacts are simulated. In addition, the data, analytical tools, and techniques developed to support the implementation of MAP-21 could yield new approaches that can be adapted to refine or replace HERS, NBIAS, and TERM.
With these issues in mind, the Federal Highway Administration (FHWA) initiated the Reimagining the C&P Report in a Performance Management-Based World effort in late 2012. Preliminary scoping work was conducted in 2013 to document who uses the C&P report, to assess the utility of the report to FHWA program offices in communicating key information, and to identify options for presenting information more effectively. This effort identified two areas of potential improvement to align better with performance measures: methodology and communication. Two major research projects were initiated in 2014, with the objectives of improving estimation methodologies to compute investment needs and enhancing communication approaches, respectively.
Methodology Improvement
Simulation modeling inherently involves compromises, as the desire for detailed, reliable predictions must be balanced against data collection burdens and computational tractability. The tools and methodologies currently used in the C&P report reflect several analytical shortcuts and simplifications introduced to accomplish the desired analysis with the available data and resources. Since the initial introduction of these tools, a new generation of analytical tools and models has been developed that provide advanced methodologies in asset management and performance management.
HERS, NBIAS, and TERM have been constantly revised and updated to incorporate newly developed data and tools. Building on this ongoing improvement effort, a research project is currently underway to scan and compare methods for assessing investment needs and to propose new and improved methods for more precise and comprehensive needs estimation in the C&P report. Several analytical frameworks are being explored to identify potential alternative methodologies and upgrades to the current BCA approach. This project includes a systematic review of performance management tools that States and local governments currently use. The goal is to identify practical approaches for improving the C&P report methodology in the future.
Evaluation of Alternative Methodologies
The first stage of this research effort involves evaluating alternative methodologies that could be used to replace or supplement the BCA-driven tools currently used in the C&P report. Two specific decision methodologies that will be reviewed are the multi-criteria decision method (MCDM) and value for money.
MCDM allows for consideration of performance objectives that are difficult to monetize, and therefore MCDM frequently includes some performance measures that are not limited to monetary terms or condition matrices. It is a flexible tool, enabling the evaluation of projects based on multiple performance measures such as environmental sustainability, livability, and safety. MCDM is a viable potential method for enhancing a revised C&P report that is better aligned with MAP-21 and strategic goals. Its application, however, hinges on the selection of appropriate performance measures and assignment of weight to each performance measure, which could be challenging for national investment analysis.
As defined in the Eddington Transport Study of the United Kingdom,[1] value for money is another methodology that measures wider economic and reliability benefits. It assesses the economic, environmental, social, distributional, and fiscal impacts of an investment based on both quantitative, monetized information and qualitative information at the project level. Although this approach helps guide the modeling of reliability and economic impacts, scaling the findings from individual projects to the national system and obtaining a strategic allocation of resources for infrastructure investment could be challenging.
Other methodologies that could be studied include impact analysis tools that attempt to estimate the economic impacts of highway investments on the overall economy. Alternative methodologies for evaluating indirect user benefits not currently captured in HERS or NBIAS also might be explored.
Identification of Alternatives for Refining Benefit-Cost Analysis
The next stage in this research effort involves identifying alternatives for refining the current BCA approach to align with performance management principles. Two specific options under review are the potential for integrating needs analysis of pedestrian and cycling infrastructure into the C&P process and the feasibility of integrating network analysis into the C&P highway needs assessment.
Local and regional stakeholders are increasingly demanding consideration of active transportation modes (i.e., pedestrian and cycling) in needs assessment. Data availability issues have hampered such efforts in the past, but significant advances in recent years could make this option more feasible.
Although HERS currently incorporates some limited procedures for estimating network effects, the system is fundamentally a highway segment-level evaluation tool. Potential alternatives are the adoption of a more corridor-focused analysis process or a complete network analysis. The NPMRDS (National Performance Management Research Data Set) discussed in Chapter 5 might prove useful in identifying existing corridor conditions and in calibrating forecasting procedures.
Other potential enhancements that could be explored include options for estimating needs specific to freight movements and the direct integration of operations treatments and assets within the core procedures for highway investment analysis.
Integration of Performance Management and Needs Estimation
The next stage of this research effort will involve identifying existing State- and local-level tools that incorporate performance management principles, possibly leading to additional future refinements to the C&P report analytical procedures. Later stages will involve combining these findings with those identified in the assessments of BCA refinements and alternative decision methodologies. This combination would enable a detailed evaluation and comparison of several comprehensive approaches to upgrade the current national needs estimation process. Once appropriate analytical frameworks are identified, new components could be added to HERS and NBIAS or a new generation of analytical tools could replace these models.
Enhanced Communication
Currently, the C&P report is issued in paper form and the entire report is posted online using standard Adobe Acrobat and HTML formats. The look of the C&P report, however, has remained largely unchanged, despite the wide adoption by FHWA offices and several other government agencies of enhanced communication tools for presenting complex data. Preliminary scoping work conducted in 2013 revealed several basic concerns about the current approach.
Although the C&P report contains useful information and serves as a valuable reference document, its sheer size creates some problems for users. Because writing and reviewing the document is a lengthy process, the report is often transmitted to Congress after newer data have been published elsewhere. Even when this is not the case, many of the data in the biennial report and many of the data sets upon which the report relies are updated annually, which means that readers must often look elsewhere to find the latest available data.
One option under consideration is to develop a more robust website to complement the paper report. Under this approach, some of the more detailed, supplementary analyses currently presented in the report could be migrated to the website, allowing the paper version to focus on key findings. Such an approach also would facilitate more frequent data updates than are currently possible for the C&P report.
A research effort is underway to explore alternatives for enhancing the current report, focusing on data visualization and an interactive Web-based design. The underlying goal is to facilitate ease of use by a wider audience of readers and enable the alignment of performance-based information in the C&P report with the information obtained from State and MPO performance management processes.
Data Visualization
Data visualization is the representation of data in a pictorial or graphical format. It is the easiest way for the brain to receive and process large amounts of information quickly and intuitively. As part of this research effort, alternatives are being explored to improve the communication of data on both paper and the Web through advanced data visualization tools and infographics. For the paper version of the C&P report, new static graphics could be developed to help visualize complex information on highways, bridges, and transit that is easier to understand at a glance. Contents of each chapter could be condensed into a format that is more accessible to the public, such as bullet points, at-a-glance boxes, and content optimization for print layout.
For the online version, selected contents could be presented through interactive data visualization to convey information from in-depth and complex analytics. For example, an online platform might support the use of more dynamic and interactive graphics, such as customized dashboards and charts as the underlying data change according to the user's unique needs. Through the intuitive interfaces, data visualization tools enable customized analytical views with flexibility and ease by multiple users with diverse demands.
Web-Based User Interface
As part of this research effort, discussions are ongoing about how to upgrade the Web page of the report to inform, attract, and retain visitors through new methods of electronic communication. The goal of any Web page improvements is to combine good information architecture with the art of expression to guide users to contents grouped into appropriate categories. A new digital publishing platform could integrate traditional format like PDF with many interactive elements such as embedded video/audio and interactive graphs. To attract and maintain the attention of an increasingly mobile audience, an upgraded Web page could use a responsive Web design to accommodate data exploration and communication across myriad devices, including touchscreen and mobile devices.
Recognizing the current shifts in media and technology preferences, a suite of communication methods could be used to improve user experience via a highly interactive Web-based platform for the C&P report. Such a platform could enable users to extract information relevant for their specific purpose and produce customized data and reports for distribution. Functions of the website ultimately could be substantially expanded to support requests like search information, zoom in and out on maps, and sort and filter databases in real time.
A critical part of developing an enhanced future C&P report website is ensuring that it complements existing online resources and potential new resources coming on line in response to the MAP-21 State and MPO performance reporting requirements. In many cases, providing links to information posted in other locations might be sufficient so that limiting the content of C&P Web page focuses mainly on elements unique and central to the report.
Moving Forward
Although FHWA began the particular research initiatives described in this appendix, the Federal Transit Administration (FTA) as a full partner in the development of the C&P report is closely involved in these efforts. FTA has initiated its own reviews regarding future analytical approaches and report presentation and content. As potential enhancements become more fully refined through the current research efforts, external outreach will be conducted to ensure that any changes to the report content and structure will improve its utility for the members of Congress and other key readers. Although the objectives of the report will remain unchanged, the goal of this effort ultimately is to provide a multimodal product with cutting-edge analytics that improves user experience.
-
The Eddington Transport Study (2006). The case for action: Sir Rod Eddington's advice to Government. Available at http://webarchive.nationalarchives.gov.uk/20090104005813/http:/www.dft.gov.uk/about/strategy/transportstrategy/eddingtonstudy/. ↑