• Case Studies
• Newsletter
• Resources

Performance-based Planning and Programming - White Paper

3.0 Performance-based Planning and Programming

3.1 Common Elements of Performance-based Planning and Programming

While the three agencies described in Section 2.0 each have a unique approach to performance-based planning and programming, there are common elements from these and other examples that provide a framework for advancing performance-based planning and programming. The discussion below reflects these examples, the experience of other agencies in conducting performance-based planning and programming, and input from participants at the 2011 National Workshop held in Chicago, IL.

This section describes the common elements of an analytic approach to transportation planning and programming that represents a performance-based approach. These elements are rooted in a system or program level analysis that asks a basic question: what are the expected impacts of a set of strategies on future system performance? This analysis is a vital part of long-range planning and mid-range investment planning that State DOTs, MPOs, and transit agencies are increasingly using in statewide and metropolitan transportation planning to help guide future investments.

Setting Direction

Each of the examples cited in Section 2.0 began by identifying a strategic direction that consists of goals and objectives, and is supported by performance measures:

• Goals and objectives. A fundamental concept in performance-based planning and programming is that it be based on an agency's strategic direction. Agencies first need to determine their priorities and use these to shape the decisions they make about policies and investments.
• Performance measures. Any performance-based planning effort will necessarily include measures. Each of the examples described above define performance measures within their goals and objectives.

Long Range Planning

Most agencies that conduct performance-based planning and programming also conduct some type of strategy evaluation, though the specifics vary from one agency to another. The purpose of planning is to guide investment and policy decisions over a long time frame (10 to 20 years). For performance-based plans, there is an expectation that agencies describe expected performance outcomes for a package of strategies across several goal areas and use this information to help them determine how best to focus their resources. However, many agencies still struggle with quantifying future performance outcomes. The components of strategy evaluation include:

• Desired targets and trends. Before evaluating the package of strategies, many transportation agencies identify targets to provide direction to strategy evaluation. At this stage, targets and trends may be aspirational, though they would be expected to build on previous iterations of target setting and strategy analysis. The desired targets may include both specific numerical targets and/or desired trends. Targets help make individual goals and objectives concrete. Developing targets is a well known challenge and it may not be possible to develop realistic, achievable targets in all areas. Table 3.1 presents several targets Mn/DOT has established for its infrastructure preservation programs. Other agencies use combinations of arrows and colors or other qualitative methods to indicate the direction they are trying to achieve in a particular performance area.

Source: Performance-Based Planning and Programming at Mn/DOT, presentation by Tim Henkel, September 2011

• Strategy identification. Performance-based plans will need to indicate how State DOTs, MPOs, and transit agencies are working to improve performance in each goal – including policies, strategies, and investments. This step is not specific to performance-based planning and programming, but it is common to all performance-based planning efforts. Many complementary planning processes (SHSP, CMP, etc.) already identify strategies and it is important for performance-based plans to be consistent with and build on these other efforts. In addition, it is important to recognize that strategies may impact multiple goal areas and it will be important in reviewing strategies to ensure coordination across goals and programs.
• Strategy evaluation. Where possible, performance-based plans would evaluate expected future performance of packages of strategies at a system level, rather than at a project level. The Mn/DOT Highway Investment Plan described in Section 2.0 provides a detailed analysis of the expected performance outcomes. This information may be presented in quantitative or qualitative terms. In fact, Mn/DOT summarizes the performance expectations in qualitative terms (see Figure 2.3). Similarly, WMATA used a qualitative approach to present the impact different funding levels would have on competing agency priorities (see Figure 2.7). The purpose of the strategy evaluation is to relate how the set of strategies and investments proposed are expected to improve performance of each of the goal areas. One key aspect of the strategy evaluation conducted by many transportation agencies is an examination of system-level trade-offs. SEMCOG provides a clear example of this process, which looks at how performance is expected to change based on varying the distribution of funds to major program areas. Predicting specific performance outcomes based on a set of strategies currently remains outside of many agencies' capabilities for a variety of reasons (e.g., externalities, data availability, modeling constraints).

Programming

Resource allocation is the process of selecting specific investments in the transportation system. These investments are typically listed in an agency capital plan and/or in a STIP or TIP, which are federally-mandated documents that enable receipt of Federal funds. The S/TIP development process varies widely from region to region. In a performance-based framework, the resource allocation step links the system-level analysis of strategies described above to the programming of specific projects. There are three key elements of resource allocation:

• Investment plan. Increasingly, transportation agencies or individual functional units are developing overall investment plans for specific program areas that identify the funding needs and likely overall focus of a program given funding constraints. These plans help describe the tradeoffs that must be made across program areas and provide a system level understanding of the size and mix of investment in a given area.
• Programming. Short range programming documents, like S/TIPs, serve as a means to document the specific commitments made by a transportation agency. In a performance-based planning and programming approach, the specific project decisions are explicitly linked, one way or another, to the system level investment plan.
• Resource constrained targets and trends. While targets and trends are often set up front as aspirations, they are also commonly used to communicate what is possible and expected given the strategies and investments being made in the transportation system. At this stage, targets can present the expected future performance of the system.

The agencies described in Section 2.0 use a variety of methods to make the linkage between system performance and project programming, including the step that translates system performance analysis into specific projects. This takes:

• Consistency of outcome. SEMCOG tracks the consistency of the projects in their TIP with the investment levels identified in their long range plan. The long range plan identifies preferred funding levels for each major program. Over the course of the plan, SEMCOG maintains a page on their web site that tracks the consistency of actual projects programmed compared to the preferred levels.
• Consistency of measurement. WMATA has developed an analysis of proposed initiative that uses measures that are consistent with, though not exactly the same as, the measures used to evaluate the system in the Vital Signs report. Both sets of measures are based on WMATA's goals. Although WMATA's performance tools are used by different audiences (see Figure 2.6), the set of key performance indicators tracked in the Vital Signs Report permeate across all performance management efforts. In addition, all of the performance tools developed by WMATA are connected to agency goals.
• Hierarchical distribution. Mn/DOT first distributes funding to programs and districts using a performance-based formula and then prioritizes projects within those program areas.
• Corridor approach. Some agencies use a corridor-based approach that develops preferred investment strategies for major transportation corridors and then prioritizes and programs across those corridors based on various funding and other constraints.
• Optimization approach. Finally, some agencies use optimization techniques in some of their program areas to prioritize and program projects. These techniques identify not the best projects, but the projects that maximize performance of a full program of projects subject to a set of constraints (typically funding). These techniques are commonly used within bridge and pavement management systems to identify programs of projects that minimize the lifecycle cost of investments.

Each of these methods has performance-based aspects and, in practice, hybrids of these approaches are likely to be used to support resource allocation. For example, pavement and bridge management programs typically use a system optimization approach within their individual programs, while a corridor approach may be well suited to examine the range of capacity, transit service, highway operations, and certain safety improvements.

Fundamental to a performance-based approach is the recognition that agencies should first identify projects that are consistent with their goals and performance targets, and then determine the appropriate funding source for those projects. Unlike a traditional programming and budgeting process that identifies funding sources first, this approach first identifies the set of projects that best help the agency meet its goals or targets. The second step, then, is to identify what potential funding sources can be used to fund the preferred set of projects. Many Federal and State funding sources have some flexibility and can be used for multiple purposes or can be combined on projects to meet multiple goals.

Implementing

While the previous sections describe how performance measures are used in preparing plans and programs of future investment, another key element of performance-based planning and programming is tracking actual performance through ongoing reporting. Two types of tracking are relevant:

• Tracking trends in performance. As transportation agencies implement their plans and programs, it is important to monitor how performance is changing, especially in relationship to forecasted performance described above. This step provides a means to address both the success of all types of strategies (investment, policy, resource allocation) and potentially identify gaps in tools and data that may improve forecasts in the future. For most measures, because strategy implementation and travel behavior change relatively slowly, performance is tracked as a trend over several years.
• Comparing performance with targets and expected levels. In addition to tracking the actual performance, part of performance-based planning and programming is tracking performance against a set of targets and/or expectations described above.

Finally, any planning process includes evaluation of the types of information used and the elements of the process. This evaluation includes providing feedback to other elements of the overall process and improving how the planning process is used to make decisions.

3.2 Performance-based Planning and Programming Process

Table 3.2 summarizes the elements described previously. This framework represents a vision for the future of planning and programming. This information would evolve over time as tools, data, and processes improve.

Figure 3.1 presents how these elements are arrayed into the planning process. As suggested by participants at the 2011 National Workshop, the figure describes how performance management would enhance the established transportation planning and programming process. The figure builds off of the FHWA and FTP Transportation Planning Capacity Building Team's The Transportation Planning Process Key Issues: A Briefing Book for Transportation Decision Makers, Officials, and Staff. The figure identifies the steps of the federally required planning process, including:

• Setting vision, goals, and objectives to guide all planning efforts.
• Deriving performance measures that capture the fundamental outcomes of the agency's goals and objectives.
• Identifying strategies that are consistent with those identified in other planning efforts, including the CMP, SHSP, and other required and voluntary planning efforts (corridor plans, modal plans, and others).
• Evaluating and prioritizing packages of strategies through trade-off or scenario analysis.
• Using the results of the scenario analysis to guide setting resource constrained targets or trends for these measures and to develop preferred program investment levels as part of a long range transportation plan for the State or region covered by the plan.
• Conducting resource allocation exercises that translate an overall investment plan into a specific set of projects for a short range S/TIP or an agency's capital program.
• Project delivery and system operations that provide feedback into the planning process and provide data and information for performance measurement.
• Feedback of the decisions made and information used into early elements of the overall process.
• Ongoing coordination and collaboration with stakeholders and the public throughout the process.
• Data and tools used to support all aspects of planning, including the evaluation of strategies, development of targets, and allocation of resources to specific projects.

Figure 3.1 Performance-based Planning and Programming Process

Figure 3.2 graphically illustrates the types of information that might be included in a performance-based plan for a given performance measure. It includes:

• The performance trend (medium gray). This provides a basis for understanding how performance is changing over time using historic estimates of performance for a given measure. For some measures, State DOTs, MPOs, and transit agencies may have this information already; for others it would require multiple cycles to provide.
• A comparison of performance forecast to a target (dark gray). This would require being able to forecast the expected performance impacts of projects and/or investment levels. Is the gap between current performance and the projected target level closing? Or is current performance, assuming it meets a target, being sustained?
• Performance monitoring (light gray). Over time a State DOT, MPO, or transit agency would compare observed performance to forecasts provided in long range planning and to the performance target.

Figure 3.2 Example of Information Expected in a Performance-based Planning and Programming Process

For all of this information, it is anticipated that plans and programs would provide substantial narrative to explain the quantitative findings and document the policies and strategies being implemented in each area.

3.3 Other Issues

This section summarizes several key issues that were identified in the development of the framework, including:

• Implementation;
• The resources needed to support the framework;
• The relationship of the framework to various planning analysis time frames; and
• The varying roles for State DOTs, MPOs, and transit agencies within this framework.

Implementation

In addition to the elements described above in the framework, important implementation steps include:

• Project delivery. As plans and programs are developed, it becomes important to ensure that they are delivered. Having as much quality information as possible about project costs, risks, and other information during planning and programming is important to ensure that the set of projects that is selected can actually be delivered. From a practical standpoint, changing circumstances (e.g., funding availability, commodity price fluctuations, political factors) frequently result in changes to plans and programs, impacting the ability to achieve targets.
• Evaluation. As agencies develop polices and make investments in the transportation system, they should examine how policies and investments impact system performance. At the same time, national organizations - TRB, FHWA, FTP, AASHTO, AMPO, APTA, NADO, and others – should examine policy and investments across a range of contexts to develop new approaches and share best practices for improving system performance. A strong evaluation process will help refine strategies and improve the tools used to analyze strategies. Today, post-project implementation evaluation is not common practice.

Data and Tools

While not a specific element in a planning process, data and analytic tools are critical resources for performance-based planning and programming. Conducting system or project analysis will require a suite of approaches, tools and methodologies to help DOTs, MPOs, and transit agencies both measure current performance and predict future performance, given a set of strategies and investments.

Tools and data vary by performance area and agency type and size. Some performance areas have well established tools (e.g., pavement and bridge management systems) or significant data collection efforts (e.g., the Fatality Accident Reporting System, or FARS) that can help in measuring and evaluating expected future performance. Transit agencies operate numerous modes (e.g., heavy rail, bus, light rail, streetcar) in diverse conditions, making data collection and analysis more complex and less comparable across transit agencies. For a performance-based planning and programming approach to work, it needs to build on the success and information that is currently available, such as State DOT pavement and bridge management systems or transit asset management systems and planning processes like the SHSP and CMP. The range of approaches that may be acceptable will vary by the level of maturity, size, and other differences among State DOTs, MPOs, and transit agencies.

The ability to forecast future performance varies significantly depending on the type of strategy and investment. Additional capacity building will be needed to improve forecasting tools or to identify areas where a qualitative approach is more appropriate.

In the previous version of this effort, tools and data were identified as elements of the overall framework. At the suggestion of participants in the 2011 National Workshop, these have been moved to be identified as critical supporting resources for the overall framework, but not separate elements. Data and tools are required throughout the planning process, but are not themselves an element of the process.

Planning Analysis Time Frames

Actual delivery of performance-based planning and programming will likely occur across the range of planning efforts and will apply to several analysis periods, including:

• Long range. Transportation agencies typically examine how the State and/or region will respond to the trends and challenges facing the transportation system over a 20 or more year period. At this stage, performance-based planning and programming is focused on defining goals, objectives, and measures, and targets to track overall performance. Current Federal regulations require State DOTs and MPOs to develop long range transportation plans with at least a 20-year horizon, though the specific requirements differ substantially for MPOs and State DOTs.
• Mid range. The framework presented above suggests the need to examine the expected performance of investment in major State and metropolitan transportation program areas over time. Several State DOTs, transit agencies, and MPOs have been developing these types of programs, which often examine performance over a 10-year period. Current Federal regulation does not require planning analysis over this time horizon, though some agencies have begun to use it to support their planning and programming efforts.
• Short range. Over the short range, often a four to five year time horizon, project evaluation and prioritization guides the selection of a program of projects. This program should be linked back to the long range and investment planning steps described above. Current Federal regulations require State DOTs to develop STIPs and MPOs to develop TIPs.
• Ongoing. In addition to these three time frames, performance management programs generally include regular, likely annual, performance monitoring and reporting for tracking current system performance.

How these planning horizons are addressed within specific planning documents will vary from one agency to another.

Roles for State DOTs, MPOs, and Transit Agencies

While the framework presented above is meant to apply broadly to transportation planning and programming, some aspects of the framework may apply more readily to certain types of agencies than others. Specifically:

• MPOs have traditionally not focused on infrastructure issues or even on safety, focusing their efforts on congestion management and other issues. Because most MPOs do not own or operate infrastructure, they rely on State DOTs and transit agencies to provide information about infrastructure condition and investment levels. An increasing number of MPOs have started to use Federal tools such as the Highway Economic Requirements System-State (HERS-ST) and the National Bridge Investment Analysis System (NBIAS) to help examine scenarios for investment. Scenario analysis helps some MPOs plan for the long term and enables the evaluation of trade-offs across different types of investment (i.e., should more be invested in infrastructure condition or congestion mitigation?). MPOs play highly variable roles in regional planning depending in many cases on State statutory authority. In some regions, the State DOT, a regional transit agency or another entity may take the lead on scenario planning.
• Transit agencies traditionally participate in the metropolitan planning process through MPOs and in the statewide planning process through State DOTs, although significant planning efforts may occur at the transit agency or through another entity. FTP and others are actively developing tools and methods to improve how transit agencies track the condition of transit infrastructure (including rail, vehicles, facilities, etc.).
• Some specific goal areas touch multiple agencies that may not traditionally be included in the planning process. For example, safety investments and policy are managed by several agencies, including State DOTs, governors highway safety offices, and others. Transportation system operations includes State DOT and other transportation staff that are not typically involved in the planning process, as well as other groups (police, emergency response, tow truck operators, etc.) who are not regular participants in the planning process.

For any given agency, it may be appropriate to begin development of a performance-based planning and programming process by focusing on a subset of goals and expanding efforts over time. As described above, it will be important to coordinate with and build on work conducted through complementary planning processes (such as for strategic highway safety plans (SHSP)) because all units within a State DOT, MPO, or transit agency will have roles in the achievement of performance outcomes.