Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

Office of Planning, Environment, & Realty (HEP)
Planning · Environment · Real Estate

HEP Events Guidance Publications Glossary Awards Contacts

Performance Based Planning and Programming Guidebook

11. Case Studies

This section includes case studies of a diverse set of agencies that are using elements of a performance-based planning and programming approach:

These examples highlight how State DOTs, MPOs of various sizes, and transit agencies can begin to incorporate a PBPP approach into their decisions, and how the various components of a PBPP approach - setting goals and objectives, selecting performance measures, setting targets, analyzing strategies, and evaluating results - tie together.

Minnesota Department of Transportation

The Minnesota Department of Transportation (MnDOT) integrates performance management into planning, programming, and project selection through a Family of Plans (Figure 4) that includes:

Figure 4. MnDOT Family of Plans

Title: Figure 4: MnDOT Family of Plans - Description: The MnDOT Family of Plans includes 4 main steps: 1. Policy Direction 2. Modal Investment Plans 3. Capital Programs/Operating Plans 4. Implementation< /> Policy Direction entails Minnesota GO, Long-range Vision, Guiding Principles, and Statewide Multimodal Transportation Plan. Policy Direction links to the Modal Investment Plans' Metropolitan, Regional, Tribal & Local Transportation Plans & Investment Programs. This program includes Satewide Freight & Passenger Rail Plan, Greater MN Transit Investment Plan, State Aviation System Plan, and State Highway Investment Plan. There are 3 componenets that link to the Capital Programs/Operating Plans' 4-year Statewide Transportation Improvement Program: 1. Port Development Assistance Program, Rail Service Improvement Program, Airports Capital Improvement Program 2. "District ATIPS" which entails "Years 1-4 of 10-yr Highway Investment Plan" 3. Metropolitan, Regional, Tribal & Local Transportation Plans & Investment Programs under the Modal Investment Plants. The Transportation Finance Advisory Committee links to the Policy Direction, and the Maintenance & Operations 4-year Plans under the Capital Programs/Operating Plans.The Advisory Committee includes - Freight System Plan, Freght Studies, IRC Evaluation, Intercity Bus Study, Transit Human Services Coordination Plans, Bicycle & Pedestrian Registry & Guidance, ADA Transition Plan, ITS Architecture Plan, Asset Management Plan, Strategic Highway Safety Plan, District Highway Safety Plans, and Highway Systems Operations Plan. Capital Programs/ Operating Plans lead into Implementation which includes Construction, Modal Programs, and Maintenance & Operations. Based on the results of the implementation, the Policy Direction is reevaluated and changed if necessary.

The 20-year Minnesota State Highway Investment Plan (MnSHIP)

The most mature application of performance-based decision-making within MnDOT's Family of Plans is the 20-year Minnesota State Highway Investment Plan (MnSHIP), which links SMTP policies and objectives to a set of fiscally constrained investment strategies that guide capital improvements on the state highway system. In the past, MnSHIP and its precursors have employed performance measures,

targets, and predictive models to express MnDOT's aspirational performance goals and describe the gap between desired and anticipated fiscally constrained outcomes in major investment areas. The next iteration of MnSHIP, due out the summer of 2013, is further advancing the use of performance management techniques through a scenario-based planning process that encourages stakeholders and decision-makers to consider cross-cutting risks and performance trade-offs when setting investment priorities.

Using performance information to set investment priorities

A key role of performance information in MnSHIP's scenario-based planning approach is to support the development of "performance levels." Similar to levels of service, MnDOT uses performance levels to signify a set of strategies, outcomes, and risks associated with a given level of investment in one of ten distinct investment categories. As proposed investment in a category increases, that category's performance level is adjusted upwards to represent the more costly set of strategies and better outcomes made possible with additional investment.

Figure 5 shows how the performance level concept is used to construct and describe alternative investment scenarios. Developed through the MnSHIP planning process, these fiscally constrained investment scenarios reflect spending priorities and performance outcomes associated with one or more broad objectives, such as maintaining existing infrastructure or improving mobility and advancing local priorities. While total investment is held constant, the level of investment in a specific investment category (represented by the performance level) varies depending on the extent to which the category contributes to a scenario's objective. The three scenarios displayed in Figure 5 and discussed below were developed for planning purposes only. In its adopted form, MnSHIP will articulate spending priorities and performance outcomes that reflect a blend of these three options.

Figure 5. MnSHIP investment scenarios (total capital investment over the 2014-2033 planning period, excluding the STIP)

Title: Figure 5: MnSHIP investment scenarios (total capital investment over the 2014-2033 planning period, excluding the STIP) - Description: A pie chart representation of 3 different MnSHIP Investment Scenarios. The pie chart has 5 segments - assest management (green), traveler safety (orange), critical connections (blue), regional and community improvement priorities (pink), and project support (grey). For all three approaches assest management covers the highest percentage of the pie chart. Approach A - Focus on maintaining existing infrastructure. About a half of the pie chart is highlighted to represent Pavement Condition (PL 2). This implies 2% interstate, 11% non-interstae PA, and 17% NPA is predicted to be in poor condition in 2033. Approach B - Continue 2012 spending policies. About a third of the pie chart is highlighted to represent Pavement Condition (PL1). This implies 2% interstate, 13% non-interstae PA, and 43% NPA is predicted to be in poor condition in 2033. Approach C - Focus on mobility for all modes and local priorities. About a quarter of the pie chart is highlighted to represent Pavement Condition (PL 0). This implies 2% interstate, 20% non-interstae PA, and 56% NPA is predicted to be in poor condition in 2033.

As Figure 5 demonstrates, investment in pavement condition is sufficient to achieve a performance level of "2" under a scenario in which MnDOT pursues the objective of maintaining existing infrastructure (Approach A). At this performance level, MnDOT would expect just 2% of the state's Interstates to be in Poor condition in the year 2033. However, even with nearly half of MnDOT's anticipated available revenue allocated to pavement condition, the percentage of the state's non-Interstate principal arterials (PA) and non-principal arterials (NPA) in Poor condition in 2033 is expected to be 11% and 17%, respectively. This represents a significant decline from current condition in Minnesota, where 4.3% of PAs and 7.5% of NPAs were in Poor condition in 2012. Under a scenario in which MnDOT pursues the objectives of improving mobility and advancing local priorities (Approach C), investment in pavement is sufficient to achieve only the minimum performance level of 0. At this performance level, Interstate condition is maintained at 2012 levels, but 20% of the state's non-Interstate PAs and 56% of its NPAs is expected to be in Poor condition by the end of the planning period.

Figure 5 omits information about the other nine investment categories for the sake of simplicity, but as with Pavement Condition, most of the other categories achieve higher performance levels in one scenario than in the others. In many cases, performance in these categories moves in the opposite direction of performance in Pavement Condition. For example, the investment category "Regional & Community Improvement Priorities" (represented by the pink slice) achieves a performance level of 0 under Approach A and a performance level of 3 under Approach C.

The development of alternative investment scenarios based on a range of possible performance levels has enabled MnDOT to facilitate a robust and informed discussion of anticipated performance in investment areas where aspirational targets are increasingly unachievable. This discussion has encouraged stakeholders to distinguish between outcomes that are sub-optimal but acceptable and those that pose a severe risk to critical objectives. In the context of a fiscally constrained plan, the development of performance levels has also helped to clarify the performance trade-offs that occur when investment is shifted from one category to another.

MnDOT uses the external and internal input obtained through discussions of MnSHIP investment scenarios to arrive at a combination of 10- and 20-year performance outcomes that reflect public priorities and manage key risks as effectively as possible. These outcomes constitute MnDOT's investment priorities and serve as the basis for resource allocation decisions.

Improving the linkage between planning, programming and project selection

Once MnSHIP has established a set of anticipated performance outcomes and category-specific spending levels, the next step is to plan improvements that will make these outcomes a reality. Figure 6 illustrates the role of performance in this process. Every year, MnDOT's eight districts develop draft 10-year plans of projects and programs using MnSHIP investment priorities and associated strategies as a guide. These draft plans are then evaluated using the same performance measures, targets, and predictive models that inform the establishment of statewide performance outcomes through MnSHIP. The purpose of this evaluation is to determine whether the projects and planned investments contained in the districts' 10-year plans will result in the same or similar results that were anticipated when MnDOT set its investment priorities. Draft 10-year plans that result in outcomes significantly above or below what is anticipated in MnSHIP are subject to additional scrutiny and review by senior leadership.

Figure 6. MnDOT's Annual Performance Management Cycle

Title: Figure 6: MnDOT's Annual Performance Management Cycle - Description: The first stage of the cycle is "MnSHIP establishes investment priorities." The second stage is "Districts create 10-year plan of projects & programs." The third stage is "Projects implemented annually through programming schedule."

An example of how anticipated performance outcomes are being used to guide MnDOT investment decisions is the Statewide Performance Program (SPP), which has been developed through MnSHIP in response to MAP-21 performance requirements. In its current form, MnSHIP allocates approximately 45% of total available revenue to the SPP (the remaining 55% is allocated to MnDOT districts for the purpose of managing key risks identified at the regional-level). Most of the investment allocated to the SPP is associated with 10-year statewide performance objectives that anticipate MAP-21 targets for Interstate pavement condition, non-Interstate NHS pavement condition, bridge condition, and congestion reduction. MnDOT's pavement and bridge specialty offices, in collaboration with MnDOT districts, used current condition information, deterioration curves, and predictive performance models to identify a list of improvements that will enable MnDOT to achieve these statewide objectives within the constraints of SPP funding.

At the project level, MnDOT uses performance measures, performance-driven investment strategies, and ongoing performance monitoring to identify eligible uses for resources that have been allocated to particular investment categories. As an example, MnDOT limits eligibility for investment out of the pavement and bridge investment categories to assets that have fallen below a specified condition threshold. In the case of the Traveler Safety investment category, MnDOT uses historical crash rate data relative to the statewide average to determine which intersections are eligible for moderate-to-high cost safety improvements.

Target setting: aspirational vs. risk-based targets

Historically, MnDOT has set aspirational targets designed to achieve optimal or desired performance levels in particular investment categories. These targets have typically been based on lowest life-cycle costs and/or customer expectations. Others have been trend-based - set by looking at trends and outcomes associated with historical spending levels. While MnDOT continues to use some of these targets to estimate its unconstrained investment needs, the current funding reality has made aspirational targets unachievable in most cases. As a result, MnDOT has moved toward a risk-based approach to target setting. Unlike many aspirational performance targets, MnDOT's risk-based performance targets are:

MnDOT's first experience with risk-based target setting occurred as a result of an agency risk assessment conducted in 2011. This risk assessment identified deteriorating pavement condition as the most serious problem facing the agency. It also established a range of condition levels at which MnDOT was willing to accept the risks associated with poor pavement. At the time, the percentage of the state highway system with Poor pavement condition exceeded statewide targets for principal arterials (actual: 3.7%; target 2.0%) and non-principal arterials (actual 6.8%; target 3.0%). Based on the range of acceptable condition levels identified through the risk assessment, MnDOT made a commitment to maintain the percentage of the entire system with Poor pavement condition between 5 and 9%.

Although the percentage of pavement in Poor condition in 2011 was only slightly over 5%, MnDOT's predictive performance models indicated that maintaining system condition within the 5-9% range would require significantly more pavement investment than was planned under existing policies. In response, MnDOT developed the Better Roads for a Better Minnesota program that allocated an additional $400 million between 2012 and 2015 to improve 750 miles of pavement in Poor condition. Unlike MnDOT's base pavement program which is programmed at the district level in consultation with MnDOT's local partners, Better Roads was centrally programmed using innovative engineering and delivery techniques that maximized its cost-effectiveness in terms of system performance. The successful implementation of Better Roads is expected to keep the percentage of the system in Poor condition within the 5-9% band throughout the 2013-2016 STIP.

Evaluation and Review

MnDOT has institutionalized several tools to ensure that the highway planning and programming process remains consistent with department goals and policies. The most important tool is the periodic review of system performance by senior leadership. This review results in frequent adjustments to MnDOT's 10-year capital program.

Another important tool is the publication of the Annual Minnesota Transportation Performance Report, which compares historic and anticipated performance to targets using a combination of symbols, colors, and contextual analysis (Figure 7). This report provides an opportunity to publically evaluate MnDOT's progress toward Statewide Multimodal Transportation Plan objectives. It also has proven to be an effective forum for describing the challenges facing the state transportation system and the rationale behind MnDOT's decision-making. Information in the performance report is frequently used to demonstrate the impact of specific funding requests in MnDOT's biennial budget request to the Minnesota State Legislature.

Figure 7. Annual Minnesota Transportation Performance Report

Title: Figure 7: Annual Minnesota Transportation Performance Report - Description: The scorecards uses symbols as a measure of score. A green circle represents "at or above target" A yellow triangle represents "moderately below target" A red hexagon represents "seriously below target" A Minnestoa Department of Transportation symbol represents "MMnDOT primarily responsible" A green line represents "target" The scorecard is divided into 6 columns - measure, score, results, targer, trend, and analysis. The "Measure" column includes traveler safety and infrastructure preservation. The colored symbols are assigned under score. The trend column has graphical representations. "Minnesota Traffic Fatalities: All State and local roads" is given a score of "at or above target" (green circle). The trends show a decrease in the number of fatalities since 2007. By 2011, there were 368 fatalities, the lowest number in a generation. "Bridge Condition: % Good and Satisfactory, State principal arterial" is also given a score of "at of above target" (green circle). There is a downward sloping trend, with the rates at 85.4% in 2011. This rate is close to the target of 84%. "Bridge Condition: % Poor, State principal arterial" is given a score of "moderately below target" (yellow triangle). the trends show an increase by 0.2%, bringing the rates to 3.3% in 2011. This value is higher than the target of 2%. "Pavement: Ride Quality Poor, All state highways, % of miles" is given a score of "at or above target" (green circle). This measure witnessed an increase from 2007. 6.6 percent of state highway miles were in poor condition in 2011.

Southeast Michigan Council of Governments

The Southeast Michigan Council of Governments (SEMCOG) is the MPO for the Detroit, Michigan urbanized area. When adopting the 2035 regional transportation plan, SEMCOG's elected officials approved a regional investment strategy with performance measures, targets and funding for achieving the targets.

Developing goals, objectives, and performance measures for SEMCOG was a collaborative effort with stakeholders. At a work session in March 2011, stakeholders were organized into 14 different breakout groups to brainstorm ideas. SEMCOG compiled and synthesized the suggestions from each breakout group to identify its six regional outcomes:

Performance measures were developed under each desired outcome. For example, under the "Reliable, quality infrastructure" outcome, performance measures included measures on pavement condition, failure rates, and service reliability. In the summer of 2011, SEMCOG published its final outcomes and performance measures (Goals and Objectives) and began to connect these with analysis of project needs. Figure 8 shows the primary performance measure for SEMCOG's program areas.

Figure 8. Sample of SEMCOG Performance Measures

Program Area Performance Measure
Pavement Preservation Percent of pavement in good or fair condition
Highway Capacity Hours of delay per 1,000 vehicle miles
Bridge Preservation Percent of bridges in good or fair condition
Safety Fatalities per 100 million vehicle miles
Transit Extent of transit network
Nonmotorized Population % within 1/2 mile of a facility
Roadway Operations Not currently addressed 

As part of its regional transportation planning efforts, SEMCOG has focused on providing information to decision makers to help them understand the consequences of different levels of investment. Actual project selection takes place at a later step, or during programming. SEMCOG's process focuses on making informed trade-offs between key program areas. The process begins by identifying key performance goals areas and developing metrics for each. Then, a variety of investment scenarios are developed. Then, the relationship between each investment scenario and future expected performance is analyzed. Performance of scenarios is compared and presented to decision makers. SEMCOG staff then work with decision makers to select a preferred alternative (or modify scenarios and begin the process anew).

SEMCOG potential funding scenarios organized around several key themes. These include:

Figure 9 summarizes the scenarios used, including both the distribution of funding and expected future performance for each program area. These scenarios were presented to decision makers, who used this information to identify preferred funding levels for each program.

Figure 9. SEMCOG Scenario Analysis

Title: Figure 9: SEMCOG Scenario Analysis - Description: The table provides information on 7 Program Areas - Transit, Pavement, Bridge, Expansion, Safety, Nonmotorized, and Roadway Operations. For each of there areas, the Current Allocation, Public Opinion, Preservation First, Transit First, and Maximum Performance is recorded. Each of these categories are sub-divided as 2030 Target and Funding Split. The values are provided in percentage. The table aslo provides the 2010 Performance for each of the Program Areas mentioned above.

SEMCOG leadership chose a modified scenario that emphasized maintenance and preservation. SEMCOG produces an annual list of deficiencies (e.g. pavement condition, safety, bridge condition, etc.), which are shared with member local government transportation departments. One purpose of these reports is to equip local governments with information so that they can apply for discretionary funding from state and federal sources (e.g., bridge funds, congestion mitigation air quality funds, safety/high risk rural road funds, etc.). Several tools were used for this analysis, including SEMCOG's travel demand model and national tools such as the Highway Economic Requirements System-State Version (HERS-ST) and the National Bridge Investment Analysis System (NBIAS).

Figure 10 shows the final funding splits and expected performance returns on the system in the SEMCOG region. The table compares the 2030 plan to the 2035 update, and projects performance returns for the 2035 plan. Note the relatively large funding split dedicated to transit and pavement, which are expected to return improved performance. However, these gains will be achieved by making tradeoffs. Program areas like operations and safety are expected to perform at about the same levels. Areas like highway capacity and bridge are expected to decline.

Figure 10. Funding Scenarios and Change in Performance

Title: Figure 10: Funding Scenarios and Change in Performance - Description: The information corresponding to each Program Area in the table is listed below, Program Area: Transit Current RTP Allocation - Funding Split: 21 - 2035 Performance: Current System Final Committee Allocation - Funding Split: 23% - 2035 Performance: Current System + 3 Rapid Transit Lines Change in Performance (Committee to Current Allocations): increase Program Area: Pavement Current RTP Allocation - Funding Split: 21% - 2035 Performance: 57% in good or fair condition Final Committee Allocation - Funding Split: 24% - 2035 Performance: 69% Change in Performance (Committee to Current Allocations): increase Program Area: Bridge Current RTP Allocation - Funding Split: 6% - 2035 Performance: 100% in good or fair condition Final Committee Allocation - Funding Split: 4% - 2035 Performance: 91% Change in Performance (Committee to Current Allocations): decrease Program Area: Highway Capacity Current RTP Allocation - Funding Split: 10% - 2035 Performance: 2.6 hours of delay per 1,000 VMT Final Committee Allocation - Funding Split: 8% - 2035 Performance: 2.8 Change in Performance (Committee to Current Allocations): decrease Program Area: Safety Current RTP Allocation - Funding Split: 0.5% - 2035 Performance: 0.74 fatalities per 100 million VMT Final Committee Allocation - Funding Split: 0.6% - 2035 Performance: 0.74 Change in Performance (Committee to Current Allocations): constant Program Area: Nonmotorized Current RTP Allocation - Funding Split: 0.5% - 2035 Performance: 44% of people and jobs within 1/2 mile Final Committee Allocation - Funding Split: 0.9% - 2035 Performance: 58% Change in Performance (Committee to Current Allocations): increase Program Area: Road Operations Current RTP Allocation - Funding Split: 41% - 2035 Performance: N/A Final Committee Allocation - Funding Split: 40% - 2035 Performance: N/A Change in Performance (Committee to Current Allocations): constant

A key step in SEMCOG's approach is to then examine the relationship between investment levels and future performance. Figure 11 presents a set of graphs used by SEMCOG to describe the relationship between investment and performance of pavement condition. The purpose of the graph is to show the percentage of roadway segments that will have pavement in good or fair condition over the time horizon of the plan. Four scenarios are presented, with each scenario having different splits for capital preventative maintenance (CPM), rehabilitation, and reconstruction. The color of the line corresponds to a dollar figure of available funding. Using this information, SEMCOG can see that a blend of all three maintenance strategies yield the best returns. Also, higher levels of investment provide a better outcome. These graphs were generated by AssetManager NT, which is available through the American Association of State Highway and Transportation Official's AASHTOWare program.

Figure 11. SEMCOG Investment Analysis

Title: Figure 11: SEMCOG Investment Analysis - Description: Four SEMCOG Pavement scenarios are graphically compared in this figure. The capital preventative maintenance (CPM), rehabilitation (Reh), and reconstruction (Rec) varies in each case. In each graph the x-axis represents the years from 2010 to 2030 and the y-axis represents percent pavement in good or fair condition. Each graph compares four different funding criteria - do nothing (red line), $200 million/year (blue line), $400 million/year (green line), and $600 million/year ( purple line). In all 4 cases the red line representing "do nothing" decreases to 0%. The four scenarios are: 1. CPM - 52%, Reh - 24%, Rec - 24% (State default) There is an increase in the green and purple line. The blue line stays constant. 2. CPM - 0%, Reh - 0%, Rec - 100% There is a decrease in all 4 criterias 4. CPM - 0%, Reh - 50%, Rec - 50% There is a decrease for all three criterias. 5. CPM - 33%, Reh - 33%, Rec - 33% (Compromise) There is an increase in the green and purple line, and a decrease in the blue line. This division represents the best scenario among the four.

red Do Nothing

blue $200 million/year

green $400 million/year

purple $600 million/year

SEMCOG develops an annual monitoring report of plan outcomes. Dubbed Creating Success, the report monitors and evaluates project implementation. SEMCOG is in the midst of developing a dashboard for our website but even now you can view on our website numerous performance measures that we track (see SEMCOG Community Profiles at http://www.semcog.org/).

To link their planning and programming efforts, SEMCOG evaluates funding by category between the LRTP and TIP. This allows the agency to compare investment level targets identified in the long range plan with the splits in funding for projects that are actually built. Figure 12 presents the tracking of investment levels through actual construction. This information is updated along with the TIP and is posted on the SEMCOG website (http://www.semcog.org/TIP_Consistency_Direction2035.aspx).

Figure 12. SEMCOG Transportation Improvement Program Investment Levels

Project Type Estimated Funding (in millions) Share of Funding TIP Share of Funding Direction2035
Pavement $520.9 14.5% 24%
Bridge $203.5 5.2% 5%
Road Expansion $138.1 3.5% 8%
Safety $25.4 0.6% 1%
Transit Capital $276.3 7.0% 8%
Nonmotorized Facilities $41.6 1.1% 1%
Operating $2,685.0 68.1% 53%
Total $3,940.8 100.0% 100.0%

Source: Southeast Michigan Council of Governments

Champaign Urbana Urbanized Area Transportation Study

The Champaign-Urbana Urbanized Area Transportation Study (CUUATS) is the MPO for the Champaign-Urbana, Illinois area and is part of the Champaign County Regional Planning Council. Over the past decade, CUUATS has adopted an increasing number of performance-based elements in its long-range plan and frequently updates information in a robust performance tracking and reporting system to measure its progress toward system-wide goals and objectives.

Background

CUUATS began to consider incorporating performance elements into its long range planning and more regularly tracking and reporting on performance during the development of its 2025 long range plan, which was adopted in 2004. Around that time, the MPO developed its first in-house travel demand model, which for the first time allowed CUUATS to quantify the impacts of its policies, such as VMT and congestion. New modeling capabilities in particular spurred the conversation among the MPO's staff about adopting more specific goals and objectives that could be associated with metrics. In 2009, the MPO adopted a 2035 transportation plan that got much closer to the desired level of detail and allowed CUUATS to set targets in a consensus-based process. Through effective communication with representatives of the MPO member agencies regarding and model's capabilities and limitations and with the financial support of IDOT, the MPO has been able to obtain funding to update the travel demand model and develop land use change, mobile source emissions, social cost of development and local affordability, and livability index models that allow development and tracking of new area-specific performance measures and facilitate better informed transportation planning.

Developing Goals, Objectives, and Performance Measures

Goals and objectives were developed by CUUATS staff and approved by the LRTP steering committee, based on public input, local knowledge, and best planning practices. During each LRTP update, goals and objectives are revisited. The current 2035 LRTP, completed in 2009, contains 12 goals, which are group according to the eight SAFETEA-LU planning factors.

Each goal contains objectives, and each objective has one or more associated "measures of effectiveness" (MOEs) that allow CUUATS to measure whether the objective has been met; these objectives meet FHWA's criteria to be considered SMART objectives (Specific, Measurable, Agreed, Realistic and Time-bound). CUUATS staff state that SMART MOEs help facilitate the creation of measures during the planning process because they provide an easily understandable guide on which member agencies and the public can agree. MOEs, both output- and outcome-based, are defined in the three areas covered by the plan: land use, environment, and transportation. Within transportation, measures are divided by mode. Some measures of effectiveness established in the 2009 LRTP had no previous data collection associated with them and thus were rated neutrally until trends could be established through regular data collection. Data for the MOE evaluation often comes from readily available sources and generally does not require complex calculations, but some data comes from CUUATS' models or through GIS analysis. The data is made readily understandable to facilitate the process of tracking performance by CUUATS, its member agencies, and the public.

Setting Targets

Once goals, objectives, and performance measures are identified and agreed upon, CUUATS sets targets for 5- or 10-year levels based on the MOE and type of objective. In order to set targets for the 2035 LRTP, the MPO's staff held a series of meetings with stakeholders to discuss each performance measure, the level of performance that could be expected based on funding levels and the specific projects that were expected to be delivered within the next 3-5 years. Discussions continued until a general consensus was reached on an appropriate target for each measure. In some cases, the MPO set realistic targets, while others were more aspirational, especially those for which data was less precise or available.

Linking Planning to Project Selection Decisions

CUUATS has worked over the past ten years to identify and implement ways to better link its planning and performance tracking efforts to the selection projects. In previous cycles, the MPO identified the goals and objectives that would be furthered by projects already in its TIP. In recent years, CUUATS has created a Project Priority Review Guidelines document based on the goals and objectives set for in its LRTP and the eight SAFETEA-LU planning factors. It plans to use these guidelines to identify and evaluate projects for receiving priority for federal funding in the future.

Monitoring, Evaluating, and Reporting on Performance

CUUATS publishes an annual "Report Card" that provides a summary of progress toward each target and background information that helps to provide context to performance results (see Figures 13-15 below for examples of the type of information in the Report Card). The document strikes a careful balance between being data-driven while also being user-friendly and not overwhelming the reader. The MPO has found that the Report Card has been a useful tool in communicating with the public, especially during public meetings related to the LRTP update process. By demonstrating the link between the public input it received in previous LRTP planning cycles and subsequent results, CUUATS has heightened attendance and participation at these meetings. In recent years, CUUATS has also used performance information to involve the bicycle community and demand response service providers, who now serve on the LRTP steering committee. In addition to heightening interest and participation in its long-range planning efforts, CUUATS has found that enhancing its modeling, usage of data, and performance reporting has improved its credibility among the community's highly technically-savvy university population. Whereas some community members and stakeholders had previously been critical of the MPO's data methods and sources, CUUATS now has a cooperative relationship with the university departments and professors that involves exchange of data and information that is factored into the MPO planning process. The MPO's member organizations (local governments) have also been able to use the Report Card to approach board members in their respective jurisdictions to demonstrate what was accomplished based on funding the jurisdictions invested in previous cycles. Demonstrating the funding-performance link has been useful for communicating needs and impacts, providing greater incentives for local governments to invest in transportation projects that address issues the public cares about.

Data considerations

CUUATS staff emphasize that there is a need to have easily accessible, long-term, and consistent data sets over time to help track performance in the region. Once the MOEs are established, coordination with the member agencies and the use of CUUATS' models are vital to the update of the LRTP Report Card each year. U.S. Census and American Community Service data is essential for establishing baseline data and data updates for a metropolitan region like Champaign-Urbana, which has a population of less than 200,000.

Figure 13. Example of Reporting on Performance for a Specific Measure of Effectiveness, Improved Roadways

Title: Figure 13: Example of Reporting on Performance for a Specific Measure of Effectiveness, Improved Roadways - Description: This MOE provided in the example received a positive rating because more than 50% of the roadwats improved were withing the municipal boundaries of a jurisdiction. The table is divided into 4 rows. The number represents each row, followed by the corresponding infromation in the table. 1. Goals - To the greatest extent possible, improvements will be made to the existing roadway network to preserve or improve upon its current condition and to add pedestrian, bicycle and transit facilities where needed. 2. Objective - Ensure that no more than 50% of all roadway improvements or construction projects, between 2009 and 2014, fall outside the 2009 municipal boundaries of Chamgpain, Urbana, Savoy, and Bondville. 3.MOE - Miles of new roadway constructed or improved inside municipal coundaries 4. Status - more than 50% of the roadways improved were within municipal boundaries - positive rating.

Figure 14. Summaries on CUUATS's Performance on 16 Measure of Effectiveness

Title: Figure 14: Summaries of CUUATS's Performance on 16 Measure of Effectiveness - Description: 1. Bicycle Facilities - increased by 21% 2. Accessible Pedestrian Signals - increased by 50% 3. Bus Routes & Ridership - 9.75% increase in ridership 4. Parcels Near Bus Routes - increase in coverage from 89.9% to 90/4% 5. Amtrak Ridership - increased by 24% 6. Willard Airport (negative rating) - projects loss in number of flights and emplanements.

Figure 15. Summaries on CUUATS's Performance on 16 Measure of Effectiveness (continued)

Title: Figures 15: Summaries on CUUATS's Performance on 16 Measure of Effectiveness (continued) - Description: This diagram is a continuation of the previous figure. 7.Residential Density - increase in residential density 8. Ozone (negative rating) - 8-hour annual ozone current standard crossed 9. Wetlands - 0.55% increase 10. New Roadways - 0.24% increase within municipal jurisdications 11. Improved roadways - increase of roadways in excellent and good conditions 12. Total Crashes - crashes per 100 M VMT decresed by 19.7% 13. Total Fatalieies (negative rating) - increased from 4 to 6 14. "A" Injuries (negative rating) - increased by 6% 15. Pedestrian Crashes - decrease by 5% 16. Bicycle Crashes - decreased by 26%

For more information about CUUATS's performance-based planning efforts, see http://www.ccrpc.org/transportation/. For more information on the CUUATS Modeling Analysis Package, see http://cuuats.org/models.

Washington Metropolitan Area Transit Authority

The Washington Metropolitan Area Transit Authority (WMATA), the public transit provider of bus, rail and paratransit services in the Washington, DC region, has established many of the performance-based planning and programming elements necessary to become a more strategic, accountable and transparent organization (see Figure 16).

Figure 16. WMATA's Performance-Based Planning and Programming Approach

Title: Figure 16: WMATA's Performance-Based Planning and Programming Approach - Description: There are 5 steps to the Performance-Based Planning and Programming Approach. 1. Goals, that is board adopted 2. Performance Measures, from the GM business plan 3. Targets, set by GM/Executive leadership team 4. This step branches out into two directions, a. Current operations b. Momentum 2025 The fourth step involves Business Plan Actions and Expansion Projects that require Resource Allocation, Base Operating + Capital Budget, and New Funding. 5. Evaluate results and manage risks "Driving Accountability & Decisions" leads the 5th step back to step 1,2, and 3.

By establishing a framework for setting goals, developing measures, determining targets, allocating resources and evaluating results, WMATA has delivered better, safer, more reliable service to customers. Comparing 2012 results to 2011, WMATA's performance improved in 10 of the 12 measures tracked in its monthly monitoring report. Keeping safety the agency's top priority led to improvements in both customer and employee injury rates. Both rail and bus on-time performance improved due to schedule adjustments to reflect actual travel times, real time monitoring and a more reliable fleet. Railcar reliability experienced a particularly notable improvement in the final months of 2012 after the root cause of door system failures was solved, resulting in 70% more miles delivered before a delay incident. Escalators, one of WMATA's highest profile assets, demonstrated that better maintenance pays off. WMATA hired 18 more mechanics in 2012, focused resources on stations with the lowest availability, and replaced units at two key stations to drive down average time to repair. The positive impact of a performance focus was also demonstrated by an increase in the customer commendation rate.

A key to WMATA's progress in becoming a performance-based organization was the establishment of a standalone Office of Performance in 2010. This office is dedicated to expanding the use of performance information to guide decisions, to promote WMATA's benefits in the region and to unify employees to accomplish agency goals. Since its inception, the Office has developed a range of performance tools that connect day-to-day work of WMATA's employees to agency goals. As illustrated in Figure 17, the types of performance tools vary by level of specificity, by audience and by usage. The Office of Performance has been the main driver of WMATA's adoption of performance-based planning and programming.

Figure 17. WMATA's Performance Tools

Title: Figure 17: WMATA's Performance Tools - Description: A pyramid divided into 3 section. The top triangle represents the "Public/Board," The middle section represents "General Manager/CEO" and the bottom section represents the "Departments/Offices." A double-sided arrow is placed along the left and right edge of the pyramid. The left arrow, starting from the bottom and up, is labelled "More Detailed," "Performance Measures," and "Less Detailed." The right arrow, starting from the top of the pyramid and down, is labelled "Tactical Focus" and "Strategic Focus." On peak of the pyramid there is a box labelled "Strategic Goals and Objectives." "Different Audiences Call for Differing Performance Tools" are key componenets to the Strategic Goals and Objectives. The Public Board are responsible for Performance Tools such as vital signs report, performance spotlights, and annual performannce report.The General Manager/CEO is responsible for GM/CEO Business Plan, 1-on-1s with the executives, and special performance analysis.The Deparments/Offices is responsible for departmental business plans, performace tracking and analysis, and cross-department collaboration.

Source: Performance-Based Planning and Programming: A Transit Perspective, presentation by Patricia Hendren, September 2011.

Setting Strategic Goals

In its most recent long range plan, Momentum: The Next Generation of Metro, the Board of Directors drove the development of WMATA's mission, vision and goals for building a transit system that supports a competitive region. The strategic plan presented in Momentum reflects thorough technical analyses, extensive outreach and feedback from regional stakeholders. Reflecting WMATA's broad reach across the region, the outreach plan was extensive and sought input from WMATA's customers, the general public, jurisdictional and federal funders, key regional civic organizations, WMATA's own employees, and stakeholders. Business and advocacy groups further extended the initiative's reach. WMATA's partners simultaneously joined the effort to promote maximum exposure, regional reach, and breadth of input. Through this process, WMATA identified four strategic goals:

  1. Build and Maintain a Premier Safety Culture and System
  2. Meet or Exceed Expectations by Consistently Delivering Quality Service
  3. Improve Regional Mobility and Connect Communities
  4. Ensure Financial Stability and Invest in our People and Assets

These four strategic goals define where WMATA wants to go and provides guidance for decisions across the agency. The "WMATA 2025" component of Momentum identifies seven pivotal investments in the capacity of the rail and bus systems to keep up with today's demands and continue to support the region's economic competitiveness and quality of life. Investments include running eight-car trains versus six-car trains during peak periods, core station capacity expansion, and improvements on 24 bus priority corridor networks which serve half of Metrobus ridership.

Developing Performance Measures

For each goal, the Office of Performance has worked with Departments across the agency to develop measures that demonstrate departmental contribution to these goals. In selecting performance measures, the Office considered the following criteria:

In addition, Office of Performance worked with the General Manager/Chief Executive (GM/CEO) Officer and Executive Leadership to select a small set of key performance indicators to which the Board of Directors holds the GM/CEO accountable (See Figure 18). Measures developed for Departments, Offices and the GM/CEO are annually evaluated to assess how well they are meeting the criteria listed above and when new data becomes available, new measures are considered.

Figure 18. CY2013 - 2015 GM/CEO Business Plan Measures

Title: Figure 18. CY2013 - 2015 GM/CEO Business Plan Measures - Description: The table in the figure has been extracted from the "General Manager/Chief Executive Officers Business Plan CY 2013-2015." The table is divided in to 2 columns labelled "Strategic Goal" and "GM/CEO Performance Measure." The number below corresponds to each row in the table. Each number is followed by a strategic goal and its performance measure. 1. Safety - Employee injury rate, customer injury rate, crime rate 2. Quality Service - Bus on-time performance, rail on-time performance, access on-time performance, escalator availability, customer commendation rate, customer complaint rate 3. Invest in People & Assets - Operating expenses on budget, capital funds expended, number of positions filled 4. Connect Communities - TBD

Determining Targets

On an annual basis, the Office of Performance facilitates a half day session with the GM/CEO and the Executive Leadership Team to determine targets for the key performance indicators. The executives review historical data trends, activities planned for the coming year, resource constraints, externalities that may impact results (e.g. major construction projects on bus routes) and performance results from peer agencies. Given that the audience for these targets is the Board of Directors and the public, WMATA strives for targets that push progress, but are realistic to attain. The table below demonstrates how the CY2013 escalator availability target was set. WMATA first calculated that the impact of planned replacement, rehabilitation and other scheduled work (e.g., jurisdictional inspections) would bring system availability down to 94.6%. If WMATA applied the average level of unscheduled maintenance over the past two years (8.25%), this would suggest a CY2013 target of 86.4%. Given recent maintenance improvements, the assumption was made that unscheduled work would continue to decline. In addition, lowering the escalator target from 89% to 86.4% did not seem feasible in the existing stakeholder climate so, the target was kept at 89%.

For Departments and Offices, targets are typically evaluated on an annual basis with assistance by the Office of Performance when requested.

Table 2. Historical Escalator Availability

 

CY11 Data

CY12 Data

CY13 Estimate

Max Escalator Availability

100%

100%

100%

Less Availability due to:

     

Scheduled replacements and rehabilitation

2.7%

3.2%

4.4%

Other scheduled maintenance

1.8%

1.0%

1.0%

Unscheduled maintenance

10.0%

6.5%

5.6%

Average Availability

85.5%

89.3%

89%

TARGET

89%

89%

89%

Resource Allocation

To guide the allocation of resources, WMATA uses business plans to link employees' day-to-day activities to agency goals. The business plans contain actions planned under a constrained budget that are necessary to make progress towards agency goals. In addition, specific measures and targets are set in each plan to evaluate and define success. An action owner is listed to clarify who is responsible for each action's execution and cross-agency dependencies are mapped out (see Figure 19). Combined, these business plan elements provide:

Over the years, WMATA has learned to keep these plans simple and to keep the focus on a small set of measures and actions. Business plans are intended to be the road map to improving performance.

WMATA has made several attempts to strengthen the linkage between budgetary decisions and agency goals. In 2009, WMATA used agency strategic goals to prioritize each capital project in the more than $11 billion FY2011 - 2020 Capital Needs Inventory. This enabled WMATA to present to the Board of Directors which capital needs should be funded first and which would need to be deferred beyond FY2020 at different funding levels. Using prioritization results, WMATA staff could clearly communicate the impact of funding constraints to its Board. For example, if WMATA's funding continued at recent levels (about $550 million per year) only 70% of WMATA's basic maintenance needs would be met and zero funding would be available to meet growing ridership or improve customers' experience. The funding agreement signed with the region in 2010 increased WMATA's annual capital budget to $800 million per year.

During WMATA's annual budgetary process, departments requesting unfunded initiatives are required to link the initiative back to an agency goal and describe how progress will manifest through a particular performance measure. The difficulty in quantifying outcomes for the various initiatives and internal resistance to using data to drive funding decisions has hindered efforts to tightly link initiatives to agency goals. However, each year brings the budget and strategic goals closer together.

Figure 19. WMATA Business Plans Link Day-to-Day Activities to Goals

What you Do

Strategic Goals

 

Business Plans

 

Benefits

Title: What you do - Description: A WMATA staff helps a customer determine the correct metro line to take.

 
right arrow 

A WMATA staff at work

Title: What you do - Description: A WMATA bus driver gives directions to a passenger.

Build and Maintain a Premier Safety Culture and System

Meet or Exceed Expectations by Consistently Delivering Quality Service

Improve Regional Mobility and Connect Communities

Ensure Financial Stability and Invest in our People and Assets

plus

Performance Measures

Targets

Actions

Who

Dependencies

equals

Performance IMPROVES

SHOW what you do

MOVE from reactive to strategic

FOSTER unity around goals

FOCUS staff and resources

ADVOCATE for support/resources

Evaluating Results

The public and Board of Directors receive frequent performance updates from WMATA through various means. The Office of Performance's Vital Signs Report covers a small number of key performance indicators (KPI's) to monitor long term progress in the strategic areas of safety, security, service reliability and customer satisfaction. A detailed performance analysis is presented in the Vital Signs Report through answers to two prime questions: Why did performance change? What actions are being taken to improve performance? WMATA is focused on these two questions to continually drive improvement. This report documents performance results and strives to hold WMATA's management accountable for what is working, what is not working, and why.

On its website, WMATA has published a "WMATA Scorecard," which provides information to the general public in a dashboard-style, user-friendly interface that allows users to click on icons for more detailed information about performance measures. The Office of Performance considers the Scorecard a supplemental tool and cautions that such simplified tools makes it difficult to convey the various factors that shift performance; thus, the agency hopes the Scorecard will pique readers' interest in the Vital Signs Report materials, which provide contextual information to better explain the results.

The Office of Performance also teams up with Departments to present "performance spotlights" at the request of the Board. Spotlights can take the form of Board presentations or memos and have addressed a wide range of issues including: escalator availability, bus and rail on-time performance, rail service standards, contractor vs. in-house maintenance results, effectiveness of customer satisfaction measures, use of "hot spot" data to reduce bus collisions, and impact of mid-day track maintenance on service delivery.

On an annual basis, the Board evaluates the GM/CEO based in part on progress made towards agency goals as documented by the results of the performance measures contained in the GM/CEO business plan. The GM/CEO in turn keeps the agency focused on the goals through monthly 1-on-1 meetings with each member of the Executive Leadership Team to discuss performance measure results and actions leading to results. The Vital Signs Report, Scorecard, performance spotlights and the GM/CEO annual evaluation demonstrate WMATA's commitment to being transparent and accountable to the Board of Directors, jurisdictional stakeholders and the public.

At WMATA, the consistent use of performance based management varies by Department, but the approach has been gaining acceptance as its value becomes more apparent. As an example, the Bus Maintenance department has a long history with performance management, including established of measures that clearly tie back to WMATA's goals, regular review of results, and discussions of how organizational actions are impacting these results. Similarly, the Metro Transit Police Department (MTPD) has established METROStat, a process for systematically collecting crime data and analyzing incidents to identify and prioritize trends and/or hot spot locations. METROStat results, assessed bi-weekly, serve to increase accountability within the MTPD, foster better cooperation and coordination across and between departments, and modify crime reduction approaches.

More recent initiatives in the departmental implementation of performance management include the Bus Transportation department's Superintendent Meetings, which are patterned after the GM/CEO's "1-on-1s." This approach engages departmental management to regularly review safety and service delivery results and assess actions taken. An initiative with a broader reach includes the 2013 release of an organizational asset management policy. This policy requires the use of life-cycle data to guide maintenance activities across the organization, something new to WMATA.

Even with these notable examples of evaluating performance results, there still remains a gap between the existence of data and the use of data at WMATA. For example, WMATA yard and shop personnel had used an application to manage the movement of rail cars in the yards. Only recently was the device enhanced to enable system rail managers to monitor car availability allowing them to manage fleet deployment on a system-wide basis, in order to have cars strategically located for balanced operations and delay response. Although this was an example of a successful translation of data into information, there remain many more examples of untapped data.

In a mature performance-based approach, the direction set by the GM/CEO through the measures, targets and actions in his/her business plan would cascade down to the Department level business plans and individual performance plans. At WMATA however, creating a strong tie between individual performance plans and agency measures and even strategic goals remains a challenge. As a result the evaluation of individual performance is separated from efforts to achieve agency goals.

Impact of a Performance-Based Approach: Moving from Reactive to Strategic

WMATA's transition to a performance-based organization has enabled the agency to move away from being primarily reactive to being more strategic. Over the past few years, WMATA has regularly used performance data to inform operational decisions to advance its key goals, and determining how to prioritize goals if and when they conflict. Below are a few examples of ways in which the agency has used performance measurement data to enhance its overall performance.

In the last few years WMATA has established many of the performance-based planning and programming elements necessary to become a more strategic, accountable and transparent organization. A key to this success has been commitment from the agency's GM/CEO. Nevertheless, WMATA recognizes that more work remains (e.g., strengthening linkage between goals and budget, increasing use of data to drive decisions and tying individual performance evaluation to agency goals and key indicators). WMATA's experience demonstrates that becoming a performance-based organization does not happen overnight. Instead, positive performance results come from a steady, incremental approach to establishing the five key PBPP elements: goals, measures, targets, resource allocation and evaluation.

Updated: 10/10/2013
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000