The Metropolitan Transportation Commission (MTC) serves as the Metropolitan Planning Organization (MPO) for the San Francisco Bay Area, which covers nine counties and the major cities of San Francisco, San Jose, and Oakland. MTC's work is governed by a nineteen-member policy board comprised of local elected officials, and representatives of regional and state agencies.
MTC produces a variety of reports that reflect a performance-based approach to multi-modal transportation planning, including: Regional Transportation Plan (RTP), Transportation Improvement Program (TIP), Public Participation Plan, Regional Bicycle Plan for the San Francisco Bay Area, Transit Connectivity Plan, Financing Transit-Oriented Development in the Bay Area. During the most recent RTP update, MTC also prepared a Performance Assessment Report that specifically analyzed the performance results for potential long-range transportation investments in the region. In addition to preparing plans, MTC screens transportation project requests from local agencies for State and Federal funds to assess their compatibility with MTC plans. MTC's extensive use of performance measures at major planning stages is reflected in these major documents.
MTC defines performance-based transportation planning as an approach that "focuses on the measurable outcomes of potential investments and the degree to which they support stated policies." Performance-based transportation planning is further defined as both "systematic and analytic" in that it:
MTC uses performance measures, formally and informally, at distinct but connected stages of the planning process, supporting Board decision-making, public engagement, and the evaluation of policies and investments.
MTC's formal performance analysis focuses on the performance of the overall regional transportation system and the outcomes of projects included in agency plans and programs. The Commission has formally used performance measures for RTP preparation since 2001. The Commission expanded its use of performance measures following the passage of Senate Bill (SB) 1492 in 2002, which requires that MPOs use performance criteria to evaluate and prioritize any new investments for consideration in the RTP at both the project and corridor levels. MTC expanded its use of performance measures again when developing the current RTP, which was adopted in spring 2009. MTC incorporates lessons learned from past performance evaluations into each subsequent RTP update, creating a dynamic feedback loop from one cycle to the next.
Figure 7. Transportation 2035 Performance Assessment Process
Source: MTC Performance Assessment Report
Although MTC does not formally evaluate the success of the planning process itself, it does evaluate the success of the planning process informally. Staff reported that the most important indicator of a successful planning process is whether or not the process influences final decision-making, but this is not formally assessed.1 Another important indicator of a successful planning process for MTC is incorporating the perspectives of diverse stakeholders in the plan. To this effect, MTC has built performance criteria into its public participation plan.
Although MTC employs performance measures across multiple plans and programs, the agency's core performance-based planning approach is focused on the long-range planning process. MTC considers its current long-range plan, Transportation 2035: Change in Motion (adopted in April 2009) as the agency's first truly "performance-based" RTP. Performance assessments were conducted during prior RTP updates in 2001 and 2005 (see Discussion, page 7), but evaluation results were not available until after key decisions had been made, which meant that they had limited impact on final decision-making.
For Transportation 2035, the Commission committed resources to conduct and complete an extensive performance analysis well ahead of key decision points so that evaluation results could help to shape final decision-making. The results of MTC's 3-step process are detailed in a Performance Assessment Report, which was published in December 2008 (see Table 10).
The first step in MTC's performance assessment was to develop a set of specific, measureable, and ambitious, outcomes-based performance objectives that support the eight overarching goals of the RTP, as well as the "Three Es" of sustainability - economy, environment, and equity. Prior RTPs had always projected worsening regional emissions and delay over the life of the plan, based on current trends. With Transportation 2035, however, MTC decided to articulate a very different kind of vision for the region, in which transportation conditions improve and emissions and delay are reduced over the life of the plan. To demonstrate what it would take to accomplish this vision, the Commission identified eleven outcome targets against which the performance of alternative investments and policies could be modeled (see Table 10 below). It is notable that these targets combine traditional and non-traditional transportation goals, such as those related to livable communities and environmental sustainability: to reduce regional CO2 emissions to 40 percent below 1990 levels; reduce vehicle miles traveled (VMT) per capita 10 percent from 2000 levels; and reduce the share of earnings spent on housing and transportation by low and moderately-low income households by 10 percent from 2009 levels by 2035. Table 10 demonstrates how MTC uses the key dimensions of sustainability ("the Three E's"), to organize its long range performance objectives.
Table10. Transportation 2035 Performance Objectives
|Economy||Maintenance & Safety||Improve Maintenance|
|Local Streets and Roads: maintain pavement condition index of 75 or better|
|State highways: distressed land-miles no more than 10% of system|
|Transit: average asset age no more than 50% of useful life and average distance between service calls of 8,000 miles|
|Sources: State and local plans|
|Reduce injuries and fatalities|
|Motor-vehicle fatalities: 15% from today|
|Bike and pedestrian injuries and fatalities: 25% from 2000 levels|
|Source: California State Strategic Highway Safety Plan|
|20% per capita from today|
|Freight||Source: California Strategic Growth Plan|
|Environment||Clean Air||Reduce vehicle miles traveled and emissions|
|Vehicle miles traveled: 10% per capita from today|
|Fine particulate matter (PM2.5): 10% from today|
|Climate Protection||Coarse particulate matter (PM45): 45% from today|
|Carbon Dioxide (C02): 40% below 1990 levels|
|Sources: State regulations and laws|
|10% reduction from today in share of earnings spent on housing and transportation costs by low and moderately low income households|
|Livable Communities||Source: Adapted from the Center for Housing Policy|
Source: MTC Performance Assessment Report
MTC's performance analysis demonstrates that RTP projects and policies alone will not be able to achieve the agency's aggressive performance objective outcomes. The Commission modeled the long-term performance of three fiscally unconstrained packages of infrastructure investments (freeway operations, regional transit and rail, and high-occupancy toll network with significant bus enhancements), as well as a mix of pricing and land use policies and technology advances in fuels and vehicles. It found that none of these alternatives on its own would achieve MTC's adopted performance objectives. Instead, MTC views its performance objectives as "stretch" targets, which call attention to the need for the Commission to develop effective partnerships with other regional and state agencies, as well as local municipalities, and collaborate in designing and implementing policies and projects that support the RTP's strategic direction.
The second part of MTC's analysis was to conduct a project-level performance assessment for all potential RTP projects that would be funded through MTC's "discretionary" sources2 (i.e., funds over which MTC has final decision-making power). All projects submitted for discretionary funding in the RTP (about 700 total) received a qualitative analysis to screen for how well they support RTP policy goals (e.g., whether the project serves a key freight corridor, supports focused growth, improves access for youth, the elderly or disabled). "High-cost" projects (i.e., over $50 million), which represented 70-80 percent of total discretionary investment decisions in the RTP, were also submitted to a quantitative performance analysis, which estimated the project's cost/benefit in relation to RTP performance objectives. The goal of the project-level performance analysis was to identify outliers (i.e., the major projects that best and least supported RTP goals) to help frame decision-making about which projects to include in the plan, which projects to exclude, and why (see Figure 8 on the following page).
Figure 8. Quantitative & Qualitative Project-Level Performance Outliers in Transportation 2035
The third and final component of MTC's performance analysis was to do a program-level assessment that modeled the impact of the entire Draft RTP against the Commission's eleven adopted performance objectives. Assessment results showed that Draft RTP investments help to shift the trend for each performance objective in the right direction. However, Draft RTP investments alone fall far short of overall targets for transportation system performance in 2035. As a result of its performance assessment, MTC concluded that "policies and strategies addressing land use, pricing, and technology will be required" to close the gap and reach the full vision for system performance in 2035.
MTC updates its 4-year Transportation Improvement Program (TIP) every 2 years. The current TIP, adopted in May 2008, includes $12.8 billion and a prioritized list of 1,066 projects to be funded through 2012.
MTC prepares a fiscally constrained RTP, which serves as a long-range programming document (not just a strategic, long-range vision or plan). The RTP directly influences TIP preparation because overall programming for the TIP flows directly from the results of RTP decision-making. All projects and programs funded in the TIP must support RTP goals and objectives, and conform to RTP policies and funding allocations. For this reason, MTC does not conduct a second level of performance assessment to guide overall TIP preparation.
State planning guidelines require MTC and other regional transportation planning agencies to conduct a performance assessment for their portion of the state TIP (STIP) in order to demonstrate how the program of projects in the TIP are linked to the goals and objectives in the long range plan (see top of page 4 in guidelines). The state guidelines call for a quantitative and/or qualitative assessment of both current system performance and projected future impact of investments, commenting on a specific list of performance measures. Regional agencies may report substitute measures, and MTC does so based on the measures used in the RTP analysis and the availability of data on current conditions. The RTP and TIP are closely linked, making it difficult to separate any single funding source from the broader RTP funding strategy. This requires multiple, targeted funding sources to work together to achieve regional goals. The Commission does not conduct additional TIP-specific quantitative performance analysis of project impact; instead, MTC relies on its RTP performance analysis, which reflects the projected impact of the comprehensive investment strategy. To demonstrate how the TIP relates to the goals in the RTP, the Commission identifies example projects in the TIP that address each goal. The state guidelines also call for project-level performance assessment of major expansion projects with total cost over $50 million. In the past MTC has relied on project sponsors to provide this analysis; however, with expanded project performance in the RTP, MTC may be able to use the RTP analysis to meet this part of the requirement in the future.
This is an important example of a performance-based approach to planning, relying on performance measures in the Plan both to set strategic directions and then link these policies directly to decisions that can realistically be funded in the TIP.
Except for large, regionally-significant projects, the MTC board does not directly engage in individual project selection and implementation. Most TIP dollars at the Commission are programmatic in the sense that they are awarded to pooled funding programs - such as the Transportation for Livable Communities (TLC) program - instead of to individual road widening projects. Each MTC program has its own steering committee, which recommends program-specific criteria to the Commission for prioritizing, selecting, and funding individual projects. Historically, the project selection criteria for specific funding programs related strongly to the RTP Goals; however the current plan, Transportation 2035, represents the first instance in which the RTP itself includes specific performance measures. MTC staff expect that these criteria will carry through in fund programming guidelines in the future.
MTC does not conduct a separate performance analysis on the annual unified planning work program (UPWP). UPWP decision-making is influenced by the RTP goals and the TIP. By establishing policies that closely link long-range planning (RTP) with shorter-term programming (TIP), project selection, and implementation, MTC is able to focus its performance analysis on one phase of the project planning and implementation cycle (i.e., RTP preparation), while still influencing decision-making throughout the cycle as a whole (e.g., TIP and UPWP).
MTC has worked in collaboration with the California Department of Transportation (Caltrans) to prepare an annual State of the System report since 2001. The State of the System report presents data and summary information on key aspects of performance for freeways, local roadways, transit, goods movement, and bicycle and pedestrian travel in the Bay Area region. Currently, MTC relies on data collection from local and regional agencies for the State of the System effort, but has not formally tied it to the performance assessment that is conducted during RTP preparation. Moving forward, however, MTC plans to strengthen the ties between the two efforts by more formally linking data collection and monitoring metrics to the performance targets in the RTP. In addition, during development of the current RTP, the Commission expressed considerable interest in setting interim targets, to track progress in shorter, more concrete intervals. If the Commission goes this direction in the future, it will establish a closer link between monitoring and long range planning. In this way, the State of the System report could be used to monitor progress towards achieving RTP goals and objectives, and formalize the links between project planning, programming, and implementation, as well as better inform subsequent planning efforts.
MTC's approach to performance-based planning has evolved over time, but has always focused on the RTP as the key stage of analysis. The first year that MTC formally incorporated performance measures into the RTP beyond the impact analysis required in the RTP Environmental Impact Report, which is required under California law, was 2001. At that time, performance analysis was conducted at the project level. MTC staff reported that it took a tremendous amount of work but did not yield the most useful results (i.e., focused more on measuring current performance in a given corridor rather than on the performance impacts of a project on that corridor). Another challenge was that the analysis took a long time to complete, so results were not available in time to influence some key decisions. MTC conducted its second performance analysis with the 2005 RTP. Instead of focusing on projects, staff conducted a "programmatic-level" assessment which modeled the performance of alternatives from the environmental impact report. Staff felt that it was an improvement over the 2001 assessment, but still left significant room for growth. MTC staff reported that the Commission's third performance analysis was more meaningful and effective than prior efforts, and has yielded MTC's first truly "performance-based plan." In this analysis, MTC used performance criteria to develop the RTP vision and then conducted a rigorous and comprehensive performance assessment, using the results to shape and justify final RTP decision-making (see Long-Range Planning section above for details).
Several issues have influenced the evolution of MTC's performance measures approach in recent years. State legislation first required the Commission to adopt performance measures in 2002, but subsequent state policy, including land use and climate change legislation (e.g., SB 375, AB 32), is pushing agencies to go to further lengths in adopting and integrating performance measures into their work. Staff reported that the changing "rules of the game" at the state level have made its board and partner agencies more open to the use of performance measures, and more engaged in the process of designing an effective approach. In addition, MTC's Executive Director served on the National Surface Transportation Policy and Revenue Study Commission, which staff felt influenced his leadership and determination to move the agency further in its performance analysis.
MTC includes a discussion of caveats and challenges to the use of performance measures in its reports and publications, which is a valuable communication tool for elected officials, stakeholders, and the public:
MTC staff noted several lessons learned from the evolution of the agency's performance measures approach:
MTC staff also provided feedback, recommendations, and requests regarding national performance measures:
1 MTC staff suggested that this could be made more formal by instituting an after-action de-brief, to go back at the end of each planning process and examine if and how planning considerations and evaluations upfront influenced final results at the end of the process.
2 Of the $218 billion in Transportation 2035, only about $32 billion were discretionary funds (e.g., major state funding, CMAQ, STP). The remaining balance of $186 billion in the RTP were non-discretionary "committed" funds over which MTC did not have decision-making authority (e.g., transit fares, local sales tax, locally committed projects, projects already included in the TIP).