Office of Planning, Environment, & Realty (HEP)
Selection of performance measures is closely tied to development of goals and objectives, since performance measures are used to assess progress toward meeting objectives and in turn goals. Therefore, performance measures are central to implementing a performance-based planning process, since how performance is defined and measured will significantly affect the types of projects and strategies that are advanced. Moreover, performance results inform agencies if the types of projects and strategies that are implemented are in fact helping them achieve their strategic goals.
Performance measures serve five critical purposes within PBPP - they are used:
Two well-known challenges associated with defining performance measures are difficulties associated with data availability and difficulties in developing quantitative measures for factors such as economic vitality and livability. Although performance-based planning and programming focuses on specific, quantifiable measures, it is not meant to discourage the inclusion or consideration of goals, principles, or policies that are difficult to measure or quantify, such as quality of life. Some key themes or lessons include the following:
Selecting performance measures requires considering what specific metric will be used and how measurements will be taken. In selecting performance measures, several factors should be considered:
Moreover, the selection of measures should address key issues, accounting for factors such as risk, importance to the public, and implications for policy. For example, system-wide pavement smoothness ratings do not distinguish heavily traveled Interstates from lesser-traveled state highways, nor crucial links from those that may have alternative routes. A selected measure, therefore, might focus on pavement quality on a subset of a state's most important roads. The relevance of the selected measure is an important consideration, given limited funding and the need to prioritize investments.
Virginia's SHSP: Focusing on the Right Measures
Virginia's SHSP, prepared in 2006, acknowledges, "In the past, the Commonwealth of Virginia has viewed motor vehicle crashes as mainly a transportation issues. Injuries and death were compared using traditional transportation-oriented measures such as the number of vehicle miles traveled (VMT), the level of congestion, or the type of facility…However, this death rate reduction has been due to increased [VMT], not from reducing the number of people injured or killed. Safety experts…are adopting a multi-perspective approach by changing from a transportation-based measure (crashes per million VMT) to a health-based measure (per 100,000 population)."
Consequently, the plan uses total deaths and injuries as its primary performance measures, rather than rates. It identifies a realistic target to "reduce annual deaths by 100 and annual injuries by 4,000 from motor vehicle crashes statewide within the next five years" by taking a multi-perspective approach, and identifies plan elements addressing a full range of strategies including driver behavior, special users, pedestrian and bicyclist safety, intersection safety, roadway departures, work zone safety, traffic records, and transportation safety planning.
Source: Virginia Strategic Highway Safety Plan, http://www.virginiadot.org/info/hwysafetyplan.asp.
Tip for Selecting Performance Measures - Carefully Select a Limited Number of Useful and Meaningful Measures
The MPO for the Portland, Oregon area, Portland Metro, initially chose to track over 100 performance measures as part of its PBPP, but quickly found that collecting information on so many measures was resource-intensive and made it difficult to draw meaningful and broad conclusions regarding the system's performance. As a result, the agency identified 10 "used and useful" measures that it believes best support its specific objectives to guide its performance analysis. This may also increase the comprehensibility of Metro's performance measurement system to the general public and increase transparency.
There often can be value in using multiple measures to address multiple dimensions of a problem. At the same time, it is advisable to start with a limited number of measures since it can be overwhelming to address hundreds of different measures. Experience suggests the importance of keeping the measures simple. It has been noted in many places: "Measure what is important; do not measure everything." Traditionally, many measures used and reported by State DOTs and transit agencies reflect specific operational considerations (e.g., National Transit Database reporting measures, system usage measures) that provide useful information but may not be key considerations in relation to identified goals. Too many measures can be cumbersome to deal with, making an agency "data rich but information poor."
While this discussion is focused on broad system-wide performance measures, it is important to recognize that the plan level measures used to evaluate investment scenarios and implementation level measures used to inform project selection and track performance over time may not be the same. One example is a performance measure, "share of population and employment within walking distance of a transit stop," which may be used to support a regional goal related to livability or accessibility. The Atlanta Regional Council (ARC) used this metric in its 2030 Regional Transportation Plan to compare results for the current year (2005), for 2030 without strategies suggested by the plan; and for 2030 with the implementation of the plan. However, as a system-wide level, this metric cannot directly be used for project selection, except in a qualitative way to evaluate projects that support attainment of this measure.
As with developing goals and objectives, it is also vital to engage the public and stakeholders in developing performance measures (in association with objectives). For some issues, such as safety, key concerns are generally well documented - i.e., reducing fatalities and injuries. For other issues, such as sustainability, livability, quality of life, and economic vitality, the most appropriate way to define an objective and associated performance measures is often unique to each state or region, so it is important to gain input from the public on what is most important to them.
Public engagement may take place through a variety of mechanisms including the public involvement aspects of the long-range transportation planning process, as well as through what stakeholders articulate at the local level, such as through corridor studies and project-related efforts. Some regions have also used public opinion surveys and social media tools to understand the priorities of the public, and stakeholder work groups as a basis for developing objectives.
As an example, the CMP traditionally has focused on traffic congestion and used engineering measures focused on motor vehicles, such as volume-to-capacity ratios. In defining appropriate congestion management objectives for a CMP, planners and decision-makers are beginning to consider questions such as: How high of a priority is traffic congestion in the region? What type of congestion is most problematic for the public and freight shippers? And what aspects of congestion are most important to address other goals, such as livability, safety, and economic vitality? Answering these questions can lead to objectives that are quite different from a traditional approach focusing on addressing level of service (LOS) deficiencies or easing vehicle traffic congestion. Moreover, given population growth, many transportation agencies recognize that reducing traffic congestion may be difficult to achieve, and that congestion may be a sign of economic vitality - as a result, some regions are focusing on improving transportation system reliability, increasing multimodal options so that people have greater choices and the ability to avoid traffic congestion, or focusing attention on strategic freight corridors or economic development corridors.
As an example, the Capital District Transportation Committee (CDTC), the MPO in the Albany, New York, region, recognized in its CMP that reducing traffic congestion was not the highest priority for the public, given limited funding to address all transportation needs. Through surveys and public involvement activities, CDTC has learned a key public opinion: the public has said that quality of life factors such as bike and pedestrian improvements, improved landscaping, and safety improvements were more important than reducing congestion in the metro area, and that travel time reliability is the most important congestion issue for travelers in the region. Consequently, CDTC has chosen to focus on "excess delay" and reliability/predictability of delay rather than aiming for free-flow traffic speeds during peak hours.