Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram
Office of Planning, Environment, & Realty (HEP)
HEP Events Guidance Publications Glossary Awards Contacts

SAFETEA-LU Evaluation and Assessment Phase I

4. Project Analysis and Selection Practices that Support Effectiveness

In the previous section of this report, data gathered from the national CMAQ database and local project sponsors were presented to document reported congestion and emissions benefits, as well as characteristics of the broad types of strategies. This section focuses on using information from the selected set of projects to assess the projects' air quality cost-effectiveness and to examine how some areas are using this type of information for program prioritization and decision making.

This section is divided into two parts. First, a discussion of the cost-effectiveness of the selected projects at reducing emissions of each of the primary pollutants – VOC, CO, NOx, and PM10 and PM2.5 – is provided. In order to calculate air quality cost-effectiveness in a way that allows appropriate comparisons, project costs and emissions effects have been recalculated to fill in gaps in reported emissions reductions and to "normalize" the results to a common year, 2008.

Second, initial observations on good practices that States and MPOs have used to analyze, prioritize, and select CMAQ projects; including use of cost-effectiveness analysis and consideration of other factors are provided. Phase II of this evaluation and assessment study further expand upon this information through development of case studies of specific locations to understand State DOT and MPO practices and to help enhance the effectiveness of the program.

Emissions Reduction Cost-Effectiveness

The Role of Cost-Effectiveness Assessment

Understanding the cost-effectiveness of CMAQ projects should be an important consideration in project selection decisions at the State and local level. SAFETEA-LU directs that States and MPOs give priority to "diesel retrofit projects and. . .other cost-effective emission reduction activities, taking into consideration air quality and health effects" and to "cost-effective congestion mitigation activities that provide air quality benefits."41 Moreover, States and MPOs, as good stewards of public dollars, will maximize the value of their investment of CMAQ funds by targeting it toward projects that provide the most benefit per dollar. Indeed, conducting a cost-effectiveness assessment provides States and MPOs with the ability to stretch limited transportation funding resources across a wide range of projects that demonstrate congestion, energy, environment, air quality, and mobility benefits.

Given the role of the CMAQ program as a key funding source to help transportation agencies meet air quality goals consistent with attainment of regional air quality plans, cost-effectiveness at reducing air pollutant emissions is often considered an important metric of CMAQ program effectiveness. At the same time, it is important to recognize that the benefits of the CMAQ program go well beyond emissions reduction, and States and MPOs often take into account these other considerations in making project selection decisions. In this study, cost-effectiveness for the 67 sample projects was calculated in regard to emissions reductions alone due to the availability of information on emissions reduction estimates for a wide variety of CMAQ-funded transportation projects. However, emissions reduction cost-effectiveness may not be the only measure of cost-effectiveness for a project, just as air quality is not the only benefit that may be considered in project selection.

In many urban areas and states with severe traffic congestion problems, a project's cost-effectiveness at alleviating traffic congestion will often be an important consideration. A project that reduces traffic congestion in a targeted corridor may be viewed as more beneficial than another project that reduces the same level or more emissions but does not provide congestion relief benefits. These congestion relief benefits are difficult to quantify using a standard metric (such as hours of traveler delay reduced) across all projects, based on the complexities of modeling and assessing these impacts, particularly for small projects.

CMAQ projects generate a wide range of other benefits, which may also be important factors in project selection. These benefits include, among others, enhancing mobility and access, creating more reliable travel times and transit services, encouraging physical activity, reducing greenhouse gas emissions, creating better connections between transportation and land use, and fostering a more multi-modal transportation system. Most of these benefits are difficult, if not impossible, to quantify in a standard metric, and thus are not usually considered in a cost-effectiveness framework. However, these benefits may be very important in the context of regional transportation goals. The flexibility inherent in the CMAQ program allows local areas to determine their own procedures and criteria for project assessment. States and MPOs are using a suite of evaluation criteria, including air quality and energy conservation benefits, local cost participation share, and intermodal, multi-modal, and social mobility concerns, to ensure all are being addressed in regional transportation planning and programming.42

Methodology for Analyzing Emissions Reduction Cost-Effectiveness of the Selected Projects

The study team calculated cost-effectiveness of the sample projects with respect to reductions of VOC, NOx, CO, PM10, and PM2.5. Cost-effectiveness figures were developed for each pollutant independently, rather than as a composite figure. This was done for two primary reasons: 1) At the national level, it is difficult to determine the most appropriate means of weighting each pollutant, given that some pollutants are of more concern in some parts of the country than others. 2) Some strategies are targeted toward reducing individual pollutants, such as dust mitigation projects, which focus on PM10 reduction. Lumping together the reduction of a full set of pollutants, therefore, would not show how different types of strategies can be more or less effective at reducing different pollutants.

In order to increase the comparability of emissions and cost figures across the sample projects, the study team recalculated project costs and emissions effects. Recalculations were conducted largely because for many projects, data were missing on specific pollutants – commonly CO and PM. In addition, the projects were implemented in a wide range of different locations, at different times, and emissions benefits were reported for different years. Since the U.S. vehicle fleet is on average, much cleaner today than it was 10 years ago, a project that eliminates a mile of travel will have less emissions reduction benefit in 2010 than the same project in 2000. Consequently, it was useful to standardize the emissions effects to a common year using a standard set of default emissions factors for purposes of analyzing cost-effectiveness across the selected projects. Project costs, including operating and capital costs, were also adjusted to reflect constant 2008 dollars to enable better comparisons. Costs were standardized using the Consumer Price Index (CPI) which may result in reduced cost-effectiveness for multi-year projects.

The following discussion provides more detail on the calculation procedures used in this study. The procedures for "normalizing" emissions and costs are described below first. This is followed by a description of the general steps in conducting cost-effectiveness analysis, which could be used for any calculations of project cost-effectiveness, including those conducted at the State or local level. In fact, the study team found that a number of State DOTs and MPOs were using the same basic approach to calculate cost effectiveness for their proposed projects.

Procedures Taken to Ensure Comparability of Cost and Emissions Data from Sample Projects

The "normalization" procedures used to standardize the projects in this study included three main steps.

These steps are described below.

1) Develop "Normalized" Emissions Factors. In order to improve comparability of results, a common set of emissions factors for CO, NOx, VOC, PM10, and PM2.5 emissions was developed using MOBILE6.2 for analysis year 2008. MOBILE is EPA's approved model for estimating pollution from highway vehicles. The model calculates emission factors (in grams per vehicle-mile) for a variety of pollutants from passenger cars, motorcycles, light- and heavy-duty trucks. Some of the emissions factors are based on testing of tens of thousands of vehicles and account for changes in vehicle emission standards over time, changes in vehicle populations and activity levels, and variation in local conditions such as temperature, humidity and fuel quality.43

In the data collected from State DOTs and MPOs, some projects' emissions effects had been calculated with an earlier version of the model, MOBILE5a, or EMFAC, the California emissions model. In this analysis, the 2008 emissions factors from MOBILE6.2 were applied to all the selected projects, where feasible, reducing differences due to the time the project was implemented, local weather and vehicle fleet mix, and/or region-specific modeling assumptions. For additional information on the assumptions and inputs used to develop the normalized emissions factors, please see Appendix B. It should be noted that while this normalization was helpful for purposes of this study, State DOTs and MPOs should not apply this procedure in their own project assessments. They should use the best available data at the local level to develop appropriate emissions factors for conditions in their area.

Emissions factors for some types of CMAQ projects, such as diesel engine retrofits, bus replacement, and dust mitigation projects, were not standardized. These technologies vary widely in their ability to achieve emissions reductions and depend on specific local conditions (e.g., road dust levels depend on precipitation and silt loadings). Consequently, the emissions factors reported by project sponsors, or the most recent EPA certification data for retrofits, were used in the calculation of cost-effectiveness.

2) Recalculate Emissions Reductions, as appropriate. Using the normalized emissions factors, emissions reductions were then recalculated (as kg/day). This generally was done using the project's reported travel impacts (e.g., VMT reductions, speed changes), using the methodologies that the project sponsors had used.

3) Adjust Total Public Costs to Constant Dollars. CMAQ and non-CMAQ project costs reported by local sponsors were converted to a 2008 base by using the average annual Consumer Price Index published monthly by the U.S. Bureau of Labor Statistics.44 CPI values for respective years in relation to 2008 are shown in Table 3, along with the corresponding adjustment factor. The CPI for January 2008 is 211.08. To convert dollar values in one year to constant dollars in a second year, multiply the first-year dollar value by a factor whose numerator is the average annual CPI of the second year and whose denominator is the average annual CPI of the first year. For instance, to convert $10,000 in 2000 dollars to 2008 constant dollars, multiply $10,000 by the average annual CPI in 2008 divided by the average annual CPI in 2000:

$10,000 * (211.1 / 172.2) = $10,000 * 1.226 = $12,259 in 2008 constant dollars.

Table 3. Consumer Price Index (CPI) Factors.
Year CPI Factor
1999 166.6 1.267
2000 172.2 1.226
2001 177.1 1.192
2002 179.9 1.173
2003 184.0 1.147
2004 188.9 1.118
2005 195.3 1.081
2006 201.6 1.047
2007 207.3 1.018
2008 211.1 1.000
Calculating Emissions Reduction Cost-Effectiveness Using the Normalized Results

Once normalized emissions figures and costs were developed, cost-effectiveness at reducing each type of emissions was calculated using a standard approach, as listed in the three steps below:

The cost figures used in the calculations represent the total public costs associated with implementing a project. This includes funding from the CMAQ program for capital and operating costs, as well as any other Federal, State, or local sources. Some individual projects were funded over multiple years or multiple States, and so the cost used in the calculations reflected all of these components. In some cases, CMAQ only paid for a small portion of a project's total costs. Some State and MPO analyses that the study team reviewed involved calculations of both overall project cost-effectiveness (based on the full costs of the project) and cost-effectiveness associated with CMAQ dollars alone (not including other Federal, state, and local funding sources). However, the results presented in this study reflect full project cost-effectiveness at reducing emissions using total public funds. The measure of cost-effectiveness reported in this study is dollars per ton (although it can also be reported as dollars per kg, or another similar metric). The steps in this process are described below.

1) Convert to Annualized Emissions Reductions.

Emissions reductions per day should be converted into annualized values as kg per year (which the study team converted to tons per year). Although the national CMAQ database reports emissions reductions in kg per day, in most cases the emissions benefits are not realized on all 365 days of the year, but are restricted to only work travel/weekdays or to a smaller number of days when the program is in effect (e.g., on ozone exceedance days, days when bicycling is considered most feasible, or days of application of de-icing chemicals). In most of the project examples, the project sponsor included an estimate of the number of days during which the strategy would be effective. In these situations, the local figure was used. However, in instances where it was not provided, the following standard scaling factors were used:

2) Calculate Annualized Cost of the Project.

To calculate the annualized cost of the project, two pieces of information are needed:

The annualized cost of the project is calculated by multiplying total project funding by the CRF, as shown in the following equation:

Annualized cost = Project funding x CRF

The discount rate reflects the rate at which society (taxpayers) values future benefits in terms of resources that it must give up now. As a result, it "discounts", or places a lower value on, future benefits from the investment compared to current benefits. A lower discount rate increases the effective value of future benefits (emissions reductions) by lowering the annualized cost used in the comparison.45 A 7 percent discount rate was used in this analysis, which is the value used by the Federal Transit Administration (FTA) in its New Starts program, and is the rate recommended by the Office of Management and Budget (OMB) for Federal investment analysis.

"Project life" represents the period of time over which a project remains effective at reducing emissions and congestion levels, and varies by the type of project. For example, a standard transit bus is expected to provide service for 12 years, whereas the service life of a vanpool vehicle may be only 5 years. For some projects, effects last for many years; in other cases, the effects continue only for the length of time when direct funding is provided. Individual project life periods, determined by the specific circumstances of each project and local jurisdiction, were sometimes reported by project sponsors, and these were typically used in the calculations. However, some general rules are provided in Table 4, based on a review of project life periods used by other sources, and these were generally used where no other data were provided.46

Table 4. Project Life Periods Used for Evaluating Projects.
Category Subcategory Project Life
Expectancy (years)
Traffic Flow Improvements Traffic Signalization 10
Freeway Management 10
High-Occupancy Vehicle Lanes 20
Shared Ride Programs Regional Ridesharing 1 to 2
Vanpool Programs  
ongoing assistance 1 to 2
purchase of vans 5
Park-and-Ride lots 12
Travel Demand Management Regional Approaches/Employer Trip Reduction programs 1 to 2
Bicycle/Pedestrian Facilities - 15
Transit Improvements New Bus Services  
purchase of new buses 12
operations 1
New Rail Services  
Railcars 20
Track/stations 30
Service Upgrades  
Amenities 2
Bus shelters, etc. 10
Technology Improvements (primarily transit) Conventional Bus Replacements and Alternative Fuel Buses (assumed remaining life of vehicles) 4
Dust Mitigation Projects - 20
Freight/Intermodal Projects - 20
Engine Retrofit Technologies Diesel Engine Retrofits Varies
Truck Stop Electrification 10

3) Calculate Cost-effectiveness. Once air pollutant emissions and costs were standardized into annualized values, a cost-effectiveness calculation was determined for each project sample. Cost effectiveness is calculated using the following equation:

Cost Effectiveness ($/ton) = (Annualized Cost) / (Annual Emissions Reduction)

A project is more cost effective when it achieves its results at the lowest possible cost. For each project, cost effectiveness was calculated according to the estimated reductions of VOC, NOx, CO, PM10, and PM2.5 emissions individually.

Results

The results of the cost-effectiveness analysis for each pollutant are presented below, in Tables 7 and 8, which summarize the minimum and maximum cost-effectiveness figures for individual projects studied within each category and subcategory. Given the small number of projects studied, the median value has not been provided.

In examining emissions reductions by individual pollutant, it is important for State DOTs and MPOs to consider the specific air quality issues that are faced in their areas. Moreover, the health effects, emissions inventories, and control sources for each pollutant are also different. For instance, transportation sources produce significantly more CO than PM; correspondingly, reducing a ton of PM often costs more than reducing a ton of CO. The benefits of reducing a ton of PM may also be more valuable, based on health studies showing the significant effects of PM on human health.

Across the project categories, some patterns emerge, although the results are limited due to the small number of projects studied, and cannot be used to determine statistically significant median cost-effectiveness values or confidence intervals. The projects profiled in this study are intended to be illustrative of typical projects funded through the CMAQ program, but do not represent a statistical sampling of the CMAQ database. The largest sample category size in this study is seven diesel engine retrofit projects. It is important to note that these figures are not directly comparable to the results from some other studies, such as the TRB study on the CMAQ program, cited in the Appendix to the CMAQ Interim Guidance.47

Table 5. VOC, NOx, and CO Cost-Effectiveness of Selected CMAQ Projects by Strategy.
Category No.
Projects
VOC ($/ton) NOx ($/ton) CO ($/ton)
Low High Low High Low High
Traffic Flow Improvements              
Traffic Signalization 7 $2,000 $5.6 M $5,000 + $500 +
Freeway Management 4 $1,000 $98,000 $10,000 + $2,000 +
High-Occupancy Vehicle Lanes 1 $18.9 M $40.5 M $1.3 M
Shared Ride Programs              
Regional Ridesharing 3 $86,000 $494,000 $78,000 $440,000 $7,000 $39,000
Vanpool Programs 4 $34,000 $158,000 $29,000 $160,000 $3,000 $13,000
Park-and-Ride Lots 5 $14,000 $8.5 M $12,000 $4.9 M $1,000 $384,000
Travel Demand Management 4 $16,000 $2.9 M $15,000 $2.9 M $1,000 $223,000
Bicycle/Pedestrian Facilities 4 $551,000 $6.0 M $667,000 $7.4 M $46,000 $453,000
Transit Improvements              
New Bus Services 3 $130,000 $1.5 M $222,00 $1.4 M $9,000 $15,000
New Rail Services 3 $88,000 $416,000 $89,000 $380,000 $7,000 $33,000
Service Upgrades/Amenities 5 $11,000 $1.5 M $7,000 $1.5 M $1,000 $116,000
Bus Replacements/Technologies              
Conventional Bus Replacements 2 $852,000 $1.5 M $134,000 $231,000 $706,000
Alternative Vehicles/Fueling Facilities 4 $152,000 $2.9 M $82,000 $316,000 $124,000 $734,000
Dust Mitigation Projects 3 -- -- -- -- -- --
Freight/Intermodal Projects 6 $37,000 $424.2 M $2,000 $213,000 $7,000 $3.7 M
Diesel Emissions Reduction              
Diesel Engine Retrofits 7 $7,000 $677,000 $21,000 $1,000 $174,000
Truck Idle Reduction 3 -- -- $2,900 $4,600 $6,800

NOTE: Cost-effectiveness calculations noted with a plus sign (+) indicate that project(s) in the category showed an increase in the pollutant of concern. Projects with (--) indicate categories where a cost effectiveness calculation was not applicable due to zero pollution reduced.

Table 6. PM Cost-Effectiveness of Selected CMAQ Projects by Strategy.
Category No.
Projects
PM10 ($/ton) PM2.5 ($/ton)
Low High Low High
Traffic Flow Improvements          
Traffic Signalization 7 $287,000 $68.9 M $442,000 $106.2 M
Freeway Management 4 $279,000 $15.7 M $430,000 $135.9 M
High-Occupancy Vehicle Lanes 1 -- -- -- --
Shared Ride Programs          
Regional Ridesharing 3 $2.0 M $11.1 M $4.2 M $24.1 M
Vanpool Programs 4 $695,000 $3.8 M $1.5 M $8.3 M
Park-and-Ride Lots 5 $285,000 $128.2 M $616,000 $277.5 M
Travel Demand Management 4 $390,000 $79.8 M $845,000 $172.9 M
Bicycle/Pedestrian Facilities 4 $22.8 M $259.6 M $49.4 M $562.1 M
Transit Improvements          
New Bus Services 3 $6.1 M $6.1 M $13.3 M (+)
New Rail Services 3 $2.3 M $9.7 M $5.0 M $21.2 M
Service Upgrades/Amenities 5 $184,000 $41.6 M $398,000 $90.1 M
Bus Replacements/Technologies          
Conventional Bus Replacements 2 -- -- -- --
Alternative Vehicles/Fueling Facilities 4 -- -- $676,000
Dust Mitigation Projects 3 $15 $700 -- --
Freight/Intermodal Projects 6 $66,000 $10.8 M $80,000 $13.2 M
Diesel Emissions Reduction          
Diesel Engine Retrofits 7 $7,000 $1.7 M $8,000 $2.1 M
Truck Idle Reduction 3 $110,300 $173,600 $110,300 $173,600

NOTE: Cost-effectiveness calculations noted with a plus sign (+) indicate that project(s) in the category showed an increase in the pollutant of concern. Projects with (--) indicate categories where a cost effectiveness calculation was not applicable due to zero pollution reduced. One figure reported between the high and low categories indicates that only one project reported emissions effects for that pollutant.

As seen in the tables, a high level of variability is found in the results for each individual category of projects, indicating that local context and project-specific factors are an important determinant of cost-effectiveness. The range of estimated figures for air quality cost-effectiveness within individual categories is very large, with some individual projects showing very strong cost-effectiveness for certain pollutants, while others clearly appear to have lower cost-effectiveness for certain pollutants, as indicated by costs of several million dollars per ton.

This finding seems to indicate that some projects are better suited for reducing certain pollutants and likely were selected for reasons other than emissions reductions (e.g., congestion mitigation, social effects). Indeed, while air quality cost-effectiveness is an important aspect of transportation agencies' project selection, these other benefits can have significant impacts on overall urban mobility, livability, and sustainability initiatives.

Observations regarding the various categories of projects are noted below.

Traffic Flow, Shared Ride, and Demand Management Projects.

Some traffic flow improvements and projects that target reductions in single-occupancy vehicle travel – such as shared ride and travel demand management programs – were very cost-effective in reducing the ozone precursors, VOC and NOx, as well as CO. Due to the relatively limited contribution of personal motor vehicles to PM, in comparison to VOC, NOx, and CO, none of these strategies appeared to be very cost-effective at reducing PM. Moreover, the MOBILE6 model used to generate the emissions changes for this analysis does not take into account the impact of changes in vehicle speeds on PM emissions levels.48 Therefore, the PM emissions reductions reported from traffic flow projects in this analysis were only calculated based on reductions in vehicle idling due to reductions in incident-based or intersection delay. These projects often have important non-emissions benefits, including travel time savings, reductions in greenhouse gases, and supporting increased non-motorized travel.

Transit and Technology/Fuels Programs

Transit improvements that target reductions in motor vehicle travel, such as new rail or bus services and service upgrades/amenities, appear to offer the potential for relatively high cost-effectiveness at reducing VOC, NOx, and CO emissions, but fared poorly in reducing PM. Overall, bus replacement projects fared poorly in cost-effectiveness at emissions reduction. The costs are used to purchase vehicles that will last 12+ years in service, but emissions benefits can only be credited for a limited number of years, not the full service life of the new bus.

Dust Mitigation

Projects focused on dust mitigation offered some of the most effective means measured in this study for reducing PM10 and PM2.5 emissions in locations where they were practical. These projects, including paving unpaved roads and application of deicing chemicals to reduce sand application, achieved substantial reductions in particulate matter (in the form of wind-blown dust) for far less public resources than other types of project categories.

Diesel Emissions Reduction and Freight/Intermodal Projects

Diesel retrofits, truck idle reduction, and freight/intermodal projects are categories of projects that have received increased emphasis in recent years. These categories had some of the most cost-effective projects within the reviewed projects at reducing both ozone precursors and particulate matter. However, there was a very large range, with some projects fairing poorly when focusing solely on the cost-effectiveness of emissions reductions. This may be due in part to the fact that different retrofit technologies target different pollutants. For instance, one retrofit project showed high cost-effectiveness at reducing NOx, whereas some retrofits showed no impact on NOx.49 Use of idling reduction technology to reduce long-duration truck idling showed the best cost-effectiveness at reducing NOx emissions.

Examples of Good Practices

States use a variety of processes and procedures to identify, select, and evaluate projects for inclusion in the CMAQ program. Drawing on the observations and results of the project analysis and information from State and local project sponsors, the following sections provide examples to illustrate the range of approaches taken by States and MPOs. Examples of good approaches identified through this research include:

These practices are discussed below.

Standardized Tools or Emission Calculation Methods

State and local transportation and air quality agencies have the flexibility to conduct CMAQ project air quality analyses with different analytical approaches. While FHWA does not specify a single set of methods for use in CMAQ emissions estimation, every effort should be taken to ensure that determinations of air quality benefits are credible and based on reproducible and logical analytical procedures.

Use of Accepted Emissions Calculation Approaches

An important first step in making decisions is to base those decisions on appropriate methodologies and reasonable assumptions. There are several online resources and published guides available to State and local transportation practitioners that describe the modeling tools and other methods that can be used to assess the emissions benefits of projects applying for CMAQ funds.

The most recent and comprehensives of these is a guidebook, Multi-pollutant Emissions Benefits of Transportation Strategies (2006). This compendium includes sketch planning methods for 35 different categories of transportation strategies, based on a review of many guidance documents and analytical tools. The report includes calculations of emissions impacts for sample projects, based on real project examples, and identifies EPA and FHWA guidance documents that should be referenced. It also reports on the direction of emissions impacts (increase, decrease, neutral or uncertain) that are typically expected for each transportation strategy on the following seven pollutants: CO, PM10, PM2.5, NOx, VOCs, SOx, and NH3. The report is available at: www.fhwa.dot.gov/environment/air_quality/conformity/research/mpe_benefits/.

The report A Sampling of Emissions Analysis Techniques for Transportation Control Measures (2000) includes a brief overview of 19 methods which include pre-packaged and customizable software tools as well as worksheets or other procedures for calculating benefits. They collectively address a wide range of potential CMAQ projects, including travel demand management, traffic flow improvements, and vehicle and fuel technology strategies. The report also includes references to other sources of information on CMAQ program effectiveness. The report, including information on the source and availability of the methods is available online at: www.fhwa.dot.gov/environment/cmaqeat.

EPA has published a number of methodology guides for calculating emissions impacts of different types of strategies, notably diesel retrofits and program to reduce long-duration truck idling. The national Clean Diesel Program sponsored by EPA has published information and materials that relate to on- and off-road diesel engines. In particular, the Diesel Emissions Quantifier is an interactive tool developed by EPA to help State/local governments, fleet owners/operators, and others estimate emissions reductions and cost effectiveness for clean diesel projects. The Quantifier uses emissions factors and other information from EPA's National Mobile Inventory Model (NMIM) which includes the MOBILE 6.2 and NONROAD2005 models. For further information access: www.epa.gov/otaq/nmim.htm.

A number of the methodologies identified through the review of the selected projects referenced these documents, particularly EPA guides and certification data related to emissions benefits of diesel retrofits and long-duration idle reduction.

Standardized Approaches for Comparisons of Projects

Some State DOTs and MPOs have developed their own guidebooks or emissions modeling tools to assist in documenting and evaluating proposed CMAQ projects and programs. These tools can help the State DOT or MPO in evaluating projects, reduce calculation errors, and ensure that local project sponsors provide information that is consistent and comparable with other similar projects. Several States have provided project sponsors with a spreadsheet into which sponsors can enter project-specific assumptions and receive back emissions benefits calculations. These guidebooks and tools often contain default parameters viewed as appropriate to the region.

Table 7 highlights several States and MPOs that provide emissions calculation aids or tools.

Table 7. Selected States and MPOs with Standardized Tools or Emission Calculation Methods.
State DOT or MPO Standardized Tools or Emission Calculation Methods
Maricopa Association of Governments (Phoenix area, Arizona) "Methodology for Evaluating Congestion Mitigation and Air Quality Improvement Projects" report provides standardized methodologies for calculating direct emissions effects (kg/day reduced) and cost-effectiveness at reducing emissions ($/metric ton)
Birmingham Regional Planning Commission (Alabama) "A Guide for Estimating the Emissions Effects and Cost-Effectiveness of Projects Proposed for CMAQ Funding" includes standardized methodologies that are used to assess emissions impacts of different types of CMAQ projects, as well as cost-effectiveness
California "Methods to Find the Cost-Effectiveness of Funding Air Quality Projects" guidebook and automated database contains standardized methods for estimating the emissions benefits and cost-effectiveness of different types of CMAQ projects; Access database files automate calculation procedures.
North Front Range MPO (Fort Collins area, Colorado) A CMAQ Air Quality Benefit Program Excel workbook that includes a spreadsheet which allows project sponsors to select the Type, Area, and Category for the project being submitted. Based on those selections, the spreadsheet directs the sponsor to provide category-specific evaluation criteria and then it automatically calculates the emissions benefits and cost-effectiveness of the project. Two measures of cost-effectiveness are used: total current year project cost/annual emissions reduced, and CMAQ funds/annual emissions reduced. Although the calculation is automatic, within the workbook is another spreadsheet that provides the formulas used for the calculations.
Massachusetts Executive Office of Transportation Excel workbook automatically calculates emissions benefits based on sponsor-provided assumptions; also provides sample air quality analysis methods.
New York State DOT "CMAQtraq" application feeds into the DOT's database tool to determine air quality results. Local project sponsor provide input data with the application, and the DOT enters the information into the Microsoft Access database tool to determine the project's air quality results. The current version of CMAQtraq (ver. 6.2) has MOBILE6.2 emissions factors embedded in the calculations.
Pennsylvania DOT "PAQONE" software analyzes a variety of transit, non-motorized travel, and roadway improvements using standardized methods
Wasatch Front Regional Council (Salt Lake City area, Utah) Excel workbook automatically calculates emissions benefits based on default values or sponsor-provided assumptions.

Cost-Effectiveness Calculations

Analyzing the cost-effectiveness of CMAQ projects for both emissions reductions and congestion mitigation effects should be an important step in the project selection process, both in terms of the benefits that accrue to the States or MPOs receiving CMAQ funding and the net benefits achieved nationally by the funds distributed through the Federal CMAQ program. Broad statements about the types of projects that will or will not be funded in certain areas should be avoided because the types of strategies that are most cost-effective will vary due to local factors. Rather, States and MPOs may use cost-effectiveness calculations as a mechanism to objectively compare projects during review and selection. Examining project cost effectiveness can also be a way of bringing attention to the design or proposed application of the project, and can provide help in judging its suitability or most effective implementation strategy.

Several of the State DOTs and MPOs that provided project information for this study also had calculated cost-effectiveness, and were using standardize procedures to calculate cost-effectiveness. While the estimates of project duration, discount factors, and pollutants of concern varied, these methods allow projects to be evaluated across strategies and geographies to determine the most appropriate for funding.

For instance, in Alabama, standardized emissions calculation worksheets for common CMAQ strategies are provided to local project sponsors by the Birmingham MPO. The MPO, State DOT, and other agencies input information and assumptions for their projects into the spreadsheet model to determine travel impacts, emissions reductions, and cost effectiveness. The cost effectiveness is calculated using the following equation: Cost Effectiveness = (Annualized cost) / (Annual Emissions Reduction). Annualized costs include a 7 percent discount rate and a capital recovery factor to account for the project service life multiplied by the total capital cost of the project to estimate the average annual cost. Cost effectiveness calculations are provided for HC, NOx, PM10, (HC + NOx), (PM10 – NOx), and (PM10 + NOx) in both dollars per lb per year and dollars per kg per year. In the case of the North Front Range MPO in Colorado, two measures of cost-effectiveness were calculated: total project cost/annual emissions reduced, and CMAQ funds/annual emissions reduced. The second calculation takes into account the source of project funding, and enables projects with a higher non-CMAQ funding share to shower better cost-effectiveness.

Transparent, Inclusive Selection Processes

While the CMAQ program is intended to enable local agencies the flexibility to select projects that meet the transportation infrastructure, political, and geographic needs of local areas, FHWA CMAQ Interim Guidance includes language requiring that,

"The CMAQ project selection process should be transparent, in writing, and publicly available. The process should identify the agencies involved in rating proposed projects, clarify how projects are rated, and name the committee or group responsible for making the final recommendation to the MPO board or other approving body."50

Although the collection of data did not reveal readily available documentation of the CMAQ project selection process in most cases, it did identify several States and MPOs that appear to have consistent and robust project selection procedures. In addition, SAFETEA–LU encourages State DOTs and MPOs to consult with State and local air quality agencies about the estimated emissions reductions from CMAQ proposals. States which seek guidance and/or evaluation assistance from these agencies will also ensure more accurate air quality analyses for CMAQ projects. Table 8 provides examples of State DOTs and MPOs that appear to have documented, transparent project selection methods.

Table 8. Identified States and MPOs with Transparent Project Selection Methods.
State DOT or MPO Selection Process
North Front Range MPO
(Fort Collins, Colorado)
Current process utilizes a three-tiered scoring system to rank projects: 50 percent of the score is assigned to short-term air quality impacts (rankings based on VMT and carbon monoxide reduction estimates for year one); 20 percent for long-term benefits (estimated for years two through five), and 30 percent for bonus features (e.g., overmatch, multi-agency or public/private partnerships, and multi-modal projects). Standardized calculation procedures are used to analyze emissions effects. CMAQ Project Selection Committee includes representatives form the Colorado Department of Transportation, Colorado Air Pollution Control Division, FHWA, FTA, and U.S. EPA. Selection Committee develops list of recommended projects based on project scoring, as well as other intangible elements (which may include regional equity, project readiness, synergies with projects funded from STP or other sources, and project mix).
Hillsborough MPO
(Tampa, Florida)
CMAQ projects are evaluated by a committee of representatives of the MPO, FDOT, Florida Department of Environmental Protection, and the Hillsborough County Environmental Protection Commission (EPC) based on a series of qualitative and quantitative measures. Final project ranking is based on the average total score assigned by each of the four reviewing agencies. The ranking is based on 5 criteria, each scored on a scale one to five: 1) projects that remove vehicles from the road or reduce travel delay; 2) outreach projects that change the public's driving behaviors; 3) projects with the most efficient dollar per ton cost/benefit figure for reducing NOx; 4) projects with air quality benefits to be realized within 3 years of funding; and 5) projects identified in CMS Study and/or 2025 LRTP Interim Plan.
Georgia DOT A project selection process, developed by GDOT, the Environmental Protection Division, the Georgia Regional Transportation Authority, and Georgia Environmental Facilities Authority – together known as State Air Quality Partners – is used consistently across the State. Previously, CMAQ funds were confined to Atlanta area, but with PM2.5 designations, a new approach was developed. The process does not sub-allocate funding to specific MPOs but instead seeks to support the most beneficial projects for reducing emissions and meeting air quality goals across the state.
Rouge Valley MPO
(Oregon)
The Rouge Valley MPO awards points for meeting certain evaluation criteria outlined in a Project Evaluation Questions & Intent questionnaire form. Criteria include emissions reduction, and other considerations, such as: long-term air quality improvement, potential to reduce reliance on automobiles, potential to mitigate congestion, completes a multi-modal transportation system, located in city limits or inside Urban Containment Boundary, and diesel retrofits. Points awarded for the criteria are used to develop an overall score for each project.
Southwestern Pennsylvania Commission
(Pittsburgh MPO)
The MPO provides potential project sponsors with a CMAQ application and instruction package, including schedule, guidelines, and selection criteria to guide sponsors of candidate projects through the CMAQ process. Application forms can be filled out electronically. Candidate CMAQ projects are placed into appropriate investment categories and sent to appropriate SPC members, PennDOT Districts, and transit agencies as well as SPC's CMAQ Evaluation Committee. Projects are evaluated for effects on emission and cost-effectiveness based on standardized models developed for PennDOT. A scorecard is completed by SPC staff for each project rating each candidate project on consistency with priority project types (e.g., diesel retrofits, traffic signal improvements, TDM, commuter bicycle/pedestrian improvements) and 9 ancillary selection factors to develop total weighted score. CMAQ Evaluation Committee members use this information to develop recommendations for each investment category.
Wasatch Front Regional Council
(Salt Lake City, Utah MPO)

The MPO has adjusted its evaluation criteria and procedures over time. In the past, a score was calculated using a weighted ranking system that considered the following:

  • (10%) Project in a congested corridor
  • (15%) Length (years) of project effectiveness
  • (25%) Emissions reduction
  • (25%) Congestion reduction (VHT)
  • (25%) Cost

This objective ranking was then combined with subjective rankings by staff and 3 different committees consisting of city planners and elected officials.

Currently, the MPO uses air quality cost-effectiveness rankings as a primary criterion for project selection within different categories of projects. The MPO generally allocates a certain percentage of funding for each major project category (e.g., bicycle/pedestrian projects, transit projects) in order to ensure a variety of project types are implemented, and ranks project cost-effectiveness within each category. Field visits are also conducted of projects proposed for funding.

Collecting Project-Specific Data and Conducting Project Evaluation Studies

Regardless of the model or methodology used to calculate emissions benefits of CMAQ-funded projects, good inputs are needed to produce good outputs. States and MPOs should take efforts to gather data through surveys and other data collection methods to justify and/or make assumptions. Some States, such as Michigan and New York, require project sponsors to provide the source and justification of all inputs and assumption used in the emissions calculations. Not only does this ensure that the project demonstrates an air quality benefit, but it allows the State to evaluate the accuracy of the analyses. Many of the project samples cited local data in their calculations, including factors such as average trip lengths, park-and-ride utilization rates, number of actual vanpool riders, transit riders, number of rideshare matches, etc.

A comparison of forecasted impacts (via project selection methodologies) to actual results (based on ex post evaluation) can help inform the rigor and accuracy of calculation methodologies and project selection procedures. State DOTs report the status and effectiveness of the CMAQ programs in their States to the U.S. Department of Transportation. The information from these reports is entered into the CMAQ database and can provide States with an effective tool for monitoring and evaluating the results of CMAQ-funded projects. Performing project evaluation studies allows States to periodically review their project selection criteria to ensure it remains appropriate and up-to-date. Evaluation studies also provide States and MPOs with new and more accurate data to be used in future emissions analysis calculations. For example, data on the actual speed improvements along freeways due to an ITS system implementation may lead to an increase in the baseline speeds along freeways in the entire region.

Although post-project analysis is not commonly conducted on a program-wide basis for all CMAQ projects within a state or MPO area, in some cases, post-project evaluations are conducted by States and MPOs for specific projects or types of projects, especially those that are included as part of a regional conformity analysis. Post-project evaluation is a good practice in helping to provide information on the accuracy of emissions forecasts and assumptions used in emissions calculations. In some cases, however, post-project analysis may not be practical, such as for a small project where conducting a rigorous evaluation might cost nearly as much as the project itself. Some examples of post-project evaluations that have been conducted are listed in Table 9 below.

Table 9. Selected States and MPOs that Conduct or Call for Post-Project Analysis.
State or MPO Types of Post-Project Analysis Conducted
California The California Air Resources Board uses post-project evaluation reports generated by regional air districts as part of a state grant process to update the California emissions methodology guidebook.
Georgia DOT Detailed evaluations have been conducted for the regional Clean Air/TDM program.
New York State DOT New York State DOT conducts an annual evaluation of its Clean Air / Ozone Action Days outreach program.
Metropolitan Washington Council of Governments (Washington, DC area) MWCOG conducts a regular evaluation of its Transportation Emissions Reduction Measures (TERMs), which include a number of programs funded in part by CMAQ (from allocations from Maryland, Virginia, and the District of Columbia). The TERMs report includes collection of data on participation rates in programs, including collection of survey data.

Next Steps

The study team's collection of data on the selected set of projects revealed a number of strengths and limitations in the analysis of CMAQ projects. On the one hand, many of the project analyses were conducted based on relatively limited data, using sketch planning methodologies, with limited verification of results. This is perhaps not surprising given the limited scope of many projects, limited data and tools available for analyzing many of these projects, and the costs and effort associated with conducting detailed evaluation studies. On the other hand, it appears that a number of states and MPOs have implemented good practices to help standardize the emissions analyses, collect local data for use in calculations, rank project cost-effectiveness, and implement systematic procedures for evaluation. These procedures often take into account multiple factors beyond emissions reduction cost-effectiveness.

In Phase II of this evaluation project, FHWA, in consultation with EPA, conducted a set of limited on-site case studies and/or program analyses. These case studies add to the national understanding of how the CMAQ program operates at the state and local levels, and may build on five case studies (Los Angeles, Chicago, Houston, Washington, DC, and Albany) conducted as part of the TRB study on the CMAQ Program. The Phase II case studies provide information on how States and MPOs are analyzing, prioritizing, and selecting projects, and implementing the CMAQ program to meet State and local objectives. The insights gained from these studies help to inform States and MPOs about best practices, and a variety of potential ways to improve the effectiveness of their CMAQ program implementation efforts.

  1. 41 SAFETEA-LU 1808(d).
  2. 42 Integrating Air Quality and Transportation Planning: A Compendium of Workshop Summaries for Regional Councils and metropolitan planning organizations. 2001-2005. Available at: www.narc.org/uploads/File/01Workshop_Summaries_2005_Edit.pdf.
  3. 43 U.S. Environmental Protection Agency. Office of Transportation and Air Quality. Description and History of the MOBILE Model. (2004) http://www.epa.gov/OMS/mobile.htm.
  4. 44 U.S. Department Of Labor, Bureau of Labor Statistics. Consumer Price Index for All Urban Consumers - (CPI-U), U.S. city average, All items. http://www.bls.gov/news.release/cpi.toc.htm.
  5. 45 For example, imagine two projects, each reducing 1 pound of emissions per year. The first project costs $1,000 and has 5 years of effects; the other project costs $2,800 but has 20 years of effects. At a 7 percent discount rate, the first project appears more cost effective, while with a 5 percent discount rate, the second project appears more cost effective.
  6. 46 See: California Air Resource Board, "Methods to Find the Cost-Effectiveness of Funding Air Quality Projects," 1999; Birmingham Regional Planning Commission, "A Guide for Estimating the Emissions Effects and Cost-Effectiveness of Projects Proposed for CMAQ Funding," 2002; Maricopa Association of Governments, "Methodology for Evaluating Congestion Mitigation and Air Quality Improvement Projects"; U.S. Environmental Protection Agency, "Summary Review of Cost and Emissions Information for 24 Congestion Mitigation and Air Quality Improvement Projects," 1999; Transportation Research Board Special Report 264, "The Congestion Mitigation and Air Quality Improvement Program: Assessing 10 Years of Experience."
  7. 47 For instance, in the TRB study, emissions benefits for NOx and VOC were combined and weighted, resulting in a composite cost-effectiveness figure, whereas this study presents separate figures for each pollutant. Moreover, in the TRB study, emissions benefits expected to occur in the future were "discounted", but for this study, all emissions benefits were counted equally. The inherent economic logic of discounting presumes that short-term benefits are preferable to benefits in the long-term. This study, however, values a ton of emissions reduction in year 1 as equivalent to a ton of emissions reduction in year 10. This approach makes the reporting of dollars per ton reduced more intuitive when reporting emissions reductions for individual pollutants. It also is consistent with the fact that emissions reductions in a nonattainment area need to be continued into the future in order for compliance with the ambient air quality standards.
  8. 48 See Preamble to 40 CFR Part 93 for additional discussion of EPA's conformity decisions.
  9. 49 For comparison purposes, EPA's document, "The Cost-Effectiveness of Heavy-Duty Diesel Retrofits and Other Mobile Source Emission Reduction Projects and Programs," May 2007, and "Diesel Retrofit Technology: An Analysis of the Cost Effectiveness of Reducing Particulate Matter and Nitrogen Oxides Emissions from Heavy-Duty Nonroad Diesel Engines through Retrofits," May 2007, estimated a range of $18,700 to $87,600 per ton of PM emissions reduced, and a range of $1,900 to $19,000 per ton of NOx reduced for various retrofit scenarios.
  10. 50 Sec. 1808: Addition to CMAQ Eligible Projects. Publication of Interim Guidance on the Congestion Mitigation and Air Quality Improvement (CMAQ) Program. Dec. 19, 2006 Federal Register.
  1. Table of Contents
  2. Executive Summary
  3. Section One
  4. Section Two
  5. Section Three
  6. Section Four
  7. Appendix A
  8. Appendix B
  9. Appendix C
  10. Appendix D
Updated: 6/28/2017
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101