Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

Office of Planning, Environment, & Realty (HEP)
Planning · Environment · Real Estate

HEP Events Guidance Publications Glossary Awards Contacts

Evaluation - Guidance for the Preparation of TCSP Evaluation Plans - June 2001

Table of Contents

  1. Importance of Evaluation
  2. Grantee Roles and Responsibilities
  3. General Approach to Evaluation
  4. Detailed Evaluation Guidance
  5. Evaluation References

List of Figures

  1. Steps in Developing an Evaluation Plan
  2. Use of Control Group in Before-and-After Data Collection

List of Tables

  1. Examples of TCSP Goals and Objectives
  2. TCSP Process Evaluation
  3. TCSP Product Evaluation: Planning Grant
  4. TCSP Product Evaluation: Implementation Grant
  5. TCSP Outcome Evaluation
  6. Potential Existing Data Sources for Evaluation
  7. Methods for Collecting New Data

1. Importance of Evaluation

The purpose of the Transportation and Community and System Preservation (TCSP) Pilot Program is to fund innovative projects that will increase the knowledge of the costs and benefits of different approaches to integrating transportation investments and strategies, community preservation, land development patterns, and environmental quality. Planning and implementation projects may be undertaken at the neighborhood, local, metropolitan, State, and regional levels by States, local governments, and metropolitan planning organizations (MPOs) working in cooperation with non-traditional partners. The TCSP is a pilot program explicitly designed to encourage innovative strategies and techniques, the results of which can then be used by other public and private organizations throughout the country. While TCSP funding is not sufficient to implement projects on a nationwide basis, all organizations nonetheless will benefit by being able to easily tap into the experience of others to learn what might be applicable for their own situations and how these new transportation strategies and techniques can be most effectively implemented.

To accomplish this learning and the desired resultant transfer of experience, the evaluation of individual projects is a key component of the TCSP program. Evaluation of projects which are new or experimental in character will indicate the success of various activities at achieving the desired transportation, community, and system preservation objectives. The lessons learned from this process will be used in evaluating the overall TCSP program and will help develop more effective TCSP projects in the future. As a result, the TCSP program will provide an important nationwide learning experience.

In keeping with the TCSP program's emphasis on learning and evaluation, grant applicants should bear in mind the following points:

In presenting this guidance for the preparation of evaluation plans, it is recognized that, "one size does not fit all." This guidance, therefore, provides ideas for evaluation plans rather than a mandated approach.

[Top of Section, Table of Contents, Top of Document]


2. Grantee Roles and Responsibilites

Evaluation Plan

TCSP grantees are responsible for conducting a systematic evaluation of their TCSP project. Each grant application should include an evaluation plan which describes how the grantee proposes to evaluate the project. This will assist in demonstrating the applicant's commitment to the evaluation component. The strength of the evaluation component, including identification of resources required, will be an important factor in the selection of final grant awards.

FHWA will use the results from individual evaluations, in conjunction with other overall program evaluation criteria and methods, in assessing the overall effectiveness of the TCSP program. As results and lessons learned from individual TCSP grant awards become available and the overall program can be assessed, the FHWA will coordinate and disseminate results, tools, and information developed through the program.

In the evaluation plan submitted, grantees should identify program goals and objectives, performance measures, measurement techniques, potential data sources, and schedule milestones. Proposals should identify existing sources of information which will be utilized (either qualitative or quantitative), and should also identify any new data collection efforts which may be required or useful for evaluating the effectiveness of the program. The evaluation plan also should contain clear roles, responsibilities, commitments by participants, and a budget estimate. The resources required for evaluation activities should be included in the overall grant budget proposed for the project.

As a component of the TCSP program evaluation, a grant workshop is planned for the Spring of 1999, at which grantees will share experiences and initial results from their projects. Budgets for grant applications should include travel for the key investigator to this workshop, as well as a second such conference, as part of the evaluation component.

Assistance with Evaluation

The remainder of this document provides guidance relating to the development of an evaluation plan. The purpose of this guidance is to provide ideas rather than a mandated approach, and agencies should not be discouraged from applying for TCSP program funding simply because they lack expertise in particular evaluation methods. It is more important that grant applicants commit to undertaking a systematic evaluation, including the designation of project resources, than they demonstrate proficiency in any particular evaluation method. Grant applicants not already having the desired level of in-house evaluation expertise may want to consider working in cooperation with another agency or a university.

[Top of Section, Table of Contents, Top of Document]


General Approach to Evaluation

This section provides an overview of how to develop an evaluation plan, with more detail on how to structure and conduct the evaluation described in Section 4.0. In particular, Section 4.0 identifies specific techniques that may be used, issues to consider, and key questions to ask in evaluating a TCSP project.

Steps in Developing an Evaluation Plan

Grant applicants are encouraged to take the following steps in developing an approach to project evaluation, as illustrated in Figure 1:

Figure 1. Steps in Developing an Evaluation Plan

Figure 1. Steps in Developing an Evaluation Plan

[List of Figures]

  1. Define Project goals and objectives. What is the motivation for undertaking the project? What is the project intended to accomplish? Table 1 shows examples of general goals and objectives for the overall TCSP program. Goals and objectives for individual TCSP projects may be a subset of these program goals and objectives. In addition, grant applicants may have additional goals and objectives which are important for the project to achieve locally.

  2. Identify performance measures. Performance measures are either quantitative or qualitative measures which indicate the success of the project at achieving its stated goals and objectives, e.g., total emissions per capita or land consumed per unit of development. Examples of performance measures for the identified TCSP program goals and objectives are shown in Section 4.0. Applicants, however, should resist the temptation to establish a "laundry list" of performance measures, but instead should identify a few key measures which best reflect the impacts of the program. It is also important to select performance measures which are simple to understand, are as objective as possible, and can be constructed from available data sources.

  3. Identify data and information sources and evaluation methods. Grant applicants should identify data and information sources to support each performance measure. In the case of quantitative data, applicants should identify both existing sources and potential new data collection efforts. In the case of qualitative data, proponents should identify key sources of information (people, agencies, committees, etc.), along with appropriate techniques for obtaining and evaluating information (interviews, direct observation, etc.) Some potential data sources and evaluation techniques are identified in Section 4.0. Consideration also should be given to identifying the baseline condition from which changes will be assessed.

Table 1. Examples of TCSP Project Goals and Objectives

  • Improve efficiency of transportation system
  • Maximize use of existing infrastructure
  • Reduce impacts on environment
  • Reduce costs of infrastructure investment
  • Ensure efficient access to jobs, services, centers of trade
  • Encourage private sector land development patterns to achieve above goals
  • Involve non-traditional partners
  • Integrate transportation, community preservation, and environmental activities
 

[List of Tables]

Once potential performance measures, data sources, and evaluation methods have been identified, an overall evaluation plan should be developed for collecting and analyzing the required information. This includes identifying the individual work tasks required to carry out the evaluation and establishment of the associated budget and timeline for these tasks.

What Should Be Evaluated?

A TCSP evaluation should focus on identifying both the magnitude and the distribution of the costs and benefits of a project, and on those aspects of the planning and implementation process that will be useful to other organizations in deciding whether or not to implement similar strategies. Thus, evaluations can focus on three different aspects of a TCSP project: process, products, and outcomes. Appropriate goals and objectives, performance measures, and evaluation methods will differ for each, as will the timeframe over which the evaluation is conducted.

These three aspects of a project are interrelated and important to the evaluation of a TCSP project. Outcome goals are of ultimate interest to society, but achievement of process and product goals can indicate the likelihood of success at achieving the desired outcomes. Process and product goals are also desirable for their own sake. For example, an open and participatory process is important for ensuring that all viewpoints and potential impacts are considered. The involvement of non-traditional partners will help to identify strategies that encourage private sector development patterns that are consistent with the goals of the TCSP program. Examining the linkages among process, product, and outcome also can be useful. For example, desirable outcomes can be facilitated by the relationships developed during a planning process. Conversely, difficulties encountered during implementation may be traceable to the unintentional omission of an important factor during the planning stage. Finally, evaluation of all aspects of a project serves as an important learning tool, helping to identify both successful and unsuccessful approaches to a problem.

Evaluation Reports

An initial evaluation plan is to be included by an applicant as part of the application for a TCSP grant. This initial plan then may be refined in negotiating the terms of a grant awarded to the applicant. While the evaluation plan is expected to cover the basic approach proposed for evaluating a TCSP planning or implementation grant, the details of an evaluation plan, such as the statistical basis for a stratified sampling plan, may not be fully developed until after a project is actually underway.

The evaluation activities associated with a TCSP grant should result in one or more reports. The purpose of these reports is to provide the information needed by other organizations throughout the country to decide whether similar projects would be beneficial within their jurisdictions, and how they should go about planning or implementing this particular kind of action.

The initial evaluation report should document the process by which the TCSP grant project was developed or implemented, as well as the final product of the grant. This report can be produced shortly after completion of the project. Initial information on the results or outcomes of the project may also be available soon after completion and can be documented in this initial evaluation report. It is possible, however, that the full impacts of a project will not occur immediately, and that additional documentation of project outcomes will be appropriate in the future as data on longer-term impacts become available. The proposed approach to reporting should be explained in the evaluation plan portion of the grant application.

[Top of Section, Table of Contents, Top of Document]


4. Detailed Evaluation Guidance

This section provides more detailed guidance on evaluating the process, product, and outcomes of TCSP projects. For process and product evaluations, key questions for obtaining information as background to the evaluation are identified. For outcome evaluations, specific techniques and issues to consider in either estimating or measuring the impacts of the TCSP project are identified. For all three types of evaluations, examples of goals and objectives, performance measures, and evaluation methods relevant to TCSP projects are provided.

It is important to identify in each case a baseline from which a change is being determined. For a process evaluation, this can be simply a comparison of the new or TCSP planning process with the existing or traditional approach. For a product evaluation, this can include an assessment of how the final project differs from what was initially proposed. Baseline considerations in estimating project outcomes include issues of time scale and differentiating project impacts from parallel changes in other significant factors, as discussed in Section 4.3.

Process Evaluation

Evaluation of the process by which the TCSP plan or project was produced or implemented can serve a number of useful functions. Process evaluation can identify reasons for success or failure of the plan or project as well as specific strategies and tactics which were most effective. Evaluation of specific aspects of the process, such as who participated and their respective roles, also can help indicate how likely the product is to achieve success. For example, extensive participation of a variety of affected parties or groups may mean that the project is more likely to be successful, since potential obstacles and stumbling blocks can be resolved.

A number of techniques can be used to gather information for evaluating the process, including:

Questions that can be asked as a basis for evaluating the process include:

Documenting answers to the above questions can determine the degree to which the process met its defined goals and objectives. Some process-related goals and objectives for the TCSP program, as well as associated performance measures, are shown in Table 2. Local agencies may also hold other goals and objectives for activities carried out under the TCSP program. Documenting the answers to these questions also will help in identifying circumstances or actions that influenced the level of success of the final product.

Table 2. TCSP Process Evaluation
Sample Goals/Objectives and Performance Measures

Goal/Objective

Performance Measures

Involvement of non-traditional partners

Number/type of groups involved:

  • Public utility operators Social services agencies
  • Community groups
  • Environmental organizations
  • Non-profit organizations
  • Public health agencies
  • Economic development agencies
  • Private land development organizations
  • Home builder associations
  • Real estate investors
  • Zoning commissions
  • Other public or private groups

Contribution (policies, actions, ideas) and commitment (financial and other resources) of each group

Consistent with Statewide and MPO planning process

Construction projects are ultimately included in approved State or MPO Transportation Improvement Program

Project included in air quality conformity analysis if required

Changes to State or MPO plans are coordinated with other affected jurisdictions

Other demonstrated linkages to planning process

Broadens scope and impact of planning process to integrate transportation, community preservation, environmental activities

Number/type of interests involved:

  • Public sector
  • Community/interest groups
  • Private sector

Elements of process/plan/project that affect or consider:

  • Land development planning
  • Community preservation
  • Environmental impacts
  • Economic development
  • Social equity
  • Private sector activities

New ways of doing business

Evidence of common goals

Achieves stakeholder commitment and buy-in

Endorsement of results by:

  • Participants
  • Other affected parties

Participation of stakeholders in plan development:

  • Attendance/participation at meetings
  • Other participation/communication

Individuals/organizations/groups not supporting plan

Commitment to implementation (through responsibility, funding, etc.)

Process led to learning and innovation

New approaches taken

Innovative ideas generated

New relationships formed (formal or informal) for implementation

Process is directed at achieving desired TCSP outcomes

Development of background information and analysis to support plan development or project selection:

  • Empirical evidence based on implementation of other, similar plans or activities
  • Modeling/forecasting
  • Surveys

Other qualitative assessment of potential impacts

Evidence of consideration of this information in planning process

Development and implementation of evaluation plan and activities


[List of Tables]

Evaluation of improved linkages to metropolitan or statewide planning process, as encouraged by TEA-21, is of particular importance, although this may not be relevant to all TCSP grants. As applicable, grantees might evaluate their ability to improve connections through the funded project with the broad metropolitan or statewide transportation planning processes at the center of TEA-21. Linkages to the planning process can be flexible, and could be demonstrated, for example, by:

With respect to the public involvement process for transportation planning in particular, federal guidelines suggest the following desirable outcomes of public involvement (Reference: A Guide to Metropolitan Transportation Planning Under ISTEA: How the Pieces Fit Together. U.S. Department of Transportation, Washington, D.C., 1995.):

Product Evaluation

Product evaluation focuses on what was produced by the planning or implementation activity. A description of the project as it was actually produced or implemented can serve as an interim step in identifying the likely outcomes or impacts of the project. Some general questions that can be asked about the product include:

Evaluation of the product of a TCSP activity will differ significantly depending on whether the activity is a planning or implementation grant. In the case of an implementation activity, product evaluation can focus on describing what was actually built, or what service was developed, and why it is significant. In the case of a planning activity, product evaluation will focus on the content of the plan, agreement, etc. (e.g., what will be achieved if the plan is implemented or the agreement carried out); adoption of the plan; and on provisions to ensure successful implementation of the plan or agreement. While development of the plan or project consistent with the original scope of work, timeline, and budget may be a criterion, this should not limit flexibility in making mid-course modifications to a project. As planning and implementation progresses, it is possible that changes to the project may be incorporated that result in an improved product compared to the original proposal.

Table 3 shows examples of goals and objectives and performance measures for evaluating the product of a planning grant. Table 4 shows examples of goals and objectives and performance measures for evaluating the product of an implementation grant.

Table 3. TCSP Product Evaluation: Planning Grant
Sample Goals/Objectives and Performance Measures

Goal/Objective

Performance Measures

Adoption of plan or agreement

Adopted or revised plans, policies, ordinances, processes (by everyone with implementation responsibility)

Adopted agreements, memoranda of understanding, etc.

Provisions to ensure plan implementation

Legal authority to implement plan

Funding/resources identified to implement plan

Provisions for management/oversight of plan implementation

Implementation timeline with specific implementation responsibilities

Feedback process to monitor/adjust implementation as needed

Other indicators of likelihood of successful implementation

Plan is consistent with other state and locally adopted plans

Stakeholder commitment/buy-in

Political legitimacy to implement plan:

  • Outcome of accepted planning process
  • Support of legislative bodies required to implement plan

Who does not support the plan

Plan or agreement is consistent with Statewide and Metropolitan planning processes

Implementation through collaborative partnerships, for example, involving the MPO, state transportation and environmental agencies, city planning agencies, transit, or non-traditional partners

Contributes to alleviation of priority area transportation and related problems identified in the 20 year plan and any "visioning"

Includes projected life-cycle costs developed through financially constrained planning

Includes performance indicators and provisions for monitoring, possibly including those in transportation management systems

Includes public involvement consistent with federal guidelines for metropolitan planning (see A Guide to Metropolitan Transportation Planning Under ISTEA: How the Pieces Fit Together, U.S. Department of Transportation, 1995.)

Plan is directed at achieving desired TCSP outcomes

Clear statement of purpose and need

Consistency with defined goals and objectives


[List of Tables]

Table 4. TCSP Product Evaluation: Implementation Grant
Sample Goals/Objectives and Performance Measures

Goal/Objective

Performance Measures

Project is innovative/provides a learning experience

Something has been accomplished/learned that has not been done before:

  • Similar projects implemented and/or evaluated elsewhere
  • External inquiries about the project

Changes to improve project during development/implementation phases in response to new information, analysis, etc.

Project can be replicated in other areas

Project was successfully completed

Time schedule of completion

Cost of project versus what was achieved

Project-specific indicator

Examples:

  • Miles of non-motorized trails completed connecting activity centers
  • Decrease in intermodal transfer costs, improved economic development potential, and improved community environmental quality resulting from an improved marine/rail freight terminal connection
 

Project is consistent with Statewide and Metropolitan planning processes

Contributes to alleviation of priority area transportation and related problems identified in the 20 year plan and any "visioning"

Includes projected life-cycle costs developed through financially constrained planning

Associated performance indicators and provisions for monitoring, possibly including those in transportation management systems

Plan is directed at achieving desired TCSP outcomes

Project has clear statement of purpose and need

Consistency with defined goals and objectives


[List of Tables]

Outcome Evaluation

Outcome evaluation focuses on determining the effectiveness of the project at achieving particular transportation, community, and system preservation objectives, such as reductions in emissions or preservation of open space. Measuring the outcomes of a project is, in many ways, the most difficult aspect of evaluation. Numerous factors must be considered, such as distinguishing the impacts of the program from other concurrent changes and identifying the time scale over which impacts occur. Measurement of outcomes, however, is ultimately of critical importance to determining whether a project is worthwhile. Therefore, grant applicants are encouraged to give careful thought to how the impacts of the proposed programs can be directly assessed. Applicants are encouraged to seek agreement with both traditional and non-traditional partners regarding the specific set of outcome measures to be evaluated.

This section provides guidance regarding issues to consider in evaluating the outcomes of projects funded through TCSP. Section 5.0 provides additional references on how to design an evaluation program and implement specific evaluation methods.

Approaches to Measuring Outcomes

Three general approaches may be taken to measuring the outcomes of a project:

As appropriate, grant applicants should identify a balanced set of techniques that allow evaluation of the economic, environmental, mobility, and social equity effects of strategies or investments.

In developing proposals, grant applicants are encouraged to predict - at least from a qualitative standpoint - the potential impact of the proposed project on each of the outcome performance measures which have been identified. Proponents are also encouraged to predict impacts on a quantitative basis, using available modeling or sketch planning tools, although in many cases appropriate tools may not exist or may not be readily usable.

The applicant also should develop a plan for measuring the impacts of the project once it has been implemented as part of the evaluation plan in the applicant's proposal. Ideally, this plan will include data collection and/or analysis which is capable of quantifying the impacts of the project on identified performance measures. It also may include, in some situations, development or refinement of analytical models to predict the impacts of the project. In many cases, however, it is likely that accurate quantitative measurements or forecasts will either be difficult to obtain or will not be relevant to the type of project being implemented. In this case, qualitative assessments should be performed in order to gauge the magnitude and nature of project impacts.

General Measurement Issues

Important issues to consider in designing an evaluation plan - whether quantitative or qualitative - include:

Collection of before-and-after data on both the affected population and control groups can be a particularly effective means of isolating the effects of a program, as illustrated in Figure 2.

Figure 2. Use of Control Group in Before-and-After Data Collection

Figure 2. Use of Control Group in Before-and-After Data Collection

[List of Figures]

Data collection plans, where possible, should account for seasonal fluctuations in the variables being measured, in addition to identifying longer-term trends. For example, many areas experience higher levels of pedestrian and bicycle activity in summer than in winter.

Available Evaluation Methods and Data Sources

Table 5 illustrates examples of outcome-related goals and objectives of the overall TCSP program, along with associated performance measures and methods for evaluating these measures. These performance measures are provided as examples and may not be relevant to all projects or measurable in all situations. Grant applicants are encouraged to define their own short list of meaningful, performance measures, as well as those goals and objectives which may be important locally. Applicants are further encouraged to identify the most appropriate and feasible evaluation methods for developing these performance measures.

Table 5. TCSP Outcome Evaluation
Sample Goals/Objectives, Performance Measures, and Evaluation Methods

Goal/Objective

Performance Measures (examples)

Evaluation Method(s)

Improve efficiency of transportation system (maximize use of existing infrastructure)

Percent of trips by non-SOV modes

Before/after counts & ridership surveys

Stated preference surveys

Modeling

Person-miles of travel per vehicle-mile of travel

Regional travel model

Transit passenger-miles per vehicle revenue-mile

National Transportation Database

Avoid need for new major construction:

  • Lane-miles per person
  • Avoided lane-miles of construction
  • Maintain LOS without new facilities
  • Lane miles per registered driver
 
  • TIP analysis under "baseline" versus "TCSP" condition
  • Regional travel model: lane-miles required to maintain base level of performance ("baseline" versus "TCSP" condition)
 

Total annual infrastructure cost per unit of travel (declining over time)

Analysis of TIP, LRTP, and travel forecasts

Reduce impacts on environment

Total VMT and VMT/person

Surveys or modeling to determine changes in mode shares, total trips, trip lengths

Criteria pollutants, greenhouse gas emissions

Emissions models based on travel impacts (trips, VMT)

Fuel consumption (total and per person)

Energy models and fuel utilization factors

Community impacts (aesthetics/ design, noise):

  • Community satisfaction
 

Satisfaction surveys

Focus groups

Interviews with key local officials

Land consumption per unit development (square feet or acres per dwelling unit, job, etc.)

Zoning regulations - permitted densities (with versus without program)

Actual versus expected development statistics

Accommodation of expected growth within existing urbanized area

Land use databases, mapping of building permits

Wetland/other habitat preservation/fragmentation:

  • Amount of preserved habitat space (with versus without program)
  • Connectivity/fragmentation of natural areas
 

Pre: Zoning regulations - allowable land use/development patterns (with versus without program)

Post: Actual versus expected preserved land

Maps showing natural areas/ ecosystems

Reduce costs of infrastructure investment

Projected life-cycle cost savings:

  • Costs of "baseline" versus "TCSP" projects in TIP
 

Analysis of TIP (Baseline versus TCSP conditions)

Life-cycle infrastructure cost analysis

Development of method and/or research study for relating travel or land use changes to infrastructure costs

 

Ensure efficient access to jobs, services, centers of trade

Quantitative accessibility measures (by type of activity, population segment), trips per person for all trip purposes

Travel demand models - before/ after accessibility measures

Proximity analysis using GIS or manual calculation

Travel time savings (passenger or freight movements)

Travel demand models

Project-specific calculations

Improvements in access for specific populations/needs:

  • Total population served
  • Number of users of new transit service
 

Usage measurements

Interviews with planners, service providers, etc.

Economic impacts of project:

  • Property values
  • Business Sales
  • Employment
 

Time-series analysis (before/after studies)

Qualitative analysis (surveys of businesses and property owners)

Encourage private sector land development patterns to achieve above objectives

Implementation of policies/incentives to affect development patterns

Review of changes in general plan, zoning, tax policies, impact fees, etc.

Agreements with private developers

Interviews with local officials

Review of other agreements

Changes in development patterns/trends:

  • Types and character of land use
  • Densities
  • Location of new development
 

Compare new developments to existing developments

Compare new developments in area to those elsewhere in region

Evidence of developer interest in affected area

Impacts on performance measures identified for above objectives

Quantitative assessment methods as identified above


[List of Tables]

Table 6 identifies potential existing data sources that can be used for project evaluation. Table 7 identifies methods for collecting new data as well as applications for each method.

Table 6. Potential Existing Data Sources for Evaluation

Type of Data

Existing Sources

Traffic data (volumes, speeds)

Highway Performance Monitoring System (HPMS); local monitoring stations (Metropolitan Planning Organization, or city or county traffic engineering department)

Transit ridership

Systemwide data: National Transit Database

Route or area-specific data: Local transit agency

Personal and household travel characteristics (mode shares, travel time, etc.)

U.S. Census of Population and Housing

National Personal Transportation Survey

Metropolitan area household travel survey (Metropolitan Planning Organization)

Worksite travel characteristics (mode choice, etc.)

Local Transportation Management Associations, ridesharing agencies

Business sales, employment, income

U.S. Census of Retail Trade

County Business Patterns

Land use and development

Local or regional land use databases (Metropolitan Planning Organization, or city or county planning department)

Aerial photography (Metropolitan Planning Organization, or city or county planning department)

Parcel-level data (city or county assessor's office)

Building permits (city or county planning department)


[List of Tables]

Table 7. Methods for Collecting New Data

Method

Uses of Method

Quantitative Data Collection

Field observations of traffic volumes or speeds, transit ridership, pedestrian activity, etc.

Before-after or control group comparison

Random sample telephone/mail surveys

Determine travel behavior (mode choice, trip-making, etc.)

Determine satisfaction, awareness, etc.

Workplace, establishment, and visitor surveys

Determine travel characteristics of travelers to specific sites

Transit onboard surveys

Determine transit ridership, trip characteristics, traveler characteristics

Surveys of businesses

Determine sales, employment, property value, development impacts, etc.

Stated preference (hypothetical choice) surveys

Determine what people would do in a hypothetical situation (use for forecasting)

Windshield surveys

Determine land uses and development patterns through observation

Qualitative Data Collection

Interviews

Obtain information from key persons

Focus groups

Use of a structured group discussion to gather information from multiple participants (either key players or a random selection)

Observation of points of common agreement as well as disagreement

Field observation methods

First-hand observation of activities, behavior, etc.


[Top of Section, Table of Contents, List of Tables, Top of Document]


5. Evaluation References

The following documents provide additional guidance on designing and implementing a data collection and evaluation plan. References also are provided on qualitative analysis methods and on the design of planning processes. In addition to addressing generic evaluation issues and methods, many of these documents describe evaluations of specific transportation programs.

Transportation-Related Data Collection, Evaluation, and Experimental Design

Institute of Transportation Engineers. Manual of Transportation Engineering Studies. H. Douglas Robertson, ed. Prentice Hall: Englewood Cliffs, NJ (1994). This reference manual discusses data collection methods for traffic volumes and speeds, public transportation, pedestrian activity, goods movement, environmental impacts, and other transportation data. The manual also discusses general methodological issues including experimental design, survey design, and statistical analysis methods. Available through the Institute of Transportation Engineers bookstore at 525 School Street, S.W., Suite 410, Washington, D.C. 20024-2797; Phone: 202/554-8050; Fax: 202/863-5486; Internet: http://www.ite.org.

Cambridge Systematics, Inc. and Barton Aschman Associates. Travel Survey Manual. Prepared for the U.S. Department of Transportation and U.S. Environmental Protection Agency, Publication No. FHWA-PL-96-029 (Manual) and FHWA-PL-96-030 (Appendices), (1996). This guidance manual discusses the design, implementation, and uses of various types of surveys used in transportation planning, including household travel surveys, vehicle intercept and external station surveys, transit onboard surveys, commercial vehicles surveys, workplace and establishment surveys, visitor surveys, parking surveys, and stated response surveys. The manual can be ordered from the U.S. Department of Transportation at: TASC Subsequent Distribution Office, Ardmore East Business Center, 3341 Q 75th Avenue, Landover, MD 20785; Fax: 301/386-5394; e-mail: SDS.Info@OST.DOT.GOV. Refer to complete title, Travel Survey Manual and Appendices, and publication numbers FHWA-PL-96-029 and FHWA-PL-96-030 when ordering.

Richardson, Anthony, E.S. Ampt, and A.H. Meyburg. Survey Methods for Transport Planning. Wiley-Interscience Publications: New York, NY (1995). This book discusses elements in designing and implementing various types of surveys used in transportation planning. Specific elements include selection of survey method, sampling procedures, survey instrument design, survey administration, and data processing and analysis.

Cambridge Systematics, Inc., Economic Impact Analysis of Transit Investments: Guidebook for Practitioners. Transit Cooperative Research Program: Report 35, National Academy Press, Washington, D.C. (1998). This reports presents 12 evaluation methods for use in evaluating the economic impacts of transit projects. The report describes uses of each method, advantages and disadvantages, data sources, examples, and provides guidance for selecting methods. Many of the methods and issues discussed are generically relevant to the evaluation of all types of transportation-related projects, as well as to the evaluation of impacts other than economic impacts. Transit Cooperative Research Program (TCRP) reports can be ordered through the Internet at http://www.nas.edu/trb/index.html or by writing: Transportation Research Board, National Research Council, 2101 Constitution Avenue, NW, Washington, D.C. 20418.

Casey, Robert F. and John Collura. Advanced Public Transportation Systems: Evaluation Guidelines. Prepared by the U.S. Department of Transportation, Volpe National Transportation Systems Center, for the Federal Transit Administration, Publication Nos. FTA-MA-26-0007-94-2 and DOT-VNTSC-FTA-93-9 (January 1994). This report provides guidelines for evaluating Advanced Public Transportation Systems, including; identification of performance measures; techniques for collection, deriving, and analyzing data; issues in experimental design; survey methods and execution; and statistical methods. Much of the guidance is relevant to the evaluation of transportation programs in general. The report is available through the National Technical Information Service, Springfield, VA, and on the Internet at http://ntl.bts.gov/DOCS/ate.html.

Qualitative Assessment Techniques

Krueger, Richard A. Focus Groups: A Practical Guide for Applied Research. Sage Publications: Thousand Oaks, CA (1992).

Mishler, Elliot G. Research Interviews: Context and Narrative. Harvard University Press: Cambridge, MA (1986).

Yin, Robert. Case Study Research: Design and Methods. Sage Publications: Beverly Hills, CA (1992).

Planning Processes

United States Department of Transportation. A Guide to Metropolitan Transportation Planning Under ISTEA: How the Pieces Fit Together. Publication No. FHWA-PD-95-031 (1995). Available on the Internet at: http://ntl.bts.gov/DOCS/424MTP.html

United States Department of Transportation. Statewide Transportation Planning Under ISTEA: A New Framework for Decision-Making. Publication No. FHWA-PD-96-026 (1996).

United States Department of Transportation. Metropolitan Transportation Planning Under ISTEA: The Shape of Things to Come (1997).

United States Department of Transportation, Volpe National Transportation Systems Center, Enhanced Planning Reviews of 14 Metropolitan Areas, prepared for FTA and FHWA, 1991-1997. Available on the Internet at: http://www.fta.dot.gov

Innes, Judith. Planning Through Consensus Building. Journal of the American Planning Association (Autumn 1996).

Ozawa, Connie. Recasting Science: Consensual Procedures in Public Policy-Making. Westview Press (1991).

Susskind, Lawrence, and J. Cruikshank. Breaking the Impasse: Consensual Approaches to Resolving Public Disputes. Basic Books: New York, NY (1987).

[Top of Section, Table of Contents, Top of Document]

Updated: 08/01/2013
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000