Evaluation

Guidance for the Preparation of TCSP Evaluation Plans - June 2001

Table of Contents

1. Importance of Evaluation
2. Grantee Roles and Responsibilities
   • Evaluation Plan
   • Assistatnce with Evaluation
3. General Approach to Evaluation
   • Steps in Developing an Evaluation Plan
   • What Should be Evaluated
   • Evaluation Reports
4. Detailed Evaluation Guidance
   • Process Evaluation
   • Product Evaluation
   • Outcome Evaluation
     • Approaches to Measuring Outcomes
     • General Measurement Issures
     • Available Evaluation Methods and Data Sources
5. Evaluation References
   • Transportation-Related Data Collection, Evaluation, and Experimental Design
   • Qualitative Assessement Techniques
   • Planning Processes

List of Figures

1. Steps in Developing an Evaluation Plan
2. Use of Control Group in Before-and-After Data Collection

List of Tables

1. Examples of TCSP Goals and Objectives
2. TCSP Process Evaluation
3. TCSP Product Evaluation: Planning Grant
4. TCSP Product Evaluation: Implementation Grant
5. TCSP Outcome Evaluation
6. Potential Existing Data Sources for Evaluation
7. Methods for Collecting New Data

1. Importance of Evaluation

The purpose of the Transportation and Community and System Preservation (TCSP) Pilot Program is to fund innovative projects that will increase the knowledge of the costs and benefits of different approaches to integrating transportation investments and strategies, community preservation, land development patterns, and environmental quality. Planning and implementation projects may be undertaken at the neighborhood, local, metropolitan, State, and regional levels by States, local governments, and metropolitan planning organizations (MPOs) working in cooperation with non-traditional partners. The TCSP is a pilot program explicitly designed to encourage innovative strategies and techniques, the results of which can then be used by other public and private organizations throughout the country. While TCSP funding is not sufficient to implement projects on a nationwide basis, all organizations nonetheless will benefit by being able to easily tap into the experience of others to learn what might be applicable for their own situations and how these new transportation strategies and techniques can be most effectively implemented.

To accomplish this learning and the desired resultant transfer of experience, the evaluation of individual projects is a key component of the TCSP program. Evaluation of projects which are new or experimental in character will indicate the success of various activities at achieving the desired transportation, community, and system preservation objectives. The lessons learned from this process will be used in evaluating the overall TCSP program and will help develop more effective TCSP projects in the future. As a result, the TCSP program will provide an important nationwide learning experience.

In keeping with the TCSP program's emphasis on learning and evaluation, grant applicants should bear in mind the following points:

• An evaluation plan for each grant is to be included as part of each proposal.
• The evaluation plan should describe roles, responsibilities, project objectives, performance measures, evaluation methodologies, data sources, schedule milestones, and budgets.
• Efforts to reflect an effective approach to evaluation will receive a higher priority in the evaluation of proposals.
• The TCSP is a pilot program to develop and evaluate new strategies. Falling short of expected goals is acceptable, as long as the evaluations identify barriers encountered and efforts undertaken to overcome them. Documentation of both successes and problems encountered in carrying out TCSP activities will help other areas develop more effective approaches in the future.

In presenting this guidance for the preparation of evaluation plans, it is recognized that, "one size does not fit all." This guidance, therefore, provides ideas for evaluation plans rather than a mandated approach.

[Top of Section, Table of Contents, Top of Document]

2. Grantee Roles and Responsibilites

Evaluation Plan

TCSP grantees are responsible for conducting a systematic evaluation of their TCSP project. Each grant application should include an evaluation plan which describes how the grantee proposes to evaluate the project. This will assist in demonstrating the applicant's commitment to the evaluation component. The strength of the evaluation component, including identification of resources required, will be an important factor in the selection of final grant awards.

FHWA will use the results from individual evaluations, in conjunction with other overall program evaluation criteria and methods, in assessing the overall effectiveness of the TCSP program. As results and lessons learned from individual TCSP grant awards become available and the overall program can be assessed, the FHWA will coordinate and disseminate results, tools, and information developed through the program.

In the evaluation plan submitted, grantees should identify program goals and objectives, performance measures, measurement techniques, potential data sources, and schedule milestones. Proposals should identify existing sources of information which will be utilized (either qualitative or quantitative), and should also identify any new data collection efforts which may be required or useful for evaluating the effectiveness of the program. The evaluation plan also should contain clear roles, responsibilities, commitments by participants, and a budget estimate. The resources required for evaluation activities should be included in the overall grant budget proposed for the project.

As a component of the TCSP program evaluation, a grant workshop is planned for the Spring of 1999, at which grantees will share experiences and initial results from their projects. Budgets for grant applications should include travel for the key investigator to this workshop, as well as a second such conference, as part of the evaluation component.

Assistance with Evaluation

The remainder of this document provides guidance relating to the development of an evaluation plan. The purpose of this guidance is to provide ideas rather than a mandated approach, and agencies should not be discouraged from applying for TCSP program funding simply because they lack expertise in particular evaluation methods. It is more important that grant applicants commit to undertaking a systematic evaluation, including the designation of project resources, than they demonstrate proficiency in any particular evaluation method. Grant applicants not already having the desired level of in-house evaluation expertise may want to consider working in cooperation with another agency or a university.

[Top of Section, Table of Contents, Top of Document]

General Approach to Evaluation

This section provides an overview of how to develop an evaluation plan, with more detail on how to structure and conduct the evaluation described in Section 4.0. In particular, Section 4.0 identifies specific techniques that may be used, issues to consider, and key questions to ask in evaluating a TCSP project.

Steps in Developing an Evaluation Plan

Grant applicants are encouraged to take the following steps in developing an approach to project evaluation, as illustrated in Figure 1:

Figure 1. Steps in Developing an Evaluation Plan

[List of Figures]

1. Define Project goals and objectives. What is the motivation for undertaking the project? What is the project intended to accomplish? Table 1 shows examples of general goals and objectives for the overall TCSP program. Goals and objectives for individual TCSP projects may be a subset of these program goals and objectives. In addition, grant applicants may have additional goals and objectives which are important for the project to achieve locally.

2. Identify performance measures. Performance measures are either quantitative or qualitative measures which indicate the success of the project at achieving its stated goals and objectives, e.g., total emissions per capita or land consumed per unit of development. Examples of performance measures for the identified TCSP program goals and objectives are shown in Section 4.0. Applicants, however, should resist the temptation to establish a "laundry list" of performance measures, but instead should identify a few key measures which best reflect the impacts of the program. It is also important to select performance measures which are simple to understand, are as objective as possible, and can be constructed from available data sources.

3. Identify data and information sources and evaluation methods. Grant applicants should identify data and information sources to support each performance measure. In the case of quantitative data, applicants should identify both existing sources and potential new data collection efforts. In the case of qualitative data, proponents should identify key sources of information (people, agencies, committees, etc.), along with appropriate techniques for obtaining and evaluating information (interviews, direct observation, etc.) Some potential data sources and evaluation techniques are identified in Section 4.0. Consideration also should be given to identifying the baseline condition from which changes will be assessed.

Table 1. Examples of TCSP Project Goals and Objectives

[List of Tables]

Once potential performance measures, data sources, and evaluation methods have been identified, an overall evaluation plan should be developed for collecting and analyzing the required information. This includes identifying the individual work tasks required to carry out the evaluation and establishment of the associated budget and timeline for these tasks.

What Should Be Evaluated?

A TCSP evaluation should focus on identifying both the magnitude and the distribution of the costs and benefits of a project, and on those aspects of the planning and implementation process that will be useful to other organizations in deciding whether or not to implement similar strategies. Thus, evaluations can focus on three different aspects of a TCSP project: process, products, and outcomes. Appropriate goals and objectives, performance measures, and evaluation methods will differ for each, as will the timeframe over which the evaluation is conducted.

• Process evaluation focuses on the approach through which a project is developed and implemented. A process evaluation can focus on questions such as the number and types of both traditional and non-traditional groups or persons involved, the manner in which these groups have been involved, the degree to which stakeholder commitment and buy-in were achieved, and the nature of the issues which emerged as being important in the deliberations.

• Product evaluation focuses on what was produced by the process or activity. For example, a description of the plan that was developed or project that was implemented, and how it compares to what was originally implemented. How many miles of sidewalk were built connecting residential neighborhoods with employment and activity centers, public transportation systems, or recreational areas?

• Outcome evaluation focuses on determining the effectiveness of the project at achieving the defined transportation, community, and system preservation objectives. How much are emissions reduced? What is the reduction in infrastructure cost per unit of person-travel?

These three aspects of a project are interrelated and important to the evaluation of a TCSP project. Outcome goals are of ultimate interest to society, but achievement of process and product goals can indicate the likelihood of success at achieving the desired outcomes. Process and product goals are also desirable for their own sake. For example, an open and participatory process is important for ensuring that all viewpoints and potential impacts are considered. The involvement of non-traditional partners will help to identify strategies that encourage private sector development patterns that are consistent with the goals of the TCSP program. Examining the linkages among process, product, and outcome also can be useful. For example, desirable outcomes can be facilitated by the relationships developed during a planning process. Conversely, difficulties encountered during implementation may be traceable to the unintentional omission of an important factor during the planning stage. Finally, evaluation of all aspects of a project serves as an important learning tool, helping to identify both successful and unsuccessful approaches to a problem.

Evaluation Reports

An initial evaluation plan is to be included by an applicant as part of the application for a TCSP grant. This initial plan then may be refined in negotiating the terms of a grant awarded to the applicant. While the evaluation plan is expected to cover the basic approach proposed for evaluating a TCSP planning or implementation grant, the details of an evaluation plan, such as the statistical basis for a stratified sampling plan, may not be fully developed until after a project is actually underway.

The evaluation activities associated with a TCSP grant should result in one or more reports. The purpose of these reports is to provide the information needed by other organizations throughout the country to decide whether similar projects would be beneficial within their jurisdictions, and how they should go about planning or implementing this particular kind of action.

The initial evaluation report should document the process by which the TCSP grant project was developed or implemented, as well as the final product of the grant. This report can be produced shortly after completion of the project. Initial information on the results or outcomes of the project may also be available soon after completion and can be documented in this initial evaluation report. It is possible, however, that the full impacts of a project will not occur immediately, and that additional documentation of project outcomes will be appropriate in the future as data on longer-term impacts become available. The proposed approach to reporting should be explained in the evaluation plan portion of the grant application.

[Top of Section, Table of Contents, Top of Document]

4. Detailed Evaluation Guidance

This section provides more detailed guidance on evaluating the process, product, and outcomes of TCSP projects. For process and product evaluations, key questions for obtaining information as background to the evaluation are identified. For outcome evaluations, specific techniques and issues to consider in either estimating or measuring the impacts of the TCSP project are identified. For all three types of evaluations, examples of goals and objectives, performance measures, and evaluation methods relevant to TCSP projects are provided.

It is important to identify in each case a baseline from which a change is being determined. For a process evaluation, this can be simply a comparison of the new or TCSP planning process with the existing or traditional approach. For a product evaluation, this can include an assessment of how the final project differs from what was initially proposed. Baseline considerations in estimating project outcomes include issues of time scale and differentiating project impacts from parallel changes in other significant factors, as discussed in Section 4.3.

Process Evaluation

Evaluation of the process by which the TCSP plan or project was produced or implemented can serve a number of useful functions. Process evaluation can identify reasons for success or failure of the plan or project as well as specific strategies and tactics which were most effective. Evaluation of specific aspects of the process, such as who participated and their respective roles, also can help indicate how likely the product is to achieve success. For example, extensive participation of a variety of affected parties or groups may mean that the project is more likely to be successful, since potential obstacles and stumbling blocks can be resolved.

A number of techniques can be used to gather information for evaluating the process, including:

• Direct observations of process activities;
• Interviews or discussions with facilitators of the process and process participants;
• Reviews of documents, including process schedules, timelines, and workplans; participation and attendance lists; meeting agendas and minutes; plans and reports produced; and letters of support.

Questions that can be asked as a basis for evaluating the process include:

• Who participated (organizations, titles, level of authority to act on behalf of organization, etc.);
• Who did not participate; whether they (a) opted out or (b) were not invited; and why;
• What the participants' roles were (e.g., attend meetings, read and critique materials, produce data/reports, partners in planning, partners in decision-making, etc.);
• What the process for planning was:
  • Who established the agenda and how it was done;
  • Schedule and organization of meetings and other actions;
  • When and how were goals established;
  • Development of background information and supporting analysis (what was performed; how was it used in supporting plan development or project selection);
  • Process for reaching decisions (discussion and vote, discussion to agreement, recommended options and a decision by others, consultation with others followed by decision, etc.);
  • Support in documentation of process for goals and decisions;
  • What factors influenced the decision;
• Relationship of process to existing planning processes and activities, including the metropolitan and statewide transportation planning process;
• Substantive issues covered;
• Timeframe of substantive issues (current focus, future - short-term, future - long-term);
• Actions taken;
• Legitimacy to implement plan or project:
  • Legal authority;
  • Political legitimacy;
  • Financial resources identified.

Documenting answers to the above questions can determine the degree to which the process met its defined goals and objectives. Some process-related goals and objectives for the TCSP program, as well as associated performance measures, are shown in Table 2. Local agencies may also hold other goals and objectives for activities carried out under the TCSP program. Documenting the answers to these questions also will help in identifying circumstances or actions that influenced the level of success of the final product.

Table 2. TCSP Process Evaluation
Sample Goals/Objectives and Performance Measures

[List of Tables]

Evaluation of improved linkages to metropolitan or statewide planning process, as encouraged by TEA-21, is of particular importance, although this may not be relevant to all TCSP grants. As applicable, grantees might evaluate their ability to improve connections through the funded project with the broad metropolitan or statewide transportation planning processes at the center of TEA-21. Linkages to the planning process can be flexible, and could be demonstrated, for example, by:

• Contributing to alleviation of priority area transportation and related problems identified in the 20-year plan and any "visioning"
• Applying performance indicators, possibly including those in transportation management systems;
• General or specific support from a public involvement process;
• Development through collaborative partnerships, for example, involving the MPO, state transportation and environmental agencies, city planning agencies, transit, or non-traditional partners; or
• Projecting life-cycle costs developed through financially constrained planning.

With respect to the public involvement process for transportation planning in particular, federal guidelines suggest the following desirable outcomes of public involvement (Reference: A Guide to Metropolitan Transportation Planning Under ISTEA: How the Pieces Fit Together. U.S. Department of Transportation, Washington, D.C., 1995.):

• Informed and involved citizens with access to public records and the decision-making process;
• A planning approach that is proactive and open to early participation by all;
• A process that not only encourages broad public participation but also considers and responds to public input;
• Appropriate and early interagency consultation in air quality non-attainment areas;
• Ample opportunity for public comment when the final plan or TIP differs from the draft.

Product Evaluation

Product evaluation focuses on what was produced by the planning or implementation activity. A description of the project as it was actually produced or implemented can serve as an interim step in identifying the likely outcomes or impacts of the project. Some general questions that can be asked about the product include:

• What was the product of the activity, and how does it compare to what was originally planned?
• What did the product accomplish?
• Why does it matter - what impact did the product make, with respect to both the defined project objectives and the overall objectives of the TCSP program?
• To whom does it matter - who is impacted?
• Is there anything innovative in the project? What was done that had not been done before?
• What was learned that wasn't already known? What was the added knowledge and how important is it?
• How can the lessons learned from this project be generalized to other situations?

Evaluation of the product of a TCSP activity will differ significantly depending on whether the activity is a planning or implementation grant. In the case of an implementation activity, product evaluation can focus on describing what was actually built, or what service was developed, and why it is significant. In the case of a planning activity, product evaluation will focus on the content of the plan, agreement, etc. (e.g., what will be achieved if the plan is implemented or the agreement carried out); adoption of the plan; and on provisions to ensure successful implementation of the plan or agreement. While development of the plan or project consistent with the original scope of work, timeline, and budget may be a criterion, this should not limit flexibility in making mid-course modifications to a project. As planning and implementation progresses, it is possible that changes to the project may be incorporated that result in an improved product compared to the original proposal.

Table 3 shows examples of goals and objectives and performance measures for evaluating the product of a planning grant. Table 4 shows examples of goals and objectives and performance measures for evaluating the product of an implementation grant.

Table 3. TCSP Product Evaluation: Planning Grant
Sample Goals/Objectives and Performance Measures

[List of Tables]

Table 4. TCSP Product Evaluation: Implementation Grant
Sample Goals/Objectives and Performance Measures

[List of Tables]

Outcome Evaluation

Outcome evaluation focuses on determining the effectiveness of the project at achieving particular transportation, community, and system preservation objectives, such as reductions in emissions or preservation of open space. Measuring the outcomes of a project is, in many ways, the most difficult aspect of evaluation. Numerous factors must be considered, such as distinguishing the impacts of the program from other concurrent changes and identifying the time scale over which impacts occur. Measurement of outcomes, however, is ultimately of critical importance to determining whether a project is worthwhile. Therefore, grant applicants are encouraged to give careful thought to how the impacts of the proposed programs can be directly assessed. Applicants are encouraged to seek agreement with both traditional and non-traditional partners regarding the specific set of outcome measures to be evaluated.

This section provides guidance regarding issues to consider in evaluating the outcomes of projects funded through TCSP. Section 5.0 provides additional references on how to design an evaluation program and implement specific evaluation methods.

Approaches to Measuring Outcomes

Three general approaches may be taken to measuring the outcomes of a project:

• Quantitative assessment techniques, such as measurement of traffic volumes, access to jobs, economic growth, or land preservation before and after project implementation. Quantitative assessment may be based on actual counts or other field data collection; or on surveys of travelers, potential travelers, or businesses to determine behavior before and after the project. It also may include surveys to quantify changes in "soft" variables such as satisfaction with transportation and community characteristics or awareness of the impacts of various transportation or land development alternatives.

• Qualitative assessment techniques, such as interviews, surveys, focus groups, review of minutes of meetings with non-traditional partners, and anecdotal evidence. Qualitative techniques are likely to be the primary source of evaluation when: 1) the scale of impacts is too small to be measured directly; 2) resources are not available for quantitative data collection or modeling techniques; or 3) the primary impacts are "soft" effects, such as quality of the community environment, that cannot be easily quantified or valued. Qualitative methods also are useful for verifying findings from quantitative evaluation techniques.

• Analytic procedures or models which forecast the impacts of a project. Analytical models include regional travel and land use models, simulation models, sketch-planning tools, emission models, and other quantitative forecasting methods. In addition to forecasting impacts, models can be used for converting directly measurable impacts, such as travel changes, into other impacts, such as emissions. These models also can be used to control for external factors and validate the results of before-and-after data analysis. Examples where modeling may be useful include policies which influence the nature and location of development, or actions which change the relative time or cost of travel by different modes. On the other hand, modeling will not be applicable to many types of activities, such as some very small scale projects, enhanced public involvement or the formation of regional decision-making bodies.

As appropriate, grant applicants should identify a balanced set of techniques that allow evaluation of the economic, environmental, mobility, and social equity effects of strategies or investments.

In developing proposals, grant applicants are encouraged to predict - at least from a qualitative standpoint - the potential impact of the proposed project on each of the outcome performance measures which have been identified. Proponents are also encouraged to predict impacts on a quantitative basis, using available modeling or sketch planning tools, although in many cases appropriate tools may not exist or may not be readily usable.

The applicant also should develop a plan for measuring the impacts of the project once it has been implemented as part of the evaluation plan in the applicant's proposal. Ideally, this plan will include data collection and/or analysis which is capable of quantifying the impacts of the project on identified performance measures. It also may include, in some situations, development or refinement of analytical models to predict the impacts of the project. In many cases, however, it is likely that accurate quantitative measurements or forecasts will either be difficult to obtain or will not be relevant to the type of project being implemented. In this case, qualitative assessments should be performed in order to gauge the magnitude and nature of project impacts.

General Measurement Issues

Important issues to consider in designing an evaluation plan - whether quantitative or qualitative - include:

• The time scale over which impacts are measured. In some cases, usage may increase over time as people become aware of the new project or service, and it may take a year or two for a project to achieve significant results. In other cases, such as with changes to land use and development practices, impacts may not be fully apparent for many years. Evaluation plans should identify the anticipated time scale of impacts and include provisions for both near-term and longer-term monitoring of these impacts.

• Separation of project impacts from the impacts of external factors. For short-term evaluation, data collected prior to project implementation may be sufficient as a baseline to which data collected after implementation can be compared. For longer-term evaluation, more sophisticated methods may be required to compare measured changes to a future "baseline" level which may be affected by other concurrent changes, such as changes in the economy, demographic trends, or gasoline prices. Techniques for doing this include:
  • Identification and documentation of changes in other potentially significant factors. A qualitative assessment of the impacts of these factors, including the magnitude and direction of the changes, can help indicate which factors are most significant in influencing the measured changes. For example, a sharp rise in gasoline or other travel-related prices would be expected to lead to reduced automobile travel.
  • The use of control groups. Trends in travel behavior or land development patterns, for example, can be compared between the community affected by the project and other similar communities which are not affected.
  • Time-series analysis of data. Time-series data analysis techniques can be used to predict actual versus expected changes and quantify the contribution of other factors to observed changes.

Collection of before-and-after data on both the affected population and control groups can be a particularly effective means of isolating the effects of a program, as illustrated in Figure 2.

Figure 2. Use of Control Group in Before-and-After Data Collection

[List of Figures]

Data collection plans, where possible, should account for seasonal fluctuations in the variables being measured, in addition to identifying longer-term trends. For example, many areas experience higher levels of pedestrian and bicycle activity in summer than in winter.

• Issues of sampling and statistical significance. For quantitative measurement - whether through surveys or field data collection - an appropriate population on which to measure impacts must be determined. Data collection and sampling plans should ensure that the measured impacts are representative of actual impacts on the population. Sample sizes should be selected so that results will be statistically significant given the expected magnitude of project impacts. The use of panel surveys (sampling the same people before and after project implementation) may reduce data collection requirements compared to selection of a random sample both before and after the project. Finally, non-users as well as users should be surveyed, in order to identify barriers to use.

Available Evaluation Methods and Data Sources

Table 5 illustrates examples of outcome-related goals and objectives of the overall TCSP program, along with associated performance measures and methods for evaluating these measures. These performance measures are provided as examples and may not be relevant to all projects or measurable in all situations. Grant applicants are encouraged to define their own short list of meaningful, performance measures, as well as those goals and objectives which may be important locally. Applicants are further encouraged to identify the most appropriate and feasible evaluation methods for developing these performance measures.

Table 5. TCSP Outcome Evaluation
Sample Goals/Objectives, Performance Measures, and Evaluation Methods

[List of Tables]

Table 6 identifies potential existing data sources that can be used for project evaluation. Table 7 identifies methods for collecting new data as well as applications for each method.

Table 6. Potential Existing Data Sources for Evaluation

[List of Tables]

Table 7. Methods for Collecting New Data

[Top of Section, Table of Contents, List of Tables, Top of Document]

5. Evaluation References

The following documents provide additional guidance on designing and implementing a data collection and evaluation plan. References also are provided on qualitative analysis methods and on the design of planning processes. In addition to addressing generic evaluation issues and methods, many of these documents describe evaluations of specific transportation programs.

Transportation-Related Data Collection, Evaluation, and Experimental Design

Institute of Transportation Engineers. Manual of Transportation Engineering Studies. H. Douglas Robertson, ed. Prentice Hall: Englewood Cliffs, NJ (1994). This reference manual discusses data collection methods for traffic volumes and speeds, public transportation, pedestrian activity, goods movement, environmental impacts, and other transportation data. The manual also discusses general methodological issues including experimental design, survey design, and statistical analysis methods. Available through the Institute of Transportation Engineers bookstore at 525 School Street, S.W., Suite 410, Washington, D.C. 20024-2797; Phone: 202/554-8050; Fax: 202/863-5486; Internet: http://www.ite.org.

Cambridge Systematics, Inc. and Barton Aschman Associates. Travel Survey Manual. Prepared for the U.S. Department of Transportation and U.S. Environmental Protection Agency, Publication No. FHWA-PL-96-029 (Manual) and FHWA-PL-96-030 (Appendices), (1996). This guidance manual discusses the design, implementation, and uses of various types of surveys used in transportation planning, including household travel surveys, vehicle intercept and external station surveys, transit onboard surveys, commercial vehicles surveys, workplace and establishment surveys, visitor surveys, parking surveys, and stated response surveys. The manual can be ordered from the U.S. Department of Transportation at: TASC Subsequent Distribution Office, Ardmore East Business Center, 3341 Q 75th Avenue, Landover, MD 20785; Fax: 301/386-5394; e-mail: SDS.Info@OST.DOT.GOV. Refer to complete title, Travel Survey Manual and Appendices, and publication numbers FHWA-PL-96-029 and FHWA-PL-96-030 when ordering.

Richardson, Anthony, E.S. Ampt, and A.H. Meyburg. Survey Methods for Transport Planning. Wiley-Interscience Publications: New York, NY (1995). This book discusses elements in designing and implementing various types of surveys used in transportation planning. Specific elements include selection of survey method, sampling procedures, survey instrument design, survey administration, and data processing and analysis.

Cambridge Systematics, Inc., Economic Impact Analysis of Transit Investments: Guidebook for Practitioners. Transit Cooperative Research Program: Report 35, National Academy Press, Washington, D.C. (1998). This reports presents 12 evaluation methods for use in evaluating the economic impacts of transit projects. The report describes uses of each method, advantages and disadvantages, data sources, examples, and provides guidance for selecting methods. Many of the methods and issues discussed are generically relevant to the evaluation of all types of transportation-related projects, as well as to the evaluation of impacts other than economic impacts. Transit Cooperative Research Program (TCRP) reports can be ordered through the Internet at http://www.nas.edu/trb/index.html or by writing: Transportation Research Board, National Research Council, 2101 Constitution Avenue, NW, Washington, D.C. 20418.

Casey, Robert F. and John Collura. Advanced Public Transportation Systems: Evaluation Guidelines. Prepared by the U.S. Department of Transportation, Volpe National Transportation Systems Center, for the Federal Transit Administration, Publication Nos. FTA-MA-26-0007-94-2 and DOT-VNTSC-FTA-93-9 (January 1994). This report provides guidelines for evaluating Advanced Public Transportation Systems, including; identification of performance measures; techniques for collection, deriving, and analyzing data; issues in experimental design; survey methods and execution; and statistical methods. Much of the guidance is relevant to the evaluation of transportation programs in general. The report is available through the National Technical Information Service, Springfield, VA, and on the Internet at http://ntl.bts.gov/DOCS/ate.html.

Qualitative Assessment Techniques

Krueger, Richard A. Focus Groups: A Practical Guide for Applied Research. Sage Publications: Thousand Oaks, CA (1992).

Mishler, Elliot G. Research Interviews: Context and Narrative. Harvard University Press: Cambridge, MA (1986).

Yin, Robert. Case Study Research: Design and Methods. Sage Publications: Beverly Hills, CA (1992).

Planning Processes

United States Department of Transportation. A Guide to Metropolitan Transportation Planning Under ISTEA: How the Pieces Fit Together. Publication No. FHWA-PD-95-031 (1995). Available on the Internet at: http://ntl.bts.gov/DOCS/424MTP.html

United States Department of Transportation. Statewide Transportation Planning Under ISTEA: A New Framework for Decision-Making. Publication No. FHWA-PD-96-026 (1996).

United States Department of Transportation. Metropolitan Transportation Planning Under ISTEA: The Shape of Things to Come (1997).

United States Department of Transportation, Volpe National Transportation Systems Center, Enhanced Planning Reviews of 14 Metropolitan Areas, prepared for FTA and FHWA, 1991-1997. Available on the Internet at: http://www.fta.dot.gov

Innes, Judith. Planning Through Consensus Building. Journal of the American Planning Association (Autumn 1996).

Ozawa, Connie. Recasting Science: Consensual Procedures in Public Policy-Making. Westview Press (1991).

Susskind, Lawrence, and J. Cruikshank. Breaking the Impasse: Consensual Approaches to Resolving Public Disputes. Basic Books: New York, NY (1987).

[Top of Section, Table of Contents, Top of Document]