Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration
Office of Planning, Environment, & Realty (HEP)

TMIP Peer Review Program Synthesis Report

(PDF version)

November 10, 2004

Prepared by
Volpe National Transportation Systems Center, USDOT
For the Federal Highway Administration

Table of Contents

I. Introduction

This report provides a synthesis of the nine TMIP Peer Review Panel meetings held between June 2003 and May 2004. Section I provides a brief overview of TMIP and the Peer Review Program. Section II provides a summary of each of the nine Peer Review Program Panels. Section III highlights common issues and recommendations identified among the nine Peer Review Panels that are pertinent to metropolitan planning organizations (MPO) and state departments of transportation (DOT). Section IV concludes the Synthesis by suggesting strategies for improving technical assistance offered by TMIP and the Peer Review Program.

A. TMIP Program Overview

The TMIP program provides a wide range of services to help planning agencies improve their travel analysis techniques. The program has three specific goals:

  1. To help planning agencies build their institutional capacity to develop and deliver travel related information to support transportation and planning decisions;
  2. To develop and improve analytical methods that respond to the needs of planning and environmental decision making processes; and
  3. To develop mechanisms to ensure the quality of technical analysis used to support decision-making and to meet local, state, and federal program requirements.

B. TMIP Peer Review Program

As an integral part of the overall program, the Peer Review Program provides state and local planning agencies the opportunity to solicit input from experts in the field of travel demand modeling. The purpose of the Peer Reviews is to have a process whereby experts in the field of modeling can provide guidance to ensure that agencies are developing technical products, procedures, and processes that meet an agency's needs and state, federal, and local planning requirements. In April 2003, the program began soliciting Peer Review applications from regional and state planning agencies.

II. Summary of the Nine Peer Review Panels: June 2003-May 2004

Between June 2003 and May 2004, TMIP sponsored nine Peer Review Panels. These Panels took place in MPOs and state DOTs throughout the nation. Peer Review Panels were held at the following MPOs:

Four of the Panels - two each at DRCOG in Denver and SCAG in Los Angeles - dealt with large urbanized areas developing innovative techniques to improve their current model program, while the other three dealt with other metropolitan planning issues.

Two of the Panels focused on statewide modeling efforts. Peer Review Panels took place at the following DOTs:

Table 1 below provides details on the characteristics of each of the regions that conducted a Peer Review Panel.

Table 1. TMIP Peer Review Agencies Characteristics
Type of Agency MPO MPO MPO MPO State DOT State DOT MPO
Number of Counties

8 counties

  • Butler
  • Clermont
  • Hamilton
  • Warren
  • Boone
  • Campbell
  • Kenton
  • Dearborn

9 counties

  • Adams
  • Arapahoe
  • Boulder
  • Clear Creek
  • Douglas
  • Gilpin
  • Jefferson
  • Denver
  • Broomfield

6 counties

  • LA
  • Orange
  • San Bernardino
  • Riverside
  • Ventura
  • Imperial

10 counties

  • Cherokee
  • Clayton
  • Cobb
  • DeKalb
  • Douglas
  • Fayette
  • Fulton
  • Gwinnett
  • Henry
  • Rockdale

3 models

  • Model 1: 10 counties, 4 MPOs
  • Model 2: 5 counties
  • Model 3: 4 counties
MTMUG composed of 8+ MPOs each with own model 1 county
Population 1,886,600 2,414,600 16,000,000 3,500,000 8,187,000 2,926,000 270,000; additional 66,000 in Mat-Su
Area (sq miles) 2,636 ~ 5,000 38,000 2,981 48,715 57,853 1,700

The nine Panels used a total of 27 Panel experts from a variety of backgrounds and areas of the country. The 27 Panel members included representatives from MPOs, state DOTs, private consulting firms, educational institutions, and an environmental interest group. Of the 27 Peer Review Panel members, there were:

Table 2 below shows the breakdown of the types of representatives for each of the Peer Review sessions (the two SCAG and DRCOG Panels are each grouped together). Appendix B lists the Panel Review details and provides a web link to the reports from the individual Reviews.

Table 2. TMIP Peer Review Panel Member Affiliations

MPO 3 3 3 5 4 1 4
SDOT         1    
Consultants 1 2 1     3  
Academia 1 1 1     1  
Environmental   1          
TOTAL 5 7 5 5 5 5 4

The nine meetings described in this synthesis report took place at a wide variety of settings with a variety of needs. The topics of the Peer Review Panels covered issues ranging from the Review of Denver's Integrated Regional Model (IRM) Project - which is to replace the existing land use and travel models with a state-of-the-art, fully-integrated modeling system - to the evaluation of current 4-step model updates to ensure consistency with state-of-the-practice techniques. Typically, the Peer Reviews covered a variety of technical details concerning the trip generation, trip distribution, mode choice, assignment, calibration and validation phases of modeling. Topics discussed within the trip generation and distribution phase included travel impedance, travel time, friction factors, special generators, and trip purposes. Topics discussed for the trip assignment phase included peak spreading, number of iterations for a model run, pre- and post-processing, and adjustment factors.

The Peer Review Panels took place at various stages of model development. Peer Review Panels were convened in the earlier stages of model development, where the requesting agencies sought consultation on anticipated or early developments of their model improvement work. Other agencies were further along in their model development, and utilized the Peer Review Panel to calibrate their model developments against the suggestions of the Panel Experts.

Table 3 below gives an overview of the focus of each of the Peer Review Panels, the modeling stage at which the Peer Review was held, and the model platform that each region has adopted.

Table 3. TMIP Peer Review Model Summary
Self-identified Peer Review Focus
  • Ability of models to address planning issues
  • Reliability and defensibility of forecasts
  • Consistency with state-of the-practice
  • Suggestions and comments for further enhancement
  • Outline planning issues of key importance
  • Discuss possible model improvements
  • Review work plans for auto availability and trip generation models, external trip models, mode choice models, and trip assignment models (1)
  • Review validation targets for all model components (2)
  • Provide ARC with an independent assessment of its travel demand modeling system
  • Compare the model with industry standards
  • Modeling best practices
  • Data
  • Calibration/ validation
  • Types of levels of travel demand modeling
  • Prioritized recommendations for improvements and timeline
  • Recommendations on organizational questions, such as the role of MPOs and staffing capacity
  • Assessment of current practices of model calibration/validation and reasonableness checking
  • Methodologies and best practices for enhancing accuracy and reliability
  • Review current status of travel model improvement process and
  • Provide guidance for near-term and future model development
Modeling Stage
  • Region enlarged to include Miami Valley Regional Planning Commission
  • Improving model and increasing expertise
  • Beginning blueprint for tour-based model, getting feedback
  • Re-estimating and recalibrating model with new data
  • Adding new sub-models
  • Improving model and increasing expertise
  • Developing statewide expertise, model, and data
  • Developing statewide expertise, model, and data
  • Identifying potential improvements for next round of changes
Modeling platforms
  • Homegrown solutions by pbConsult
  • Migrated to TransCAD and UrbanSIM from MinUTP and DRAM-EMPAL
  • Migrating to TransCAD
  • TP+ converted from TRANPLAN, moving to CUBE
  • Using DRAM/EMPAL
  • Gradually migrated to TransCAD from FORTRAN and TRANPLAN
  • Migrating to TransCAD from Tranplan and QRSII
  • Land use allocation model
  • TransCAD travel demand forecasting model

A. MPO Peer Review Panels

The following section provides a brief summary of each of the Peer Review Panels held over the past year.

1. Anchorage Metropolitan Area Transportation Study

The Anchorage Metropolitan Area Transportation Study (AMATS) held a Peer Review Panel meeting in May 2004 in Anchorage, Alaska. The Anchorage region has a population of 270,000, and covers an area of approximately 1,700 square miles. Because the existing Transportation Demand Model was validated to a 1994 base year, the model needs to be revalidated by 2004, in accordance with air quality conformity regulations which require serious carbon monoxide non-attainment areas to validate their models to observed counts. The primary focus of the Peer Review was to review the current status of the AMATS travel model improvement process and to provide guidance on near-term and future model development. The overall objective of the Peer Review was to seek assistance from knowledgeable transportation modeling practitioners on how to improve the AMATS travel demand model and associated land use allocation model.

The Peer Review covered a variety of topics, including:

The Peer Review Panel convened a private discussion of the AMATS model presentations provided the previous day. The recommendations were then presented to AMATS staff, as well as to members of the Technical Oversight Committee, a citizen advisory group to AMATS.

In general, the Peer Review Panel agreed that the AMATS 4-step model design is adequate and appropriate for a metropolitan area the size of Anchorage. The initial steps for data collection and the model structure were considered to be on par with Eugene, a metropolitan area in Oregon that is similar to Anchorage. The Peer Panel stated that they had difficulty in conducting an overall assessment of the AMATS model technical details because the update is still in its preliminary stages. The Panel did, however, identify specific elements of the model update process perceived as "red flags" for AMATS to consider as the model improvement process continues.

2. Atlanta Regional Commission

The Atlanta Regional Commission (ARC) held a Peer Review Panel meeting in February 2004, in Atlanta, Georgia. ARC serves as the MPO for a 10-country region around Atlanta. The region has a population of approximately 3.5 million residents and encompasses more than 2,981 square miles. ARC's current transportation model was implemented using input received in a previous Peer Review in 2000. The primary aim of the two-and-a-half day Peer Review was to provide ARC with an independent assessment of its travel demand modeling system updates and to compare the model with industry standards.

The Panel commented that over the last eight years, ARC has made many positive steps. Significant successes include:

As ARC proceeds to the next steps in their modeling process, the Panel presented ARC with 10 chief recommendations. Highlights included:

3. Denver Regional Council of Governments

The Denver Regional Council of Governments (DRCOG) held two Peer Review Panel meetings - the first in October 2003 and the second in April 2004. Both meetings were held in Denver, Colorado with the same members of the Peer Review Panel. Denver has a population of 2.5 million, with an additional growth of one million people expected over the next 20 years. DRCOG serves as the MPO for the Denver area. For the past 15 years, DRCOG has used a MinUTP-based model. Currently, DRCOG is in the early phases of replacing its existing land use and travel demand model with a fully integrated modeling system. The model improvement project, named Integrated Regional Model Project, is underway with an expected completion date of 2005. DRCOG has recently completed its Travel Behavior Inventory Project, a $1.5 million travel/activity/demographic survey of the region.

The purpose of the first Peer Review was to begin the development of a blueprint for a new modeling system for the Denver region, to include comprehensive redevelopment of all transportation and land use modeling elements. The TMIP Peer Panel was charged with helping the DRCOG project team identify approaches to the development of an integrated model. Recommendations from the Panel included:

DRCOG reconvened the Peer Review Panel in April to address general model design, potential staging issues, and other practical considerations for model development based on recommendations from the previous meeting. The second Peer Review consisted of presentations from DRCOG staff as well as its model consultants, pbConsult, on a number of issues.

The Peer Review Panel met in a separate session to develop its recommendations. Recommendations included:

4. Ohio-Kentucky-Indiana Regional Council of Governments

The Ohio-Kentucky-Indiana Regional Council of Governments (OKI) held a Peer Review Panel meeting in June 2003 in Cincinnati, Ohio. OKI is the MPO for an eight county region spanning the three states of Ohio, Kentucky, and Indiana and is located in Cincinnati. The region has a population of approximately 1.88 million people and covers 2,592 square miles. The region has conducted Peer Review Panels on their travel demand model in both 1994 and 1996. The major purpose of this TMIP Panel Review was to assess:

OKI recently expanded the coverage of its travel demand model to include an adjacent region, the Miami Valley Regional Planning Commission (MVRPC). The Panel supported the combining of the two model areas to better represent travel over the entire region. The Panel also recommended that the MPO develop a regional forecasting capability independent of the three state agencies of Ohio, Kentucky, and Indiana.

For future enhancements, the Panel recommended that OKI consider:

5. Southern California Association of Governments

The Southern California Association of Governments (SCAG) held two Peer Review Panel meetings - the first in November 2003 and the second in April 2004. Both meetings were held in Los Angeles, California with the same Peer Review Panel members. SCAG serves as the MPO for six Southern California counties. The region encompasses a population exceeding 16 million people and covers over 38,000 square miles. During the 1990's, SCAG embarked on the first comprehensive overhaul of its travel simulation capabilities since the mid-1970s. SCAG held a Panel Peer Review in January 2002 in an effort to utilize new data to update and recalibrate its travel simulation modeling process.

SCAG hired a team of consultants, lead by Cambridge Systematics, to undertake the nine-month model improvement task. The work focus was on:

The purpose of the first Peer Review Panel meeting was to present the scope of work for proposed travel demand model improvements to the existing SCAG regional model to the Peer Review Panel members. SCAG staff members and the consultants presented the work elements for each of the tasks and solicited comments on the reasonableness of the proposed work from the Peer Review Panel. Key points made by the Panel for the consultants to consider when performing the model work included:

SCAG reconvened its Peer Review Panel in April 2004 to provide staff members and the consultants the opportunity to present the work-to-date for each of the tasks: model validation, vehicle availability, trip generation models, external trip models, and mode choice models discussed at the earlier Peer Review.

The consultant tested three potential structures for the vehicle availability model - multinomial logit, nested logit, and ordered response logit. Because of the similarities in the results of the three models, the consultants proposed, and the Panel agreed, that the simplest model, the multinomial logit, be used. It was agreed that for the SCAG model, it was acceptable to use data generated by merging socio-economic zonal data with the results of a household survey. It was also recommended by the Panel that a new trip purpose-home-based work-strategic-be added to the trip generation model. As part of the mode choice model, the Panel recommended that the consultants incorporate walk distance rather than zone, if possible.

Next steps included further work by the consultant team on finishing the external trip model, fully developing the mode choice model, and beginning work on the trip assignment model. The third, and final, Peer Review session for SCAG is tentatively scheduled for later in 2004.

B. DOT Peer Review Panels

1. Iowa Department of Transportation

The Iowa Department of Transportation (IaDOT) held a Peer Review Panel meeting in March 2004 in Ames, Iowa. The purpose of the two-day Peer Review Panel meeting was to discuss model calibration and validation as it pertains to each specific area of the travel demand modeling process. The meeting agenda was organized to allow experts in the field to share descriptions of best practices for travel model calibration and validation with transportation planning professionals in the Midwest.

The recommendations given by the Peer Review Panel focused directly on the steps and needs of moving a coordinated statewide program for travel modeling in urban areas of Iowa. Recommendations included:

As next steps, IaDOT will begin the development of statewide guidelines, standards, best practices and programs for travel demand modeling in Iowa. In addition, IaDOT is taking steps to consider the development of a statewide vehicle and freight demand model.

2. North Carolina Department of Transportation

The North Carolina Department of Transportation (NCDOT) held a Peer Review Panel meeting in February 2004 in Raleigh, North Carolina to provide an independent assessment of the state DOT's travel demand modeling systems and to compare it to industry standards. North Carolina consists of 17 MPOs. NCDOT is responsible for three regional models for its largest urbanized areas. No statewide model is currently in place for North Carolina, but NCDOT has recently created a Model Research and Development Unit to provide modeling services across the state.

NCDOT staff members and its consultants presented details of the NCDOT's Transportation Planning Branch (TPB) modeling efforts to the Peer Review Panel. In addition, the Peer Review Panel had time for discussion of Panel recommendations, without the presence of NCDOT staff, before their presentation to all of the attendees. The Peer Review Panel focused on six main issues: organizational, data, trip generation, trip distribution, mode choice and assignment, and calibration and validation.

The Panel recommended exploring the creation of a statewide model of all major corridors within the state to better serve smaller urbanized areas. For data collection, the Panel urged NCDOT to encourage coordination and cooperation among MPOs. For trip generation, the Panel recommended staying with the use of the current gravity model. For mode choice, NCDOT will need to incorporate the ability to test transit options for long range planning efforts.

Among next steps, the Panel recommended that NCDOT create a statewide task force on model development to build partnerships across the state. The Panel also recommended the use of consultants to help in the development of certain portions of the model process, specifically the calibration of a mode choice model.

III. Recommendations for MPOs and State DOTs

While the composition and discussion of the Panel Reviews was unique to the individual region, several common issues were addressed during the course of the nine Peer Review meetings. These common issues were identified within Peer Panel discussion sessions; in many instances, the issues were directly included within the list of recommendations prepared by the Peer Review Panel. The following section provides highlights of common issues and recommendations. The issues and recommendations are grouped into three categories: Technical Recommendations, Managing the Modeling Process and Results, and Improvements for Conducting a Peer Review.

A. Technical Recommendations

Trip-based 4-step modeling is still the norm in the travel demand modeling industry. Trends include increasing use of detailed socioeconomic data in trip generation, continued use of gravity models for trip distribution, and increased application of outputs to air quality and environmental impact analysis and models. The following are highlights of the major technical recommendations considered within the Peer Review meetings.

Land use should be integrated with travel demand modeling wherever possible. In the nine Peer Review Panel meetings held over the past year, each has at least touched upon the integration of land use development patterns into the travel demand models. In particular, Panelists recommended that land use considerations be made at various steps within the modeling process in order to better capture the multiple ways that development patterns affect travel behavior. Land use issues are important as agencies strive to improve their capabilities to model the impact of development patterns on travel behavior to better inform decision-makers of various policy outcomes.

Each of the agencies addressed land use issues in different ways. DRCOG spent significant time on the issue because the agency is in the process of moving from a land use allocation model to one based on fundamental economic behavior theories (UrbanSim). However, panelists agreed that more sophisticated land use models are more data intensive, and thus may not be appropriate for smaller agencies. Modeled changes to transportation conditions are fed back to land-use models, enabling the modeling of efficient land use distribution, and allowing tests for reasonableness to be made. The OKI Panel pointed out that land use area type designations should be variable over time to allow changes from year-to-year. Agencies that discussed land use are: AMATS, ARC, DRCOG, IaDOT, NCDOT, OKI, and SCAG.

Understanding and incorporating freight-based activities into travel demand modeling is important, especially for state DOTs. Several Peer Review Panels recommended that state DOTs lead the development of statewide freight models in an effort to provide consistent input to regional travel demand models. Because freight modeling can be extremely difficult-requiring commercial growth forecasting, specialized surveys, and estimates of their corresponding effect on traffic-state DOTs may be best equipped to address the challenge of incorporating freight. Freight activities are important to include within transportation demand models because the total impact of freight vehicles on the operations of the network can be significant.

Several regions are trying to improve their understanding of freight-based activities within their region and are attempting to incorporate this knowledge into their travel demand models. OKI is in the process of developing a new truck model that uses the method proposed in the U.S. DOT's Quick Response Freight Manual (1996). SCAG will undertake an update of its freight model in future model improvement work. At both of the state DOT Peer Reviews, the Panel recommended that the state DOTs take the first step in developing statewide freight models, as states are better positioned to model the longer-haul freight movements throughout the major corridors of the state. Agencies that discussed freight are: ARC, DRCOG, OKI, IaDOT, NCDOT, and SCAG.

The migration to activity-based or tour-based modeling should be undertaken only with ample funding, resources, and proper documentation of the transition. Most Peer Review Panels agreed that although the modeling industry recognizes the benefits of activity or tour-based modeling, agencies with limited resources should concentrate on improving their current 4-step model rather than switching to a new model type at this time. Once the state of the practice has been further developed, existing activity and tour-based models could then be modified for other agencies. Furthermore, Panelists agreed that agencies undergoing this transition should document their experiences and perform comparisons between the 4-step model and the activity/tour-based model in order to check the reasonableness of the model outcomes. Activity and tour-based models are of interest because of their ability to use micro-simulation techniques to analyze the effects of different policies for various subpopulations (i.e., based on household income, etc.).

Yet, activity/tour-based modeling presents its own challenges-such as added complexity, air quality outputs that have not yet been validated against observed traffic counts, and the need for multiple iterations of runs-even for those leading agencies within the industry. For example, ARC is planning to run its trip-based model and tour-based model in parallel. ARC's Peer Panel strongly recommended that ARC document, compare and study the results of the two models in order to develop their own expertise and conclusions. Agencies that discussed activity/tour-based modeling are: AMATS, ARC, DRCOG, OKI, and SCAG.

Agencies should begin exploring methods like GIS in their models. Five of the seven Peer Review Panels mentioned the need to better utilize GIS in agency modeling processes. GIS has been recognized as a powerful and useful tool for data analysis and presentation of results. In addition, GIS is useful because it greatly facilitates the association of geographically based data sets that are gathered separately. Furthermore, a great deal of data is readily available in GIS format.

Peer Panels recommended GIS use for agencies ranging from a state DOT to a smaller MPO. After manually maintaining the linkage of their travel demand model to GIS through many version changes, IaDOT decided to re-develop all five of their MPO models to a platform that would more easily incorporate GIS. The AMATS Panel recommended using GIS as an easy way to get started with visualizing and comparing results even with basic GIS data. Agencies that discussed GIS are: AMATS, DRCOG, IaDOT, and NCDOT.

Agencies ought to create coordinated data collection strategies and standardization guidelines for regional modeling. Many Peer Review Panels agreed that standards and guidelines would increase the ease for coordination and sharing of data amongst neighboring agencies. Both State DOTs and MPOs can act as central collectors and distributors of transportation related data. The growth in urbanized areas over the past decade and the reconfiguration of urbanized areas has lead to an even greater need to coordinate with agencies within and outside of a region to ensure the quality of data for both external and internal trips.

State DOTs are well positioned to create statewide models to support regional models with external trip information, as well as ensure that consistent data is used throughout the state. Both the NCDOT and IaDOT Panels supported the development of a statewide travel demand model, data collection, and data dissemination program. Agencies that discussed data collection/coordination are: AMATS, IaDOT, NCDOT, and SCAG.

Data quality can be improved by supplementing existing data sources with specialized add-on surveys. Many of the Peer Panels advised that a great deal of data exists, and can be supplemented with add-on surveys or sampling. Most regions have conducted their own specialized surveys, such as transit, airport, truck, and amusement park surveys. Piecing various sources of data together is important for agencies that are finding that their data sets are not complete enough due to low participation or financial constraints for estimating models. For example, NCDOT was recommended to mine NCHRP reports, the National Transit Institute courses, the National Household Travel Survey, census data, REMI, and regional FTA representatives for more data. Agencies that discussed surveys: AMATS, DRCOG, IaDOT, NCDOT, OKI, SCAG.

Agencies should assure that the model and input data are consistent from step to step. More than half of the Peer Review Panels pointed out that consistency checks are crucial throughout the modeling process. For example, the NCDOT Panel recommended checking consistency between regional totals and local agencies; the OKI Panel advised checking for consistency between highway and transit path building; the AMATS Panel suggested checking the consistency of outputs from year-to-year; and, the SCAG Panel addressed the necessary consistency between assignment algorithms and mode choice models. Agencies that discussed model consistency are: AMATS, NCDOT, OKI, and SCAG.

Transit and alternative mode modeling should be thoroughly validated and appropriate data collected. Many Peer Review Panels recommended that additional model validation or data collection be conducted for transit modeling, especially for out-of-vehicle times, headways, and related assumptions for mode choice models. Multiple Panels recommended looking into the FTA New Starts Summit Model for transit modeling support, and conducting additional on-board surveys of transit riders. Some Panels suggested that biking and walking should be modeled at a finer scale and be geographically based. As issues of personal health and the environment become increasingly larger parts of policy considerations, agencies are confronted with the challenge of analyzing various transit alternatives within their models. Agencies that discussed transit are: AMATS, ARC, NCDOT, OKI, and SCAG.

The model should be designed to be flexible enough for a variety of toll and HOV modeling policies to be evaluated. Almost all Peer Review Panels recommended that agencies develop a method for modeling pricing policies and begin the necessary data collection to support the model. Methods of modeling tolls and HOV should include 2-person vs. 3-person HOV lanes and a range of toll prices and policies. Increasingly, tolls and HOV lane restrictions are becoming more widely used travel demand management policies that transportation models should be able to address.

For example, the OKI Panel recommended that the regional model be able to examine the options of additional transit HOV lanes. Similarly, the SCAG Panel recommended that SCAG add the ability to examine high-speed rail and high occupancy toll facilities to its modeling capabilities. Agencies that discussed pricing policies are: AMATS, ARC, DRCOG, IaDOT, NCDOT, OKI, and SCAG.

Agencies should consider the application of microsimulation and re-evaluate modeling zone sizes whenever opportunity allows. Some Peer Panels recommended that agencies begin to investigate opportunities for the use of microsimulation and smaller zone sizes. Microsimulation of many elements of traffic demand modeling (e.g. household size or traffic assignment) can be used to produce samples as input to the traditional 4-step process. The choice of zone sizes depends on the available data, data quality, and use of results. The ability to use smaller zone sizes and microsimulation will allow more detailed modeling of areas which need special attention, such as those areas which display unusual characteristics or are highly diverse and require finer grained modeling. Agencies that discussed zone sizes/microsimulation are: AMATS, ARC, DRCOG, NCDOT, and SCAG.

Consideration of time-of-day variables in models should be made. Many Peer Review Panels recommended including or improving time-of-day modeling, since the transportation system varies widely during peak and non-peak hours. Although many agencies are experiencing difficulties with time-of-day modeling, it is important to build this function into the models so that as data collection methods improve, daily averages of traffic volumes and trips can then be substituted by finer grained time-of-day variables to capture these effects and increase the realism of the models.

In particular, the IaDOT Panel recommended replacing their current daily approaches with up to four time periods (AM peak, PM peak, midday, and night). OKI has separate peak and off-peak models for several steps of their model. DRCOG is planning to use a ten-period time-of-day model. Agencies that discussed time-of-day are: AMATS, ARC, DRCOG, IaDOT, NCDOT, OKI, and SCAG.

Simple techniques for modeling can often provide adequate solutions. A few of the Peer Panels pointed out that rough estimates are acceptable when data is lacking and a full-model is difficult to estimate, or when accurate estimates are not essential, such as for low-share alternatives. In many situations, simpler techniques such as the use of sketch modeling instead of a full model, or a binomial model instead of nested logit model, is sufficient for rough estimates. However, Panelists warned not to go to the sketch model approach too quickly, as there are a large number of models for different applications available for purchase that should be considered.

The Panels recommended that NCDOT consider using simple techniques for rough transit estimates since data were limited. AMATS was also advised to use sketch planning models for quickly comparing scenarios. IaDOT uses sketch techniques for providing travel demand estimates to small urban and rural areas. Agencies that discussed simple techniques are: AMATS, IaDOT, and NCDOT.

Reasonableness checks should be made by borrowing and comparing values to values used in other models. Peer Panels recommended that other studies and models should be leveraged for impedances, elasticities, and other coefficients. These can be adapted from similar regions for use in similar models or used to check the reasonableness of derived values or resulting model estimates. When values are borrowed, it is important to justify their appropriateness.

Due to the complications SCAG experienced with its commuter rail line, Metrolink, the Panel recommended for them to consult literature values on modeling commuter rail lines. It was also recommended that DRCOG look into purchasing work already done on population microsimulation and activity-based models. When coefficients are found to be unusual, such as in AMATS and SCAG, the Panel recommended further studying the reasons behind the differences. Agencies that discussed borrowing values are: AMATS, DRCOG, NCDOT, and SCAG.

Calibration and validation should be performed to targets customized for the travel demand model application. Peer Panels recommended that calibration and validation not be done to assignment standards but to targets and goals customized to the local agency and transportation model. Targets should be set to be reasonable, depending on the particular purpose of the model, available data resources, model structure, and desired level of accuracy. The FHWA 1997 Model Validation and Reasonableness Checking Manual is a good reference for agencies desiring to improve their calibration and validation techniques. Agencies that discussed calibration/validation are: IaDOT, NCDOT, and SCAG.

B. Managing the Modeling Process and Results

The Peer Review meetings covered topics beyond the technical details of the model itself. The following are highlights of recommendations made by the Peer Panels on ways to better manage the modeling process and how to optimize the use of the model results.

Building expertise by investing in skilled experts, creating a core team of modeling specialists, and using consultants is a crucial way to combat rapid turnover of modeling staff. Peer Review Panelists discussed the challenges of limited organizational resources and rapid turnover of expert staff. The NCDOT Peer Panel suggested that skilled experts should be better leveraged across the state. The Panel recommended that NCDOT maintain a core team of modeling specialists in the central office who could then be dispatched to assist the inexperienced or more generalist practitioners in the local agencies. Moreover, the Panel recommended that NCDOT investigate increased use of consultants as a source of current expertise and indirect source of training for in-house staff. Other Peer Panels agreed that coordination with regional agencies is crucial to addressing the shortfall of modeling resources. An agency that discussed staff expertise is NCDOT.

Agencies should pool resources and share their experiences through best practice publications. Peer Review Panels recommended that as agencies identify best practices or state-of-the-practice applications, it is important for those practices to be shared both regionally and nationally so that other regions can benefit. It is also important for an agency to ensure that other agencies within the region that have parallel modeling efforts underway to coordinate. With limited resources available to regions and state, sharing resources and experiences will avoid duplicative efforts and potentially create a synergy that can optimize regional modeling efforts.

North Carolina is planning to produce a report detailing its set of best practices identified at its statewide Peer Review Panel. In the case of Los Angeles, Panelists recommended that SCAG ensure that its model was compatible with the destination choice model being created by the transit agency. A major finding of the Iowa Peer Review was the need for the state DOT to actively coordinate with other states on best practices. Agencies that discussed pooled resources are: IaDOT, NCDOT, OKI, and SCAG.

State DOTs should begin to establish uniform practices throughout the state with the assistance of an autonomous statewide technical advisory group. Peer Review Panelists stressed the need to establish uniform practices for specific elements of the model development process and to document those practices so that they could be used consistently across the state. The state DOT has a unique role that could potentially be better optimized to support the efforts of independent, regional modeling efforts by the creation of a statewide model. Such a model would produce freight activity data and other inter-regional data to ensure the consistency of assumptions throughout regional models being used across the state. State DOT Peer Review Panels recommended that a technical advisory group be created to assist state DOTs in the development of a statewide model, the oversight of metropolitan models for the state's urbanized areas, and data collection efforts. Increased coordination is important for standardization across the state and ensuring the quality of various transportation models. Agencies that discussed state technical advisory groups are: IaDOT and NCDOT.

Regions should have high-quality documentation of the components of, and assumptions in, the travel demand model. Peer Panels stressed the importance of regional modelers understanding the intricacies of models and having the ability to interpret the model results. Many areas hire an outside consultant to develop various aspects of the model, making it crucial that the regional modeling staff be given good documentation of work performed. The documentation is also a valuable resource for training and learning for in-house staff. The documentation of the model inputs and outputs are especially important when making the air quality conformity determination, and is a necessary step in ensuring acceptance of a region's long-range plan by the federal government. Agencies that discussed documentation are: AMATS, IaDOT, and NCDOT.

Early dialogs between modelers and policy makers should be conducted to ensure model sensitivity to policy initiatives that might be used in the future. At almost every Peer Review Panel, the need to better understand the needs of policy makers was raised. Many modelers shared the challenge of responding in a reasonable amount of time to proposed policy initiatives and their impacts on the network. Panelists noted that early dialog between policymakers and modelers should occur to adapt the model appropriately to reflect various policy alternatives. Because the results from the model scenarios are used to inform policy makers involved in making decisions about future investments in a region, it may be useful to extend the dialog beyond the current set of modelers and planners who make up the Peer Review Panels and bring others, especially policy makers, into the conversation. Agencies that discussed early dialog are: AMATS, ARC, IaDOT, NCDOT, OKI, and SCAG.

C. Improvements for Conducting a Peer Review

As agencies continue to conduct Peer Reviews as an integral part of their model improvement process, a variety of lessons can be learned from the TMIP Peer Reviews that took place over the past year. The following section provides recommendations on how an agency can improve its Peer Review, thereby maximizing the benefits of the model improvement process.

The host agency should ensure an adequate lead-time for the planning of the Peer Review to ensure that it is able to enlist those Panelists most familiar with the meeting's subject matter. Because each Peer Review meeting may have a different focus depending upon the host agency's needs, it is important that it clearly define the meeting's subject matter early in the meeting planning stages. Each Peer Review Panel expert brings a unique set of knowledge and skill about the different parts and phases of the modeling process. By determining its needs early in the process, the host agency can identify those experts that are most suited to its needs. Adequate lead-time will also allow the host agency the time needed to provide the proper background material to be distributed to the Panel members.

The host agency should provide Panel members with specific information about the meeting's objectives and the details of the model. Once a state or region decides to hold a Peer Review Panel meeting, it is crucial for the host agency to clearly define its modeling needs and prepare a package of background material on the model for use by the Panel members. The information will help in deciding which experts to choose for the Panel as well as educating the experts before the meeting is conducted. This information should be distributed to the Panel Review members before the meeting. Panel members should be provided with enough time to allow for proper review of the materials. With a rich source of background material and clear definition of what the host agency wishes to focus on, the Peer Review Panel members can better prepare for the meeting.

A discussion should be held with policy makers on possible policy initiatives that may need to be explored over the next decade or more prior to, or at a Peer Review meeting. Because the ultimate clients for the travel demand modeling work are the policy makers who determine the future capital and operating investment for the state or region, it is important that there be an early and on-going dialog between the policy makers and the modelers. There should be a discussion of what policy initiatives will be required to be modeled and which ones, while maybe not feasible at present, are possibilities for future consideration. Environmental considerations, such as the region's compliance with the Clean Air Act Amendments, may require the exploration of policy initiatives that were not considered in the past. Initiatives may include various modes of transit, pricing charges, high occupancy vehicle or toll lanes, and parking or land use restrictions. Documentation of potential policy initiatives would give the Peer Review experts a better understanding of the region's political landscape and the demands that may be placed upon the model.

The host agency should ensure proper documentation of the model development process for the benefit of internal staff and the Peer Review experts. This includes the documentation by any outside consultant of model development work performed, the documentation of data used as inputs into the model, and the documentation of processes and procedures for operating the model.

This documentation is important during at least four of the phases of the Peer Review process. First, it is important that the documentation of the underlying model assumptions given to the Peer Review experts be clear and concise. Second, the quality of the data used for the model inputs is crucial to ensuring quality outputs from the model. The documentation of the data used and its characteristics is needed to ensure quality control. Third, because many agencies hire an outside consultant to perform model development work, the documentation of assumptions and characteristics is important so that the host agency's modeling staff understand the assumptions and can provide guidance, feedback, and verification. Fourth, it is important that non-technical participants in the discussion be able to understand at a higher level the assumptions that are being built into the model.

The ongoing documentation of the model assumptions is important as the host agency hires new staff, includes additional consultants for additional work, and possibly adds new components to the model. As public interest groups become more involved in the certification review process, it is important that the host agency is able to document that the region has incorporated the most up-to-date data into the travel demand model.

The host agency should disseminate the results of the Peer Review to a broad range of audiences. It is important that any findings, both positive and negative, that result from the Peer Review program meetings be distributed to an audience that is wider than the host agency itself. The audience includes the traditional modeling community within the region and others outside the region that share common characteristics, as well as policy makers, politicians, interest groups, and the public at large. It is important that the results from the meeting be made more accessible to groups that may not be intimately involved in the modeling process but rely on its results to participate in the public process of deciding which projects to consider or fund.

As more meetings are held, host agencies should reach out to other agencies similar to it in either socioeconomic characteristics or in model development. The coordination with other regions and states on best practices can further the goals of the program and bring about an informal extension of the Peer Review Panel meeting program.

Appendix A. Acronyms

The following is a list of acronyms used in this report:

AMATS - Anchorage Metropolitan Area Transportation Study
ARC - Atlanta Regional Commission
DRCOG - Denver Regional Council of Governments
FHWA - Federal Highway Administration
IaDOT - Iowa Department of Transportation
NCDOT - North Carolina Department of Transportation
OKI - Ohio-Kentucky-Indiana Regional Council of Governments
SCAG - Southern California Association of Governments
TMIP - Travel Model Improvement Program

Appendix B. Index of Peer Review Panel Meetings

The following is a list of TMIP Peer Review Panel Meetings held between June 1, 2003 and May 30, 2004. Information includes the host agency, city location, date of the Peer Review Panel meeting, and a link to the meeting report contained on the TMIP website.

Atlanta Regional Commission (ARC)
Atlanta, Georgia
February 3-5, 2004

Anchorage Metropolitan Area Transportation Study (AMATS)
Anchorage, Alaska
May 24-25, 2004

Denver Regional Council of Governments (DRCOG)
Denver, Colorado
Meeting 1: October 31, 2003
Meeting 2: April 20, 2004

Iowa Department of Transportation (IaDOT)
Ames, Iowa
March 30-April 1, 2004

North Carolina Department of Transportation (NCDOT)
Raleigh, North Carolina
February 10-11, 2004

Ohio-Kentucky-Indiana Regional Council of Governments (OKI)
Cincinnati, Ohio
June 16-17, 2003

Southern California Association of Governments (SCAG)
Los Angeles, California
Meeting 1: November 3, 2003
Meeting 2: April 16, 2003

Appendix C. List of Host Agency and Peer Review Panel Members

Last Name First Name Organization OKI (6/03) DRCOG (10/31/03) SCAG (11/2/03) ARC (2/3/04) NC (2/10/03) IA (3/30- 4/1/04) SCAG 2 (4/16/04) DRCOG 2 (4/20/04) AMATS (5/24-25/04)
Bhat Chandra University of Texas P   P            
Bradley Mark Mark Bradley Consulting P   P       P    
Cervenka Ken NCTCOG         P P      
Chiao Kuo-Ann NYMTC         P        
Erhardt Gregory DRCOG                 P
Fussell Rhett NCDOT         H        
Garry Gordon SACOG       P          
Granzow Edward CH2M Hill           P      
Gunning Andrew PAG                 P
Huntsinger Leta ITRE           P      
Ismart Dane Louis Berger           P      
Ives Berry New Mexico COG         P        
Killough Keith KLK Consulting   P              
Lamb Danny FDOT         P        
Lawton Keith Portland METRO P P P P     P    
Lee Deng-Bang SCAG     H       H    
May Jeff DRCOG P     P          
Mescher Phil Iowa DOT           H      
Miller Eric University of Toronto   P           P  
Miller James CH2M Hill           P      
Morris Michael NCTCOG   P   P       P  
Purvis Charles Metro   P         P    
Quackenbush Karl Central Transportation Planning P                
Reiff Sherwin (Bud) Lane Council of Governments                 P
Replogle Michael Environmental Defense   P           P  
Rousseau Guy ARC       H P        
Sabina Erik DRCOG   H P         H  
Spielberg Frank BMI-SG   P           P  
Spring Jon AMATS                 H
Tsai Cheng OKI H                
Walker Dick Portland METRO                 P
Walton Rita MAG       P          

H = Hosts
P = Peers

Updated: 4/25/2014
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000