Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

Office of Planning, Environment, & Realty (HEP)
Planning · Environment · Real Estate

HEP Events Guidance Publications Glossary Awards Contacts

Status of TMIP Peer Reviews - Iowa Department of Transportation (IaDOT)

(PDF version)

PDF files can be viewed with the Acrobat® Reader®

Location Date Exchange Host Agency Peer Review Panelists
Ames, Iowa March 30 - April 1, 2004 Iowa Department of Transportation (IaDOT), Office of Systems Planning Ken Cervenka, North Central Texas Council of Governments (NCTCOG)
Ed Granzow, CH2MHill Consultants
Paul Hershkowitz, Wilbur Smith Consultants
Leta Huntsinger, The Institute for Transportation Research and Education at North Carolina State University
Dane Ismart, Louis Berger Consulting
Jim Miller, CH2MHill Consulting
Howard Slavin, Caliper Corporation

Disclaimer

The content of this peer review report does not represent the opinions of FHWA nor does it constitute an endorsement, recommendation or specification by FHWA. The content of the report does not determine or advocate a policy decision/directive or make specific recommendations regarding future research initiatives. The report is based solely on discussions and comments during the peer review.

Table of Contents

Executive Summary

The following report summarizes the results of a Peer Review Panel held through the Travel Model Improvement Program (TMIP), which is sponsored by the Federal Highway Administration (FHWA). The Iowa Department of Transportation (IaDOT) Office of Systems Planning hosted the three-day Peer Review. In attendance were representatives from various planning organizations in the Midwest, most in the immediate vicinity to the State of Iowa. These representatives included Metropolitan Planning Organizations (MPOs), State Departments of Roads and Transportation, University Transportation Research Center, transportation consulting firms and the FHWA. Most of these representatives are members in the Midwest Travel Model Users Group (MTMUG). The primary focus of the Peer Review was to provide MTMUG and the local travel demand modeling community, as a whole, an assessment of their current practices of model calibration/validation and reasonableness checking as well as recommending methodologies and best practices for us to follow with the goal of enhancing the accuracy and reliability of their demand models.

The Peer Review session focused on model calibration and validation as it pertains to each specific area of the travel demand modeling process. The session also focused on the Midwest travel demand modeling community and how we can further strengthen our partnerships and travel modeling processes. At the end of the first two days of presentations and discussions, the Peer Review Panel prepared a summary of recommendations for the following categories:

After preparing the recommendations in a closed session, the Peer Panelists presented their feedback to the group for clarification and discussion. The findings of both the intermediary discussions and final recommendations are summarized within the ensuing report.

Participants in the Peer Review included transportation model experts from the North Central Texas Council of Governments, CH2MHill Consulting, Wilbur Smith and Associates Consulting, The Institute for Transportation Research and Education at North Carolina State University, Louis Berger Consulting, and Caliper Corporation. The Peer Review was held March 30 - April 1, 2004 in Ames, Iowa.

Background

Peer Reviews are conducted by planning agencies, with the support of FHWA and FTA, to ensure that technical products, procedures and/or processes being used or developed meet the agency's needs, the standards of professional practice, and/or Federal, state or local planning requirements. Peer Reviews of forecasting and data collection procedures are crucial to planning agency model development and improvement efforts. As part of its program, TMIP is committed to assisting agencies in meeting these planning challenges by supporting Peer Reviews.

Prior to mainframe computer platforms and the proliferation of desktop personal computers the typical method for developing future forecasts of travel demand was by hand calculations and paper maps. During this time period and into the era of the mainframe computer, the IaDOT was generally responsible for travel demand forecasting for MPOs. During the 1980's and early 1990's travel demand software that could run on personal computers was becoming more prominent. It was this general time period that the responsibility for travel demand modeling was taken over by the MPOs. The Department aided the MPOs in acquiring travel demand software. Five of Iowa's then eight MPOs adopted Tranplan as their platform of choice while the three remaining chose QRSII. The MPOs were then on their own to develop and maintain the travel demand models. In the early 1990's the Iowa State University's Iowa Transportation Center, now the Center for Transportation Research and Education (CTRE), and the IaDOT established a travel demand model users group that provided a forum for training and knowledge sharing. The MTMUG has been successful and going strong since then.

With the increasing popularity and usefulness of Geographic Information Systems (GIS), CTRE facilitated a research project that linked our existing Travel Demand Model software to GIS. Although the projects were successful in most regards the ever-changing versions of the model software prevented the linkage to be consistently applied over time. A couple of years ago MTMUG began a search of the software market to determine if there was a commercially available package that incorporated GIS into the travel demand forecasting software. The group invited various companies to Iowa to demonstrate the capabilities of their software. MTMUG formally evaluated various platforms such as TP+/VIPER/CUBE, QRSII, VISSUM and TransCad. After a thorough evaluation, TransCad was determined to be the best fit for Iowa's travel demand modeling needs. As each MPO works to update their Long-Range Transportation Plan their existing model is re-developed using TransCad.

For the last several years, Iowa has seen the turnover rate for modeling staff at the MPOs grow substantially. This is troubling for the MPOs since it takes time for a new person to gain proficiency with the modeling process and can leave the MPO in a position where they cannot perform needed modeling work. IaDOT has made the commitment to provide a significantly increased level of technical support to Iowa's MPOs in regards to travel demand models. Current services now provided by the IaDOT include:

Planned future services based on Peer Review recommendations include:

In October of 2003, the IaDOT was awarded the opportunity to host a Peer Review Program through TMIP. TMIP is a multi-year, multi-agency program that is sponsored by the United States Department of Transportation and the Environmental Protection Agency, with the mission of supporting and empowering planning agencies through leadership, innovation and support of travel analysis improvements, to better meet current and future mobility, environmental, safety and security goals. TMIP began in 1992, and has three goals:

  1. Help build the institutional capacity of planning agencies to perform technical analyses.
  2. Support development of analytical methods that respond to the needs of planning and environmental decision making processes.
  3. Support mechanisms to ensure the quality of technical analyses used to meet local, state and federal program requirements.

This Peer Review was organized to allow experts in the field of travel demand modeling to share their descriptions of best practices for travel model calibration and validation with transportation planning professionals in the Midwest. Unfortunately, a significant shortcoming of many travel demand models is the lack of attention and effort placed on the validation phase of model development. This often results in a low level of confidence in the model output traffic volumes. It is our intention that this Peer Review will better define the professional standards we are trying to obtain with model accuracy and provide a collection of common methods and techniques to meet those goals. This will be done by identifying state-of-the-art modeling practices and discussing the best method for introducing them into the modeling process. Eventually, this will create a more consistent way to measure and evaluate the reliability of our traffic forecasting models and hopefully gain an increase in the confidence level of the output.

Attending panel members are experts in the field of travel demand forecasting with experience from work in MPOs, State Departments of Transportation, University Transportation Research Centers, Consulting Firms, and the FHWA.

As stated in the Model Validation and Reasonableness Checking Manual that was prepared in February of 1997 for TMIP and FHWA, "A major shortcoming of many travel demand models is the lack of attention and effort placed on the validation phase of model development. Validation involves testing the model's predictive capabilities. Travel models need to be able to replicate observed conditions within reason before being used to produce future-year forecasts. As metropolitan areas continue to refine and improve the travel demand forecasting process, the credibility of the process with decision makers will depend largely on the ability of analysts to properly validate procedures and models used.

The travel modeling process has undergone many changes in the past few years in order to evaluate more complex policy actions resulting from legislation such as ISTEA, TEA-21 and the Clean Air Act. As travel models have become more complex, so have the procedures needed to validate them. Often there is a tradeoff between increasing confidence in the level of accuracy of the models and the cost of data collection and effort required to validate models. Tests or checks used to evaluate the reliability of models can range from a simple assessment of the reasonableness of model outputs to sophisticated statistical techniques."

These shortcomings are also prevalent in Iowa's travel demand modeling community. Many variables are influencing the level of model validation. High staff turnover rates, steep learning curves, time constraints and lack of knowledge on calibration/validation techniques are a few examples. The Iowa modeling community has the desire to identify best practices for model validation and calibration in an effort to increase the credibility and reliability of our traffic forecasting models. Every year millions of dollars in investments to our transportation system rely on the information obtained from our traffic models. Every effort should be made to ensure that credible information is used in the decision making process. The Model Validation and Reasonableness Checking Manual illustrates many examples of these processes, however, we are looking to this Peer Review to provide expert guidance in the most reliable and cost effective validation/calibration techniques and methodologies.

Plans for Improvement

It is desired that this Peer Review will result in a description of best practices for travel model calibration/validation that the modeling community in Iowa can work to collectively implement in our respective institutions. It is our hope that this will create a more consistent way to measure and evaluate the reliability of our traffic forecasting models. This will be especially helpful from the perspective of the State Department of Transportation when analyzing traffic forecasts in different parts of the state.

The Iowa modeling community is continually making investments regarding travel demand forecasting. Approximately twelve years ago a group (Midwest Travel Model User's Group, MTMUG) was formed and currently meets quarterly. These meetings provide a forum for information sharing, formal presentations, idea exchanges and solutions to common problems. In addition, our nine MPOs in the state recently made the investment to change travel demand modeling software suites. Prior, six MPO's used Tranplan and the remaining three used QRS II.

Although sound programs for accomplishing the basics of travel demand forecasting, the models were lacking in the newer Geographic Information System capabilities as well as being more user friendly and MS Windows compliant. Currently all MPOs have made the commitment to change to TransCad from the Caliper Corporation. Soon after the software was purchased organizations participating in the MTMUG partnered together to have a weeklong customized training session from Caliper Corporation. One MPO has partnered with the IaDOT to purchase an "add-on" survey to the National Personal Transportation Survey or NPTS in order to have access to more local survey data. This data will be shared with the other MPO's in the state. The Iowa travel demand modeling community will continue in its efforts to make their models more credible and reliable.

Presentations and Discussion

After the opening remarks, introductions and an explanation of the purpose of the Peer Review, each panel member gave a presentation on their selected experiences and best practices regarding travel model calibration/validation and reasonableness checking. The following are summaries of these presentations:

Panel Member Presentations

North Central Texas Council of Governments1
Ken Cervenka

Mr. Cervenka is the Senior Program Manager for the Travel Forecasting Program Area at the North Central Texas Council of Governments (NCTCOG). NCTCOG serves as the MPO and Regional Transportation Planning Agency for the Dallas/Fort Worth, Texas region. The MPO is comprised of a 16 county region and the regional agency covers a 16 county region. NCTCOGs transportation department is divided into six program areas:

NCTCOG currently uses three travel demand model software packages for their traffic forecasting work. The mainframe-based regional multimodal model has been developed as a series of FORTRAN programs, while TRANPLAN is used for sub area traffic modeling. Recently, NCTCOG has gradually migrated to a four-step TransCad based modeling methodology. The NCTCOG TransCad model is comprised of a 4,874-zone system. Mr. Cervenka indicated this model has a trip table with approximately 23.8 million zone-to-zone pairs. NCTCOGs year 2025 model network has over 36,000 coded roadway links, 22,000 network nodes, 410 coded one-way bus lines, 36 rail lines, 14,500 bus stops and 171 rail stations for the NCTCOG region.

Mr. Cervenka has identified a series of future steps that need to be made at NCTCOG. While the full TransCad model is in place at the agency, the model has not been fully transferred to staff for all model applications work because of the need to continue legacy applications and limitations on current staff capability. NCTCOG seeks to train both in-house transportation planning staff, and planning staff from other agencies to better understand the development and operation of the TransCad model. In particular, NCTCOG desires to train several "TransCad model application champions" who have a strong understanding of GIS and travel model theory, are experienced in working directly with TransCad, and can spend the majority of their professional time on model applications projects.

In addition to a high priority on training, NCTCOG will also concentrate efforts on the preparation of additional roadway/transit "supply and demand" performance reports and the documentation of the four-step modeling process. Mr. Cervenka noted that the documentation will be crucial, as it will include a description of the model components and how they operate, as well as a detailed description of the reasoning behind how the model works. Additional next steps include ongoing improvements to the modeling procedures, greater coordination with TxDOT on the Statewide Analysis Model, and use of traffic microsimulation for detailed vehicle operations analysis.

1This summary was borrowed from the NC DOT TMIP Peer Review Report. Mr. Cervenka gave the same presentation at both Peer Reviews.

CH2MHill Consultants
Ed Granzow

Mr. Granzow is with CH2MHill consultants and has been involved in transportation planning studies for over 25 years. Types of work have included: citywide transportation model development and application, transit service/operations planning, and related software development. Mr. Granzow presented general information on model calibration/validation techniques and highlighted these with modeling experiences in Anchorage, Alaska.

Key considerations when developing, calibrating and validating a travel demand model

The best way to build a good validation vehicle starts with good input data. Input data is one of the most common sources of error in a travel demand model. Having the right engine and good model design is also very important. A good process and the right validation tools are also necessary. Having realistic/appropriate targets for calibration success. The modeler must place an emphasis on strategic value.

Examples/Lessons from Anchorage

A TransCad model was built consisting of over 6,000 links and over 600 traffic analysis zones. Modes analyzed consist of walk, bike, driver, pass car, and bus. Trip purposes consist of HBW, HBS, HBO, HBSC, NHW, and NHB. An Integrated Freight Model is included. The model is now undergoing Peer Review.

Anchorage Area Characteristics:

Input data used consists of CTPP Part 1 and Permit Data used to estimate housing and characteristics by location. Es-202 data was used to estimate employment by SIC and location. This data was geocoded to the correct geographic location. Household travel survey data from 2002 in Anchorage was utilized and consisted of 12,093 samples. Previously coded networks and link data were used.

Upon utilizing the traffic analysis zone boundaries there was found to be a difference between the CTPP and the existing zone boundaries. This was taken care of by directly allocating from the Census Block and Block Group geographies. These totals were then reconciled with local control totals. Time was taken to reconcile errors in employer location and misallocation. Special generators were also introduced.

The highway network has a representation of limited access facilities and incorporates speeds and turn penalties. Centroid connector links are sparse.

Objective for the "Engine" or model design are summarized by the following bullets:

Caliper's GISdk scripting language was utilized to create batch files to automate some of the processes of running the model. Model operation could be accomplished using these batch files or a manual step-by-step process. The menu system was developed to be straightforward.

Validation Process

The validation process used screenline reports to determine level of accuracy. Network links were categorized by functional class and evaluated by each of these classes for acceptable accuracy. Maps were created to compare link volumes and traffic counts.

It is important to understand the validation process is continuous. There is always the opportunity to find new data and process errors. Comparisons and checks for reasonableness between independent data sources should be done.

CTPP Part 3 can be used to double check:
NCHRP 255 criteria is a starting point for assessing:
Emphasizing Strategic Value

The model validation scale or criteria should fit the application. It is important to recognize critical locations or issues and determine the causal factors.

Wilbur Smith Consultants
Paul Hershkowitz

Mr. Hershkowitz has worked for the Michigan Department of Transportation for 25 years in a variety of travel demand modeling and project/corridor planning positions before beginning his current position as Manager for Traffic and Travel Demand Forecasting for the North Central United States at Wilbur Smith and Associates. Mr. Hershkowitz has extensive knowledge and experience in travel demand model development, calibration and validation.

Travel demand models are developed to provide decision makers the best possible information in which to make their decisions upon. These models are a valuable tool, especially to the Policy and Technical committee members of the MPO as they are the local decision makers. Travel demand models support Long Range Transportation Plans, corridor studies and sub area studies in the MPO region. They should be practical, useful and easy to learn and maintain. Mr. Hershkowitz indicates the sophistication of the model should match the needs.

Model calibration and validation is necessary prior to its application in transportation analyses. Model applications can be traditional ones such as corridor analyses, new land use impacts and roadway scenario analyses. They can also be useful in non-traditional applications such as construction detour evaluations or fair-share financing. You should be creative in your project applications and think outside the box.

Model calibration and validation methodologies should be a top down, systematic effort. When problems occur, you should be a detective to figure it out. Good sources for guidance in the modeling process consist of the following:

It is typically acceptable to use K factors in the Trip Distribution step, however their use should be minimized. Also, post-processing of model volumes should minimize link factoring. A basic suggestion is to make adjustments area wide, by corridor, Federal Functional Class, etc. to avoid problems later in the process.

Global validation targets:
Model development Rules of Thumb:
Common Model Calibration Problems:
Model Calibration/Validation Tips:
  1. Apply common sense. Never use raw model numbers without examining them. Do they make sense? Are they reasonable? Rationale? Logical?
  2. See #1. Trust your instincts. You're the modeling expert in your MPO. If something doesn't look right, it probably isn't.
  3. Your model stream should be easy to replicate, so that you (and someone after you) can do it over and over, easily. Don't want to have to reinvent the wheel.
  4. Regarding the network, if you're going to alter link speeds in a corridor (volumes too high or low), suggest you make small changes over many links, not a huge change on one link.
  5. Do the model documentation as you do the work. If you leave it to the end of the process, it's always tougher to accomplish.
  6. An FSUTMS-type structure would be a good idea for Iowa. Can be used as a guide at first, then later a standard.
  7. Number of zones rule of thumb - 1 zone /1,000 pop.

The Institute for Transportation Research and Education at North Carolina State University
Leta Huntsinger

Ms. Huntsinger is the Program Manager for the Triangle Regional Model (TRM) Service Bureau at the Institute for Transportation Research and Education at North Carolina State University and an Adjunct Professor at NC State. Prior to this assignment, Ms. Huntsinger was the Team Leader of the Model Research and Development Unit at the North Carolina Department of Transportation. Ms. Huntsinger presented information concerning the newly developed Transportation Model Service Bureau and local experiences with model development, calibration and validation.

The Institute for Transportation Research and Education (ITRE) has started a new program that will be responsible for providing travel model expertise, training and research to the Triangle region. The TRM Service Bureau is a partnership between the North Carolina Department of Transportation (NCDOT), the Capital Area Metropolitan Planning Organization (CAMPO), the Durham-Chapel Hill-Carrboro Metropolitan Planning Organization (DCHC MPO), and Triangle Transit Authority (TTA). The ultimate goal of this partnership is to implement an improved computer travel demand modeling process that will allow for the prediction of future traffic flows in the region. This modeling process will have the ability to identify transportation deficiencies in the Triangle region and aid decision makers in determining appropriate transportation investments to address these deficiencies as well as plan for the impacts of proposed land use changes.

Typical Types of Analyses

Travel Allocation Models- These models can produce estimates of existing and future travel patterns in a fast, low cost, and easy to apply manner. A benefit is this method teaches the general concepts of transportation planning and travel modeling.

Small Urban Studies- Studies are done for small urban areas with population between 5,000 and 50,000. These are areas that have highly anticipated growth potential. These areas typically have a current TIP project and a need for new facilities. A "Sketch" travel model is developed to analyze the area.

MPO Study- These studies are for areas with population greater than 50,000 and the need to answer more policy related questions. These questions usually have multiple transportation alternatives and high growth communities. For areas that are not in compliance with EPA air quality standards are required to perform air quality analysis. This requires the use of a full Travel Demand Model.

Regional Studies- Regional studies are typically needed where complex, cross-jurisdictional policy issues are present and will likely result in large multi-modal investments. These studies analyze regional travel flows and facilities in detail with more non-traditional travel models. (More information on these studies is provided in Ms. Huntsinger's PowerPoint presentations.)

The following are key points from Ms. Huntsinger's presentation:

There are new challenges these days as advocacy groups are challenging travel demand models. This is why good documentation of the process is very important. It may need to be used to defend the modeling process in court.

The model research and development team is focusing on a variety of issues that are typically not common to everyday modeling tasks. These consist of model literature and documentation review and also Peer Reviews. Also the team works on the development of application tools to use in the modeling process. Training is important due to the loss of qualified staff.

Performance measures and reasonableness checks should be applied throughout each step in the modeling process. Land use data is a common area for error. In the Triangle region, the MPO is responsible for collecting the data and then it is transferred to the DOT or ITRE. Even though the MPO collected the data it still needs to be checked for reasonableness. Don't assume the data is good without checking it.

The use of GIS software is an excellent way to visualize your data. Aerial photography can be utilized in a GIS environment for a variety of uses. One common use is for the location of centroids and centroid connectors.

Transportation Networks- Errors in the highway network are another common source of error in travel models. The following checks are recommended for the highway network.

Trip Generation- Typical reasonableness checks in the trip generation process consists of checking the land use and socioeconomic data. These areas are the most likely to contain errors. Make sure the unbalanced productions and attractions are within a (+/-) 10% range. A population to employment ratio is typically between 40-60%. Rule of thumb for external trips is approximately 10-20%. Trip generation totals should be deemed reasonable prior to advancing to the trip distribution phase.

Trip Distribution- Travel impedances reflect the shortest travel path between all zones. These travel paths should be checked for reasonableness. A variety of methods exist to accomplish this. Model trip length frequency distribution by trip purpose should be compared to observed trip lengths. If quality observed data is not available, you should use reasonable estimates based on local field observation. You should make sure your friction factor table goes out at least to the maximum zone-to-zone travel time in your network. Intrazonal trips are important because they directly affect the volumes on the network:

Highway Assignment- Validation tests for highway assignment typically consist of system wide checks such as vehicle miles of travel (VMT,) vehicle hours of travel (VHT,) cordon volume summaries, and screen line summaries. Cut lines and screen lines are used to check the validation on more corridor specific areas. Root mean square error is also used to compare all count locations to traffic assignments at the same location.

Various factors affect the level of calibration in a travel demand model. Some factors influence the model in a global manner while others are more specific locations. It is imperative that processes used to calibrate can be transferred to the forecast model and remain effective over time. As previously noted most significant calibration problems generally stem from errors in the employment, housing, or network data and from errors in the traffic counts. These items should always be checked first.

Partnering- Partnering is important in North Carolina, thus the development of the Travel Model Research Program. Recommendation for Iowa is to develop and adopt modeling guidelines and best practices as a community. Partner with consulting when needed.

Survey Data- There should be an emphasis placed on the investment of travel survey data to support modeling efforts.

Summary Points:

CH2MHill Consultants
Jim Miller

Jim Miller is Vice President-Project Manager for Ch2MHill in Chicago, Illinois and is a registered Professional Engineer in the State of Illinois. Jim Miller's presentation covered a recent project in Sioux City, Iowa that entailed the conversion and re-development of the MPOs travel demand model. The MPOs travel demand model was converted from a Tranplan to a TransCad environment. In addition to the conversion process, a new base year model was calibrated to existing traffic counts and a new forecast year 2030 model assignment was created. Key items in the calibration/validation phase consisted of:

The travel demand model was successfully calibrated to meet all FHWA standards.

Jim also discussed various projects consisting of model calibration/validation, sub area planning studies, Fox River Bridge Crossing in Kane and Lake counties in the Chicago, Illinois region.

Model Calibration Tips:
Essential items:

Louis Berger Consultants
Dane Ismart

Dane Ismart is senior associate with the Louis Berger firm of Florida. He has 25 years experience with the FHWA and is the author of Calibration and Adjustment of System Planning Models, December 1990.

Mr. Ismart covered various sections of the model calibration manual, describing its application to current travel demand models. A summary of the points presented is as follows:

The following is a synopsis trouble shooting techniques when calibrating a travel demand model. Our common measure of effectiveness is the traffic count. This is what we compare our base model runs to in order to measure how well our model is working. An important note is to realize that traffic counts can have significant error. We must judge traffic counts for reasonableness. When adjusting models to meet calibration standards, it is encouraged to make changes on global levels, rather than at specific locations. We want model modifications to be transferable to future models. Special care should be taken when using K-factors to adjust model volumes. The K-factor is the "nuclear weapon" for model assignment. Use as a last resort.

As modelers, we should first determine what types of errors we have. Are they universal or local in nature? Are they system wide, link specific or screenline specific? The first approach is take a macro approach and look at system totals such as Vehicle Miles Traveled and Vehicle Hours of Travel.

In the trip generation stage the productions and attractions should match within +/- 10% prior to balancing. If they are not within these standards there may be error in the socio-economic data. Special generators should be accounted for, however, they will not have any effect when balancing the productions and attractions.

Intrazonal trips should be about 3-5% of total trips. Decreasing the number of intrazonal trips can allow more vehicles onto the network and increase traffic volumes. Zone size can be reduced to cause less intrazonal travel. Homogenous land use within the TAZ can also reduce the number of intrazonal trips. Pass by trips and internal capture for larger shopping developments should be accounted for also. The nearest neighbor/shortest path method is most common for determining the number of intrazonal trips.

Travel demand modeling can be done without expensive surveys, however the confidence levels are reduced.

Additional methods to consider: Centroid connector speeds can be changed or the centroid can be moved to adjust traffic loading. Certain centroid connector locations can cause "lumpy" assignments. Adjusting the friction factor table can modify average trip lengths for each trip purpose. For traffic analysis zones, the population should be approximately 1,000 persons per zone.

The travel demand network should reflect intersections penalties and prohibitions. It is important for link attribute data to be representative of the actual street system.

Caliper Corporation
Howard Slavin

Howard Slavin is the President of Caliper Corporation. Caliper Corporation, founded in 1983 and headquartered in Newton, MA, is a technology leader in the development of GIS and transportation software. Caliper is the developer of TransCAD® (http://www.caliper.com/tcovu.htm), a commercial geographic information and analysis software packages.

Mr. Slavin's presentation is summarized by the following bulleted list:

Individual MPO Overview and Discussion

In order for the panel members to get a better idea of what the state of the practice is regarding model development, application, calibration and validation in Iowa, each MPO gave a synopsis of their travel demand modeling efforts.

Once there was an understanding of the best practices from the panel members and an overview of the state of the practice in the Iowa MPOs, discussion ensued regarding the application of the best practices in Iowa.

TransCad Software Demonstration

Howard Slavin and Andres Rabinowicz of Caliper Corporation gave a software demonstration of the newest version of TransCad.

Summary of Panel Recommendations

Iowa Statewide Travel Modeling Peer Review Recommendations

All of the following recommendations focus directly on the steps and needs of moving a coordinated statewide program for travel modeling in urban areas forward...

1) Develop a statewide program framework for best practice.

IADOT should, in partnership with other participants, lead an effort to define and develop an overall framework for statewide modeling support. This framework should address how to establish standards in the areas identified below and how to address ongoing and evolving needs in areas such as statewide needs in data development.

1a) Establish standards and uniform practices for specific elements of model development, application, evaluation and implementation.

Through the participant group (MTMUG), specific groups should be identified to determine ways to support and standardize specific elements of the statewide program. The Peer Review panel has identified a list of specific areas to address:

1b) Develop technical support facility.

To ensure adequate software and modeling support to participant agencies, the program should identify and establish a local technical support resource to respond to day-to-day questions regarding model development, implementation and application. This resource should be knowledgeable in travel modeling design and application and in TransCad technical issues and related matters.

1c) Actively coordinate with and draw on similar programs and efforts in other states and areas.

Statewide programs such as this have been initiated and developed in a number of other states including Florida, North Carolina, Michigan, Ohio and Texas. Iowa should contact the groups responsible for these programs, collect documentation on their approaches and experiences, and develop technology transfer and coordination mechanisms with them.

2) Documentation of travel modeling guidelines, standards and best practices.

IADOT, CTRE and the MPO's should make every effort to document the evolving consensus on appropriate practices, standards and criteria for travel modeling in Iowa.

3) Dissemination of information to support and define the travel-modeling program.

Means are needed to ensure that information about guidelines, standards and ongoing activities of the state travel-modeling program is widely distributed to participants and other involved parties. Continuation of regular meetings of the Midwest Travel Model User Group, as well as, an up-to-date website, email discussion lists and other appropriate means should be explored as ways to ensure timely and accurate dissemination of information to support the program.

4) Create a statewide initiative to design and implement a data development program to ensure generation and availability of travel behavior survey and other related data resources required for ongoing participant agency travel modeling activities and updates.

5) The following are recommendations to direct the statewide effort's resources into those areas, which will provide immediate and ongoing benefits to the program's foreseeable needs.

Integration of the elements listed below into the program should be considered in light of immediate needs and judicious allocation of program resources, but should be applied where feasible.

However, planning should consider the need for multimodal models in the future and accommodate introduction of detailed transit models as and when it becomes useful.

To support this and other non-auto mode modeling in the future, a path for migration from the current vehicle based modeling environment toward person trip based modeling should be identified and implemented as an early action. In the interim, consideration should be given to other separate tools and procedures for transit planning.

Summary of Next Steps

This TMIP Peer Review was intended to provide an assessment of our existing travel demand model calibration and validation methodologies and result in guidance from the panel experts on how we could improve our process and ultimately our travel demand models. The IaDOT and the MTMUG will begin outlining a methodological approach to prioritizing and implementing the panel's recommendations. The following are brief descriptions of the efforts we intend to begin in the near future to achieve goals set forth in the panel recommendations:

Statewide Travel Demand Model

The IaDOT is taking the first steps to consider the development of a statewide vehicle and freight demand model. A statewide travel model will be particularly useful in the departments statewide planning efforts and providing a technical basis for decision makers. This is an area that has received consideration in the past and seems to have support of upper management. Should funding be available, this project could begin soon.

Travel Modeling Guidelines, Standards, Best Practices and Expectations Program and Documentation

Over the past few years, IaDOT and MTMUG have been working steadily to enhance the quality of our travel demand models and provide examples of best practices through presentations and training sessions. To be most effective, this process will become more formalized in the near future. Cooperatively, the IaDOT and MTMUG intend to begin the development of a statewide Guidelines, Standards, Best Practices and Expectations program that identifies and documents the currently accepted practices for travel demand modeling in Iowa. This program and documentation will identify standards and uniform practices for travel demand model development, application, evaluation, and implementation. The program will be ongoing and updated frequently to reflect the state-of-the-art practices in the field.

MTMUG and the IaDOT anticipates keeping in touch with Ms. Leta Huntsinger at The Institute for Transportation Research and Education at North Carolina State University as they work to implement the same type of program and documentation in North Carolina.

Development of a Technical Support Facility

The IaDOT's Office of Systems Planning has been given the task by the Director to provide more technical support and assistance to the MPOs, specifically in the area of travel demand modeling. The Department is in the process of adding additional staff to take on this initiative. Department staff are currently providing technical assistance to the majority of Iowa's MPOs as well as providing training to new modeling staff. It is anticipated the effort will continue to grow as we take on more endeavors resulting from the Peer Review recommendations.

Continue and Enhance the MTMUG and the MTMUG Website

The IaDOT and MTMUG are committed to further enhance the attendance and quality of the quarterly MTMUG meetings. This can be accomplished by continually providing useful information to the group and maintaining the strong sense of community we currently have. It will also be beneficial to provide more opportunities for professionals from other states to give presentations on their experiences and best practices.

There is also a commitment to provide more frequent updates and new information to the MTMUG website. Additionally, the MTMUG listserv will be used as a basis for asking questions and obtaining input from MTMUG members on travel demand modeling issues.

Learn and Draw From Similar Programs

The IaDOT and MTMUG are committed to be more proactive in researching and learning from similar programs in other States. It is planned to contact other states to find out what elements of their programs have been most successful. Various states with operational programs were identified in the Peer Review Panel recommendations. This will be followed by an effort to incorporate proven practices into our own operations.

Partner in OD Survey and Data Development Program

The IaDOT and MTMUG will investigate ways of pooling funds and resources to collect and maintain data sources and update our outdated Origin/Destination surveys. This information will then be made available for use in MPOs within the state.

The previous descriptions are some of the planned efforts the IaDOT and MTMUG plan on implementing in the near future in order to achieve goals set forth in the panel recommendations.

Appendices

Agenda

Travel Demand Model Calibration/Validation and Reasonableness Assessment

Tuesday, March 30, 2004
1:00 pm Welcome - Phil Mescher, Iowa Dept of Transportation
1:10 pm Purpose of the Peer Review and Charge of the Panel - FHWA
1:15 pm Introductions and MPO Descriptions
1:30 pm Panel Presentations on Model Calibration and Validation Best Practices
3:15 pm Afternoon Break
3:30 pm Panel Presentations on Model Calibration and Validation Best Practices
5:15 pm Open Questions and Answers
6:00 pm Dinner at Gateway Center Provided
Wednesday, March 31, 2004
7:45 am Pastries and Rolls
8:00 am

Model Calibration Discussion Topics

  • General Description of Steps in Calibration and Adjustment Process
  • Networks
    • Capacities
    • Speeds
    • Intrazonal Travel Times
    • Intersection Penalties/Prohibitions
    • Centroid Placement and Connectors
9:45 am Morning Break
10:00 am

Model Calibration Discussion Topics (continued)

  • Socio-economic Data
    • Census/CTPP
    • ES 202 Employment
    • Survey Data (NHTS)
  • Trip Generation
    • Trip Purposes (HBW, HBO, NHB, EI, EE)
    • EI as Part of HBW, HBO, NHB
    • Cross Classification/Regression
    • Trip Rates
    • Special Generators
    • Trip Balancing
    • TransCad Quick Response Method
11:30 am Lunch at Gateway Center Provided
12:30 pm

Model Calibration Discussion Topics (continued)

  • Trip Distribution
    • Friction Factors
    • Gravity Model K-Factors
    • Trip Length Distribution Adjustments
  • Traffic Assignment
    • Types of Assignment Methodologies
      • All-or-nothing
      • Equilibrium
      • Capacity Restraint
      • Stochastic
2:30 pm Afternoon Break
2:45 pm

Model Calibration Discussion Topics (continued)

  • Performance Measures
    • Expected and Required Accuracy
    • Reasonable Expectations
    • Percent Error Region Wide
    • Percent Error by FFC
    • Correlation Coefficient
    • RMSE
    • VMT/VHT Ranges for Size of Area
    • Screenlines/Cut Lines/Cordon Lines
    • Post processing procedures NCHRP 255
5:00 pm Dinner at Gateway Center Provided
Thursday, April 1, 2004
8:00 am TransCad Software Demonstration by Caliper Corp.
Panel Deliberation and Recommendation Development
10:00 am Summary of Peer Review and Panel Recommendations
10:30 am Discussion
12:00 pm Adjourn

List of Participants

* Ken Cervenka - North Central Texas Council of Governments
* Ed Granzow - CH2MHill Consulting
* Paul Hershkowitz - Wilbur Smith and Associates Consulting
* Leta Huntsinger - The Institute for Transportation Research and Education at North Carolina State University
* Dane Ismart - Louis Berger Consulting
* Jim Miller - CH2MHill Consulting
* Howard Slavin - Caliper Corporation
Ed Christopher - Federal Highway Administration (FHWA) Resource Center
John Cater - FHWA Iowa Division
Holly Liles - FHWA Iowa Division
Andres Rabinowicz - Caliper Corporation
Tom Kane - Des Moines Area Metropolitan Planning Organization (MPO)
Adam Garms - Des Moines Area MPO
Kevin Gilchrist - Des Moines Area MPO
Dean Wheatley - Linn County Regional Planning Commission Cedar Rapids Area MPO
Sam Shea - Linn County Regional Planning Commission Cedar Rapids Area MPO
Lalit Patel - Bi-State Regional Planning Commission Quad Cities Area MPO
Chandra Ravada - East Central Intergovernmental Association Dubuque Area MPO
Sheldon Harrison - Siouxland Interstate Metropolitan Planning Council Sioux City Area MPO
Greg Youell - Metropolitan Area Planning Agency Omaha/Council Bluffs Area MPO
Steve Wallace - Metropolitan Area Planning Agency Omaha/Council Bluffs Area MPO
Kevin Woodard - Ames Area MPO
Anissa Williams - Johnson County Council of Governments/Iowa City Area MPO
Garrett Pedersen - Iowa Northland Regional Planning Commission/ Waterloo Cedar Falls Area MPO
Ahmed Kaja-Mohideen -Champaign Urbana Urbanized Area Transportation Study
Rodger Tomasek - Nebraska Department of Roads
Reg Souleyrette - Iowa State University/Center for Transportation Research and Education
Tom Maze - Iowa State University/Center for Transportation Research and Education
Phil Mescher - Iowa Department of Transportation (IaDOT) Office of Systems Planning
Andy Loonan - IaDOT Office of Systems Planning
Brian Squier - IaDOT Office of Systems Planning
Stan Peterson - IaDOT Office of Systems Planning
Lee Benfield - IaDOT District 6
Jason Carbee - URS Consulting
Brad Chambers - HWS Consulting
Brian Ray - HWS Consulting
Kevin Pape - Howard R. Green Company Consulting
Richard Storm - CH2MHill Consulting

* Denotes Peer Review Panel Member

Presentations

Standard Rates and Values for Modeling (HTML or PDF)

Calibration and Validation (HTML or PDF)

Wilmington Model Development Steps.

Travel Demand Modeling At NCTCOG.

Travel Model Validation - Key Considerations.

Iowa Peer Review for Travel Demand Model Calibration/Validation and Reasonableness Checking.

Planning Process.

Travel Demand Model Calibration/Validation.

For more information about these presentations, please contact the TMIP Moderator.

Questions/Topics Posed

  1. How do other MPO's address model calibration and validation?
  2. What types of data are needed to perform model validation?
  3. What Measures of Effectiveness (MOE's) should be used to determine model validity?
    1. Trip Generation
      1. P's and A's within 10% prior to balancing
    2. Trip Distribution
    3. Traffic Assignment
      1. Correlation Coefficient
      2. Root Mean Square Error
  4. At what level should the MOE's be based?
    1. Functional Class
    2. Volume Ranges
  5. How do we know when the model is calibrated "good enough"? When to we reach the point of diminishing return?
  6. What are some of the calibration "tricks-of-the-trade" to get a model validated?
    1. Speed Changes
    2. Moving Centroid Connectors
    3. Changing Capacity
    4. Assignment Methodology
      1. All-or-nothing
      2. Capacity Restraint
      3. Equilibrium
      4. Stochastic
  7. How to best apply and use results from Screen lines, Cordon lines, and Cut lines.
  8. What are some common sources of model error?
  9. What are some common troubleshooting strategies?
  10. How much of the network should be coded with actual traffic counts. I read that 65% of the network links should be counted and that sounds high and expensive.
  11. How to get parallel links to calibrate, when the flow seems to toggle from one link to the other.
  12. What cross-classification data is used most often and which data is proven to generate the most reliable Productions & Attractions?
  13. What to do when trip productions and attractions are outside the +/- 10% parameter prior to balancing and updated survey info for the area is not available?
  14. Is there a process can be used to force zonal trips to utilize all centroid connectors?
  15. Is there a maximum that a speed should be changed to manipulate the model assignment?
  16. How many cut and or screenlines should be utilized during calibration?
  17. What percentage of total trips should be intrazonal for residential zones, commercial zones, industrial zones and CBD zones?
  18. When validating traffic assignment to actual ground counts, which is more important, the individual link volume to count ratio or the totaled link type (FFC or volume) volume to count ratio? And if the individual link analysis is more important, what percentage of link volume/count ratios needs to be within parameters?
  19. What is the predominant measure of effectiveness level for validation and calibration? Count comparisons based on totals by FFC or simply by individual links?
  20. Should a mixture of link volume to count analysis and cut line and screen line analysis be used to calibrate and validate the base network assignment?
  21. How to check and make sure centroid connectors are not being used for through traffic. Traffic should be in and out only.
Updated: 03/28/2014
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000