Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration
Office of Planning, Environment, & Realty (HEP)

Status of TMIP Peer Reviews - Memphis Metropolitan Planning Organization (2006)

(PDF version)

PDF files can be viewed with the Acrobat® Reader®

Location Date Peer Review Host Agency Peer Review Host Agency Contact Peer Review Panelists
Memphis, TN March 27-28, 2006 Memphis Metropolitan Planning Organization (MPO) Paul Morris, Memphis MPO Principal Planner, Memphis, Tennessee Ken Cervenka (Chair), North Central Texas Council of Governments, Arlington, TX
Ed Granzow, CH2MHill, Oakland, CA
Leta Huntsinger, ITRE/NCSU
Guy Rousseau, Atlanta Regional Commission, Atlanta, GA
Howard Slavin, Caliper Corporation, Newton, MA


The content of this peer review report does not represent the opinions of FHWA nor does it constitute an endorsement, recommendation or specification by FHWA. The content of the report does not determine or advocate a policy decision/directive or make specific recommendations regarding future research initiatives. The report is based solely on discussions and comments during the peer review.

Table of Contents

Executive Summary

On March 27 and 28, 2006, the Memphis Metropolitan Planning Organization (MPO) hosted the second travel demand model peer review meeting1 at the Memphis Area Transit Authority (MATA) Central Station offices. The two-day peer review was held as part of Travel Model Improvement Program (TMIP), sponsored by the Federal Highway Administration (FHWA). The primary purpose of the peer review was to help the Memphis MPO successfully implement an updated travel demand model by discussing the issues, obstacles, and solutions for achieving the goals described in the Memphis MPO Travel Demand Model Study Design2.

1 The first peer review meeting for the Memphis MPO, sponsored by the TMIP program, was October 27-28, 2004. The summary report for the first peer review is available at:
2 Kimley-Horn and Associates, Inc., Cambridge Systematics, Inc., and HNTB, The Memphis MPO Travel Demand Model Study Design, December 2002.

On the first day, the consultant team gave presentations to peer review panel members, the Memphis MPO staff, and stakeholders on the project schedule, trip generation validation, trip distribution validation, mode choice results and validation, transit assignment validation, highway assignment validation results, and model interface. Discussions followed each presentation.

The peer review panel met in closed session on the second day to confirm that the travel model is in line with the original scope, study design, and commitments; confirm that validation measures and criteria are consistent with industry standards; brainstorm additional troubleshooting measures; determine acceptable calibration measures; confirm that the current progress is acceptable; and define what kind of output will determine acceptable progress.

The panel reiterated that the Memphis MPO has a strong model consultant team in place and that the strengths identified by the previous peer review are still valid. Panel members then presented a set of recommendations to the MPO and the consultant team on how to complete the validation phase, how to facilitate the transfer of the model to the Memphis MPO, and issues to consider for the future.

1. Background

The Memphis MPO uses a validated travel demand model and is currently updating its modeling practices by developing new travel demand forecasting tools for use with various transportation planning, programming, and project evaluation activities; air quality analyses; and transportation and land use policy decisions. During the first phase of the travel demand model update, the Memphis MPO used data from Census 2000 and a 1998 household travel survey. During the second phase, the MPO took inventory of the available travel data and created the Memphis MPO Travel Demand Model Study Design document (the "Study Design"). The MPO is now developing a model that follows the Study Design and contains "best practice" approaches supported by available data. Figure 1 illustrates the old model boundary and the proposed travel demand model boundary in Shelby County and western Fayette County in Tennessee and northern DeSoto County in Mississippi.

To help implement Phase Two, the MPO hired the consulting firm Kimley-Horn & Associates, Inc (KHA) and Cambridge Systematics, Inc. and HNTB as sub-consultants. The consultant team completed the Study Design in 2002. In 2004, it began to develop the travel demand model. The Memphis MPO used the Federal Highway Administration (FHWA) Travel Model Improvement Program (TMIP) Peer Review Program to obtain feedback on the model from a panel of experts to ensure the model meets the MPO's needs, meets the standards of professional practice, and meets Federal, state or local planning requirements.

First TMIP Peer Review

The first TMIP peer review took place in October 2004. The panel was tasked with advising the Memphis Model Steering Committee, consisting of representatives from various local transportation agencies, and Memphis MPO staff on developing a new travel demand model for the Memphis area to successfully implement a state-of-the-practice four-step model that can be used in developing the Regional Transportation Plan (RTP), the Transportation Improvement Program (TIP), and in air quality conformity analyses.

The Memphis MPO asked the peer review panel to:

Proposed Model Boundary
Figure 1. Current and Proposed Memphis MPO Travel Demand Model Boundary

The panel felt that the Memphis MPO has a strong consultant team and a good working relationship with them, which suggests that the travel demand model is in a very good position for being completed within budget and on schedule. They concluded the proposed model structure represented a reasonable state-of-the-practice model development. The panel's primary recommendations for model improvements in the first TMIP peer review were:

Second Peer Review

The second TMIP peer review panel, presented in this report, met in March 2006. The panel's charge was unchanged from the first peer review. Presentations and discussions focused on project schedule updates, trip generation validation, destination choice validation, mode choice validation, transit assignment validation results, highway assignment validation results, and model interface.

Panelists were:

The consultant team requested the following input from the panelists during this meeting:

2. Presentations on the Status of the Memphis Travel Demand Model

Members of the consultant team from KHA and Cambridge Systematics presented the status of the Memphis travel demand model to peer review panel members, the Memphis MPO staff, and stakeholders.

At the time of the meeting, the consultant team was calibrating the model to meet validation targets. The assignment results had generated noticeably lower vehicle miles traveled (VMT) than the target by 17 percent. The consultant team had several ideas for troubleshooting the model but had asked the peer review panel for input before proceeding. As a result, presentations focused on calibration and validation. Topics included assignment validation results and model interface, in addition to a project schedule update. Discussions followed each presentation. Presentations on the project schedule, the travel demand model, and validating the model are available at

Project Schedule Update

The consultant team distributed a detailed project schedule3 describing tasks completed since the previous peer review meeting:


The consulting team is currently validating the components of the model and expects this phase to be completed by the end of April 2006. Future tasks include quality control of the forecasting, output, and model documentation in June 2006. The consultant team noted that air quality analysis is not within their scope and did not include the task in their budget; this remains the responsibility of the Memphis MPO.

Trip Generation Model Validation

Troubleshooting low VMT in the model began with the trip generation model. In comparing productions and attractions by trip purpose, no large imbalance was found, indicating that the data is valid. The consultant team aggregated trips per household results into three categories to validate the share of trips by purpose.

As shown in Table 1, the trips per household in all three categories fall within the expected range.

Table 1: Trips per Household Validation

Trip Category Trips per Household Per Day Percent Expected Range4
Journey to work 2.0 25 18-27
Home based non-work 4.2 54 45-56
Non-home based 1.6 21 20-30
Total 7.9 (8.7 adjusted for JTW stops)

4National Cooperative Highway Research Program (NCHRP) Report 365, Travel Estimation Techniques for Urban Areas, 1998.

Initially, the consultant team thought the trips-per-household rates were low. To compare the results to trip rates from other cities, they adjusted the total number of trips to reflect the extra stops included in journey-to-work (JTW) trips, using the definition that one home-based non-work trip plus one or more non-home based trips are equivalent to a JTW trip. The consultant team compared these rates with the Atlanta, Detroit and Salt Lake City travel models. The findings showed that the Memphis rates were in the appropriate range and did not differ systematically from the rates in the three other models. Therefore the consultant team did not see a need to increase the trip rates.

The consultant team found that refinement in the modeling of rural areas explained some of the differential in the VMT, but ruled out geographic stratification by testing different cross-classifications, including three-way cross-classifications. Cross-classification measures the change in one variable when other variable(s) are taken into account. The consultant team used vehicles per household instead of income as a classification variable for trip generation because it was statistically a better indicator for most trip purposes. They also surveyed other models to determine the frequency of use for both variables and found the models to be evenly split between the two. The consultant team is planning to compare the expanded survey and the model output by geographic districts.

Panelists suggested using a tourist model to capture the additional attraction, beyond regular employment, of tourist areas such as Beale Street. Based on review of a Parson's Brinkerhoff study5 performed for MATA on the magnitude of tourist trips, tourism alone would not result in such a level of difference, but it could slightly increase VMT. The consulting team will check the Parson's Brinkerhoff model (which used transferred 1996 rates from Las Vegas) to assess the potential magnitude of tourism trips and their effects (e.g. availability of parking lots).

5 Parsons Brinkerhoff Quade and Douglas, Inc., Memphis Alternatives Analysis, Downtown-Airport Corridor, Travel Forecasting, Methodology and Analysis Report, April 2002.

Trip Distribution Model Validation

Following trip generation, the model is separated by time of day so that the appropriate travel times are applied for the appropriate period. The Memphis travel model uses a logit-destination choice formula. The utility of choosing a destination, zone j, is a function of the impedances that account for distance, the log-sum of zone size, and a set of production or attraction dummy variables that represent area types.

Utility of choosing destination zone j =

B1 * (impedanceij)


+ B2 * ln (size variablej)

  + B3 * (prod or attr zone dummy variable 1)

+ ...

  + Bn * (prod or attr zone dummy variable n-2)

All zones are included in the calibration-year destination set. Given the likely changes in the number of employees in each destination zone, during the forecasting stage the consultant team will reevaluate the need to include all zones and the calibration factors used. The consultant team added factors for work trips (constants) to ensure that the number of work trips to each zone matched the number of workers in each zone. The consultant team calibrated the destination and mode choice models together. They also calculated true free-flow speed since travel time studies showed that free-flow speeds are often greater than the posted speed limit.

The consultant team used the following steps to calibrate the trip distribution model:

  1. Calculate, for each zone, the probabilities of travel to all other zones
  2. Set the distance coefficients for 0 and 1+ vehicles; this number is allowed to vary since people with vehicles are less sensitive to distance than those without vehicles
  3. Ensure that coefficients applied to distance variables will always be positive
  4. Use the unweighted geo-codable trip-records (representing 80 percent of all records) to estimate the origin-destination (OD) matrix for the different time of day categories
  5. Run the model four times

The consultant team found the destination model coefficients were reasonable by:

Figure 2 illustrates the predicted and observed frequencies of trip lengths for zero-vehicle households. The observed trip length histogram displays two peaks. The behavior was most prominent for JTW. The consultant team and peer panelists speculated that the peaks might suggest a multi-nucleated region or separation by mode availability. Without understanding the source of the peaks, the travel model will model it simply as one peak since a two peak model would result in forecasting problems.

this is a histogram of the journey to work trip length

Figure 2: Journey to Work Trip Length Histogram

Another segment that was difficult to model was the home-based university (HBU). Observed histogram data for 0 and 1+ vehicle households also showed multiple peaks. Since the home-based university market segment was small, the consultant team did not spend a large amount of time analyzing the observed data. One peer panelist suggested that in the future the MPO should take a higher number of traffic counts for special, low market segments such as universities and airports. Another peer panelist suggested constructing a district-to-district flow table to better understand the data. The consultant team had already identified this as an action item. All other trip length histograms showed the model output was close to the observed data.

The consultant team concluded that even though the distribution model could be adjusted to more closely match the observed data, it could not be the main source of the low VMT. Since travel times matched well, other changes should be considered first for addressing the problem.

Mode Choice Model Results and Validation

Since the first peer review, the transit on-board survey has been completed. The consultant team summarized the survey data set, which provided the majority of the data for the model estimation. The mode choices considered were: transit with auto access, bus with walk access, trolley with walk access, non-motorized, shared-ride, and drive alone. Some trip purposes were separated or combined in the modeling if the survey had too few data points. Table 2 lists trip purposes and mode options.

Table 2: Modes Modeled by Trip Purposes

Trip Purpose Modes Available
HBO (Home-based other)
NHBW (Non-home-based work)
NHO (Non-home-based other)
Transit Auto Access
Bus Walk Access
Trolley Walk Access
Shared Ride
Drive Alone
HBSc (Home-based School) Transit
School bus
Drive alone
HBSh (Home-based shop) Transit with auto access
Bus Walk Access
Trolley Walk Access
Shared Ride
Drive Alone
HBPD (Home-based pick-up and drop-off) Non-motorized
HBSR Transit

1Drive-alone is the portion before and after when the passenger has been picked up and dropped off.

Similar to the trip generation model, the consultant team tested the use of income versus vehicle availability as an explanatory variable. They decided not to include income as a direct variable because it is difficult to forecast; however, income is used for vehicle availability forecasting.

Many coefficients, such as the in-vehicle time (IVT), out-of-vehicle time (OVT), and cost are manually constrained in the mode choice model because they fell outside of the range of the FTA's New Starts requirements.6 Since constraints prevent the estimates from fitting the data, to minimize the number of constraints the model is first run without constraints, then run with the minimum number of constraints on variables. The consultant team noted that the model was made more difficult to calibrate due to the FTA constraint on the on-vehicle time.

6 FTA New Starts information is available at:

Peer panelists noted that it is important to document the constraints applied. The consultant team said they would probably provide the capability to run the model with or without the constraints so that the effect of the constraints can always be determined.

The multinomial model was established for all trip purposes. Tests showed that nesting structures were not supported by the data (i.e., the nested coefficients were close to 1). Trolley and bus trip behaviors were very similar as they both have captive ridership due to the high number of 0-vehicle households. While large cities with rail systems are easier to nest due to the distinctness of the options, Memphis does not currently have a light rail system and therefore nesting was not implemented. However, the model does have a placeholder for a future new mode.

Rho-squared for the model was 0.3, indicating an acceptable goodness of fit. Target shares were based on the transit survey and the control totals were based on ridership data calculated from fare box counts in the late 1980s. Data should be valid since route levels have changed very little. Non-rider share data was based on the 1998 household survey. If ridership levels have changed, it could affect the VMT, although not enough to account for the low VMT. Panelists suggested verifying ridership by analyzing the current revenue data.

Panelists also suggested that the percentage of transfers used in the model could significantly affect mode choice. The model uses between 50 and 60 percent transfers based on data from the on-board survey.

As a rough gauge for the reasonableness of the coefficients, the consultant team compared their results to the averages and ranges of coefficients for the same variables in other mode choice models and coefficients listed in the FTA New Starts Guidelines. The consultants noted that coefficients from logit models couldn't be technically compared because they are meaningful only in relation to the other variables in the model. The ranges of values for some variables was so large and the underlying model structures sufficiently different that the comparisons did not provide much meaningful input.

The consultant team presented the mode choice validation for JTW and home-based other (HBO) results by car ownership levels. The majority of the results was very close to the observed data. The panelists asked about the higher-than-typical share of shared rides (18.5 percent for JTW) since nationwide vehicle occupancy is decreasing and shared rides outside of the family are decreasing. The consultant team acknowledged these numbers were high but that they were consistent with the survey data.

The non-motorized share was larger than the transit share, which is consistent with a city the size of Memphis. The majority of the share data is from 1998 and the sub mode data is from the on-board survey. Non-motorized trips are mostly walk trips with very few bike trips.

Trip Assignment Model - Transit Assignment Validation

The transit assignment model uses the TransCAD Pathfinder transit assignment method for every mode and period. The consultant team presented data on observed and modeled line-by-line ridership on MATA and identified large differences. Panelists felt that even though such differences are common, they should be investigated, particularly where the modeled results are over 100 percent of the observed ridership, to ensure that the reason behind the difference is understood.

There was some confusion during the meeting about the reporting of number of trips as opposed to number of boardings. The model was estimated based on MATA's boarding and trip data from 2003. MATA informed the consultant team that they have revised their ridership numbers to reflect the decreasing ridership, bringing the total boardings to 42,000 (as opposed to 60,000 to 64,000 boardings per year previously reported). MATA will verify their numbers and forward them to the consultant team. This means that the observed transit mode share is actually lower than model output data, which will slightly increase the number of car trips and VMT.

The model uses four time periods to validate the transit assignment. All of the routes are maintained in one file and used by the system to calculate the four periods. It is not clear whether the MATA peak and off-peak route and headway data were properly used. The consultant team will work more closely with MATA to ensure that the transit data used in the model is current and accurate.

The consultant team was also concerned that the trolley and bus surveys were inconsistent. They proposed using just the bus data and expanding the 2004 on-board survey for the transit assignment.

Panelists made several validation suggestions:

Trip Assignment Model - Highway Assignment Validation

The consultant team presented data from the highway assignment validation targets, shown in Tables 3-6. The first three tables measure the percentage difference for VMT and volume by functional classification and by daily volume grouping.

Table 3: Percent-Difference Targets for VMT by Functional Classification

Facility Type Target
Freeways 8-12%
Principal Arterials 18-22%
Minor Arterials 27%
Colletors 33%

Table 4: Percent-Difference Volume Targets by Functional Classification

Facility Type Target (+/-)
Freeways 7%
Principal Arterials 10%
Minor Arterials 15%
Colletors 25%
Local 25%

Table 5: Percent-Difference Volume Targets by Daily Volume Groupings (entire group totals)

Volume Target (+/-)
<1,000 200%
1,000-2,500 100%
2,500-5,000 50%
5,000-10,000 25%
10,000-25,000 20%
25,000-50,000 15%
>50,000 10%

Another target is the percent of links within a specified percent of count by facility type, to measure how many links are performing on target as opposed to the system-wide measures. In Table 6, 75 percent of the freeway links need to be within 20 percent of the counts and 50 percent of the freeway links need to be within 10 percent of the counts.

Table 6: Percent of Links within a Specified Percent of Count by Facility Type

Facility Type Target within Count Range Compared to Counts
Freeway 75% 20%
Freeway 50% 10%
Major Arterial 75% 30%
Major Arterial 50% 15%
Minor Arterial 75% 40%
Minor Arterial 50% 20%

The consultant team identified the following problems:

Panelists suggested the consultant team troubleshoot the data based on the assumption that the counts may be incorrect. Since aggregate data can be misleading, the data sets should be checked separately, focusing on the data sets that were not specifically gathered for this model. They also suggested separately analyzing data from different sources to identify patterns in validating the data, such as directional time period. The first step should be a basic analysis, such as identifying where counts are greater than capacity, zero-volume links that are not loading, and screenline checks. Panelists described experiences where they found counts to be four times the capacity on certain links. Targeting these links can help to identify network miscodings. Memphis capacities are calculated using facility-based capacity equations that closely matched counts. To emphasize the importance of checking data, one panelist said many modelers are finding that their models are better than they thought after determining their data was bad.

A member of the consultant team said they collected travel time data using global positioning system (GPS) units and analyzed data by comparing speed limits to counts from the peak and midday periods. They obtained free flow speeds using floating car surveys and then used congested PM peak speeds to calculate travel time factors that looked reasonable.

Panelists also suggested that the consultant team check whether seasonal adjustment factors and average daily traffic-based truck factors have already been applied to the count data; these should be confirmed and validated to ensure that the input count data is interpreted correctly. In addition, the consultant team should consider a tourist model, although this may not result in a significant VMT change since the Memphis model is a weekday model. Panelists suggested that construction of district flow tables and checking trip directions to help visualize patterns will help with troubleshooting.

The consultant team presented the tasks they were troubleshooting or planning to troubleshoot. These checks are being built into the interface so that they can be processed rapidly to make the troubleshooting/calibration activities more efficient and simpler to document.

Table 7 summarizes the tasks and troubleshooting status.

To troubleshoot the assignment bias toward interstate rather than non-interstate facilities (an issue that was particularly pronounced in urban areas), the consultant team plans to review the following issues: ramp attributes, volume delay functions, free-flow speed factors by facility type, and signal density factors by facility type.

Peer panelists agreed that the proposed troubleshooting/calibration plan was reasonable. They suggested running changes individually and logging the results in a matrix to evaluate the impacts of each change separately, since some of these changes could counteract each other.

Table 7: Troubleshooting Tasks for Globally Low Modeled Volumes Versus Observed Volumes

Task Description/Issue Status
Review Model Script Review model script modules. Complete

Unlikely that trips are being "lost" anywhere
Review Network Perform a standard quality assurance on the network, for example, identifying zigzags inside a single link that should be interpreted as U-turns. In Progress

One of the peer panelists performed a preliminary check of the model network and found:
  • Some connectivity issues exist on the network
  • Alignment inaccuracies of the line layer
  • Too few centroid connectors being used. Four or more should be used for each zone, instead of two in the current model. In general the addition of centroids reduces VMT but it will affect how the new network is loaded. Consultants will also be checking centroid positioning.
Review Internal Trip Rates Planned
Review Internal/External Trip Lengths Check trip lengths against survey.
Perform checks at the level of geographic districts

Peer panelists suggested that the HPMS VMT data might be based on a low sample size.
Review QRFM Truck Trip Rates Truck rates are based on the Quick Response Freight Manual. They probably do not include service trips using commercial autos. In Progress
Review Traffic Count Data Review the traffic count process and assumptions used by TDOT to ensure that data is being interpreted correctly. Researching whether TDOT calculates the number of vehicles by dividing the number of axles by two. In Progress
Review External Station Trip EE\EI Splits Check the quality of the statewide model on which the data is based.

Check VMT on through-trip tables since non-resident through-trips might significantly increase the number of internal trips as people who enter the area might also make secondary (internal) trips.
In Progress
Review Special Generator Rates Planned
Review Vehicle Occupancies Planned

Model Interface

The consultant team demonstrated some of the functionalities in the user interface. The model interface is based on a standard TransCAD interface. The model flow is a standard four step model with a full mode choice component. It also incorporates a feedback loop that closes based on the method of successive averages. Iterations are initialized with v0, the estimated congested speeds based on posted speeds with factors from the travel time runs applied to them. Each complete loop through the model takes about four hours. The model can be run fully or partially. It was noted in correspondence preceding the meeting as well as during the meeting that dual processor computers could improve the model run time, potentially cutting it in half.

Results are exported to text files. Figure 3 illustrates a sample model results text report. The consultant team asked peer panelists to recommend which performance measure output fields should be included and which would be most useful for the client since the Memphis MPO does not currently have a modeler on staff to make these decisions. The model could also generate a report on the measures of effectiveness for long-range planning.

The consultant team will provide training and documentation on model use. The MPO requested that information on how the network is coded and how to perform analysis using the model be included in the training. All scripts and memos generated by KHA during the model development will be handed over to the MPO. All software and files will be handed over on a CD. The consultant team will also keep an electronic archived copy of the CD.

Peer panelists suggested that the MPO have an approval process in place to evaluate the model and address problems or requests prior to accepting the model from the consultant team. All changes should be clearly documented.

Comp Summary screenshot
Figure 3 : Sample Model Results Text Output

Panelists recommended the following additions to the interface:

MATA requested the ability to view modeled transit boardings along transit lines.

One panel member suggested that the best method for checking convergence is to check if the difference between skims is lower than 1 percent. This can be done through TransCAD code.

Since the Memphis MPO does not yet have a staff person to run the model, peer panelists were concerned that operational details of the model could get lost during the transition period. Two different paths that MPOs are taking for model management are retaining in-house expertise to perform modeling or contracting with a consultant to run the model.

Panelists speculated that since the model might change hands many times and be used for many different purposes, it is important that the user is aware that parameters should not be changed casually. This could be enforced by an "expert mode" and a "general user mode" where some parameters are locked. It is also important that results can be reproduced. The model runs should produce log files to store the parameters used for a run.

The Memphis model uses data management to perform multi-year network coding. The TransCAD network management tool is used to store fields and allow parameters for multiple years and alternatives to be seen all at once. All geography is maintained in one line layer. Link widenings and changes in connectivity are handled by including born and end dates in the link or node properties in the link editor. Multi-year transit layers are more challenging to include since there is only limited information on future year routes in the long-range plan.

The panel discussed several current projects that are developing a solution to multi-year network coding, which becomes more complicated as an MPO grows in size, the number of projects increases, the network expands, and versioning is introduced. TransCAD is working on a prototype to help manage all the different geographic elements, scenarios, projects, and project elements associated with a network. A member of the consultant team asked how they can set up the Memphis model to help the MPO with the transition to these potentially new versions of TransCAD. Panelist Howard Slavin, of Caliper (the developer of TransCAD), suggested coding all future links with date attributes.

The panelists commented that the coding of all local roads in the network, requested by stakeholders for modeling local roads as collectors, would be useful data in the future, especially when preparing the TIP. Currently the model only uses a subset of all streets.

3. Panel Recommendation and Action Plan

The TMIP peer review panel convened in closed session the morning of the second day to discuss the Memphis travel model and review the previous day's meetings. The panel identified the strengths of the model and then developed a set of consensus recommendations for the consultant team and the Memphis MPO. Recommendations were based on the panel's charge to advise the Memphis Model Steering Committee in successfully implementing a state-of-the-practice four-step model that can be used in developing the RTP, the TIP, and air quality conformity determinations.


The panel reiterated that the Memphis MPO has a strong model consultant team in place and that the strengths identified in the previous peer review were still valid. These included incorporating time-of-day modeling early in the model stream, using a logit-based destination choice formulation for trip distribution with impedances based on the mode choice logsum variable, and implementing multi-year roadway and transit TransCAD-based networks. The peer review panel felt that the consultant team has been very responsive and open about identifying where things stand. The panel emphasized that the focus should be on "doing things right" even if the completion date must be extended.

Recommendations/Action Plan:

  • Validate the highway and transit assignment model
  • Perform travel time checks
  • Conduct sensitivity tests
  • Facilitate MPO application
  • Forecast modeling improvements

Recommendations and Action Plan

The panel made the following recommendations.

1. Validate the Highway and Transit Assignment Model

The panel did not feel it was their role to prescribe how to validate or troubleshoot the assignment model but suggested using the following checks:

2. Perform travel time checks

The peer review panel noted that the consultant team was already checking travel time, but offered this recommendation to emphasize its importance. They particularly encouraged additional checks of modeled versus observed travel time for peak and off-peak, highway, and transit.

3. Conduct sensitivity tests

The panel suggested that once the issues with the low volume results are resolved, sensitivity tests should be conducted. One suggestion for a sensitivity test was to make an additional (hypothetical) roadway or rail line network change and verify that the model response is reasonable. Other suggested changes are land use, parking cost, transit headway, and fare changes. An example of a land use change is the addition of a shopping mall (e.g. Bass Pro Shop in the Pyramid). Elasticities should also change appropriately when changing inputs such as fare or headway.

Some sensitivity tests can be used as "gut-checks" but some tests should be formalized and input assumptions and elasticities should be documented. Tests should be performed on both mode choice and destination choice models.

MATA expressed interest in checking sensitivities to parking costs changes. A member of the consultant team said that this can be tested but there is the caveat that in the vast majority of zones, the current parking cost is $0 and such a sensitivity test could be difficult to conduct.

4. Facilitate MPO Application

The panel reminded the MPO and consultant team of the importance of documenting, making available, and archiving everything related to the project. A national trend is that MPOs are out-sourcing the travel demand modeling so it is important that the MPO has all required data and procedures to run the model, including:

A control mechanism might be necessary if the model is to be used by many parties. It could be developed with this in mind. For example, a novice user would not have permission to access and change a look-up table for capacity, whereas a more experienced user would. There could be a two-tiered model where one is "locked down" and certain parameters cannot be changed. This would not require two interfaces to be developed. In addition to locking portions of the program, the MPO needs to avoid the problem where the consultant team is passing back and forth different versions of the model among themselves. The MPO should keep a controlled or official version of the model at all times.

The consultant team verified they will submit everything (documentation, memos, software, spreadsheets, etc.) on a CD.

The panel also suggested that the MPO implement an acceptance testing program. At a minimum, the MPO must have some level of acceptance testing in place to ensure the model is usable and workable. This is standard procedure in any software development process because it gives the client an opportunity to ensure the product was delivered as requested.

The consultant team emphasized that prior to delivery the MPO must have someone in place to run the model. This person must already have some expertise with the four-step model and TransCAD and can either be an experienced modeler on the MPO staff or an on-site contractor. The project can also be out-sourced.

Since the model currently takes four hours to run at KHA, the panel recommended that the MPO make a modest investment in an updated computing environment for running the model. A dual-processor computer could cut processing time in half.

5. Forecast modeling improvements and next steps

The panel reminded the MPO and consultant team of the importance of documenting, making available, and archiving everything related to the project. A national trend is that MPOs are out-sourcing the travel demand modeling so it is important that the MPO has all required data and procedures to run the model, including:


Appendix A: List of Participants

Ken Cervenka*, North Central Texas Council of Governments (NCTCOG)
James Collins, KHA
Mark Dunzo, KHA
Tom Fox, Memphis Area Transit Authority (MATA)
Ed Granzow*, CH2M Hill
Craig Gresham, KHA
Zhiyong Guo, KHA
Tom Hammer, HNTB
Leta Huntsinger*, ITRE/NCSU
John Lancaster, MATA
Ging Ging Liu, U.S. DOT Volpe Center/Cambridge Systematics
Martha Lott, Memphis MPO
Jim McDougal, Desoto County
Kenneth Monroe, KHA
Paul Morris, Memphis MPO
Clark Odor, City of Memphis
Tom Rossi, Cambridge Systematics
Guy Rousseau*, Atlanta Regional Commission
Howard Slavin*, Caliper Corporation
Michelle Stuart, Memphis MPO
Sarah Sun, RTC

Appendix B: Agenda

Memphis MPO Travel Demand Model Development

Peer Review Meeting
March 27-28, 2006

Monday, March 27
8:00 am Welcome and Introductions (James Collins, KHA)
Define Success of Meeting Schedule (Kenny Monroe, KHA)
8:30 am Results from Model Component Validation
Destination Choice Validation (Tom Hammer, HNTB)
Model Re-estimation with Congested Skims
Validation by Trip Purpose, Auto Ownership Level
Mode Choice Validation (Tom)
Model Re-estimation with Congested Skims
Validation by Trip Purpose, Auto Ownership Level
9:30 am Break
9:45 am Results from Model Component Validation (cont.)
11:00 am Highway and Transit Assignment Validation Results
Trip Assignment (Craig Gresham, KHA)
Highway Assignment
Transit Assignment
12 noon Lunch (to be provided)
1:00 pm Highway and Transit Assignment Validation Results (cont.)
2:00 pm Demonstration and Discussion of Model Interface (including the roadway and transit network coding tools) (Craig and Zhiyong)
2:45 pm Break
3 pm Demonstration and Discussion of Model Interface (cont.) (Craig and Zhiyong)
3:30 pm Follow-up Discussions Directed by the Panel Members (James)
5:00 pm Adjourn
Thursday, October 28
8:00 am Peer Review Committee Discussion (Panel Members Only)
10:00 am Break
10:30 am Presentation of findings and recommendations - Committee
12 noon Lunch (to be provided)
1 pm Follow-up and conclusions
2:30 pm Adjourn
Updated: 3/25/2014
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000