April 27, 2009
Prepared by
Volpe National Transportation Systems Center, USDOT
For the Federal Highway Administration
This report summarizes the results of an evaluation of the Travel Model Improvement Program (TMIP) Peer Review Program. The purpose of the evaluation was to understand the value to transportation planning host agencies of a peer review of their travel demand model. The study involved interviewing modeling staff from eight metropolitan planning agencies that have hosted a TMIP-sponsored peer review of their travel demand model.
Major findings are detailed below, grouped into four categories: motivations for conducting a peer review, planning the peer review, implementing technical recommendations, and overall satisfaction with the process. Following the findings, recommendations for enhancing peer reviews and the peer review process are presented.
One of the major advantages of peer reviews is that each host agency can frame the meeting to address its unique modeling needs. Nonetheless, there were some common reasons for holding peer reviews:
Interviewees agreed that planning their peer review was very time consuming. However, they also agreed that good planning is a critical factor in a successful peer review. Planning tasks that interviewees found particularly important were:
The most common technical issues addressed in peer reviews were:
Agencies implemented approximately 70 percent of peer panelists' recommendations. The major reasons for not implementing recommendations were:
Interviewees were very satisfied with their peer review and the peer review process. They felt that panelists were highly skilled and that the recommendations were appropriate and helpful. The post-meeting report was helpful in documenting these recommendations. Interviewees said that the peer review helped build their staff's modeling skills and help improve their travel demand model.
The following recommendations are drawn from the interviews as well as from observations of the Volpe Center team.
This report summarizes the results of an evaluation of the Travel Model Improvement Program (TMIP) Peer Review Program. The purpose of the evaluation was to understand the value to transportation planning host agencies of a peer review of their travel demand model. This report presents strengths and weaknesses of peer review meetings and the peer review program as described by modeling staff from agencies that have hosted a peer review. It also provides recommendations for improvements to the TMIP Peer Review Program and to aspects of the overall TMIP Program that could help grow the educational impact of peer reviews.
TMIP is a partnership between the Federal Highway Administration (FHWA) and the Federal Transit Administration (FTA). The program's goals are to:
Improved modeling techniques allow planners to generate better, more robust transportation forecasts that decision-makers can use when making important decisions about transportation services under various scenarios changes in population, economic growth, and land use.
TMIP's peer review component gives transportation planning agencies the opportunity to have their model reviewed by modeling experts from around the country. These experts make recommendations on how to proceed with model enhancements. Peer reviews are particularly valuable because each host agency customizes its peer review to meet its unique modeling concerns.
This study examined peer reviews conducted by eight metropolitan planning organizations (MPO) across the country. Section II of this report describes the methodology used in the study. Section III presents findings drawn from major themes in the interviews. Section IV presents recommendations for peer review program enhancements, followed by the conclusion in Section V.
This evaluation was conducted by the U.S Department of Transportation (DOT), Research and Innovative Technology Administration (RITA), Volpe National Transportation Systems Center (Volpe Center) at the request of the FHWA Office of Planning, Environment, and Realty. It consisted of eight semi-structured telephone interviews with travel demand modeling staff from eight transportation agencies that have hosted peer reviews. Five of these eight agencies conducted their peer review after 2005 when the most recent peer review evaluation was conducted. These five agencies, therefore, have not been evaluated previously.
FHWA chose three additional agencies to provide a more balanced perspective of the program. Each of these additional agencies conducted its peer review at least 5 years ago, giving them substantially more time to address peer review recommendations than the five agencies that conducted peer reviews since 2005. This evaluation focused more on agencies that are in the process of developing their model rather than refining an existing model. Four agencies studied have conducted more than one TMIP-sponsored peer review. This evaluation focused on the most recent peer review.
Modeling staff from the following agencies were interviewed:
Interviews focused on:
The interview questions are included in Appendix A.
Table 1 provides basic information about these agencies and their peer reviews.
| Agency | Metro Area | No. of Peer Reviews | Dates of Peer Reviews | Primary Goal for Peer Review | Modeling Platform |
|---|---|---|---|---|---|
| Baltimore Metropolitan Council | Baltimore, MD | 2 | Sept. 23-24, 2004 Feb. 28, 2005 |
Evaluate and feedback on the effectiveness of the current model; identify short- and long-term model enhancements. | TP+ 3.0; TRNBUILD 3.0.6 (transit) |
| Community Planning Association of Southwest Idaho | Boise, ID | 1 | June 5- 6, 2007 | Feedback on the technical aspects of the travel demand model including recommendations for components needing improvement. | Cube Voyager and TP+ |
| Denver Regional Council of Governments | Denver, CO | 2 | Oct. 31, 2003 April 20, 2004 |
Identifying key planning issues and developing a vision for a new modeling system | Migrated to TransCAD and Urbansim from MinUTP and DRAM/EMPAL |
| East-West Gateway Council of Governments | St. Louis, MO | 1 | Dec. 7-8, 2006 | Provide guidance on model fixes and enhancements for its compliance review. | Cube, TRNBUILD |
| North Jersey Transportation Planning Authority | Northern NJ, metro NYC | 2 | Oct. 27-28, 2005 | Assess the plans for the development of the newly enhanced travel demand model and recommend both near- and long-term model enhancements. | 4-step CUBE/Voyager TP+ with FORTRAN routine for mode choice |
| San Diego Association of Governments | San Diego, CA | 1 | June 23-24, 2005 | Provide guidance on near- and medium-term model enhancements to its existing four-step model. | 4-step, TransCAD |
| Southeast Michigan Council of Governments | Detroit, MI | 1 | Dec. 6-7, 2004 | Assess the current travel demand model and recommend both near-term and long-term model enhancements. | 4-step, TransCAD |
| Southern California Association of Governments | Los Angeles, CA | 3 | Nov. 3, 2003 April 16, 2004 January 9-10,2006 |
Examine current model and focus on assessing the model updates, including trip distribution, mode choice, trip assignment; assess plans for model improvement and recommend near- and long-term enhancements, review validation targets for all model components. | Migrating to TransCAD |
This section lays out the common themes expressed by interviewees. They are divided into four categories: motivations for holding a peer review, planning the peer review, implementation of technical recommendations, and overall satisfaction.
Agencies were looking to the peer review panelists for expert advice on the best approaches to improve their model's capability in high-priority areas. Common reasons for requesting a peer review included:
Assess the model with respect to the state of the practice.
Several interviewees said that one of the reasons for their peer review was to get an "independent eye" to review their model and comment on it with respect to the state of the practice. COMPASS, for example, requested that the panel provide general commentary and advice on its model. NJTPA, which was in the process of making major enhancements to its existing model, wanted "to make sure it was on the right track; be sure we were using the best modeling techniques available." At SEMCOG, management wanted the peer review "to be sure that the modeling process was robust and have people in the region feel comfortable with the process."
Provide expert opinion to help agencies respond to controversies about the model, model components, or modeling process.
Output from a travel demand model can have a substantial effect on transportation policy and, subsequently, air quality, land use, and economic development planning. By running different scenarios, the projected transportation conditions and needs are used to inform short- and long-term transportation planning.
Occasionally, stakeholders-usually environmental or neighborhood-based groups-object to local plans based, at least in part, on model output. Some stakeholders question the model itself, contending that the data, assumptions, or algorithm are not consistent with the state of the practice. Many agencies requested a peer review in part to address the concerns of these stakeholders. When determining the charge to the peer review panel, these agencies specifically requested comments on controversial model elements either to demonstrate that the model is reasonable or to identify model deficiencies and suggest improvements.
For example, an environmental group in Baltimore questioned the output of the travel demand and air quality models in forecasting the impact of the transportation system on air quality. For BMC, the peer review process helped to provide legitimacy to the model outputs based on an assessment by objective, outside modeling experts. In San Diego, SANDAG requested a peer review partly to respond to criticism about its model by soliciting panelists' comments addressing mode share for future transit systems.
Receive technical assistance on particular model components.
Most agencies made specific requests that the panel review one or several technical model components. For example, SANDAG requested guidance on the best forecasting methodologies for evaluating new transit services and smart growth and land use policies. EWGCOG asked for comments on its volume-delay function and on detailed aspects of the mode choice model. Several agencies requested advice on model calibration and validation.
Identify and build support for funding modeling activities.
Some agencies wanted to use the peer review comments and recommendations to demonstrate the agency's modeling needs as assessed by objective outside experts. They hoped that this would encourage decision-makers to provide sufficient resources to maintain and enhance modeling activities. For example, DRCOG, which conducted a deliberate and careful visioning process for its new approach to modeling, said that "the weight of the [peer panel] experts' names" helped build political support for its new modeling effort.
Receive comments and advice on the model's compliance with Federal requirements.
SANDAG and BMC each specifically requested comments and advice on their model's sufficiency with respect to FTA's New Starts requirements. While these were the only agencies that specifically requested comments on New Starts funding criteria, most of the peer reviews included some discussion of New Starts modeling requirements. Even agencies that were not planning to apply for New Starts funding wanted a model that would meet FTA criteria in case the agency decides to apply for New Starts money in the future. Although complying with New Starts requirements is important to many transportation agencies, in order to avoid debating modeling and modeling requirements with FTA, TMIP does not approve peer review requests in which the agency's only goal is to receive feedback on its model with respect to FTA New Start requirements.
In addition to New Starts requirements, travel demand models must meet Federal requirements for certification of the region's planning process. Because transportation modeling is highly technical and every model is unique, there are no specific Federal modeling requirements. When evaluating the planning process, Federal agencies generally consider a model peer review as evidence that the model and modeling process is consistent with the state of the practice or that the agency is working to improve its model. One interviewee noted that the peer review "provided good input to the certification process."
Help determine priorities for model enhancements.
Every agency studied asked the panel for recommendations on short- and long-term modeling priorities. On this request, agencies generally fell into two categories: those that had a good idea where they were going but wanted reassurance that they had chosen the most efficient way to get there; and those that were at a standstill and did not know which direction to take for model enhancements.
Of the former, some of the agencies were looking to the panelists' recommendations in part to validate their plans for model improvements, hoping to reassure politicians and senior managers of the legitimacy and importance of the planned enhancements. This, they hoped, would convince decision-makers to provide sufficient funding for implementing model enhancements. Agencies in the latter category-COMPASS, for example-wanted a general sense of the state of their model relative to other travel demand models and advice on how to allocate their resources to meet their agencies' modeling objectives.
Receive comments and advice for modeling prospective policy alternatives.
Many metropolitan areas are considering major changes to their transportation network and its operations. Some changes are pro-active policy initiatives-such as congestion pricing or high-occupancy vehicle lanes-to control congestion and generate revenue for today and the future. Other proposed policy alternatives are intended to address problems largely outside the control of planning agencies such as the merging of neighboring metropolitan areas into a single urban conglomerate.
Modeling many of the major new policy initiatives can be extremely complicated and risky since these techniques are still in their infancy. For agencies that do not intend to develop a new model to accommodate these policy initiatives-most of those included in this study-peer panelists have suggested less complicated techniques such as post processing or add-ons.
Grow the skills of the agency modeling staff.
Many of the Nation's foremost experts in travel demand modeling have served as peer reviewers. This has given agency modeling staff an important opportunity to increase their modeling expertise-formally through the peer review meeting, discussions, and peer panelists recommendations and informally by building relationships with nationwide experts and expanding their professional network.
Planning a peer review requires significant effort on the part of the host agency. Interviewees made several observations about the process of planning the peer review:
Involving senior management and policy-makers in planning the peer review strengthens the process and improves outcomes.
Some interviewees said that it was important to communicate to senior managers and policy-makers the strengths and weaknesses of the agency's current model, to describe the implications of these characteristics, and to demonstrate how a peer review can help improve its model and modeling process. Senior managers and policy-makers should not be expected to weigh in on specific model elements or on the panel's technical recommendations. However, by helping senior managers and policy-makers understand the value and limitations of the peer review, they can make better-informed decisions about allocating resources to improve their model.
Planning a peer review requires substantial time and effort but is critical to a successful peer review.
Agencies are responsible for developing the charge to the panel, setting the meeting schedule and agenda, selecting panel members, providing background material, developing presentations to be given at the peer review, and planning meeting logistics. One interviewee noted that the host agency "needs to be clear about the agenda and do advance preparation... to help clarify key points of confusion and determine the magic number of participants and narrow the panel discussion." Interviewees agree that this required a lot of effort, but that doing this well is necessary to get the most out of their peer review. The following are detailed findings related to these planning activities:
Choosing panelists with expertise in topics or circumstances important to the host agency. Interviewees said that, when choosing panelists, they wanted at least one person with experience modeling for agencies in areas similar in size and geography of the host agency. Panelists representing a broad range of expertise were also essential to a successful meeting. The typical panel included representatives from consultants, academia, MPOs, and other government agencies. This range of experience was important to provide a broad perspective on the intricacies of travel demand modeling.
However, recruiting panelists is more than locating modelers with a desired expertise. Once panelists have been identified, each has to be available to participate; and the host agency must choose a date for the peer review meeting that is convenient for all panelists. Though the effort was complicated, interviewees agreed that getting the right panelists is very important.
Deciding the length and number of peer review meetings. Some agencies chose to split the process into two or three meetings. After the first meeting, the agencies had time to act on the recommendations. The second and subsequent meetings consisted of discussion of these recommendations and the agency's progress in addressing them. It also provided agencies the opportunity to ask the panel to address new modeling issues that have come up since the first meeting.
BMC held two peer review sessions. The first meeting in September 2004 focused on a review of the status of its model improvement process. Panelists provided guidance on near- and long-term model development issues. The second meeting, held in February of 2005, involved BMC presentations of progress made from the first meeting and included discussion on developing its work plan to address the panel's recommendations.
DRCOG hosted two meetings while it was in the process of developing a completely new model. It used the first meeting to discuss the agency's visioning process with the panelists. DRCOG wanted to use the panel's expertise to educate itself on advances in forecasting models beyond the recommendations it had received from its consultant. Its second panel, held six months later, focused on the preliminary suggestions for the new model blueprint.
Providing panelists with technical details of the model well before the peer review meeting. The host agency is expected to provide detailed documentation of its model and modeling process several weeks before the peer review so that peer panelists and other attendees arrive at the meeting with a good understanding of the model's technical details and the specific problems the peer review will address. Interviewees said that writing this background material was very time consuming. However, they felt that this documentation was very important because it eliminated the need to use meeting time to explain model intricacies.
Table 2 lists the most common topics for recommendation and their status of implementation. It shows that host agencies implemented approximately 70 percent of the panels' recommendations. In addition, agencies either have implemented or are considering implementing 23 percent of the recommendations. Seven percent were not addressed.
Agencies were most likely to implement recommendations related to basic model elements such as modeling boundaries and network revisions. Other frequently implemented recommendations were related to mode choice, validation, and calibration. Altogether, agencies implemented over 80 percent of the recommendations that address basic model elements.
| Agency/ Recommendation Topics | Model Platform, Land Use Integration | Activity-Based Modeling | General Model Details/Data Needs | Freight Component | Mode Choice/ New Starts | Validation/ Calibration/ Feedback | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BMC | √ | + | √ | - | √ | + | √ | o | ||||
| COMPASS | √ | + | √ | o | √ | - | √ | o | ||||
| DRCOG | √ | o | √ | + | √ | + | √ | - | √ | + | √ | + |
| EWGCOG | √ | o | √ | + | √ | o | √ | + | √ | + | ||
| NJTPA | √ | + | √ | o | √ | o | √ | + | √ | + | ||
| SANDAG | √ | o | √ | o | √ | + | √ | + | √ | + | ||
| SCAG | √ | + | √ | + | √ | + | √ | + | ||||
| SEMCOG | √ | o | √ | o | √ | + | √ | o | √ | + | √ | + |
Key:
Several recommendations were partially implemented or delayed because of:
None of the interviewees said they could not implement a particular recommendation because agency decision-makers disagreed with it. This is to be expected since most agencies vetted issues with management before deciding the charge to the peer review panel.
The list below presents details on the most common technical topics addressed in peer reviews:
Model Platform/Land Use Integration: The larger MPOs discussed with panelists the possibility of integrating land use with the travel demand model to better capture development patterns. In some cases, the panels encouraged agencies to work toward this goal. However, interviewees agreed that the state of the practice is evolving; and agencies must be aware of the resources and risks involved.
Activity-Based versus four-Step Modeling: A few panels recommended that agencies begin to develop a tour- or activity-based model but warned of the costs, expertise, and risks involved. One agency included in this study has begun tour- or activity-based modeling. Two other agencies have plans to implement tour-based modeling with future model updates. A fourth is staying with its four-step model because it felt that it was too far along in the process of developing its four-step model to make such a dramatic change. This agency will consider tour- or activity-based modeling for its next set of major model improvements.
General Model Details and Data Needs: Each of the peer reviews studied for this evaluation included discussion of general improvements to basic model components such as networks, traffic analysis zones, and model boundaries. Many discussed the need for additional, current, and accurate data on elements such as travel speeds, traveler behavior, regional employment forecasts, external trips, transit use, and commercial vehicle travel. In most cases, these recommendations were implemented or are included in the agency's Unified Planning Work Program for future implementation. A few interviewees said financial constraints prevented them from moving forward on data collection, but they planned to collect data as their budget allows. Panelists strongly encouraged organizations to share data if possible.
Freight Model: Panels consistently recommended that agencies work to understand the effect of commercial vehicles on the transportation network. Some interviewees said their agency has not yet developed a freight component because it does not have the necessary funding or management support. Panels recommended working with the state department of transportation, although in some cases, states do not have the necessary data or staff to support a freight model.
SCAG, for example, indicated that the panel "did a very good job of identifying future areas of improvement for our heavy-duty truck model...and the improvements listed in the peer review report are a basis for model improvements we are seeking to implement through this effort." However, this interviewee also noted that "obtaining quality freight and commodity flow data at a detailed geographic level and quality survey data remains a major challenge."
Mode Choice: Generally, interviewees said that the panelists made recommendations to improve their mode choice model, especially by collecting data to validate it. Agencies from large metropolitan areas were consistently concerned with meeting the modeling requirements for the FTA New Starts Program. A few interviewees noted that they wanted advice on how to best integrate bus-rapid-transit into their model. In a few locations, panelists recommended including non-motorized transportation. Every agency felt these were worthy recommendations although financial constraints have hampered their implementation in some areas.
Validation, Calibration, and Feedback: Every agency asked its peer review panel to address model calibration and validation. Panelists recommended calibrating and validating the model to customized targets rather than assignment standards with feedback. Most agencies implemented these recommendations. One agency was planning to incorporate the work into the next fiscal year's work program. Another agency felt that the panel did not understand its roadway system so it did not plan to make all the recommended changes.
Interviewees agreed that the technical recommendations were helpful and resulted in significant improvement to their models. However, one agency acknowledged that the comments and advice provided by the panel could be a "double-edged sword." With so many possible model enhancements, panels occasionally presented an overwhelming list of options that the agency could not even consider given staffing constraints and the maturity of its current model.
The evaluation interviews asked about overall satisfaction with the peer review process. Interviewees were generally very satisfied with the process and made the following key points:
Most of the technical recommendations were helpful and resulted in significant model improvements.
When asked to rate the suitability of the TMIP process in achieving the agency's major outcomes on a scale of 1 to 5 (with 5 being the best rating), interviewees gave the process an average rating of 4.25. In the discussions, they noted specific examples of how implementing the panel's recommendations improved its model:
One interviewee cited an example of a recommendation that was not helpful. The panel's comments on the rail constant in mode choice were very different from what FTA wanted for New Starts funding. As a caveat, however, the interviewee noted that the model as presented in the peer review was too far off the targets for the panel to give effective, informed recommendations.
The peer review format was very effective.
When asked to rate the effectiveness of the meeting format on a scale of 1 to 5 (with 5 being the best rating), interviewees gave the process an average rating of 4.44. Because each agency created its own agenda, staff were able to develop the format that worked best for them. All interviews agreed that this arrangement worked well.
Panelists were very professional and provided valuable technical assessments.
When asked to rate the suitability of the panelists on a scale of 1 to 5 (with 5 being the best rating), interviewees gave the panelists an average rating of 4.69. Interviewees said that the panelists' honest and objective judgment was an important part of the peer review. The panelists' breadth of experience in topics including land use, transportation policy, and technical modeling components gave agencies a broad perspective on the potential for their models.
One interviewee noted that the high-caliber reputation of the panelists was helpful in encouraging its consultants to participate in the peer review. Before the agency began preparing for the peer review, the consultant discontinued work on the agency's model until receiving additional funding. The panel's potential influence on the consultant's professional reputation gave the agency the leverage it needed for the consultant to complete the work necessary to present a respectable model to the peer experts.
One interviewee noted that one panelist was somewhat distracting to the process because of his narrow focus on a particular issue that was not part of the questions asked of the panel. This example was an exception to the overall positive feedback on the panelists' professionalism and technical expertise.
The peer review report was a valuable resource for implementing recommendations.
Interviewees said that their agencies refer back to the report frequently for guidance on how to move forward with changes to the model. Every interviewee said that a timely and accurate report on the meeting was extremely important, particularly because it contained details of the panel's recommendations. One interviewee said he was "glad to have an independent party write the report to make sure that the outside ideas were represented. It could have been tempting for our agency to represent what it wanted to hear rather than what actually happened." Interviewees agreed that receiving the report within about 2 months after the meeting was important so that agencies could quickly move forward in addressing recommendations.
Participation in the review process helped agency staff and peer panelists to expand their professional network.
Peer reviews brought together a variety of planning and modeling professionals. Interviewees said that this helped to build relationships among the various peer review attendees. The opportunity for networking was especially beneficial for agencies located outside of major metropolitan areas where they are not likely to interact with other modeling professionals locally. Some interviewees indicated that occasionally host agency staff continued to follow up with members of the panel after the meeting for technical advice or guidance on recommendations. They felt that the peer review was a successful technique for creating sustained relationships among planners and modelers.
The following recommendations are drawn from the interviews as well as from observations of the Volpe Center team.
Develop outreach materials for senior managers to inform them of the purpose and potential benefits of a peer review.
Interviewees felt that it was extremely important that senior managers and decision-makers understand and approve of the peer review process prior to the meeting. Managers should appreciate the potential benefits of a peer review but also have realistic expectations for the outcomes of the peer review. TMIP staff should consider developing outreach materials stressing the following points:
Some interviewees suggested that it would be useful for TMIP to develop a standard PowerPoint presentation or memorandum that agency staff could customize and use as a tool to help communicate to senior managers and policy-makers the benefits of a peer review and to set realistic expectations on what it can and cannot do.
Provide more guidance and technical assistance to agencies for hosting a peer review.
Planning and hosting a peer review is a time-consuming, lengthy, and sometime complicated process that generates extremely useful results. By creating outreach material with tips on planning a peer review, TMIP could help smooth out the planning process for host agencies. Topics could include:
This outreach material could be developed using ideas and advice from agencies that have conducted peer reviews. The material could include "lessons learned" and "best practices." The outreach material could include hard copies for distribution at conferences and elsewhere and be available on the TMIP website.
Conduct pre- and post-panel conference calls with panelists.
TMIP staff should consider scheduling a pre-panel conference call or webinar with the panelists and agency modeling staff. This call would allow the host agency to reinforce its modeling concerns and topics it wants the panel to address. It would also provide panelists the opportunity to ask host agency staff about technical modeling details. Logistical details such as invitational travel and the location of agency's office could also be addressed in the call.
A post-panel conference call or webinar could also be useful to conduct after participants have had some time to absorb the material presented at the meeting. Questions that arise after the meeting could be discussed. The note taker should participate in this call to provide details that the agency and the panelists might have missed.
Identify and inform panel of agency constraints prior to the meeting.
TMIP staff should work to ensure that the peer review does not produce a list of recommendations that the agency is unlikely to implement because of resource constraints, priorities, or policy issues. The agency should identify up front its limitations so that all peer review participants have a realistic notion of the agency's capabilities and so that recommendations are realistic and actionable under the agency's circumstances.
Provide more technical details in the post-panel report.
The post-panel report should have detailed technical information on the reasons for making specific recommendations. Agencies frequently refer back to the report when implementing changes. Additional technical details would be helpful to understand the reasons for specific recommendations. TMIP staff could assist this effort by providing clear guidelines on the expectations for the content in the report, particularly as it relates to the technical discussion. Technical discussion might be most appropriate in appendices or using hyperlinks so that details are available while maintaining the accessibility of the report for multiple audiences.
Assist agencies in developing strategies for implementing recommendations.
TMIP staff should consider providing technical assistance to agencies to help them prioritize peer review recommendations and develop an action plan for implementation.
Create a database that includes agencies and the issues discussed in their peer review for other agencies to use in planning peer review meetings.
Agency staff could use this database to identify organizations that have discussed similar topics at their peer reviews. Although the peer review summary reports are accessible on line, a database could be useful in narrowing the search to specific agencies for specific topics.
Provide agencies suggestions and advice for how to prepare a Request for Proposals (RFP) or interview consultants.
TMIP could provide advice to agencies that are issuing RFPs for travel demand models. This would help agencies develop RFPs that are technically accurate and have sufficient detail. Assistance could include:
Continue to conduct periodic evaluations to monitor the effectiveness of the peer review program.
Monitoring the program enables TMIP staff to identify elements that are working and where improvements are needed. It is also an effective way to see whether agencies are following through on implementing recommendations or not and the reasons behind their actions. In addition, TMIP should require host agencies to produce periodic status reports on their activities related to the peer review.
Continue to develop and disseminate materials on "state of the art" and "state of the practice" in travel demand modeling.
Because travel demand modeling is very technical, it can be difficult for agency staff to keep current with its latest developments. TMIP should continue to provide educational materials to help modelers understand the current trends in travel modeling and how agencies are implementing new tools.
Continue to work with modeling agencies to build peer networks.
As the peer review program has demonstrated, learning from peers has been valuable to agencies, modelers, and peer panelists. TMIP should continue to develop and promote networking opportunities to take advantage of formal and informal opportunities to learn from peers.
The results of this evaluation indicate that the TMIP Peer Review Program offers an effective process for agencies to improve their travel demand models. Agencies found the program to be very valuable for improving travel modeling in their regions. The process was instrumental in providing a valuable forum for them to identify and address important regional model components. When asked what type of agency would benefit from the TMIP Peer Review Process, interviewees indicated that it would be good for any agency involved in travel modeling, particularly MPOs, transit agencies, and state departments of transportation.
The following is a summary of the peer reviews held by the agencies included in this evaluation.
Baltimore, Maryland
The Baltimore Metropolitan Council (BMC) held two peer review meetings. The first focused on the status of the BMC travel model improvement process and provided guidance on near-term and long-term model development issues including demographic forecasting, incorporating new model functions such as managed lanes and truck traffic, and proposed new mode choice model. At the second peer review meeting, presentations and discussion focused on BMC's work plan to implement changes to its model based on the panel's recommendations from the first meeting. Both meetings were held at the office of the BMC in Baltimore, Maryland, with the same panel members.
The TMIP peer review was requested by the Baltimore Regional Transportation Board (BRTB), the MPO for the Baltimore metropolitan region. BMC staff provides support to the work activities of the BRTB. The Baltimore metropolitan region has a population of about 2.5 million people and covers an area of approximately 2,260 square miles.
The Baltimore model utilizes the traditional four-step process of travel demand forecasting. TP+ software (version 3.0) is used for running the model, except for the transit pathbuilding mode TRNBUILD. The purpose of the peer review was to provide the decision-makers in the region the opportunity to have the model reviewed by a panel of experts to determine how the model could be improved to meet future policy needs. Specifically, BMC asked the peer review panel to:
Major recommendations from the second peer review meeting included:
In an August 2005 memorandum, BMC addressed its planned next steps to incorporate these action items in a future work plan. Our interview with BMC focused on the extent to which it had made progress in moving these recommendations forward.
Meridian, Idaho
June 5-6, 2007
http://www.fhwa.dot.gov/planning/tmip/resources/peer_review_program/compass/
The Community Planning Association of Southwest Idaho (COMPASS) is the MPO for northern Ada and Canyon counties of Idaho, including the city of Boise. The area has a population of approximately 481 thousand people. At the time of the meeting, COMPASS had recently transitioned from a three-step to a four-step model, including a mode choice component that was borrowed and adapted from Wasatch Front Regional Council (Salt Lake City MPO). The purpose of the peer review was to provide COMPASS with feedback on the technical aspects of their travel demand model, including recommendations for components that needed improvement.
Recommendations included:
Denver, Colorado
DRCOG is the MPO for the Denver area. The agency's planning area has a population of 2.5 million, with an additional growth of one million people expected over the next 20 years.
DRCOG held two peer review meetings in Denver, Colorado with the same panel members. The first meeting focused on identifying key planning issues and developing a vision for a new modeling system for the Denver region to include comprehensive redevelopment of all transportation and land use modeling elements. At the time of this meeting, DRCOG was in the early phases of replacing its existing land use and travel demand model with a fully integrated modeling system. The purpose of the second peer review was to discuss the elements to be implemented for DRCOG's model update.
The panel was charged with helping the DRCOG project team identify approaches to the development of an integrated model. The model improvement project, named the Integrated Regional Model Project, is essentially complete. As part of this project, DRCOG completed a Travel Behavior Inventory Project, a $1.5 million travel/activity/ demographic survey of the region.
Recommendations from the first meeting included:
DRCOG reconvened the peer review panel in April 2004 to address general model design, potential staging issues, and other practical considerations for model development based on recommendations from the previous meeting. The peer review panel recommendations included:
St. Louis, Missouri
December 7-8, 2006
http://www.fhwa.dot.gov/planning/tmip/resources/peer_review_program/ewgcg/
The East-West Gateway Council of Governments (EWGCG) is the MPO for the St. Louis, Missouri, metropolitan area. It has a population exceeding 2.4 million people and includes three counties in Illinois and four counties in Missouri.
The primary purpose of the peer review was to provide guidance to the EWGCG on model fixes and enhancements for its compliance review. Specifically, the panel was asked to:
The panel's main recommendations included:
Newark, New Jersey
October 27-28, 2005
http://www.fhwa.dot.gov/planning/tmip/resources/peer_review_program/njtpa/
The North Jersey Transportation Planning Authority (NJTPA) region encompasses an area of 4,200 square miles with a population of 6.5 million people. NJTPA is the MPO for the 13-county Northern and Central New Jersey region, including the city of Newark, comprised of 384 municipalities. It is the Nation's fifth largest planning region.
NJTPA requested that the panelists assess the current travel demand model and recommend both near- and long-term model enhancements including:
At the time of the peer review, NJTPA had a standard four-step transportation model using TRANPLAN software. However, one of the main purposes of the peer review was to receive comments on implementing a four-step CUBE/Voyager TP+ model with a special FORTRAN routine for mode choice, a process the agency had already begun. The panel recommended that the NJTPA keep its near-term focus on using the validated Phase I CUBE model for its conformity analysis and concentrate on improving basic model elements such as networks and travel times. Other recommendations made by the panel included:
San Diego, California
June 23-24, 2005
http://www.fhwa.dot.gov/planning/tmip/resources/peer_review_program/sandag/
The San Diego Association of Governments (SANDAG), the MPO for the San Diego, California, metropolitan area, encompasses an area of 4,200 square miles with a population of 3 million people.
SANDAG requested that peer review panelists provide guidance on near- and medium-term model enhancements to its existing four-step TransCAD model including:
The panel felt that SANDAG's current model is consistent with the state of the practice, and the biggest question for SANDAG as it moves forward is how much to invest in revisions to the existing model versus the development of a new activity/tour-based model. The panel's primary recommendations included:
Detroit, Michigan
December 6-7, 2004
http://www.fhwa.dot.gov/planning/tmip/resources/peer_review_program/semcog/
The Southeast Michigan Council of Governments (SEMCOG) is the MPO for Detroit, Michigan. The region has a population of approximately 4.9 million people.
SEMCOG requested that the panelists assess the current travel demand model and recommend both near-term and long-term model enhancements. SEMCOG has a relatively new TransCAD-based four-step travel demand model. The model includes consideration of external, commercial vehicle, and transit trips. It is based on a variety of data sources, including census information, a SEMCOG household travel survey, traffic counts, and a transit-on-board survey.
The first day of the meeting focused on SEMCOG presentations on the current travel demand model and its plans for model improvements. Overall, the panel felt that the current model represents the state of the practice, addressing time-of-day, commercial vehicle, and external trips well. For future model enhancements, the panel's recommendations included:
For the long term, the panel recommended considering implementing an activity-based model.
Los Angeles, California
The Southern California Association of Governments (SCAG) hosted a total of three peer review meetings. The first two were held in November 2003 and in April 2004. Both meetings took place in Los Angeles, California, with the same panel members. The evaluation focused on the third meeting, held in January 2006, which had a different set of panelists.
The purpose of the third peer review meeting was for the panelists to examine SCAG's current model and assess model updates, including changes to the trip distribution, mode choice, and trip assignment phases. SCAG also requested that the panelists comment on the agency's plans for model improvement and recommend near- and long-term model enhancements as well as to review validation targets for all model components.
SCAG is the MPO for six Southern California counties in the Los Angeles metropolitan area. The region encompasses an area over 38,000 square miles and includes a population exceeding 18 million people. The current SCAG model follows the traditional four-step modeling structure with three ancillary models feeding into network assignment, including an external trip model, a regional airport demand allocation model, and a heavy duty truck (HDT) model. The panel felt that SCAG 's current four-step model is consistent with the state of the practice, particularly the plan for the freight model, the use of the "strategic work trip" purpose in trip distribution, and the use of four time periods in traffic assignment. It also indicated that SCAG has done well in data collection efforts and recommended that SCAG make its impressive survey datasets available to other agencies.
The TMIP peer panel made a number of recommendations for SCAG to consider, including:
H - Host agency contact person
P - Panelist
| Last Name | First Name | Organization | BMC | COMPASS | DRCOG | EWGCOG | NJTPA | SANDAG | SEMCOG | SCAG |
|---|---|---|---|---|---|---|---|---|---|---|
| Baber | Charles | Baltimore Metropolitan Council (BMC) | H | |||||||
| Bhat | Chandra | University of Texas at Austin | P | P | P | P | ||||
| Blain | Larry | Puget Sound Regional Council | P | |||||||
| Boyce | David | Northwestern University | P | |||||||
| Bradley | Mark | Mark Bradley Consulting | P | |||||||
| Cervenka | Ken | North Central Texas COG | P | P | P | |||||
| Chiao | Kuo-Ann | New York Metropolitan Transportation Council | P | |||||||
| Crandall | Mick | Utah Transit Authority (UTA) | P | |||||||
| Davidson | William A. | PBConsult, Inc. | P | |||||||
| Diogo | Robert | North Jersey Transportation Planning Authority (NJTPA) | H | |||||||
| Forinash | Chris | U.S. Environmental Protection Agency | P | P | ||||||
| Garry | Gordon | Sacramento Area COG | P | |||||||
| John | Jennifer | Tri-County Metropolitan Transportation District of Oregon (TriMet) | P | |||||||
| Killough | Keith | KLK Consulting | P | P | P | |||||
| Lawton | Keith | Portland METRO | P | |||||||
| Lee | Deng Bang | Southern California Association of Governments (SCAG) | H | |||||||
| May | Jeffrey | Denver Regional COG | P | |||||||
| McFarlane | Bill | San Diego Association of Governments (SANDAG) | H | P | ||||||
| Miller | Eric | University of Toronto | P | P | ||||||
| Morris | Michael | NCTCOG | P | |||||||
| Outwater | Maren | Cambridge Systematics | P | |||||||
| Pihl | Eric | FTA, U.S. Department of Transportation | P | P | ||||||
| Purvis | Chuck | Metropolitan Transportation Commission | P | |||||||
| Quackenbush | Karl | Central Transportation Planning Staff (Boston MPO) | P | |||||||
| Replogle | Michael | Environmental Defense | P | |||||||
| Rossi | Thomas | Cambridge Systematics | P | |||||||
| Rousseau | Guy | Atlanta Regional Council | P | P | ||||||
| Ryan | Jim | FTA, U.S. Department of Transportation | P | |||||||
| Sabina | Erik | DRCOG | H | P | ||||||
| Schlappi | Mark | Maricopa Association of Governments (MAG) - Retired | P | P | ||||||
| Shoaib | Lubna | East West Gateway COG | H | |||||||
| Spear | Bruce | FHWA, U.S. Department of Transportation | P | P | ||||||
| Spielberg | Frank | BMI-SG / Vanasse Hangen Brustlin, Inc. | P | P | P | P | P | P | ||
| Taylor | Stephanie | Southeast Michigan Council of Governments (SEMCOG) | H | |||||||
| Waldinger | MaryAnn | Community Planning Association (COMPASS) | H |
