PDF files can be viewed with the Acrobat® Reader®
|Location||Date||Peer Review Panelists|
|Austin, Texas||June 11-12, 2009||Erik Sabina, Denver Regional Council of Governments|
|William Woodford, AECOM|
|Arash Mirzaei, North Central Council of Governments|
|John Lobb, RSG Inc.|
|Guy Rousseau, Atlanta Regional Commission|
The views expressed in this document do not represent the opinions of FHWA and do not constitute an endorsement, recommendation or specification by FHWA. The document is based solely on the discussions that took place during the peer review sessions and supporting technical documentation provided by the peer review host agency.
A travel demand model peer review is conducted at the request of the agency hosting the peer review panel as a means of soliciting:
Moreover, as noted on the Travel Model Improvement Program (TMIP) website, "few individuals have had the opportunity to develop and apply more than one travel demand forecasting procedure. No individual can foresee all the issues that may arise in developing or applying a new model set. One approach to improving travel forecasting procedures has been the use of peer review panels. These panels, composed of individuals who have "hands-on" experience with both developing and applying travel forecasting models, assist local agency staff in both identifying possible problems and in developing workable solutions".
After a peer review panel meeting has been conducted, a summary report is prepared that documents the panel's findings, recommendations and suggested course of action. While this is the primary purpose of the report, a secondary purpose is equally valuable: to delineate the identified issues and workable solutions as a means of providing other modeling practitioners concepts and approaches to consider for incorporation into their own model set. Ideally it should offer new perspectives on standard assumptions regarding model development.
To facilitate assessing whether any noted recommendation is worth implementing, the peer review summary report does not extensively document the reviewed model's current structure. Instead, a brief summary of each model component is offered to place the topic of discussion in context. For the model currently under review, at the time this peer review was conducted the base year model calibration and validation effort had not been completed. The intent however, of the peer review was not focused on identifying and addressing model deficiencies, but rather looking forward and identifying model enhancements for the next model update given the current model structure. Consequently, most of the discussion for each given topic will focus on summarizing the current model approach and the perceived issue as a means of providing context for the panel's recommended approach or solution, as follows:
For the reader that desires a more comprehensive review of the existing model, Appendix A, CAMPO Model Documentation, lists relevant references that can be acquired.
This report, Summary Report Capital Area Metropolitan Planning Organization Travel Demand Model Peer Review, documents the travel demand model peer review panel meeting held at the Capital Area Metropolitan Planning Organization (CAMPO) office in Austin, Texas, on June 11th and 12th of 2009. The one and one half-day peer review panel meeting was held as part of TMIP, sponsored by the Federal Highway Administration (FHWA). The peer review panel consisted of six travel demand modeling experts (ref. Appendix B, List of Peer Review Panel Participants, for list of panel members and meeting participants).
Appendix C, Peer Review Panel Meeting Agenda, provides the meeting agenda. The meeting began with CAMPO staff providing an overview of the technical questions and issues that they wanted to discuss and which formed the basis of the peer review panel meeting. This was followed by staff presentations on the existing model structure and recent model improvements. Peer review panel discussion was based on questions and answers occurring throughout staff presentations, as well as model documentation provided prior to the meeting and the pre-defined set of questions provided at the inception of the meeting.
Apart from a brief model overview, the majority of this report summarizes the findings and recommendations of the peer review panel. Prior to discussing the identified issues and recommendations it should be noted that the peer review panel was appreciative and complimentary of the effort involved in developing and calibrating the CAMPO travel demand model. Panel members commended CAMPO staff for accomplishing particularly good results with limited staff resources. In addition, the panel noted that CAMPO staff is achieving consistent state-of-the-practice results which is a testimony to what can be accomplished with limited staff.
Structuring the peer review panel report to primarily focus on issues and recommendations may leave one with an impression that the model was not entirely sound; that is not the case nor is it the intent of this report. Rather, it is assumed that the typical reader is more interested in identified issues and model nuances that required thoughtful consideration and that more can be learned from discussing aspects of a model with potential for enhancement as opposed to reviewing existing model structure and what works. To that end, CAMPO staff have been gracious enough to openly share their current issues. Following the model overview, the remainder of the report documents the identified issues and peer review panel recommendations.
This section of the report offers a brief overview of the CAMPO model components to provide some context for the discussion comprising the remainder of the report.
A household travel survey comprised of 1,500 randomly selected households was conducted in 2006 to support model calibration efforts.
During 2005 and 2006 a workplace survey collected information from 1,690 establishments.
A commercial vehicle survey comprised of 342 commercial vehicles was conducted in the spring and fall of 2006.
In 2005 an external travel survey was conducted for 22 of the 42 external stations. In addition, 24-hour vehicle classification counts were collected at each external station on the survey day.
An on-board transit survey was conducted in 2005 to support the calibration of the mode choice model.
3,586 twenty-four hour traffic counts were collected to support model validation.
Posted speed limit data was inventoried.
Population and household estimates for 2005 were estimated based on 2000 Census figures and building permit information. Base year 2005 population and household estimates were 1,458,641 and 548,126 respectively. Employment data was estimated at 698,399 for the base year. Estimates for the five counties comprising the CAMPO model study area are provided in Table 1 - 2005 Demographic Data.
The CAMPO study area encompasses five central Texas counties: Bastrop, Caldwell, Hays, Travis and Williamson counties (ref. Figure 1). The five counties are divided into 1,413 internal traffic analysis zones (TAZs) with 49 external TAZs for a total of 1,462 zones.
Figure 1 - CAMPO Five County Area
The 2005 model network is comprised of all facilities functionally classified as collector and above. The 2005 network consists of 10,745 non-centroid links. Network capacities were based on the Highway Capacity Manual 2000 for level of service E.
|Demographic Data||Bastrop County||Caldwell County||Hays County||Travis County||Williamson County||Total|
|Persons per Household||2.84||2.92||2.95||2.53||2.89||2.66|
|Employees per Person||0.17||0.20||0.32||0.60||0.31||0.48|
Source: CAMPO, June 2009
The trip production model is a cross-classification model; production rates are stratified by household size (1 to 5+) and five income ranges resulting in a cross-classification table of five household income categories and five household size categories.
The trip attraction model is also a cross-classification model with attraction rates stratified by four employment types (basic retail, service, and educational) plus households and five area types:
There are twelve internal trip purposes:
And five external trip purposes:
The trip distribution model is a gravity-based model. Friction factors were calibrated to 2005 household survey trip length frequency distributions by trip purpose. K-factors were used sparingly to calibrate the model.
The CAMPO mode choice model is a nested multinomial logit model that was originally estimated based on 1997 home interview and on-board transit survey data. The model includes the following modes:
Neither light rail nor commuter rail has been implemented in the CAMPO study area, so estimation for these modes was based upon inputs from other study areas. For the current 2005 model update, the mode choice model is fundamentally based upon the 1997 estimation effort, with a recalibration based on 2005 on-board transit survey data.
A daily 24-hour assignment and two peak period assignments are run. The assignment procedures use a generalized cost multi-class user-equilibrium assignment process. The morning and evening peak period assignments represent the 7:00 to 9:00 AM and 4:00 to 6:00 PM time periods respectively. The base year 2005 model contains no toll roads; however, in 2006 the first regional toll road began operation. To ensure that the assignment process is functioning properly with respect to toll roads, CAMPO has developed a short-term interim-year 2008 forecast and has been comparing assignment results against 2008 traffic counts provided by the tolling agencies.
The Issues and Recommendations chapter concentrates on examining each of the technical issues that arose during the peer review meeting, its significance, and providing context for the peer review panel's recommendations.
The previous CAMPO process for allocating forecasted demographic control totals was a modified Delphi panel process that relied on panel member's local knowledge and the ability of the panel to achieve consensus. The two primary issues with this approach were that:
In response to those issues the CAMPO demographic allocation tool has been developed; it provides a framework for a replicable process that affords the development of alternative growth scenarios. In addition the process is integrated with CAMPO's travel demand model, using model travel times as an input variable for accessibility and supplying the relevant TAZ-level demographic data for trip generation.
The forecast demographic allocation tool is a GIS-based computer program that takes user supplied demographic control totals and allocates the data from large geographic areas (e.g. counties) to smaller geographic areas such as traffic analysis zones.
The allocation process (ref. Figure 2) begins with an estimation of developable land followed by an iterative allocation process to small grid cells (each 36 acres) until forecast and density goals are achieved. Allocation to the five counties is performed independently based on county control totals. Three criteria determine the allocation process to individual grid cells:
Factors that influence a zone's attractiveness rating include:
Finally, the forecast demographic allocation process is implemented in ten year increments.
CAMPO staff were interested in investigating recommendations for improving the current forecast demographic allocation tool. The desire for improving the allocation process arose in part from observable allocation anomalies along county boundary lines. This was because the process of independently allocating by individual counties sometimes yielded inconsistent growth patterns on separate sides of a county line (i.e. different resulting densities on either side of the county line).
A secondary issue of interest was the efficacy of pursuing a new land use forecasting model for alternative growth scenario testing.
Figure 2 - Demographic Allocation Process
Source: CAMPO, June 2009
In general, the peer review panel noted that the demographic allocation process was a very good approach; this was based on three elements:
The panel also stated that it was important for CAMPO to maintain staff ownership of the demographic allocation tool and process.
With regard to the county border anomalies resulting from independent county allocations, the panel recommended that CAMPO consider dropping the county control total constraint particularly when looking at alternative scenarios.
As a means of testing the demographic allocation tool's performance, the peer review panel suggested that CAMPO staff investigate reaction of the model to basic input changes and back cast or replicate the regional demographic changes that will have occurred between 2005 and 2010.
Additional recommendations for improving and/or supporting the forecast allocation process included:
CAMPO is a proponent of visually summarizing data in a manner that is useful to the general public. To that end the agency uses graphs and thematic mapping techniques to present complex data or model results as a means of making the data more accessible to the public.
Their current data visualization practice relies primarily on the use of static maps. Long-range plan maps utilize a set of map templates for quality consistency which were made using Arcview 3.2. The traffic count maps that are provided to the public are based upon Texas Department of Transportation (TxDOT) data: either the annual average daily traffic (AADT) counts, or the more robust sample saturation counts collected by TxDOT every five years.
CAMPO has also produced a series of isochronal environmental justice (EJ) maps that summarize modeled travel times in five-minute band widths. For a given map, time bands emanate from either a randomly chosen non-EJ or EJ identified TAZ.
A recent effort to improve and build upon the use of static maps has been through the application of GIS Viewer to offer dynamic viewing capabilities. Once the Phase I effort is completed, users will be able to search physical addresses and view geospatial information for traffic counts and Transportation Improvement Projects (TIP), as well as plan projects with selected project attributes on-line. The Phase II effort will be expanded to include additional CAMPO projects.
CAMPO is interested in implementing and applying advanced data visualization techniques to support improved communication of complex data to the CAMPO Policy Board and the public.
The peer review panel stated that the current static maps that CAMPO produces look good; however, the panel also noted that it was an advantageous idea to move towards the provision of web accessible data that incorporated more dynamic mapping features.
The panel cautioned CAMPO staff to be very careful about how forecasts (e.g. land use and transportation) are generally presented. Items to consider when developing presentation materials include:
Other panel recommendations for CAMPO included the following:
As noted previously, CAMPO's trip production and trip attraction models are both cross-classification models.
Person trip production rates for eleven trip purposes are derived from a 2005 household travel survey (the twelfth trip purpose, truck-taxi, uses a control total number of vehicle trip productions derived as a function of the regional population). The production rates are stratified by household size and five income ranges, but for the three home-based work trip purposes (direct, strategic and complex) a third stratification is used that accounts for the number of workers in each household.
The three home-based work trip purposes are intended to provide additional insight into the mode choice decision. The definition of the three purposes is as follows:
Three sub-models have been developed to process zonal demographic data to support the trip production model; these are:
The household size and income models generate required input data for the workers per household model. The output from the workers per household model (i.e. number of households with 0, 1, 2 and 3+ workers) is then input to the trip generation model.
The primary concern for CAMPO with regard to trip generation was how their process compared to the current state of the practice. Specifically of interest was whether:
The panel noted that CAMPO's current trip generation model conforms to the state-of-the-practice. Moreover, they advised that CAMPO should maintain the three distinct home-based work trip purposes, something CAMPO was considering abandoning.
Three recommendations provided by the peer review panel for improving the trip generation process were as follows:
As mentioned previously, the CAMPO trip distribution model is a gravity-based model. The current model development process has incorporated a travel time feedback process from the traffic assignment step to trip distribution (feedback is discussed further in a subsequent report section).
Two separate travel time matrices (skims) are developed as input to the trip distribution process: peak hour and 24-hour travel times. The peak hour skims represent the 7:00 AM to 9:00 AM two-hour time period and are only used for the three home-based work trip distributions. The 24-hour skims are used for all other non-work trip purposes. In the application of the feedback methodology the first iteration skims are based on free-flow travel times; all subsequent iterations use resulting peak hour or 24-hour assignment travel times from the prior iteration depending on whether it is a home-based work or non-work trip purpose application. It should be noted that travel times are skimmed only from non-transit network links.
Friction factors were calibrated to 2005 household survey trip length frequency distributions by trip purpose. The truck-taxi and external trip purposes used commercial vehicle and external survey data respectively. K-factors were used sparingly to calibrate the model.
As with trip generation, the primary interest for CAMPO with regard to trip distribution was how their process compared to the current state of the practice. Likewise, CAMPO was interested in ascertaining whether better methodologies for addressing trip distribution for the size and diversity of the region are available and should be implemented.
The primary peer panel observation was that the current trip distribution approach employed by CAMPO is consistent with the state-of-practice for trip distribution. On the other hand, since the model had not been completed by the time of the peer review panel meeting and thus documentation regarding model results and performance were not yet available, the panel also noted that it was not clear how well the model is functioning. Consequently, the peer review panel recommended that CAMPO should carefully validate the trip distribution model against available survey data and undertake the following tasks to assess model performance:
In addition, the panel suggested that CAMPO staff should investigate whether the trip lengths and comparable friction factors for downtown attractions are fundamentally different. A further suggestion was to validate resulting trip patterns for special destinations (e.g. airport, University of Texas).
In essence, the existing CAMPO model is a 24-hour model that forecasts daily travel patterns and system volumes. Morning and evening peak-period trip tables for peak-period assignments are derived however, from the application of household travel survey diurnal factors against individual output trip purpose matrices from the mode choice model.
Previous model development efforts aimed at addressing time-of-day questions have been impeded from a lack of supporting data (e.g. peak period traffic counts). In response to prior impediments CAMPO is planning a data collection effort to support the development of the next base year (2010) model that will include a time-of-day component.
Given the proposed data collection effort to support the incorporation of a time-of-day methodology in the next travel model, the primary issue of concern was how best to address time-of-day questions. More specifically, CAMPO was interested in the efficacy of:
The peer review panel's recommendations stem from the viewpoint that a 24-hour time period based model is difficult to explain; for example, 24-hour speeds and capacities. Their recommendations for subsequent model enhancements included the following:
To support these enhancements the panel also recommended the following actions:
As noted previously, the CAMPO mode choice model is a nested multinomial logit model that has been recalibrated based on 2005 on-board transit survey data. The nesting structure is graphically displayed in Figure 3.
There are four levels of nests with the first level determining the choice between motorized and non-motorized. The second level differentiates auto from transit as well as walk from bicycle trips. The next level separates drive alone and shared ride auto trips.
On the transit side, the primary modes are specified by mode of access, including walk, park and ride (PNR), and kiss and ride (KNR). On the auto side the drive alone and shared ride choices have a choice of toll or non-toll below them, and the shared ride alternatives have a further choice of using HOV. For the current model update the toll nest is not used; instead, toll road users are determined as a path choice during trip assignment.
Walk and drive access and egress times are based on actual times skimmed from the highway network. In the case of walk, centroid connector walk times are based on a coded speed of three miles per hour but capped at a maximum time of ten minutes. Drive access times to park-and-ride, and kiss-and-ride lots are based on modeled highway travel times along network centroid connectors and links. Opportunity (market segmentation) for walk access to transit is determined by the aggregate percentage of households in each TAZ that reside within quarter-mile and half-mile buffers around available transit stops. A geographic information system (GIS) program is used to estimate the percentages that fall within each buffer. These market segments are subsequently apportioned into seven categories based on the length of the walk distance at both the origin and destination of the trip (short walk, long walk or no walk (i.e. no transit stop available within the allowed walking distance)):
For the next model update, 2010, CAMPO intends to revise and improve their mode choice model. Consequently, they were primarily interested in options for enhancing the model beyond its current structure. Two issues were of particular interest:
One additional item of concern was the appropriate level of detail or mode choice model structure to support New Starts applications. The current model addresses three transit modes: local bus, express bus, and university bus service. In the near future the transit system will be expanded to include commuter rail and bus rapid transit (BRT) service. The existing model can also account for light rail and commuter rail in future alternatives. Questions for future model applications included:
The panel noted that the mode choice model is consistent with the state-of-the-practice from several years ago, but that it now needs to be updated consistent with current Federal Transit Administration (FTA) requirements. Moreover, the on-going mode choice model improvements are a step in the right direction but will not be sufficient.
Further suggestions proposed by the panel included:
One final panel recommendation was to collect additional transit ridership data after the commuter rail service begins operating to support subsequent mode choice model enhancements.
As noted previously, the CAMPO trip assignment process consists of a daily 24-hour assignment and two peak period assignments. The assignment procedure uses a generalized cost multi-class user-equilibrium assignment process. The same value of time is used for mode choice and the generalized cost assignment procedure. The morning and evening peak period assignments represent the 7:00 to 9:00 AM and 4:00 to 6:00 PM time periods respectively. Only the daily and AM peak-period assignments are run during the feedback process. Currently, the mode choice output is aggregated so that the following modes are assigned:
The base year 2005 model contains no toll roads; however, in 2006 the first regional toll road began operation. To ensure that the assignment process is functioning properly with respect to toll roads, CAMPO has developed a short-term interim-year 2008 forecast and has been comparing assignment results against 2008 traffic counts provided by the tolling agencies.
Toll road links in the highway network are identified using a cost value in a year specific toll cost field. If a link is not a part of the toll system then the cost value is coded as a null value. This offers the ability to make toll road selection sets since toll roads are not coded as a unique functional classification. The cost value that is coded is defined by the rates set by the toll agencies and is only coded on links that represent the location where the cost is incurred (e.g. toll plazas). All of the regional toll roads are "closed" systems.
CAMPO is interested in understanding if preferred methods for addressing toll roads should be implemented.
As mentioned in the Mode Choice section, the peer review panel stated that a 24-hour assignment is difficult to explain. In response, the panel suggested that in the next model update, CAMPO implement an assignment procedure comprised of four time periods.
The panel recommended that CAMPO document their volume-delay functions and network coding procedures. In addition the use of tighter closure criteria (i.e. a relative gap of 0.0001 and setting maximum iterations to 1000) during the assignment process was recommended.
With regard to model validation the panel recommended validating on the basis of both travel times and count data and not just count data as is currently practiced. The incorporation of additional count validation criteria (e.g. screenlines and root mean square error (RMSE)) was also suggested.
Finally, the panel recommended that CAMPO consider income stratification for trip tables and value-of-time with the caveat that this would require the following items:
The current CAMPO model includes the implementation of a feedback loop using travel times from the assignment process and fed back to the trip distribution model. The incorporation of a feedback procedure arose from the desire to use consistent travel times during trip distribution and trip assignment. It was also based on a recommendation from a previous peer review conducted in 2001 though no guidance was offered at the time on an approach or means of implementation.
The implementation of CAMPO's feedback procedure has evolved through various phases beginning with the implementation of a simple or naïve feedback process. As the process evolved, several methodologies and convergence criteria were assessed but subsequently rejected based upon the practicality of maintaining reasonable model run times. The feedback procedure currently in use is graphically displayed in Figure 4 - CAMPO Model Feedback Flow.
Figure 4 - CAMPO Model Feedback Flow
Source: CAMPO, June 2009
At the inception of the feedback process, trip distribution is applied using free-flow travel times for the first iteration of the feedback loop. For subsequent feedback iterations peak congested times and 24-hour congested times from the previous assignment are used for home-based work and the other trip purposes respectively. Mode choice is also run during each iteration of the feedback loop. The process continues until convergence or a specified number of iterations are reached.
The feedback methodology is structured to allow the user to use either an MSA (multiple successive averages) or constant weight approach to process the times being fed back into distribution. The current process also offers the user a choice of three convergence criteria to use (all are reported):
The feedback methodology is structured to allow the user to start with either free-flow times ("cold start") or congested times by assigning previously generated peak and 24-hour trip tables ("warm start"). Presently, CAMPO procedure specifies starting with the free-flow times.
The potential or need for additional improvements or refinements to the current feedback process was the only issue of concern.
The panel observed that the current approach is ahead of the state-of-the-practice. As a means of reducing run times the panel recommended the following:
Apart from the previous recommendations that were offered in response to specific issues, the peer review panel also suggested the following actions grouped by high and low priorities:
Confirm reasonableness of survey data by doing the following:
Validate model results against existing survey data and counts to accomplish the following:
Prepare more thorough and complete model documentation including the following topics:
Finally, investigate model run-times and hardware since the model appears unnecessarily slow.
Future methodological changes that CAMPO is considering implementing should be based on demonstrated deficiencies from high priority comparisons.
CAMPO should consider opportunities for across the board consistency by:
Lastly, CAMPO should continue to work towards self-sufficiency and in the meantime consider opportunities for efficiency improvements in the software linkages.
In addition to the recommendations noted above the peer review panel also offered the following comments in response to several additional technical questions posed by CAMPO:
Should CAMPO consider developing an activity-based model?
The panel recommended that CAMPO closely monitor the evolution of the practice and in the interim, to concentrate their resources on improving the trip-based model.
CAMPO was interested in methodologies for better answering environmental justice issues.
The panel suggested that CAMPO consider adopting the approach used in Denver and Dallas.
The question of whether the development of a freight model would benefit CAMPO was posed. Currently CAMPO has a truck trip purpose that is combined with taxis. No procedures exist to model freight movements within the region. In 2008 CAMPO undertook a needs assessment study for all modes of freight movements within the urban area.
The panel recommended separating trucks from taxis in the development and application of the truck trip purpose. The panel informed CAMPO that they could consider a freight model, but that higher priorities exist for the time being.
What is the best method for accounting for gas price effects in the model?
The panel noted that the current state-of-the-practice does not explicitly include gas prices; moreover, that it is not an easy variable to address. The panel did however recommend saving traffic count data and transit rider data before and after spikes in the gas price.
CAMPO was interested in any improvements that could be implemented with limited funds.
The panel recommended that CAMPO develop the ability to use microsimulation for a corridor or sub-area.
|Erik Sabina||Denver Regional Council of Governments (DRCOG)|
|Arash Mirzaei||North Central Council of Governments (NCTCOG)|
|John Lobb||RSG Inc.|
|Guy Rousseau||Atlanta Regional Commission (ARC)|
|Phillip Reeder||Texas Transportation Institute (TTI)|
|Joe Cantalupo||Capital Area Metropolitan Planning Organization (CAMPO)|
|Daniel Yang||Capital Area Metropolitan Planning Organization (CAMPO)|
|Kevin Lancaster||Capital Area Metropolitan Planning Organization (CAMPO)|
|Michael Dutton||Capital Area Metropolitan Planning Organization (CAMPO)|
|Cole Kitten||Capital Area Metropolitan Planning Organization (CAMPO)|
|Greg Goldman||Capital Area Metropolitan Planning Organization (CAMPO)|
|Greg Lancaster||Texas Department of Transportation (TxDOT)|
TRAVEL MODEL IMPROVEMENT PROGRAM
CAMPO PEER REVIEW MEETING
June 11 and 12, 2009-8:30AM
One Texas Center
505 Barton Springs Road
June 11-12, 2009
|JUNE 11, 2009|
|8:30||Welcome and Introductions|
|8:45||Demographic Allocation and Visualization|
|10:30||Trip Generation and Distribution|
|1:15||Mode Choice and New/Small Starts Applications|
|2:30||Time of Day, Toll Assignment, and Feedback Loop|
|JUNE 12, 2009|
|8:30||Panel Caucus - (Panelists Only)|
|10:15||Preliminary Findings/Recommendations from the Panel|