The panel discussion was structured by a list of questions WSDOT developed in advance of the review session. Those questions are listed below for easy reference and then appear as headings in the following section as a means of organizing the panel discussion.
The following section provides a summary of the responses that panelists provided to the WSDOT questions, as well as other questions and elements that were incorporated into the peer review discussion.
Dr. Horowitz remarked that a NCHRP Statewide model survey-based report[8] he authored in 2006 indicates that states use their models for a spectrum of analysis applications. That survey found that 32 states had their own models at that time--with additions since then states with models now number about 40. The state models range from modest ones used for long-range planning and facility planning to sophisticated ones, like the Oregon Statewide Model, which is used for a wider variety of applications.
Maryland's model has been used to support the development of the statewide transportation plan, to study regionally or inter-regionally significant corridors, and to study rural region travel. It produces both passenger and freight travel metrics to support such studies. Maryland has conducted extensive scenario planning analysis to understand the outcomes across various performance metrics. It has also been used recently to study extreme weather scenarios for emergency planning purposes, for example a key roadway put out of service by a storm, to conduct Transportation System Management and reliability analyses, and to support the state's Smart Green & Growing Initiative through climate change adaptation studies.
Arizona uses its model to support the state "Planning to Programming" initiative, a planning process that assesses different possible future investment focus areas in a structured fashion. It is contemplating how to use its model to study means of increasing roadway reliability, especially for freight and goods movement, and is examining recent Strategic Highway Research Plan 2 (SHRP2) reports on reliability techniques that can be used in models.[9]
Oregon's model has been used for a variety of studies. Of particular note was an application testing different staging schedules for dealing with the state's backlog of bridge preservation and replacement needs. The findings indicated that given the distribution of economic activity in the state, the traditional engineering approach for prioritizing the repair of high-volume bridges, was less effective in minimizing negative economic impacts than a strategic repair approach. The model helped Oregon DOT staff devise a staging plan that was later successfully funded by bonds approved in a statewide referendum.
An audience member raised the question of how statewide models do or can handle tourist travel. Mr. Killough responded that the Arizona model's long-distance personal travel component includes tourist destinations such as national parks as special attractions. Mr. MacIvor remarked that estimating tourist travel explicitly is not currently in the California Statewide Model since it is a low priority for the state, which at present has chosen to concentrate on ensuring that its model is useful for examining and comparing future large-scale investment scenarios.
Regarding the question of off-model analysis, Arizona uses the REMI TranSight tool for evaluating the economic effects of changes to the transportation system as reflected in its statewide travel model.
Ms. Knudson recommended that WSDOT prioritize their analysis questions but avoid creating features that constrain future analysis options, for example: hard-coding operating costs. Mr. Killough seconded the importance of prioritizing analysis questions in the development of the model.
Mr. MacIvor noted that budget and agency resources, such as staff and training, will dictate much of the model's development path. He noted that identifying a specific software platform that works for the state modeling team and will be sustainable is critical, but to recognize that models are not the only analysis tool out there and it will be helpful to investigate other options, as well.
Mr. MacIvor also remarked that WSDOT examine easily available data, which may suggest development directions for the state. After identifying the easily available data, it will be helpful for WSDOT to generate a data development plan that highlights what data that WSDOT should purchase to fill the holes in the available data.
WSDOT questioned which other methods the panelists would suggest for addressing uncertainty associated with forecasting. Mr. Killough responded that forecast staff needs to be aware of the context in which forecasts are made to judge the validity of upstream forecasts, for example: economic growth and population evolution. Mr. MacIvor offered that sensitivity analysis can be conducted to assess how key inputs and assumptions drive the forecasts results. WSDOT could push the tests to extremes, well beyond politically-acceptable ranges, for the purpose of understanding implications.
California started by assessing available budget and existing data, which included a complete data inventory as the second step in its model development process. Both Maryland and Oregon created lists of analysis needs and available data and cross-referenced them to suggest both directions for early model development and data 'holes' that further model development work planning would need to address. Dr. Horowitz recommended building on these ideas by scoping a phase 1 model that can use secondary (existing) data followed by a phase 2 that includes primary data collection in its work plan. Ms. Knudson of Oregon recommended proceeding with model development even if all data is not available, because the model evolution can then scope the data products that need to be developed.
Mr. Mahapatra responded that cooperation is valuable, for example much of Maryland's statewide data is taken from the MPOs. The statewide and MPO models are seen as complementary in that while state objectives may focus more on network implications, MPO models look other areas like land use and air quality. Long-distance and visitor travel are not at top of the MPO priority list, but these issues could be significant priorities for the state. Mr. Mahapatra suggested that WSDOT communicate the responsibilities for each model, both state and regional, clearly.
Dr. Horowitz noted that a 2006 survey of statewide models indicated that MPOs prefer that statewide models provide external station volumes. However, in Wisconsin, the Milwaukee area is covered by just a few zones due to institutional issues. Therefore, it is critical that WSDOT establish cooperation efforts early in the model development process and recognize that it is not a problem if results differ.
Ms. Knudson remarked that MPOs were partners with ODOT from the beginning of the statewide model's development. The MPOs wanted various types of information from the statewide model, including external station volumes and economic activity estimates that reached beyond scope of their models. The State of Oregon produces official statewide revenue and population forecasts that MPOs and state agencies must use for analysis. Scenario analysis is used to represent ranges of potential futures, such as optimistic and pessimistic versions that produced variations on the revenue and population estimates. These all can be common inputs to both MPO and state models.
Mr. Killough stated that Arizona's MPOs utilize external volumes, but otherwise the MPO models provide more data for the statewide model than interacting with the statewide model directly. Arizona carefully avoids "competing forecasts."
Mr. MacIvor noted that regional transportation planning guidelines were adopted in California two years ago in response to Senate Bill 375. The statewide model is the prime estimator of emissions for MPO models external-to-external trips. Also, internal-to-external trip length is difficult for MPO models, so there is impetus to use the statewide model for more air quality work. This is a work in progress at Caltrans.
Mr. Cervenka questioned if panelists had validated the statewide model 'externals' volumes to counts. Arizona validated the AZTDM at screenlines outside the MPOs. Caltrans plans to validate counts at the externals. Oregon does externals classification counts, as does Maryland. Maryland also noted that they may also use license plate checking for external-to-external trips.
The SWIM2 model represents the behavior of the land use, economy and transport system in the State of Oregon using a set of connected modules that cover different components of the full system. There are eight modules:
Reporting often relates the differences in state employment and production for alternative policy analysis scenarios. Information is reported by industry sectors, geographic regions and commodity flows.
Dr. Horowitz noted that Oregon's SWIM internalization of the economic forecasting is rare among state models and may be beyond what WSDOT wants to attempt in the first phase of Washington model development. He noted, though, that the Montana statewide model incorporates a simpler economic component in the form of HEAT (a TREDIS forerunner) that seems to work well and might be a more feasible early step for model development.
Maryland takes statewide economic forecasts produced by the University of Maryland as inputs for its land use allocation model component which in turn produces county-level population and employment control totals for the statewide travel model. At this time the economic and land use allocation processes do not take feedback from the travel model. Maryland has also applied the Surface Transportation Efficiency Analysis Model (STEAM) benefit-cost tool to its statewide model outputs for some studies.
Arizona now uses the REMI TranSight tool to estimate economic impacts of scenarios run through the statewide travel model. The Arizona TranSight implementation takes as inputs travel impedance skims and estimates effects on employment, gross state product (GSP), and the relative competitiveness of Arizona to adjacent states. REMI customized Arizona's TranSight to include a household sub model to forecast changes in household characteristics such as income and size. Mechanisms exist to feed the TranSight outputs back to the statewide travel model but Arizona does not generally do so. Mr. Killough remarked that techniques described in the SHRP CO3 and CO11 research could be used in lieu of REMI or TREDIS, and noted that Arizona aspires to fully integrated economic, land use, and transport modeling like that of Oregon.
California estimates economic impacts in a manner similar to that of Arizona since the former's state model does not now do economic forecasting internally. California statewide estimates of transport system investment economic impacts takes inputs from the MPO economic forecasts, runs the state travel model, and feeds state travel model outputs to state economists on staff who run an implementation of the TREDIS tool.
Ms. Barb Ivanov (WSDOT Director of Freight Planning) remarked that Washington has tested a prototype statewide computable general equilibrium (CGE) economic model and found that it is usefully sensitive. Until the new CGE model is fully operational Washington can use an existing REMI model suite to assess economic development strategies.
California's statewide model outputs vehicles by speed bin in EMFAC-ready format. Greenhouse gas (GHG) estimation is handled by separate post-model calculation.
In the AZTDM, fuel is a consumption factor. Air quality estimation is conducted through the application of the Motor Vehicle Emission Simulator (MOVES), while land use utilizes population forecasts from the Arizona state demographer. There is no formal requirement for employment forecasts, but the AZTDM uses the ADOT Risk Analysis Process, a Delphi process that includes a consortium of key developers, MPOs and academics.
In the Oregon statewide model, fuel is an input defined through vehicle operating costs. SWIM is not designed to report criteria emissions or GHG.
The MSTM does not do criteria emissions, but uses MOVES for GHG calculation. Land use inputs are a 'cooperative forecast,' developed through the analysis of possible effects of transport scenarios on the population and employment. The state has also experimented with sensitivity testing on fuel price changes.
WSDOT staff noted that Washington State imposed a law on GHG reduction in 1990, implying that GHG estimates will be a critical output of the statewide model. Mr. MacIvor responded that the VMT-GHG relationship is not linear, and furthermore GHG reductions are not coupled one-to-one with VMT reductions. Mr. MacIvor suggested application of the Vision tool to assess GHG scenarios, which uses travel model results as an input.
Mr. Mahapatra noted that Maryland expects large population and employment growth in the future; so the focus is on reducing VMT per capita by applying smart growth strategies. It was also noted that California has evaluated the issue of truck trips generated by denser growth and the effect that this has on already-crowded arterials.
When the panelists were asked if they had conducted lifecycle analysis on GHG, Mr. MacIvor responded that the CEC does look at lifecycle issues.
Mr. Cervenka asked if California was developing a state PECAS model. Mr. MacIvor responded that California's PECAS model incorporation is in stasis. Mr. Killough noted that Arizona is currently assessing expansion of the Maricopa Association of Governments (MAG) Sun Corridor MegaRegion AZ-SMART model to create statewide land use forecasts.
In the California statewide model, time-of-day is estimated by a separate sub model, which segments demand into five time periods. Arizona uses the diurnal distribution of travel by purpose derived from the 2009 NHTS Add-On Samples collected for the Maricopa Association of Governments and the Pima Association of Governments. The Oregon statewide model currently reports daily patterns, but it is designed to produce four time periods that may be utilized as better data becomes available. Maryland responded that they apply 2002 NHTS data, which precludes highly-detailed time-of-day models, and therefore the model does not have enough information to conduct this type of analysis.
Dr. Horowitz remarked that DTA can be a good method of accounting for time-of-day travel, but it is computationally intensive. One-hour slices would be adequate for statewide model. Dr. Horowitz noted that the MAFC Freight Model estimates departure time as part of shipment synthesis. Maryland is currently looking at half hour DTA time slices. This involves a 16-hour run time, but the process is not yet converging.
It was noted that the advantage of using a destination choice model versus gravity model is that destination choice models have more explanatory variables but these variables do not complicate the process. The gravity model is a special case of destination choice.
In an effort to allocate enough time for high priority topics, mode specifications were not discussed in isolation.
WSDOT defines "routing" as individual truck movements based on truck load. For example, routing would be determined by identifying whether a vehicle is following a fixed pattern or going to different locations every day and whether it is carrying full loads, part loads, or traveling empty. WSDOT clarified that they wanted to know how the panelists modeled trucks in general and how they were able to model route choice. WSDOT noted that truck route choice is important for incorporation in the statewide model because different truck trip types will react differently to toll deployments.
California's statewide model divides truck trips by long distance and short distance. Caltrans borrowed the Calgary structure and data, but they are currently funding their own survey. The California Air Resources Board (CARB) surveyed all trucks coming out of the Port of Los Angeles and the Port of Long Beach. When new data is available, Caltrans plans to examine truck-only strategies using DTA or microsimulation to assess effective capacity gains.
The AZTDM includes long distance truck data, which was first based on FAF data but is now Transearch-based. For commercial non-freight movement, ADOT borrowed MAG's short-haul model but removed the short-haul freight component.
Oregon's SWIM has a tour-based truck model, which starts with estimating economic activity and goods flows and then converts the good flows to trucks and assigns trips using a traveling-salesman algorithm. The SWIM model uses FAF and Oregon Commodity Survey data.
Ms. Knudson noted reasonable confidence in Oregon's economic estimates and, thus, in the accuracy of truck forecasts at certain, broader geographic levels. Caltrans developed a commodity flow model, which also functions reasonably well at a larger, broader scale.
The MSTM borrows a trip-based freight model from the Baltimore Metropolitan Council (BMC) that is validated to counts. MD SHA recently hosted a peer review of the MSTM that recommended a logistics-based approach rather than trip-based approach but noted that the former logistics-based approach continues to rest on a variety of assumptions. MD SHA experienced success in reviewing data to identify commodities that tend to use only one or a limited set of modes, which renders modeling commodity flows easier.
Dr. Horowitz recommended that WSDOT handle only long-distance movements that have reasonable data as the amount and quality of truck movement data available is limited.
Mr. Mysore commented that Florida built a supply chain method based on long-haul characteristics in a seven-step process. The process begins with "freight land use," which supports the estimation of goods moving to the freight land uses. Mr. Mysore suggested that WSDOT focus on long-distance freight trucks in a method similar to Florida. Florida worked with IMS/Global Insight to customize Transearch data by adding 'intelligence' to the data so that Florida could recalibrate the model, which was originally built based on FAF data.
WSDOT questioned capabilities of the long-haul model given the majority of congestion problems are located in the urban areas rather than trunk route segments. Mr. Mysore noted that proper integration of the MPO and state models, for example the state model loads long-haul flows into the MPO externals, will yield an urban tour-based model that handles both long-haul distribution and short-haul (freight and non-freight) estimation. Florida mined the ATRI GPS data to produce an origin-destination table for statewide model calibration.
The panelists warned that ATRI GPS data may include inaccurate origin-destination information and the samples may not be entirely representative; therefore, WSDOT should be very careful if applying this data. The panel also advised that WSDOT plan to spend money developing their understanding of the freight system and its behaviors, recognizing that freight route estimation is still within a "discovery" phase in this field.
Mr. MacIvor stated that lack of data is always a concern for model development. He suggested looking at what are the five most important questions the state has to answer, and then figure out what tool to use to answer these questions. Mr. MacIvor also noted that California has a two-year update cycle. They will be releasing the RFP for a California version of the VIUS in January 2015. Mr. Killough responded that Arizona releases model updates annually. Table 2 summarizes the responses from each of the panelists regarding model data sources.
WSDOT asked the panelists what investments their agency made for primary data development for or in cooperation with their MPOs. California's "CA VIUS" effort will coordinate with MPOs, and they also have a prospective truck count program. Arizona uses continuous count locations. The number of count locations has been expanded from 80 to 180 over last five years. The state of Maryland maintains road network centerline data and statewide traffic counts. They are examining a routable centerline network for the entire state and have an Open Data Hub initiative that is now underway at the state level.
Prof. Cynthia Chen of University of Washington observed that HHTSs are increasingly under-representative of the population, even when designed to be representative. Prof. Chen also noted that HHTSs are often costly, question-heavy, and untimely with regard to policy analysis, as the data is often outdated by time of use. Prof. Chen suggested that WSDOT consider a rolling survey approach, like the ACS, as this method enables more customized add-ons. Prof. Chen also suggested a modular instrument design that differentiates between core and optional questions to standardize and minimize questions.
It was also noted that WSDOT should be aware that both a full and random sample are not necessary to estimate all the models in a typical ABM. Household location choice may provide better estimates on a different sample than a typical HHTS. It was suggested that WSDOT assess passively collected data, such as AirSage, transit card, and Bluetooth information, while noting that this type of data is variable-sparse, noisy, and still relatively unknown.
One panelist noted that private-sector or university-based consultants bring both strengths and weaknesses for model development efforts. Academic consultants provide helpful creativity and insight but also present risks to the project schedule. Post-doctoral and student staff may move on despite the project timeframe, which can result in quality issues as the appropriate 'project memory' is no longer there. On the positive side, consultants can be helpful in their ability to honestly critique model inputs and functionality, and private-sector consultants can cost-effectively deliver a great deal of work on schedule when properly scoped and directed.
The panelists added that it can be useful to have consultants execute a preliminary needs assessment and model planning exercise, the product of which is a model development work plan. California spent about $150,000 in total for such an effort that included outreach to senior agency management, an existing data inventory, model development planning, and data development planning. Oregon funded a similar effort to launch development of its model, as it does for all its planning efforts, and noted that the work plan product can be structured in a prioritized fashion so that as funding becomes available the highest-priority tasks within the budget form the de facto project scope. Ms. Knudson also remarked that $150,000 for such an effort would "do a good job." Maryland commissioned a smaller-scale model planning exercise for less than $50,000 and found it to be useful. While Arizona DOT did not commission a model planning effort, Mr. Killough did so while at the Southern California Association of Governments (SCAG) for the sum of $75,000 for planning a land use model component and the same amount to plan for an activity-based model component.
Maryland began development of the MSTM with coordination with the MPOs, similar to the one being held by WSDOT. MD SHA established a structured cooperation process up front. Mr. Mahapatra suggested that WSDOT address foundation data, for example a routable network, early in the model development process. Mr. Mahapatra also suggested that WSDOT assess multi-resolution possibilities early on in the model development process. Mr. Mahapatra warned that "statistics are no substitute for judgment" (quoting Henry Clay), implying that WSDOT should carefully assess all input and output for reasonability. It was also suggested that WSDOT involve their intended end-user group with hands-on involvement opportunities in the early stages of model development.
Dr. Horowitz responded that the major challenges experienced in the development of both the RADIUS & MAFC Freight Models were the need for more time, more budget, and a bigger computer.
Ms. Knudson recommended that WSDOT start simple with their model development process and proceed incrementally, avoiding the "black box." Ms. Knudson stressed WSDOT should figure out how to communicate results of the model and establish buy-in from local and state planners with understandable results as soon as they are able, as well as manage end-user expectations of the modelling analysis. Lastly, Ms. Knudson recommended that WSDOT treat consultants and in-house staff members as one team.
Mr. Killough seconded comments from the preceding three panelists, particularly consultant-staff integration and encouraging coordination amongst all parties involved in the model development process.
Mr. MacIvor reiterated the importance of managing expectations, especially among management. He noted that formal training on the model may be helpful in this sense. Mr. MacIvor also suggested that WSDOT consider having an external review to simplify some model components. Additional recommendations included establishing a solid state storage for speed and cost, to quality check all data prior to utilizing it for the model, and to simplify the model in the initial stages with the knowledge that WSDOT can add additional functionality to the model once it has the basics done right. Mr. MacIvor advised that WSDOT understand when they are reaching the point of diminishing returns on investment with regard to data and to ensure that they communicate the model's availability and capabilities clearly to all potential users.
Mr. Cervenka identified the following goals for WSDOT to incorporate into their model development: taking on only what they could handle, think simple and smart, produce useful results early, for example accessibility analysis or congestion analysis, and to be honest with management regarding uncertainties in the forecasts.
[8] Alan J. Horowitz, "Statewide Travel Forecasting Models", National Cooperative Highway Research Program Synthesis #358, 2006.
[9] SHRP2 Solutions Modeling Reliability Fact Sheet. Available at: http://shrp2.transportation.org/documents/capacity/SHRP2_C04-C05_Improved_Models_for_Ops_Strategies_Factsheet.pdf
National Academy of Sciences. SHRP 2 Report S2-C04-RW-1: Improving Our Understanding of How Highway Congestion and Pricing Affect Travel Demand. 2013. Available at: http://www.trb.org/Main/Blurbs/168141.aspx
