In this Chapter we discuss the performance of the Argos system and its potential utility for planning practice. The utility of the Argos system for planning practice depends on the following factors: 1) model performance, 2) ease of updating, 3) utility for planning applications, and 4) transferability.
We tested model performance by first replicating our earlier results using 2001 data and then updating to 2007 and comparing with actual ground count data. Replicating the earlier results proved to be quite challenging. Likely causes include turnover within the research team between the two projects, and inadequate documentation of data processing and data preparation. The entire workflow was checked and rebuilt, and the entire process of generating the input data was performed multiple times. Our extensive checking revealed only one error in the original workflow, but this was not sufficient to explain differences in results.
Model performance with the 2007 data was not as good as our results from the previous project, as measured against screenline data. In our earlier research we were able to compare results both with screenline data and with SCAG regional model results. SCAG has not yet released validated results for their 2007 screenline, so we could only compare with the screenline data. The 2007 results could be the result of many factors, including the accuracy and reliability of the 2007 data, changes made within Argos planner, changes in the network, etc. Although the research team conducted many checks of the data and the workflow, we could not identify a specific reason for the difference in the robustness of the results.
In rebuilding the Argos workflow we sought to make improvements to make the flow both more efficient and more extensive. One change was to incorporate the entire conversion from dollar flows to PCEs within the workflow (recall this conversion starts with annual dollars, accounts for dollars/ton ratios by industry sector, calculates truck trips by tonnage, and converts truck trips to PCEs). This extends the workflow, but also reduces options for external consistency checks (the only external checks are from the IMPLAN totals in dollars), and does not allow the user to make adjustments to this conversion process. A future improvement would be to allow options for the conversion.
We tested ease of updating by collecting new data for 2007 and performing all the processing steps to generate the Argos input data. There were mostly minor changes in the structure of each data source, hence few changes were required in the workflow. The availability of our industry sector conversion web service facilitated all the required industry sector conversions. Changes in the data allowed us to eliminate one data source - for air cargo - because WISERTrade added air and port data. Data changes may affect model results. For example, the 2007 CFS is a smaller sample than 2001, and thus may be less reliable at the sub-county level.
Preparing the intraregional data is straightforward. Preparing the interregional data is complicated and proved to be quite time consuming, because none of the data sources directly provide the various flows required by Argos. Interregional data preparation also requires expertise with input/output modeling and with regional trade data. This could prove challenging for transportation planning practitioners.
Manual computation is also required to prepare the workflow output for TransCAD input. In this case, programs for the various steps have been developed (see Chapter 2), and the process is relatively easy.
Conventional regional transportation planning models require extensive data and calibration. The Argos system is intended to allow testing of alternative scenarios with relatively little effort. To demonstrate the sketch planning capabilities of the Argos system, we developed and tested three scenarios. The first scenario is an example of an exogenous economic shock, in this case the reduction in international trade that took place from 2007 to 2009. The second and third scenarios provide examples of policy alternatives: shifting a large portion of long distance trucking to rail, and imposing a toll on trucks arriving at the ports during peak hours. The first two scenarios could be implemented with some simple changes to the Argos workflow data. The third required some additional analysis outside Argos: estimation of a travel time/route choice model for the affected truck trips. Together these scenarios demonstrated the broad applicability of Argos to sketch planning and alternatives analysis. Many other scenarios could be imagined: growth or decline of an industry sector, shift in the location of major freight trip generators, peak truck pricing on all interstate highways, capacity expansions within major freight corridors, etc.
Because the Argos workflow is automated, processing any given scenario takes only seconds of computation time. If the scenario does not require a major change in input data, scenario preparation time is also minimal. Running the scenario through TransCAD requires far more time. One run takes several hours on a desktop PC, due to the size of the regional network and the use of multi-user assignment to differentiate truck and passenger trips.
Overall the Argos system proved to be a very good tool for sketch planning. A broad array of scenarios can be generated simply from manipulation of the workflow input data. Other scenarios can be handled by a combination of computations from outside the system and running the workflow and TransCAD.
The Argos system was intended to be transferable across both time and space. We demonstrated transferability across time by updating the entire model to 2007. Transferability across space (eg to other metro areas) is beyond the scope of this project, but a logical next step in model development.
The goal of this research was to create a tool for practitioners that would be easier to update and apply than the current state of practice. The goal has been partially met. The Argos system offers a more efficient and lower cost way to estimate freight flows at the metropolitan level. We have demonstrated that Argos can be linked with TransCAD, allowing the Argos workflow to be used with conventional transportation planning software. We have also demonstrated that the Argos system is easily updated and can be applied to many different scenarios.
On the other hand, the method for generating freight supply and demand is highly complex, and from the perspective of professional planners and modelers is a black box. We noted earlier that the Argos method requires significant expertise beyond transportation planning. Users must have expert knowledge of input/output models and of regional trade dynamics and data. Typically professional planners do not have this expertise. Thus widespread use of Argos would require additional training for professional staff and consulting resources to support staff.
The research team worked closely with SCAG modeling staff throughout Task 2. SCAG staff provided much of the 2007 data, as well as assistance with setting up our TrasnCAD data files. We developed our 2007 baseline with advice from SCAG. Like many other regional planning agencies, SCAG relies on consultants for most of their modeling needs. The SCAG planning staff are experts on the regional and data sources, but only a few staff are experienced in urban modeling. SCAG staff were interested in the Argos system, but found it difficult to understand. The Argos approach is quite different from current practice (an approximation of the 4-step model applied to freight), and SCAG staff did not have training in regional economics or input/output modeling. Our experience with SCAG suggests that significant training and consultant resouces would be required to use Argos without assistance from the university research team.