The first half of the peer review was spent by SEFTC staff members and consultants making presentations on specific items to the peer review panel. During these presentations, many topics came up which initiated discussion among peer review panel members and between panel members and SEFTC. This section documents the key points that arose during these presentations.
One of the presentations made by SEFTC to the peer review panel dealt with the data required for running, estimating, calibrating, and validating the model. SEFTC identified three areas of concern with regard to the available data sources:
SEFTC noted that there is no well-established source of regional and subregional employment totals for both the base and forecast years. Estimates of total regional county employment range from 2.3 million to 3.1 million depending on the data source. Some of the discrepancy is due to how each data source accounts for self-employed and under-employed persons. Each data source tends to have a similar distribution of households between the three MPO areas of Miami-Dade, Broward, and Palm Beach. However, when the currently developed socio-economic dataset is compared to the Census Transportation Planning Products (CTPP), the dataset, in some cases, widely under-predicts and over-predicts employment within each of the sixteen sub-regions. For resident and non-resident travel SEFTC is more concerned with making sure the distribution within the region is correct. However, for truck trips it is important that region-wide totals are correct.
The current procedure for developing the socio-economic dataset is for each MPO to put together their own estimates of employment. Then the modeling team decides if each MPOs estimates are reasonable. However, this task is not easy given the lack of well-established sources of regional employment. The modeling team in the past has attempted different approaches to develop a regional employment dataset:
The panelists were concerned that the inability of employment totals to match targets is confounding the efforts to understand the source of discrepancies in matching validation targets. The panelists stressed the need to resolve the development of both base year and forecast year employment totals and distributions before doing any further model improvements. One panelist also commented that TAZ-level distributions should reflect regional-level distributions in the forecast year rather than keeping the TAZ-level distribution the same as the base year.
SEFTC presented a number of charts and tables depicting how the model results compare to various data sources, such as NHTS, Longitudinal Employer-Household Dynamics (LEHD), CTPP, and observed vehicle miles traveled (VMT) data from traffic counts. The model, when compared to observed VMT, over-estimated morning peak travel and under-estimated mid-day travel. The VMT estimates include the truck and visitor model. There was no bias between arterials and highways. SEFTC noted that they need to adjust their tour scheduling model in order to shift travel from the peak periods to the off-peak periods, but were holding off until new data is collected.
One panelist recommended conducting further analysis to examine why the time-of-day inconsistencies occur.
The most recent household travel survey conducted for the region is the Florida 2009 NHTS add-on. This dataset contained approximately 2,600 household records. However, 10% of those records were incomplete since one or more adults from the household were missing travel information. The dataset over-sampled retired persons (i.e. 37% of the sample was composed of retired households but only make-up 20% of the population), and under-sampled the following markets:
The sample also did not have adequate spatial distribution across the region. For all of these reasons, the dataset contained very large sample weights. SEFTC, against the warning of NHTS, had to change the weights in many instances where unreasonable travel patterns existed.
Despite the issues with the household travel survey, the data was still used extensively for development of SERPM 7.0. The data was used for computing aggregate calibration targets in conjunction with other data sources. In addition, SEFTC compared the calibration targets to other regions to determine if the calibration targets derived from the household survey were reasonable.
The existing transit on-board surveys for the region are missing trip origin/destination information for many of their records. Currently, SEFTC only has access to an on-board survey for one transit system although a system-wide survey is currently underway.
One panelist was concerned that SEFTC did not account for the fact that the 2009 NHTS survey was undertaken during the peak of the recession. The panelists understood the struggles with attempting to work with incomplete surveys and stressed the need, when undertaking a household travel survey, to focus on collecting high-quality data that has a fully complete household rather than on collecting as many surveys as possible.
Another presentation made by SEFTC to the peer review panel focused on understanding how well the model was able to replicate regional conditions, given that the coefficients for each model component were transferred from the SANDAG ABM model. Many of the models were calibrated to SEFTC data sources by adjusting model constants. Other models were fully transferred from SANDAG without further calibration since the available data sources were unable to support further calibration.
The calibration effort undertaken by SEFTC involved evaluating the initial estimated travel patterns, before undertaking regional calibration, against model calibration targets to understand where adjustments needed to be made. SEFTC developed the model calibration targets using the NHTS survey and supplemented with other sources. They evaluated the targets for reasonableness by comparing the targets to other regions. Once the model was calibrated, SEFTC assessed the magnitude of the constant or parameter adjustments that were necessary to match the regional targets to ensure that adjustments were not too large.
The SEFTC presentation discussed the calibration effort and results for the auto ownership model and work tour mode choice model, average trip lengths resulting from location models, the calibration effort related to the daily activity pattern model, and model time-of-day comparisons to observed data resulting from departure time-of-day models. SEFTC also presented the results of a number of sensitivity tests that were undertaken to analyze model reasonableness and the model's ability to handle policy scenarios.
The 2010 American Community survey was used as the regional data source for calibration of the auto ownership model. Before calibration the model over-estimated zero-vehicle households, but otherwise performed well against the observed data. The additional SERPM constants that were added to the model, in addition to the SANDAG constants, were not excessive.
In contrast, the work tour mode choice model did not compare well against observed data before calibration was undertaken. This was not surprising given the different transportation options available in SEFTC compared to SANDAG. Significant adjustments to mode constants were required to more closely match observed data.
Before regional calibration, the work and school location models over-predicted short-distance locations and under-predicted long distance locations. In contrast, all other location models required adjustments to shorten the distances.
The work location model was calibrated for both full-time and part-time workers, income-level, and auto ownership, but did not include adjustments by occupation. These overall adjustments were relatively small. Additional County-to-county distance terms were added to the model to better match within- and inter-county home to work travel. Within the work location model presentation there was a discussion among SEFTC and the panel on the use of shadow pricing. It was noted that shadow pricing is not transferable between regions and that it is necessary to recompute the shadow pricing strategy for each forecast year. The shadow pricing implemented involved double constraining the work location model and matching worker occupation on demand side to occupations on the supply-side for each forecast year. SEFTC implemented manual adjustments to shadow pricing on a district basis rather than completely allowing the shadow pricing formula to fully adjust the model.
The school location model was also adjusted to lengthen school location distances and also included shadow pricing based on school enrollment data.
The panel noted that k-factors and special generators can be misused to over-specify the model, but careful, targeted, use of k-factors and special generators can be beneficial. One panelist noted the location choice model should be SEFTC's first priority of their model improvement efforts.
Without regional calibration the daily activity pattern model did a reasonable job matching observed percentages of mandatory, non-mandatory, and home activities for population segments that composed a high share of the total population (e.g., full-time workers, non-working adult, non-working senior). The population segments requiring larger adjustments during calibration were for smaller population segments such as part-time workers, university students, and pre-school children. Once calibration was undertaken both mandatory and non-mandatory tour frequencies compared very well to observed data.
The work-tour departure from home and arrival to home time-of-day distributions compared closely to observed data without further regional calibration. The shopping-tour departure time observed data tended to be more "lumpy" (i.e. small spikes in travel) than the model results. The calibration effort focused on trying to better match these spikes in travel. As mentioned in Section 4.1.2, the model, when compared to observed VMT, over-estimates morning peak travel and under-estimates mid-day travel. There is a desire to further adjust the time-of-day models to move more travel from the peak periods to the mid-day periods, but there is a concern that the existing observed data is not able to provide reliable calibration targets. Therefore, SEFTC is holding off on further calibration until more observed data can be collected.
SEFTC discussed the results of several sensitivity tests that were undertaken using SERPM 8 which produced reasonable results. They conducted a transit fare test by decreasing the base fare by 20% which resulted in a 12% increase in transit ridership. A sensitivity test which extended Metrorail into Broward County resulted in a significant increase in transit trips having destinations consistent with the new alignment. Another sensitivity test doubled the parking costs in the Miami CBD. As expected, overall trips to the Miami CBD decreased by 5%, and trips to other destinations increased. A land-use development test added households and jobs to an area of Palm Beach County, resulting in origin and destinations increasing in that area. A final sensitivity test discussed was an increase in labor force participation for persons 60-75 years old. Overall trip making and average trip lengths for this age group increased. Transit trips for this age group increased significantly.
Upon questioning from the panel, SEFTC reported that validation is an on-going process and that initial results show that screenline reports are good and that highway assignment is marginally better than the trip-based model results. The panel noted that the model is only one year old and that more effort needs to be put into validation before a complete assessment can be made on the model's performance.
The panelists were unable to comment directly on whether the entire model system, or any individual model, was successfully transferred from SANDAG and calibrated to the SEFTC region. One panelist did comment that transferring the model from SANDAG and simply adjusting the coefficients was not useful. The panelist recommended the development of a completely new activity-based model that is fully tailored to the region. Most other panelists believed that is an extreme position, but agreed that more work needs to be put into further calibrating and validating the model before any position can be taken on the success of the transfer. The panelists did note the limitations of developing a disaggregate time-of-day, demand-side, activity-based model and then coupling that model with a static four-time period, supply-side, aggregate assignment model.
The development of the SERPM 7.0 activity-based model has amplified the existing gaps and insufficiencies in the existing data. The last household travel survey was conducted over six years ago and had many issues as described in Section 4.1.3. The data does not account for recent technological changes and travel demand management strategies that have been implemented. The current set of travel surveys and observed data also lack sufficient information on attitudes (e.g., attitudes with regard to transit or managed lanes), mode choices, and willingness to pay.
In October 2014, the SEFTC board approved the development of a five-year Memorandum of Understanding (MOU) to dictate future plans for data collection. Currently, SEFTC has approximately 1.5 million dollars to spend on data collection. They have identified five different data collection efforts they would like to undertake:
SEFTC desires for the household travel survey to collect data on daily activity-travel behavior of both permanent and seasonal residents. The survey should include over-sampling of transit users, toll-paying customers, park-and-ride transit users, renters, zero-car households, and large households. They outlined a desire for the survey to include a GPS sub-sample and to use cellular devices to aid in data collection.
The panel stated that for improved data collection and given recent advances in survey data collection technology, the household survey should be a full GPS survey with validation via prompted recall. It is very difficult to compute trip purpose and mode from GPS data without the use of prompted recall. The panel also stressed that it is very important to ensure that each household has a complete set of high-quality travel records. The focus should be on collecting quality data rather than a high quantity of data records.
SEFTC would like to gather more information on individuals' attitudes toward certain travel modes such as transit and non-motorized transportation and on individuals' willingness to make changes in their current travel behavior. They also want to gauge a user's willingness to pay for faster and more reliable travel options. They, therefore, would like to implement an attitudinal and stated preference survey with a focus on certain corridors that are candidates for travel demand management improvements.
The panel noted that a stated preference survey must also include a revealed preference survey to appropriately interpret the stated preference information. Before developing the survey SEFTC must figure out exactly what they want the survey to answer and then develop the survey around answering that question. The panel recommended caution when interpreting answers to attitudinal questions.
SEFTC would like to collect origin-destination data and questioned the panel on how to utilize existing data collected from cell phones. Most of the discussion related to this topic took place during a post-meeting e-mail discussion. SEFTC during this post-meeting discussion noted their concern about the accuracy, and caveats, of using cell-phone data (e.g., AirSage) to develop observed trip tables.
The panel, in response, noted that data fusion and manipulation are required to use these cell-phone data. Just like all data, one has to understand the limitations and strengths of the data and use it carefully. The panel noted that other practitioners have had success in combining other data sources with the cell-phone data to provide a work around to many of the known limitations of cell-phone data.
SEFTC's proposed freight movement survey would focus on the commodities movement from approximately twenty major ports and freight hubs. They would assess the daily travel patterns of the truck trips and gather economic information from the establishments.
The panelists questioned the benefit of doing a freight survey that included a very low sample size of only twenty establishments.
SEFTC specified that the previous survey was conducted in 1999. It was an intercept survey in hotel lobbies and asked respondents to recall the previous day's travel patterns. The data was then used to develop a visitor travel model. The survey missed visitors not staying at hotels. SEFTC would like to update the visitor model with new survey data. The visitor survey would collect data at hotels and major attractions during the high tourist season (i.e. winter).
The panel noted that cell-phone data may be able to provide information on the magnitude and temporal patterns of visitors.
