The NYBPM is a complex model, and the New York metropolitan region is one of the most complicated regions in the world. Given this complexity, the peer review panel had to prioritize the list of topic areas and while some topics received extensive consideration, others received relatively little. Further, advanced models are full of nuanced detail and the substance of the peer discussion remained at a relatively high level in order to begin to address most of the topics identified in advance by NYMTC staff. The panel intentionally avoided being overly prescriptive, in deference to agency staff.
While the panel made numerous recommendations for model improvements that NYMTC should consider, it was important for the panel to point out that these recommendations did not necessarily imply greater complexity in each case. The panel felt that NYMTC should first aspire to handle its basic modeling responsibilities well and the agency should have reasonable expectations around the accuracy of the model at different resolutions. The panel was pleased with NYMTC's ongoing data collection plan and felt strongly that careful observation and data analysis should guide the model design as much as possible, in contrast to an overly specific design not informed by data.
The following text summarizes the panel's discussion on the topics of interest to NYMTC and the audience. The summary of this discussion follows the panel's final presentation back to NYMTC, with additional initial feedback on administrative topics and analysis needs.
One important issue raised early in the meeting revolved around the time it takes to produce a useful result from NYBPM once a request is made of the modeling staff. NYMTC's Executive Director raised this issue, as he is concerned that modeling analyses take too long.
The panel discussed this issue at some length, and the clear consensus was that NYMTC's experience (approximately 1 month to complete an analysis) was not atypical or unreasonable given the size and complexity of the region and the model. One month to do a thoughtful and accurate model run is consistent with experience at peer agencies.
The peers recommended that NYMTC consider appointing one person to interface with those who need analysis to provide:
The peer panel recommended that NYMTC staff work carefully to identify the real bottlenecks where time can be saved when using the NYBPM. Issues such as network coding, review/summary of forecasts, iterating on an analysis, translating planning questions into the modeling terms, conflicting priorities on staff time, among other things, can all be remedied but with different potential solutions.
Advanced models require sufficient in-house technical proficiency, regardless of who is using the model. The panel recommended that NYMTC consider appointing one person to interface with those who need analysis, as a means to efficiently and clearly work around issues associated with forecasting. The panel recommended that NYMTC consider ways to participate actively in the upcoming model development to enable better understanding and use of model. This involvement can take many forms but ideally NYMTC staff would work alongside the consultant team responsible for the bulk of the work (as feasible), because ultimately the agency will be best served to the degree that NYMTC staff develop a thorough understanding of the development of NYBPM. Finally, the panel recommended that NYMTC continue its model user group and work towards effective knowledge sharing and enhancement.
General Comments
Good forecasting requires the proper perspective, in addition to the necessary technical knowledge. The panel advised NYMTC staff on the importance of having the proper perspective on accuracy of the model at different resolutions. Instead of trying to solve every modeling challenge with NYBPM 2.0, the panel cautioned NYMTC to cover the necessary bases well.
The panel recommended that NYMTC let careful observation/data analysis guide the design of NYBPM 2.0. The panel sees regional models as (ideally) an effective starting point for an analysis, but there may be a need for focused refinement in any given study. The panel felt that certain types of operational flexibility desired (variable choice sets, sub-area extractions) is not practical at this time.
Prioritized Analysis Needs
The panel agreed with NYMTC that the design for NYBPM 2.0 should be based in part on key analysis needs envisioned over the next several years. The panel took a somewhat different tack than NYMTC in describing the vision for NYBPM from an application standpoint. Instead of trying to design a model that would be usable for all macro and micro analyses, the panel felt that NYMTC's core analysis needs are:
The first order of business for NYMTC is to ensure that the NYBPM is useful for regional planning and NYMTC's MPO responsibilities. The above list represents the panel's view of the most important policies and market segments for the MPO to address reliably in the model. The panel felt strongly that NYMTC should focus on achieving a model that handled the things in the above list well at a regional scale, and only then, consider use of the model for more refined analysis needs. The priority for NYBPM 2.0 must be to ensure that regional travel patterns are modeled reliably for all major traveler segments, in response to the most critical set of transportation policies and investments.
Following two days of discussion and consideration related to data availability, analysis needs and model capabilities, the peer review panel identified a list of priorities for NYMTC to focus on for NYBPM 2.0. The priorities identified by the panel are provided in the context of the need for a model that is more useful for both regional and corridor level forecasting. Implicit in all of this is the need to do a careful calibration and validation of both the demand and supply side components, and to ensure that the model is defensible and useful to the MPO and its planning partners.
Hierarchical geographic resolution
To represent land-use and localized accessibility, the panel recommended a hierarchical geographic structure that ranges from the parcel to the TAZ. Different steps in the model would be performed on the appropriate geography based on how much detail is needed and processing times. Parcels would be ideal for representing household and activity locations, but the panel recognizes that this might not be possible in the short-term given the scale of the region and jurisdictional challenges.
Better, more detailed, more disaggregate land-use data
Related to the geography, the panel recommends that NYMTC invest significant effort in building a detailed small area land-use and socioeconomic database. While the panel feels that a complete land-use model for the NY metropolitan region is not yet realistic, this land-use and socioeconomic database is an important step towards land-use modeling and more geographically explicit travel modeling.
Networks
The roadway and transit networks for the NYBPM are not on the same base map currently and the panel recommends that NYMTC change this so that interactions between autos and transit can be more easily captured. Further, the panel recommends a region-wide non-motorized network to better represent non-motorized accessibility.
Advanced daily pattern models
The panel feels strongly that with more detailed socioeconomic data, a more robust population synthesis is possible, and from that, NYMTC can better represent key travel markets and realistic intra-household interactions. Additionally, particularly in Manhattan, the visitor market is critical to represent since it constitutes a significant share of travel.
Temporal resolution
Time-of-day modeling was identified as a critical issue by NYMTC, and the panel agrees. The panel feels strongly that NYMTC should build a fine-grained demand model, but with some hesitation recommends a fine-grained regional network supply model. The panel is endorsing the idea that NYMTC investigate the feasibility of "planning-level" dynamic traffic assignment (DTA) models with a 1-hour resolution as a first step towards a fully-fledged meso-level DTA.
Improved destination choice models
With respect to demand modeling, the panel placed the utmost importance on properly modeling and validating destination choice. NYMTC's ability to validate the mode choice and assignment models hinges on getting overall travel patterns right by time-of-day. This requires collecting sufficient data (HH diary, cordon, river crossing, establishment surveys) and mining that data to understand observed trip-making and what makes specific destinations unique along with other aspects of the model (attractiveness, time-of-day, typical visitors, mode choice and accessibility, etc.).
Reconsider mode choice sets
The panel feels that the available modes should be reassessed, particularly with respect to transit. Specifically, the panel listed as priorities an investigation to first understand kiss-and-ride (drop off) access to transit, and to evaluate what mode characteristics affect demand (as opposed to mode technologies). These investigations might lead to redefined transit choice sets. Additionally, capacity constraints are a big issue in the region, both on transit vehicles and at stations (parking). These capacity constraints affect time-of-day choice, route choice and mode choice.
The panel urges NYMTC to focus on the above priorities, but believes that other potentially good ideas may not be feasible for NYBPM 2.0. These longer-term priorities are:
In a few instances, the panel is recommending important steps in these longer-term directions. While the land-use model is not a priority in the panel's opinion, significantly improved parcel-level land-use and socioeconomic data is a very high priority. Similarly, the panel is recommending exploring the feasibility of planning-level DTA models.
In the other cases, the panel either feels that NYMTC is not ready to undertake these efforts due to data limitations (non-recurring congestion, freight, parking location choice, DTA), significant methodological challenges (non-recurring congestion, freight, DTA), or relatively little need for improvement due to a small share in traffic (external travel).
Freight modeling was listed earlier as a prioritized analysis need by the panel, but while that may be true, the reality is that freight modeling requires a substantial investment in data and model development, so it falls to a longer-term priority.
