Delaware Valley Regional Planning Commission (DVRPC) Peer Review
6.0 Peer Review Panel Recommendations
On the last half day of the meeting, the peer review panel took about one-and-one-half hours in an executive session, closed to all participants of the meeting except for the panel members. The reason for this was to allow panel members to speak freely and openly among themselves while developing formal recommendations. This section details those recommendations of the panel.
The panel made the following recommendations related to DVRPC's modeling plan:
- Document the policy and planning needs of the agency;
- Identify and document the needed tools, surveys, and data and match the needs with the tools;
- Translate the planning needs into a model language;
- Circulate documentation to DVRPC's planning partners and stakeholders; and
- Consider forming a DVRPC model user group. This group would consist of users of the DVRPC model, allowing them to communicate on modeling issues.
During the meeting, information technology (IT) was briefly discussed and DVRPC asked the panel to comment on their modeling group's relationship with their IT group. The following recommendations were made:
- Assert and communicate IT needs. For instance, the new model will require additional hard drive space (and memory and equipment, etc.).
- Refine file management procedures. It would be great for the procedures to be automated if possible. Currently, since only staff from the modeling group run the model, such procedures are less important, but the panel encouraged DVRPC to expand the user group. In addition, a database system that keeps track of model runs would ensure model files stay up to date. Periodically a staff member would need to review the database and identify runs to keep and others to delete.
The peer review made several recommendations related to survey and other data sources as described below.
- Build a better understanding of survey data and expansion factors:
- The appropriate way to expand the data depends on the specific use, and therefore, multiple expansion factors will ultimately be needed; and
- Allowing others (e.g., planning partners) to have access to the data can be useful for identifying oddities in the data, and can help inform expansion and data cleaning procedures.
- Use survey data to answer planning questions.
- Prior to model estimation and calibration, fully analyze the survey data.
- Provide evidence of model validity:
- Via other data sources;
- Look at all facets of the model results, including well-accepted travel behaviors that new data may be able to validate or invalidate; and
- This also can help with imputation where household survey data may not have complete information.
- Continue investigating new data sources, including HERE data and open trip analysis.
- Make it simple. The model needs to be understandable; otherwise it will become a "black box" and will result in credibility issues.
- Check results against trip-based model to identify issues. In particular, look for clear mistakes. There could be differences of opinions on how closely the models should match, but generally the models should tell similar stories on broader issues.
- Write a model calibration and validation plan.
- Ensure accountability of consultants.
- Check results at each stage. It is critical to not wait until the very end of the model development process to look at results, particularly assignment results. Checking model results early leaves open the opportunity for making important changes when something does not work.
- Use the survey (and other observed data sources) to check the model.
- Run tests of the model using alternative scenarios.
- Use the model for storytelling.
- Develop in-house capacity to reestimate logit models, if needed, to interpret and write code in the ABMs software language, and to develop code version control.
- The key here is not to be overly reliant on consultants for these things.
- Join DaySim user group when it is up and running.
While not a formal recommendation made by the panel, they also did recommend that no expectations be set on the retirement of the existing trip-based model. DVRPC should plan on keeping the trip-based model for an extended period of time.
This section describes the panel's recommendations related to other modeling tools discussed during the meeting, including economic modeling tools, land use modeling, transit operations modeling tools, and bicycle and pedestrian planning.
- Develop an in-house model, simple and incrementally improved.
- Retain TREDIS to learn and improve understanding of economic modeling:
- Using it will help your own understanding of the economic importance of different factors. This also will be useful if/when an outsider questions your expertise in economics.
- There is a real need for a loosely coupled land use-transportation model.
- By loosely, this means there needs to be communication of some kind between the models (e.g., accessibility measures in the land use model).
- Need to inventory zoning and local land use outlook.
- Collect and maintain land use data; employment data purchase was useful.
- New, more disaggregate data can be used and compiled, particularly for the ABM.
- Consider simple or direct extensions to the existing models. The connection to the existing model is important, but this will allow for transit forecasting that can be run by nonmodelers and that does not require running the whole regional model.
- Consider and research simple or direct extensions to existing models. While the regional model may not handle bike and pedestrian trips very well, making that connection between the sketch planning tools and the model is important.
- Network modeling in the existing assignment should be improved. This includes the volume-delay functions, using multiclass assignment, and incorporating intersection, left turns, and control device delays.
- Continue the limited subarea testing of DTA.
