The term "model calibration" is the process of adjusting parameter values until predicted travel matches observed travel demand levels in the given region. The term "model validation" is the process of comparing the model predictions with information other than that used in estimating the model. Model calibration and validation data should be obtained from different sources than the data used in estimating model parameters. As a result, one needs to identify unique sources of data that can support model calibration and validation. For the purpose of this study, calibration and validation data are those data that can be used to compare with model predictions to determine the reasonableness of the model parameters. Model calibration and validation data also are used as a means to adjust the model parameter values so that model predicted travel match observed travel in the region. This is especially important when applying nationally derived model parameters to a specific region.
State vehicle registration databases often indicate whether registered vehicles are used for commercial purposes. These databases typically show vehicle weight classes, but not service use. Service use can be inferred based on vehicle make/model, weight class, owner, and possibly other data. However, this requires considerable data processing. Many states' databases also do not include odometer readings.
Vehicle registration databases that are maintained by a state can yield useful information on the number of commercial vehicles existing within a particular geographic area. For example, the California Energy Commission has been working with the California DMV and other agencies since the late 1990s in an effort to clean, organize, and analyze the State's vehicle data. The California DMV employed all key words from the 120-character owner field of each record in the database that reveal any potential business use information. The Energy Commission divided the DMV data into two main groups: 1) light vehicles and 2) medium and heavy vehicles. It further divided the light vehicle category by use, and the medium and heavy vehicle category by body type.
The project team recommends that other states explore and develop the same kind of multi-year cooperative arrangement that exists in California so that, over time, vehicle registration data can be used to support transportation planning - including, but not limited to, the movement of commercial vehicles.
Vehicle registration and new vehicle data also may be purchased from R.L. Polk & Co., a privately owned consumer marketing information company. Polk develops custom reports for customers, providing data by ZIP code, Metropolitan Statistical Area, county, state, or for the entire United States.
Vehicle registration data for New York State are available at their web site (New York State Department of Motor Vehicles 2001). These data are not as detailed as processed California DMV.
An independent regionwide estimate of vehicle miles traveled (VMT), based on traffic counts and roadway miles, can be used to validate the base year assignment of commercial vehicles produced by a travel demand model. These traffic counts are collected in most urban areas as part of the ongoing transportation planning process and are used to validate the passenger portion of urban travel demand models. In addition to any counts that might be undertaken for planning purposes, state departments of transportation are required to include Annualized Average Daily Traffic Counts and mileage for all roadways, based on a statistical sample, for each urban area as part of their annual Highway Performance Monitoring System (HPMS) submittal. The HPMS VMT can be summarized by functional classification of highways and by area type and compared to the urban area model volumes by functional classification and area type. When using HPMS estimates of VMT, it is important to understand that VMT is for all roadways, including local roads. Travel demand models, in contrast, generally do not include these local roads so this comparison should consider an adjustment for them to allow the comparison of the total observed and estimated VMT.
Generally, traffic counts are collected and VMT is calculated either for all vehicles or for vehicles classified by axle configuration. Traffic count information is predominately collected by Automatic Traffic Recorders (ATR) and thus will rarely include any other classification of commercial vehicles. That information will typically be based on a visual identification of commercial markings on the vehicle or a visual observation of the commercial registration plate.
HPMS estimates of percentages of single unit and combination trucks, based on ATRs, can be used to develop VMT for these types of trucks. Not all commercial vehicles are included in these classes, and intercity freight trucks that are excluded from the definition of urban commercial vehicles are responsible for a considerable portion of the truck travel on higher functional classes. Nevertheless, HPMS estimates of truck VMT can be used to validate commercial vehicle models. It should be noted, however, that the HPMS values for trucks are based on statistical samples. Thus, the "observed" truck VMT is in reality an estimate.
Based on accepted standards for model validation, modeled regional VMT should generally be within five percent of observed VMT.7-1 When the regional models are used to track VMT for air quality purposes, the Environmental Protection Agency requires that estimates be within three percent. However, these estimates are for the total of all vehicles irrespective of vehicle type. If commercial vehicles generally represent 13 percent of total VMT, and if a travel demand model's estimate of commercial VMT is within five percent of that value, it would be consistent with the overall validation standards.
In addition to validating modeled VMT to observed VMT by functional class, it is customary to use measures and VMT per person or per household to assess the reasonableness of urban models. Reasonable ranges of total VMT per household are 40 to 60 miles per day for large urban areas and 30 to 40 miles per day for small urban areas. If one applies the 13 percent of total VMT that is estimated for commercial VMT in this report to these household ranges, then the VMT per household for commercial vehicle demand would represent five to eight miles per day for large urban areas and four to five miles per day for small urban areas.
Travel demand models are validated by comparing observed versus estimated traffic volume on the highway network and by comparing summations of volumes at both cordons and screenlines. As described in the Task 3 Report, vehicle classification count data, which classifies vehicles according to the 13-axle-based classes of the Federal Highway Administration (FHWA), is generally available from state departments of transportation for sampled sets of streets and highways. For the 13 classes, the information includes counts by location, by hour of the day, and by date. In summary format, this information generally presents truck volumes (defined as FHWA Classes 5 through 13, six tires and above) and occasionally includes buses (FHWA Class 4). Four-tire pickup trucks, vans, and sport utility vehicles (FHWA Class 3), are usually included with passenger cars.
The project team expects that the network-based quick response techniques for developing commercial vehicle models will include methods to convert commercial vehicle trip tables into assignments of commercial vehicles by type (auto and truck at a minimum). These vehicle classification counts can be used to compare the observed auto and truck counts (and shares by vehicle type) with the estimated auto and truck volumes (and shares) produced by the urban area model. These vehicle assignments will include both personal and commercial vehicles, derived from both personal and commercial models, so calibration adjustments deemed necessary from these comparisons might be required for either the personal or commercial models or both. The project team does not recommend that vehicle classification counts be used to evaluate individual count locations, but that they be summarized by functional class, area type, or screenline.
Traffic counts may include truck "counts" for locations that are in reality estimated from the actual percentages of truck or total vehicles observed at nearby vehicle classification count stations for the same class of roadway. Care should be taken in understanding the source of truck "counts" used in validating commercial vehicle models. In many cases values for truck "counts" given at the stations comprising cutline, screenlines, and cordon lines will be derived, not observed. Since estimates of commercial vehicles derived from these estimated vehicle counts would effectively be estimates based on estimates, they should not be used for statistical analysis but rather for (cautious) qualitative comparisons.
7-1. Barton-Aschman Associates, Inc., and Cambridge Systematics, Inc., Model Validation and Reasonableness Checking Manual: Travel Model Improvement Program, Arlington, Texas, 1997.
