Updated December 2002
The process of validating and calibrating a travel model is an iterative process that takes place throughout model development. Ideally observed data is available to the analyst for making these checks. However, even in the absence of local data model validation and reasonableness checking can still take place throughout the model stream. These reasonableness checks help to assure that the model is performing within acceptable standards. The first part of this chapter provides some guidelines on performing validation and reasonableness checks for travel models. The recommendations are based on the TMIP manual on model validation and reasonableness checking prepared by Barton-Aschman Associates, Inc. and Cambridge Systematics, Inc., February 1997.
The process of validation and reasonableness checking throughout the model development process will not guarantee model calibration at the end of the process, especially in the absence of locally collected travel behavior data. The second part of this chapter covers several tips on model calibration. It is important to note that several factors affect calibration, some globally and others only change particular elements such as direction of flow or length of travel. Calibration is the delicate procedure of matching the modeled traffic volumes to the counted traffic using the simplest possible process.
The performance checks discussed below are suggested as a means to assure the development of models that are performing within reasonable standards. It is possible to "calibrate" a model within standards for observed ground counts and yet still have a model that is not actually performing within reasonable standards. These checks will help you to assess the reasonableness of your model outputs.
Errors in the land use data are generally the most common error in travel models. Care should be taken to assure that these errors are identified prior to model development. These checks should be performed at the zonal, regional, and aggregate levels. Basic checks should include:
Errors in the highway network are another common source of error in travel models. The following checks are recommended for the highway network:
The trip generation model is made up of several submodels. The results from each of these models should be evaluated for reasonableness prior to proceeding to trip distribution. The calculations suggested here could be performed in TransCAD or MS Excel. The chapter on Trip Generation discussed reasonable ranges for trip productions per household and per capita.
One of the major inputs into the trip distribution model is the travel impedance matrices. The impedances reflect the minimum path for each zone-to-zone interchange. Several techniques exist for checking the reasonableness of the minimum path values. The first of these techniques involves calculating the implied speeds for each zone to zone interchange. The implied speed is estimated by dividing the skimmed highway distance by the highway travel time and converting for units:
Sij = speed from zone i to zone j in miles per hour
Dij = shortest path distance from zone i to zone j in miles
Tij = shortest path time from zone i to zone j in minutes
60 = conversion of minutes to hours
After the above calculation is made, the results should be reviewed to determine if the maximum and minimum speeds by interchange are reasonable. A frequency distribution of the speeds for all interchanges is also a useful reasonableness measure. These calculations can be made in TransCAD using the matrix functions.
Estimated trip lengths should be compared to the observed trip lengths developed using the impedance matrix and the trip tables for the expanded travel behavior survey data (if available.) The following checks are recommended:
If observed travel data is not available, these checks should be made against reasonable estimates for your area based on your knowledge and field review.
You should always make sure that your friction factors go out at least to the maximum zone-to-zone travel time in your network. You may want to consider extending your friction factors out 2 - 5 minutes beyond your maximum zone-to-zone travel time to account for congestion if you are doing a traffic assignment that accounts for congestion.
The intrazonal travel time is found in the diagonal of the travel time matrix and represents the travel time for trips that are made within the zone. This value is generally calculated using the "nearest neighbor" technique. This technique calculates the intrazonal travel time as one-half the average travel time to the adjacent zones.
When calibrating and running our travel demand models, we are accustomed to thinking about the trips between zones because these are the trips that show up on the network. These trips are referred to as interzonal trips. However, it is also important to consider the trips that stay within each zone, or the intrazonal trips. The intrazonal trips are important because they directly affect the volumes on the network:
The number of intrazonal trips can be adjusted by adjusting the intrazonal travel times in the trip distribution process.
Validation tests for highway assignment are recommended at three levels: systemwide, corridor and link specific. Systemwide checks are generally made on daily volumes. Systemwide checks include vehicle miles of travel (VMT,) vehicle hours of travel (VHT,) cordon volume summaries, and screenline summaries. In addition to checking summations of VMT, VHT, and volumes, the average VMT and VHT per household and person should be checked. Corridor specific problems are generally identified through observation of cutline volumes or volumes on major facilities.
Validation of the model using VMT addresses all major steps in the travel demand models including trip generation (the number of trips), trip distribution (the trip lengths) and assignment (the paths taken). VMT validation is particularly important in urban areas that are designated by the Environmental Protection Agency (EPA) as non-attainment areas. The following VMT checks should be made:
Reasonable ranges of VMT per household are:
Reasonable ranges of VMT per person are:
Little data exists for areas with a population under 50,000 so these measures should not be used for North Carolina's definition of a small urban area with a population under 50,000. For these areas the census journey to work data and CTPP data may serve as an acceptable secondary check.
Table 1. Typical Urban VMT by Functional Classification
|Functional Classification||Urban Area Population|
|Small (50-200K)||Med. (200K-1M)||Large (>1M)|
Source: Christopher Fleet and Patrick DeCorla-Souza, Increasing the Capacity of Urban Highways - The Role of Freeways, presented at the 69th Annual Meeting of the TRB, January 1990
After validation of the VMT, the next level of validation of the highway assignment is the comparison of observed versus estimated traffic volume on the highway network:
C = count
M = model
N = number of counts
Checks of highway skims should include the following:
Table 2. Percent Difference Targets for Daily Traffic Volumes by Facility Type
|Facility Type||FHWA Targets (+/-)||NCDOT Targets|
Source: FHWA, Calibration and Adjustment of System Planning Models, 1990
Table 3. Percent Difference Targets for Daily Volumes for Individual Links
|Average Annual Daily Traffic||FHWA Desirable Percent Deviation||NCDOT Targets Desirable Percent Deviation|
|1,000 - 2,500||47||50|
|2,500 - 5,000||36||30|
|5,000 - 10,000||29||25|
|10,000 - 25,000||25||20|
|25,000 - 50,000||22||15|
Source: TMIP, Model Validation and Reasonableness Checking Manual, 1997 and NCDOT, Model Specifications
Many factors throughout the model development process affect calibration. Some items globally affect travel flow while others only change its direction or length. Calibration is the delicate process of matching the modeled volumes to the counted traffic using the simplest possible process, the "law of parsimony". It is possible to over calibrate a model. An "over calibrated" model is one that has used parameters in calibration that will not hold up with time or can not be predicted into the future with any degree of accuracy. Below is a discussion of some of the major parameters that affect travel flow. However, as previously noted most significant calibration problems generally stem from errors in the employment, housing, or network data and from errors in the traffic counts. These items should always be checked first.
ITEMS BELOW: Generally speaking items #1 through #4 control the total number of trips on the network and #5 through #9 control the direction of travel/trips on the network. Item #10 provides a false reading of the accuracy of the modeling process. Item #11 affects the complete process and if not corrected may lead to compensating "errors" made with the other variables to provide calibration. Two wrongs may provide calibration but as the old adage says, "two wrongs do not make a right."
External travel refers to all travel crossing the Planning Area Cordon. The percent split between through and external-internal/internal-external travel at the cordon of the planning area boundary is an important calculation that can affect the number of trips and the VMT. (Herein E→I refers to travel originating outside the planning area with destination inside and I→E refers to travel originating inside the planning area with destination outside.)
COMMENTS: The percent split between thru trips and E→I/I→E travel has an impact on VMT. It takes two external crossings (or trip ends) to make one thru trip. Each E→I/I→E crossing becomes a production with an attraction inside the planning area; consequently, each crossing becomes a trip. Converting thru trips to E→I or I→E trips hence increases the number of trips and may increase VMT.
2. Percent of Total Trips remaining Internal to the Area
The total amount of travel that is generated inside the planning area can be attributed to housing, employment, trucks, commercial vehicles, and taxis. The trips that have both origin and destination inside of the planning area are referred to as internal trips and can be denoted as I→I trips. For a typical urban area in North Carolina generally 80 to 90% of the trips generated inside the planning area stay inside the planning area. The other 10-20% of the travel has destination outside the planning area (I→E trips). Areas with an extremely strong economy may be slightly higher and areas with a weak economy or fewer opportunities to make trips may be lower. Areas that have a tourist driven economy may vary on the low side and areas that are close to other dominate urban areas, i.e., Clayton, may be lower. This is an important determination because the trips that leave an urban area each day are making some of their NHB trips in some other planning area; hence, they become a NHB trip made by a non-resident (NHBSec) in some other planning area. The housing generation rates for areas like Clayton should reflect this behavior. In addition, the I→E trips and the thru trips are subtracted from the total station crossings to determine the E→I trips. E→I trips are used to develop the NHB trips made by non-residents (see item #3.)
COMMENTS: The employment to population ratio can be used to help estimate the internal of total percentage. Most urban areas' employment to population ratio ranges from 40 to 60%. The stronger the economy of an area the more jobs it has for its own residents and the more people it will attract into the area for work (E→I trips). Hence, the E→I component of the station crossings becomes larger than the I→E travel. For example the total trips in an area can be changed by 10-15% by varying percent of internal travel that stays within the planning area.
NHBSec travel are NHB trips made inside the planning area by people coming from outside the planning area (E→I trips). These trips are added to the internally generated NHB (Statewide Planning's Internal Data Summation program handles this step for the analyst.) Generally, 40 to 60% of the E-I crossings make a NHBSec trip. A good rule of thumb is that non-residents will make NHB trips similarly to residents unless the economy of the region suggests a greater propensity for making extra trips within your area. When selecting the appropriate percentage, consider the opportunities to make trips within your planning area. Very small urban areas with low employment will likely NOT generate a high number of NHBSec trips.
COMMENTS: The NHBSec travel is a major component of urban travel. (For example, someone who lives outside of the Raleigh urban area would make an E→I trip every time they make a trip into Raleigh. Any additional trips that they make once inside the urban area would be considered NHBSec trips.) The increase in travel referred to in the comment section of item #2 is caused by more of the internally generated trips staying inside the planning area. This creates more E→I trips, and thus, more NHBSec trips. NHBSec travel generally ranges from 25-50% of the total NHB purpose. NHB trips mainly circulate in and around the employment centers. If your assignments are low in these areas, the NHBSec trip estimate may be the problem.
Changes in the trip generation rates globally affect travel volumes throughout the entire network. Screenline accuracy measures are the first checks to be made for determining the need for generation rate changes. The mathematical average of the dwelling unit trip generation rates as calculated by the IDS program should be similar to the average rate used for other current studies (some adjustments may be required to reflect differences in planning area size, income levels, and economic conditions). In a tourist area a small generation rate may be appropriate for the motel rooms. Major Universities may require a small generation rate for the students that are housed on campus (Please review 10a - University may not be in session for time frame being modeled.)
COMMENTS: Words to the wise, if screenline accuracy is considerably off, do the following: (1.) Check all generation rates and equation coefficients for data entry errors. (2.) Color code highway map to show links where the assignment is 10-15% high or low. (3.) Compute VMT per capita and per person. (4.) Compare mathematical average of the generation rates (IDS) with other current studies.
Generally, the internal travel generated by the housing units is divided into three main purposes - HBW, HBO, and NHB purposes. Some studies also have a home based shop purpose (HBS). The HBW percent generally ranges from 18-25 percent, HBO between 47-58 percent, and NHB between 18-28 percent.
COMMENTS: The sum of the individual percentages for each purpose must total 100%. Consequently, changing the percentages does not change the number of trips created. HBW and HBO trips by definition must have at least one of their trip ends at the home. NHB trips can have a trip end at a dwelling unit but they mainly circulate between the employment centers. Therefore, changing the percentage of the trips generated by the housing units for the NHB purpose will change the direction of travel. Too many trips entering the residential zones and not enough trips circulating in the employment zones could be caused by the purpose percentages.
The equation most often used for small urban studies came from Scot Leftwich's regression analysis using 16 of our O-D surveys as one database. The constant was dropped on this equation and each variable of the equation was divided by the industrial coefficient so that it could be compared to other equations we were critiquing. Scot's equation compared very favorably to a rate equation constructed using ITE's trip generation manual. This equation compares favorably to some of the equations used in other North Carolina cities. It also compares well to equations found in the ITE document "Travel Demand Forecasting Processes Used by Ten Large Metropolitan Planning Organizations, February, 1994" [please note: to compare equations they must be normalized by dividing them by the industrial coefficient].
Y= 1(industrial) + 1.83(retail) + 8.36(hwy retail) + 2.6(office) + 2.55(services) + 0.5(DU)
When using Scot's equation:
a. The Dwelling Unit (DU) coefficient for HBO, NHB and Ext-Int equations should be reviewed. When equations from various sources are normalized 0.5 is a very common coefficient for the HBO and NHB purposes and the same is true for 1.5 for the Ext-Int purpose. If the dwelling unit coefficient for the NHB and HBO is high, too many trips will circulate among the residential zones and vice-versa if the coefficient is too small. An inspection of many of our O-D studies reveals that 70-80% of the external- internal trips by vehicles garaged inside the planning area (I→E) start at the home end. The only way to bring them back to the home end is with the proper coefficient place on the dwelling unit variable in the Ext-Int purpose.
The Industrial coefficient for HBO and NHB equations should be reviewed. By inspection, we have realized that the general equation attracts too many NHB and HBO trips into industrial development. It is recommended that the coefficient for NHB be adjusted into the 0.3-0.5 range and for HBO into the 0.1-0.2 range.
c. Special variables (shopping centers, large regional hospitals, schools, etc.) Occasionally a special variable is needed to attract enough trips into a zone. Large shopping areas are the most likely candidates. Quick Response generation rate (NCHRP #187) or ITE generation manual can be used to help fine tune the appropriate coefficient value when it is deemed appropriate to have a special variable.
Check with areas that have recently conducted travel surveys to use the data analysis as a point of comparison. A Work Place survey is schedule for the Greater Charlotte region in 2003 and this data may be useful in helping us to update our attraction equations. Rates used for the Triangle Regional model are provided below.
COMMENTS: When selecting a regression equation, always consider the equation that was developed for your city if they had an O-D survey, even if it is old. Scot's equation, as well as equations from the Triad or Triangle may be considered as well. Always compare your unbalanced attractions to your unbalanced productions to be sure that you are within 0.9 - 1.1 as mentioned earlier. This can be a good reasonableness check for your equations. Be careful and do not "over calibrate" the coefficient for any variable by inspecting one or only a limited number of zones. The equations must work for all zones and if tailored for any one zone, may cause more overall harm than good. For example, if a zone consists of one particular type of industry, and if this zone was used to fine-tune the industrial coefficient, it may throw off all the other zones' industrial attractions. Do not over use special variables. NO MODEL (AND EQUATION) IS PERFECT! Simpler is better! Peculiar or unusual travel circumstances that exist in the base year may not transfer into the future. For example, Hickory's industrial base consisted mostly of textiles and furniture in the 1970s. Now they have a more varied industrial base. An equation "over" developed for this condition may not work for the design year as well as a more general equation. The next project engineer may not properly apply processes that are too complex.
Trip lengths can have an impact on VMT. For simple environments with one city center, Friction Factors (FF) can be used to change total VMT by approximately 10%. For complex areas with many city centers, the trip lengths can have a significant impact on the assignment for centrally located high-speed corridors. For example, the assignment on I-85 through the Burlington area could be changed 20% by "just" adjusting the trip lengths. The recent calibration of the Triad model was able to change the assignment on a key section of I-40 from 200,000 VPD to approximately 70,000 vpd. VMT was reduced from 36 VMT/capita to 29 VMT/capita. No amount of generation rate changes or equation adjustments would have calibrated some of the key sections of high-speed roadways in the Triad without the trip length adjustments.
FFs are a relative set of numbers that control the length of travel. In most instances they should be a descending set of numbers unless the first few need to be lowered to reduce the number of intrazonal trips. If the generation rate is within the normal range, VMT is abnormally high or low, and most roads are calibrated, the trip lengths may need to be adjusted. Another good check on the appropriateness of your trip length curves is to compare the shape of the curve and the average trip length to known or synthesized data for your area. This is a reasonableness check that basically let's you consider is the distribution seems reasonable.
Network selection can have some implications to the calibration process. The centriods of a network generally represent the local street system. When a "coarse" network is selected the major street system must be overloaded to account for all of the trips being carried by the streets left off the network, this is especially true if intrazonal trips have not been properly accounted for. On the other hand, a "fine" network will present network calibration challenges.
Centroid connections to the network should duplicate the "natural" trip patterns out of a zone. False trips out of a zone may cause calibration problems. Most calibration problems caused by centriods are network calibration problems and not model calibration problems.
COMMENTS: Centroid connector problems will most likely create network calibration problems that may only be localized.
Zones that are too large make it difficult to duplicate the "natural" travel flows from the local street system onto the arterial road system. As a general rule, the more intense the land use development, the smaller the size of the zone should be. Most urban areas have very small zones in the central area and they are bigger in the rural undeveloped areas. When these large zones begin to develop, it becomes hard to duplicate traffic flows onto the system. Future development intensity should also be considered when establishing zone size. A good rule of thumb when considering zone size is that centroid connector loading should generally be less than 10-15K vehicles per day.
When selecting a planning area, one should look for natural boundaries and contain any land mass expected to have urban characteristic during the next 20-25 years. It is also a given that development within the area should have a common attraction to each other. If one planning area infringes on the sphere of influence of another urban area it becomes hard to duplicate the travel flow characteristics.
COMMENTS: Inappropriate planning area boundary can create model and network calibration problems. Zone size problems will most likely create network calibration problems. Centroid loading should be less than 10-15K vehicles per day.
In an urban environment, a considerable amount of travel is routine travel and occurs each weekday on a regular basis (very cyclical). And yet, each day may have a special set of events unique to that particular day. In general, the larger the ADT, the smaller the percentage of variance and visa versa. The smaller volume roadways can be significantly affected by special events. Weather conditions, the economy, special events or attractions, time of year (seasonal), tourism, etc influence traffic counts. Errors in the actual count data created by the counting hardware may also exist. Counts should always be reviewed for reasonableness and if questions exist, a recount should be requested.
Another factor that can influence travel behavior and traffic count data is time frame. The time frame modeled can be average daily traffic or average weekday traffic and both of these will represent a specific period like summer, fall, winter, or spring. Most States model an average weekday for a specific time period. Know what time frame you are modeling. You should decide this before your start so that counts can be ordered for the time period you are modeling.
The FHWA manual "Calibration and Adjustment of System Planning Models, December 1990" states that the number one problem preventing calibration of travel demand models is errors in the housing and employment data. Time spent supervising, monitoring, and checking the data collection is time well spent. The errors that will haunt you are generally big ones, 100 dwelling units mis-coded as 1,000 or 10,000 units, major employment centers not counted, employees coded in the wrong category, etc.
COMMENTS: In the 1984 Hickory study too many trips were assigned to US70 between Conover and Claremont (count 7,500vpd, assignment 15,000vpd). Cause, 400 industrial jobs were coded as highway retail. In the 1990 Hickory study a major shopping center was not open when the traffic counts were made but was open when the employment survey was conducted. This one snafu caused approximately 10,000vpd difference on the main highway serving the shopping center.
Before starting a model, time spent getting to know your area is time well spent. Many calibration problems can be prevented through careful planning and knowledge of the local area. Planning area size, zone structure, count locations, and proper screenline selections are just a few of the mistakes that can be avoided by knowing your area. Getting involved in the data collection to ensure quality control can have tremendous pay back. Reasonableness checking throughout the model development process can lead to smoother model calibration. Once you have processed your data at each step in the modeling process and after highway assignment you should do the following:
This information will enable you to analyze your model and make rational decisions. All of the items covered in this section can play a significant role in the calibration process. Don't over look the importance of reasonableness checks throughout the process. During calibration of the traffic assignment, don't get overly focused on individual links that are out of range, start with the big picture, first looking at VMT, screenlines, and cutlines, and then work your way down to the link level analysis.