Office of Planning, Environment, & Realty (HEP)
The most obvious and important benefit of panel surveys is that they directly measure behavioral change at the level of the individual sample member and thus supply information that cannot be obtained in a cross-sectional survey. By virtue of this feature, they provide a rich source of information that can be used to arrive at a better understanding of the factors that influence and control personal travel behavior. This information is important whenever the purpose of the travel survey is:
The sections below discuss why panel designs are preferable in these situations.
Although cross-sectional data are well suited for assessing current levels of travel and for measuring period trends in population behavior, they provide only indirect information on the determinants of personal travel behavior. Nonetheless, this information, in the form of differences across households or individuals, forms the basis for most predictive models of personal travel behavior. These models assume that household or individual changes in personal travel behavior can be predicted on the basis of cross-sectional differences in behavior across households or persons . Since these models are based on data from a single point in time, they are often referred to as "static" models in the literature. (Models based on panel data are typically referred to as "dynamic" models, on the other hand.)
For illustrative purposes, suppose one wanted to predict how the automobile trip frequency of a one-car household would change if it acquired an additional automobile. In models relying on cross-sectional data, this change in trip frequency would be predicted on the basis of the difference in trip frequencies between one-and two-car households.
This type of inference assumes that several restrictive conditions are met:
In terms of the example above this means:
Recent studies challenge the validity of these assumptions and the suitability of cross-sectional data for predicting changes in travel behavior. A study based on data from the Dutch National Mobility Panel (DNMP), for example, offers empirical evidence that changes in trip frequency and employment status do not occur simultaneously, as assumed in cross-sectional models . In the study, sample members were divided into four groups according to their employment status in the first and second waves of the survey: employed -employed; employed -not employed; not employed -employed; and not employed -not employed. Analysis of changes in trip frequencies within these groups revealed strong inertia effects; trip frequencies did not change very much regardless of changes in employment. The average trip rate of male adults in the "not employed -employed" group, for example, was smaller than the sample average by 0.9 trips in the first wave, and remained smaller than the sample average by 0.7 trips in the second wave. The average rate of male adults in the "employed -employed" group, on the other hand, was greater than the sample average by 0.7 trips in the first wave, and 0.9 trips in the second wave. In other words, trip rate changes from the first to second wave were about equal for both groups. These results suggest that changes in employment status do not immediately produce a drop or gain in trip frequency as assumed in models of cross-sectional data. A simple example that compares regression coefficients from static and dynamic models of travel behavior also illustrates how the predictions of travel demand models may be affected when they are based on cross-sectional versus panel data . For the example, a static model was fit to the cross-sectional data from a panel survey conducted in South Yorkshire, England, between 1981 and 1984. A dynamic model (a model based on individual changes over time) was fit to the panel data from the same survey. The regression coefficients from both models are shown in Table 2.
Table 2 The Effect of Income on Three Travel-Related Measures
in the South Yorkshire Panel, 1981-1984
*the coefficient was non-significant
The dynamic coefficient from the regression of changes in total trip rate on changes in income is within the range of the two static coefficients. The static coefficients, however, differ from one another and show that the assumption of stability does not hold in these data. Moreover, the panel results for bus usage and car ownership are quite different from the cross-sectional results. The model based on cross-sectional data predicts no change in bus usage and an increase in car ownership as income increases. The model based on panel data, on the other hand, predicts an increase in bus usage and no change in car ownership with changes in income.
The panel results for income and bus usage seem out of line with results from other studies and with what one would expect to find. However, a plausible explanation for the finding lies in the relationship among income, bus usage, and employment status within the group of panel members who typically travel to work by bus. Within this group, changes in employment status are likely to produce rather marked increases or decreases in bus usage and income, depending on the direction of the change in employment status. If such changes in status occur between the waves of data collection, then the data from this group would exhibit a relatively strong relationship between changes in income and changes in bus usage. When combined with data from the other panel members- whose income and bus usage may be changing in other ways- these changes could produce a small effect of income on bus usage, such as that observed in this survey.
Repeated cross-sectional designs yield measurements of period trends in population behavior, but they do not further our understanding of why the changes occur. Panel designs yield similar information but also allow for analysis of the underlying causes by providing information on the changes occurring to individual members of the population. Aggregate measures of change tend to mask these changes and often lead to erroneous conclusions of stability, even when the behavior of the individuals is volatile.
Cross-sectional data from the South Yorkshire Panel Survey of car ownership, for example, show modest increases in net car ownership, ranging from about 2 to 6 percent during each time interval (see Table 3) . The panel data, on the other hand, indicate that ownership levels were quite volatile during this time period. Between 21 and 26 percent of the population changed their level of ownership during each time interval. Moveover, between 13 and 15 percent acquired additional automobiles, while about 7 to 12 percent reduced their level of ownership. Since these changes differ in direction, they tend to cancel one another out when measurements are based on cross-sectional data. For this and other reasons, aggregate measures of change tend to provide an inaccurate picture of changes occurring to members of the population.
Table 3 Changes in Car Ownership of the South Yorkshire Panel
Cross-sectional surveys provide sufficient data for examining travel behavior at a single point in time and for analyzing and modeling differences in travel behavior across individuals, but they reveal very little about the dynamics of personal travel behavior. Cross-sectional data, for example, show that public transportation usage is negatively correlated with automobile ownership, but they can not predict how much an individual's usage might change following a change in car ownership. Data from six waves of the DNMP illustrate this point . Static correlations between public transport usage and car ownership are in the order of -0.20 suggesting that an increase in car ownership will lead to a moderate decrease in use of public transit. The dynamic correlations are in the order of -0.05 showing that changes in car ownership have little effect on an individual's use of public transportation.
Before-and-after designs are commonly used in transportation surveys to study the impact of transport services and policy on travel behavior, attitudes, and safety. In studies of this type, the phenomenon of interest is measured before and after a change in services or policy to assess the impact of the change. Examples of such studies include:
In one such survey conducted in Almere, Netherlands sample members were asked to report their mode of transportation to the workplace, along with other information- such as car availability- before and after the opening of a new railway line [6, 7]. In a similar study conducted in San Diego, sample members were asked to report their travel behavior and attitudes before and two times after a roadway for high-occupancy vehicles was opened on Interstate 15. The second and third waves of data were used to evaluate short-and long-term effects of the roadway on personal travel behavior .
In assessments of this type, the advantages of a panel design are clear. In comparison to a repeated cross-sectional design, a panel design:
To illustrate the type of information that would be lost if a repeated cross-sectional survey was adopted instead, data from the Almere study are shown in Tables 4 and 5. Table 4 displays the type of aggregate-level information that would be obtained in a repeated cross-sectional design and in a panel design. It shows the number and percent of sample members who traveled to work by car, train, or bus before and after the opening of the new railway line. According to these data, the level of travel by car remained the same across time, while bus use substantially declined after the opening of the railway.
Table 4 Mode of Transportation to the Workplace Before and After the Opening of the Rail Line
The data in Table 5, available only in a panel survey, provide a more complete picture of the effects of the railway line on travel patterns. Of the 320 individuals who originally traveled to work by car, 27 or (about 8 percent) switched to train while 5 (or roughly 2 percent) switched to bus. But, more surprisingly, 33 (or roughly 21 percent) individuals who originally traveled to work by bus chose to drive to work after the opening of the line. Without the benefit of a panel design, these turnovers in mode use would be missed.
Table 5 Change in Mode of Transportation
In addition to providing for direct measurement of change, panel surveys offer a number of other practical and analytical benefits. These benefits include:
Statistical efficiency. When the same sample of units is used in all time periods, estimates of change over time become more precise. This is because in panel surveys comparisons across time periods are free from some of the effects of random sampling error. As a result, panel surveys require a smaller sample size than repeated cross-sectional surveys to measure aggregate change with the same level of precision. For simple statistics like averages or proportions, the reduction in sample size depends on the correlation over time in the variable of interest (for example, the number of cars available to the household).
If R is the correlation between the measurements of a variable over time, then the variance of the estimate of change (the difference between measurements at time 1 and time 2) is reduced by a factor of 1- R , while the standard error of the estimate is reduced by a factor of . This means that separate cross-sectional samples of size nc , where
are required to measure change with the same level of precision as that provided by a panel sample of size np .
In cases where the correlation between measurements is high, the gains in efficiency can be quite large. Kish, for example, reports the results of a survey on car ownership in which the measurements correlate 0.8 over time . In this case, the variance of the difference between the measurements is reduced by a factor of 0.20, the standard error by a factor of . In terms of the formula above, this means that the cross-sectional samples must be 1/or roughly 2.24 times larger than the panel sample to yield estimates of equal precision as measured by their standard errors.
Timely source of travel information. Once a panel survey is in place, it can serve as an ongoing source of up-to-date information about travel behavior. New data can be examined as they become available, and questions can be added to the survey instrument as needed to address current concerns and policy issues. It is often far easier and faster to add supplemental questions in an existing panel than to mount a whole new survey to acquire the same information. The extent to which a panel survey will serve this purpose should be decided in advance of the survey since it may affect the content and length of the core questionnaire.
Cost savings. Because panel surveys measure the same sample across time periods, sampling and respondent recruitment costs are considerably lower than those for repeated cross-sectional designs, where a new sample must be drawn and recruited during each time period. In later waves, these savings may be offset somewhat by the extra effort required to "feed and maintain" a panel sample. However, if the design includes only a few waves, a panel survey should cost considerably less than a repeated cross-sectional survey with the same number of measurement periods. The savings include some or all of the instrument development and pretesting costs, the costs of screening and recruiting the initial sample, and much of the costs of developing systems for monitoring the field effort and processing the data. Depending on the exact design, the costs of reinterviewing a panel may be 20 to 80 percent less than the costs of obtaining the same information from a new sample. Lawton and Pas estimate that the cost per sample household in subsequent waves of a travel panel survey is about 50 percent of the cost in the first wave .
If panel surveys have advantages over cross-sectional designs, they also have certain drawbacks . These include:
In a panel survey, the effects of nonresponse in the initial wave of data collection are compounded over time as initial respondents drop out in subsequent waves. The cumulating impact of nonresponse across waves of data collection is called panel attrition. As panel attrition increases, the sample becomes less and less representative of the cohort it was selected to represent.
To illustrate the cumulative effects of nonresponse, Table 6 shows the number of respondents who participated in each of the first four waves of the Puget Sound Transportation Panel (PSTP). About 33 percent of eligible members in the original sample took part in the first wave of data collection. Only about 55 percent of those original respondents completed the fourth wave of data collection in 1993. In other words, only 18 percent of the original sample of eligible members remained in the survey after the fourth round of data collection.
Table 6 Number of Respondents and Percent of Original Sample in the First Four Waves of the Puget Sound Transportation Panel
Similar information for the Dutch National Mobility Panel is shown is Table 7. After the first year of data collection, which consisted of two waves, the DNMP retained about 58 percent of the original respondents. By the end of the survey, only about one third of the original respondents remained in the panel.
Table 7 Number of Respondents and Percent of Original Sample in Selected Waves of the Dutch National Mobility Panel
Reporting errors can also increase among those who remain in the panel over time. There are several terms for such time-in-sample effects, including conditioning , rotation bias , and panel fatigue . All three terms refer to the same general phenomenon: respondents tend to report fewer trips, spells of unemployment, household repairs, and consumer purchases in the later rounds of a panel survey than in the earlier ones. This pattern of reporting is evident in data from the DNMP [17 ]. According to a regression model fit to those data, participants in the first wave reported about 2.27 fewer trips per week than expected, while participants in the seventh wave reported about 8.35 fewer trips per week than expected. Table 8 shows how the magnitude of underreporting increased over time as participants completed more rounds.
Table 8 Estimated Number of Unreported Trips Per Week by Wave
In some cases, a drop in reporting can be observed within a single round; for example, respondents tend to report more consumer purchases in the first few days of keeping a diary than in the last few days, even in the initial wave of a panel survey. A number of studies have examined whether respondents in travel surveys display this pattern of reporting as they complete multi-day diaries. The results of the studies are mixed. Analysis of 1984 data from the seven-day travel diary of the Dutch National Mobility Panel, for example, revealed that trip reporting decreased over time largely because more respondents reported no trips at all over time . Analysis of data from a three-day travel survey conducted in Seattle in 1989, on the other hand, found no evidence of decreased levels of diary reporting in the second and third days .
Another kind of reporting error may affect panel surveys that collect information about the entire period between rounds of data collection. In such surveys, respondents might be asked to report the amount they earned in each month since the prior interview. In these types of designs, there is a tendency for respondents to report changes as occurring at the beginning or end of the time interval between rounds rather than at other times covered by the interview. Changes in salary, for example, seem to cluster in the first month covered by the interview. This pattern of reporting is called the seam effect; it reflects the effect of faulty memory for when changes took place .
In summary, then, panel designs can compound the problems of nonresponse bias and reporting errors that are also found in cross-sectional surveys.