Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

Office of Planning, Environment, & Realty (HEP)
Planning · Environment · Real Estate

HEP Events Guidance Publications Glossary Awards Contacts

An Introduction to Panel Surveys in Transportation Studies

APPENDIX: COMPUTING WEIGHTS IN PANEL SURVEYS

A. 1 WEIGHTING DATA FROM THE INITIAL WAVE

Step 1: Compensating for differential selection probabilities.

Typically, the initial or base weight (Wli) for a case (e. g., a sample household) is calculated as 1i the inverse of that case's selection probability (Pri):

Wli = 1/Pri

All eligible selections- whether they went on to complete the survey or not- should receive a base weight. The selection probability (or sampling rate) is the proportion of the population selected for the study. If the sample is purchased from a vendor, the vendor should provide the selection probability for each of the sample telephone numbers. In a random-digit dial (RDD) survey, the sampling unit is a telephone number and the selection probability is the percentage of possible numbers within the study area that were actually selected for the sample.

In stratified sample designs, the population is first divided into subgroups called "strata" and separate samples are selected within each subgroup. Often different sampling probabilities are used within the different strata. For example, the study area might be divided into counties; if the telephone numbers linked to different counties were subject to different rates of sampling, then separate selection probabilities would have to be computed for each county. The PSTP sample, for example, consists of four geographic strata- King, Kitsap, Pierce, and Snohomish Counties [27].

In an RDD survey, this base weight should be adjusted to compensate for the fact that people in households with multiple telephones have more than one chance of being selected into the sample. The standard adjustment is quite simple; it is the base weight for household i divided by the number of distinct household telephone lines (ti ):

formula

In a survey in which households are first screened and then subsampled for the main data collection, the base weight should reflect the selection probabilities at both phases of selection- selection into the screening sample and retention for the main sample:

formula

in which Prli represents the case's probability of inclusion in the screening sample and Pr2i is its probability of retention for the main sample. If all eligible households are retained for the main data collection, then Pr2i is one.

Step 2: Compensating for nonresponse.

The base weight for the initial wave (W1 or W1 ') should then be adjusted to compensate for the effects of nonresponse. 1 1 Nonresponse adjustments ensure that the sum of the weights is unaffected by nonresponse; they do this by reallocating the weights originally assigned to nonrespondents to the respondents. In addition, the nonresponse adjustments can reduce the bias introduced by nonresponse by compensating for differences in nonresponse rates across subgroups of the sample.

Nonresponse adjustments are often calculated by grouping cases into nonresponse adjustment cells and finding the (weighted) response rate for cases in that cell. In a travel survey, household size or number of vehicles might be used to form the nonresponse cells if that information is available for nonrespondents as well as respondents. For each cell, the weighted response rate Rj is:

formula

in which the numerator is the sum of the weights for the respondents in cell j and the denominator is the sum of the weights for all eligible cases in that cell.

The adjusted weight (W2 ) is the base weight divided by the nonresponse adjustment:

formula

For nonrespondents and ineligible cases, the adjusted weight is set to zero. The sum of the adjusted weights for the respondents in cell j should equal the sum of the base weights for the eligible cases in that cell.

Ideally, adjustment cells should be formed using variables that are related both to the likelihood of nonresponse and to the substantive variables of interest in the survey (such as travel behavior). Often, however, the choices are quite limited because so little is known about the nonrespondents and because both respondents and nonrespondents must be classified into adjustment cells. For example, in a telephone survey, the only information available for the nonrespondents may be their area codes and exchanges (and any geographic information that can be inferred from these). Thus, the nonresponse adjustment cells have to be formed using whatever information happens to be available for the nonrespondents.

When there are two phases of data collection- a screening phase and a main interview phase- separate nonresponse adjustments should be calculated for each phase. The same adjustment cells need not be used in both phases. In fact, the screening data are generally useful for forming adjustment cells to compensate for nonresponse to the main interview. If R1j denotes the weighted response rate in the first phase of data collection and R2k the response rate in the second phase, then the adjusted weight would be:

formula

The factors in the denominator of this equation (R1j and R2k ) represent estimates of 1j 2k the probability that a given case will take part in the study. It is possible to derive these estimates from the observed response rate within a subgroup of the sample, but it is also possible to derive them through more sophisticated estimation procedures. Estimates of the response probabilities can be obtained via logit or probit models that take into account multiple characteristics of the sample members. Probit models were used to estimates response probabilities in the PSTP sample, and these estimated response probabilities were used, in turn, to adjust the PSTP weights [27].

Step 3: Post-stratifying to population estimates.

If the weights have been properly calculated, their sum represents an estimate of the size of the population from which the sample was drawn.

Sometimes independent estimates of the size of the population are available (for example, from decennial census data). The weights can then be adjusted to bring the sums into agreement with those outside population figures. This method- called post-stratification- is used to correct for two types of errors in survey estimates- random sampling error and coverage error. Random sampling error refers to chance departures of the sample from the population it is selected to represent. Post-stratification can be expected to reduce random sampling error when the population estimate is derived from the decennial census or from a survey with a much larger sample than the one in the survey being weighted. Coverage error refers to systematic problems in who is included or excluded from the sample. Post-stratification can be expected to reduce the effects of coverage error when the population estimate gives better coverage of the population than the travel survey sample does. For example, if a telephone survey was used to collect the data, the sample will necessarily exclude households without telephones. The two most commonly used sources of figures for post-stratification are the decennial census and the Current Population Survey; these are thought to achieve much higher levels of coverage of the general population than other surveys. The PSTP weights, for example, were adjusted to agree with the Public Use Microdata Set (PUMS), an extract from the decennial census data [27].

Post-stratification involves comparing the sum of the weights (i. e., W2) for a given subgroup with the population estimate for that group. For example, the PSTP weights were adjusted to agree with the PUMS totals for income-household size-number of vehicle groupings within each county. The post-stratification adjustment is calculated by multiplying the current weight for cases in a subgroup, say subgroup j, by the ratio between the population estimate for that subgroup (Nj) and the sum of the weights for sample cases in that subgroup:

formula

The adjustment cells are typically defined in terms of areas (such as townships) and one or more demographic variables (such as household size).

Population figures for poststratification adjustments (the values for Nj in the equation) can be obtained from decennial census data, the CPS, or other Census Bureau estimates. Which source to use will depend on how recent the data are, whether they are based on sufficient sample sizes (in the case of the CPS), and whether they provide appropriate grouping variables.

In general, both household-level and person-level weights can be calculated. Both sets of weights can incorporate nonresponse and post-stratification adjustments (and different adjustment cells can be used in developing weights for households and persons).

A. 2 DEVELOPING PANEL WEIGHTS

Step 1: Defining longitudinal households for weighting purposes.

One practical approach to defining longitudinal households is to continue to treat as a household all the persons who made up the household at the time of the first wave of data collection, regardless of what other changes that household subsequently undergoes. That is, each household is a treated a collection of persons and the longitudinal weight is applied to this collection of persons even if they no longer live together. For instance, if a couple divorced after the initial wave of data collection, the longitudinal household weight would still be attached to both of the households they now form. (Of course, analysts might want to take into account that this "household" now encompasses two separate residences, each of which may include other persons.) Although this approach has some obvious drawbacks, it is preferable to the main alternatives discussed in the statistical literature (for example, restricting the analysis to intact households) and we recommend that it be adopted in travel panel surveys.

Step 2: Developing longitudinal base weights.

Having defined longitudinal households, the next step in creating a longitudinal weight is to calculate a base weight, reflecting the household's selection probability. Generally, the households retained for follow-up in subsequent waves of the panel are drawn from the sample of responding households in the first wave. Sometimes all Wave 1 respondents are included in the sample for Wave 2; in other cases, only a subsample is retained for follow-up. When the sample for later waves is drawn from Wave 1 respondents, the base panel weight (Wp1) can be computed by dividing the final Wave 1 household weight by the probability of retention for later waves:

formula

Since this initial weight is based on the final Wave 1 weight, it incorporates corrections for Wave 1 nonresponse and adjusts for any discrepancies between the composition of the Wave 1 subsample and the population from which it was drawn. If all Wave 1 respondents are retained for follow-up, the initial panel weight is simply the final Wave 1 weight.

If new units are added to the sample in later waves, they must also receive a panel weight. If the new households represent in-movers (that is, immigration to the study area) or other additions to the Wave 1 population (e. g., births, returns from an institution), then the procedures outlined for weighting the Wave 1 households apply to the households added in later rounds as well. The new cases represent a new component of the population, one that was not previously eligible for inclusion in the sample.

When the new households are added because of changes in the composition of Wave 1 households, the situation is more complicated. Members of sampled households in the initial wave of a panel survey are sometimes referred to as "key" members of the sample. Other persons who join the households of key members after the initial wave but who were part of eligible population at the time of the first wave of the study are referred to as "non-key" members of the sample. Our recommended definition of a longitudinal household implies that data should be collected for key members in subsequent waves even if they move out of the household that was sampled initially. For weighting purposes, these persons remain linked to their Wave 1 households. However, data should also be collected for non-key members while they are part of a household containing a key member; the data for non-key members can be used understand the context of the responses of the key members [28].

All key members of the sample constitute the core sample forperson-level analyses, both cross-sectional and longitudinal. Non-key members are included in person-level analyses only in those waves when they were members of a household that included a key member of the panel survey. An alternative to the above methodology is to includeonly key members in all person-level analyses as done in the National Medical Expenditure Survey (NMES) [28].

Step 3: Compensating for nonresponse.

Once a longitudinal household has been defined for weighting purposes, it is necessary to develop rules for classifying the household as a panel respondent or nonrespondent. The simplest rule is to classify a household as a panel respondent if it provides the necessary data in each wave. For example, in a three-wave survey, households would be classified as respondents if they completed data collection in all three waves. All other households would be treated as nonrespondents. The base weights of the responding households can then be adjusted by the weighted response rate (that is, the weighted proportion of households that completed all three waves of data collection):

formula

This definition of respondents would produce a single set of longitudinal weights for all three waves. However, the definition may be too stringent for some purposes. For example, an analyst may be interested only in data from the initial and most recent waves of data collection. For that purpose, it may be useful to treat households that completed those two waves of data collection as respondents, even if they failed to take part in some intervening wave. This definition would produce a set of pairwise weights for the first and most recent waves. Pairwise weights for the second and third waves could be generated in the same way.

Updated: 03/28/2014
HEP Home Planning Environment Real Estate
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000