Jennifer Symoun:
Good afternoon or good morning to those of you to the West. Welcome to the Talking Freight Seminar Series. My name is Jennifer Symoun and I will moderate today's seminar. Today's topic is Establishment Surveys. Please be advised that today's seminar is being recorded.
Today we'll have three presentations, given by John Fowler of the U.S. Census Bureau, Joy Sharp of the Bureau of Transportation Statistics, and Johanna Zmud, Mia Zmud, and Chris Simek of NuStats.
John Fowler is Chief of the Commodity Flow Branch in the Service Sector Statistics Division of the US Census Bureau. Throughout his career at Census he has worked on monthly and annual survey programs covering service industries, wholesale, and transportation in addition to his work on the Commodity Flow Survey since the initial 1993 program.
Joy Sharp is the Assistant Director for Survey Programs at the Bureau of Transportation Statistics. She has worked at BTS for 8 years and is the co-manager of the 2007 Commodity Flow Survey. While at BTS, she has also worked on a number of other survey projects including the National Census of Ferry Operators, the National Household Travel Survey, and the Omnibus Household Survey. Prior to coming to BTS, she worked 12 years at the U.S. Census Bureau on a variety of demographic surveys, and she has also worked as a survey research consultant for PricewaterhouseCoopers. Her academic background is in mathematics, statistics, and survey methods.
Dr. Johanna Zmud is president of NuStats, a survey science consultancy, specializing in transportation studies. She has 24 years of experience in survey design, implementation, and statistical analysis. She is an active project manager, typically for large-scale travel behavior surveys. In total, she has managed more than 30 household travel surveys. She has also managed 100+ other surveys, including: visitor surveys, external station surveys, on-board surveys, and tolling and road pricing surveys. She currently serves as chair of the TRB Data and Information Systems section.
Mia Zmud, a Research Director at NuStats, has 22 years of experience in environmental improvement, integrating communications and research methods. She has 9 years of survey research experience with NuStats. Prior to this, she spent nearly 15 years on staff at the US Environmental Protection Agency. Much of her work at NuStats is in the area of establishment surveys, primarily for the purpose of inventorying and surveying businesses regarding their on- and off-road vehicles for air quality emissions modeling.
Chris Simek is a Survey Research Manager with 8 years of survey research experience and 11 years of professional experience. Over his tenure with NuStats, he has specialized in several different types of environmental and transportation related projects, including tolling and road pricing studies, household travel surveys, transit on-board surveys, user/non-user surveys, and commercial freight surveys. He currently serves as co-chair of the TRB Freight Surveys subcommittee of the Travel Survey Methods committee.
I'd now like to go over a few logistical details prior to starting the seminar. Today's seminar will last 90 minutes, with 60 minutes allocated for the speakers, and the final 30 minutes for audience Question and Answer. If during the presentations you think of a question, you can type it into the smaller text box underneath the chat area on the lower right side of your screen. Please make sure you are typing in the thin text box and not the large white area. Please also make sure you send your question to "Everyone" and indicate which presenter your question is for. Presenters will be unable to answer your questions during their presentations, but I will start off the question and answer session with the questions typed into the chat box. Once we get through all of the questions that have been typed in, the Operator will give you instructions on how to ask a question over the phone. If you think of a question after the seminar, you can send it to the presenters directly, or I encourage you to use the Freight Planning LISTSERV. The LISTSERV is an email list and is a great forum for the distribution of information and a place where you can post questions to find out what other subscribers have learned in the area of Freight Planning. If you have not already joined the LISTSERV, the web address at which you can register is provided on the slide on your screen.
Finally, I would like to remind you that this session is being recorded. A file containing the audio and the visual portion of this seminar will be posted to the Talking Freight Web site within the next week. We encourage you to direct others in your office that may have not been able to attend this seminar to access the recorded seminar.
The PowerPoint presentations used during the seminar are available for download from the file download box in the lower right corner of your screen. The presentations will also be available online within the next week. I will notify all attendees of the availability of the PowerPoints, the recording, and a transcript of this seminar.
We're now going to go ahead and get started. Today's topic, for those of you who just joined us, is Establishment Surveys. Our first presentation will be given by John Fowler of the U.S. Census Bureau.
As a reminder, if you have questions during the presentation please type them into the chat box and they will be answered in the last 30 minutes of the seminar.
John Fowler:
Thank you, Jennifer, and thank you everybody for joining us today. I will be talking a little bit about the 2007 Commodity Flow Survey today giving you a little background on that and how we conducted it and look forward to any questions you might have at the end of all the presentations today. 2007 I'm sure at this flow survey is a little unique survey at the Census bureau. It is part of our economic Census conducted every five years but also conducted through a partnership with the bureau of transportation statistics at DOT. Our goal is to measure the movement of goods in the U.S. by all modes of transportation at a very high level of U.S. state level aggregation.
A little bit of background on how we conducted the survey itself. We collected all the information throughout the data year in 2007 with a little bit of overlap in the end of 2006 and start of 2008. The way we collected the information is each establishment in the survey is mailed a questionnaire once in each calendar year and a mailout mailback survey for the most part. Editing and data re review and resolution began in early April of 2007 and really continued through just last month in preparation for releasing our preliminary data which I will talk about more later. Commodity flow survey is a sample survey and actually we have three levels of sampling. I would like to touch on each of those this afternoon. First it is a sample of establishment. And the establishments are selected among those in mining, manufacturing, wholesale, and selected retail and service industries. We also include what we refer to as auxiliaries, and these are really warehouses of retail and other multi-unit establishment companies and support only those parent companies. The second level of sampling for the Commodity Flow Survey is the weeks of the year. We make sure we cover an entire 52 weeks of the year by breaking up the year into one week components and each respond he wanted is responsible for reporting for one week in each calendar quarter, and that's how we get the four weeks of reporting from each establishment.
Probably the most unique feature of the survey is we ask respond ends on do the third and final stage of sampling in the survey. We ask them and give them instructions how to conduct this, a sample of their outbound shipments for the one-week reporting period we assign to them. The sample is selected on a random basis and on average respondent is going to report for approximately 23 shipments on any given report form. The most they can report on in the form based on the survey design is 40.
On each questionnaire and for each sampled shipment the respondents are asked to report information about each shipment or agenda nation, all modes of transportation and the order they occurred, the major commodity of the shipments, value of the shipment, the weight and pounds of the shipment accident, and a couple new items on the Commodity Flow Survey for 2007 one is we ask if it is intermodal shipment or not and at the end of the fourth quarter we ask all respond ends a series of questions about third party logistics providers, three P L questions the slide there, just to ask them about the supply chain information to see perhaps how that is changing. We had hopes about the intermodal status helping us identify that component of transportation but unfortunately the responses weren't of a quality we hoped for, so we probably won't be able to publish anything on that. The problem was the respondents didn't quite understand the terminology as clearly as we would like.
In addition to those items we ask for additional information if the shipment is hazardous material shipment. We ask for the four digit United Nations and UNNA code on that. We've had great success with that, finding that we really don't have to supply a lot of additional information to these respondents if they're shipping hazardous material shipments they know what the code is and can easily provide it, and we have a high response rate on that item. For export shipments we also ask the respond end to provide the export mode, foreign city and country of destination. Both the UNNA code and the additional export informational allow us to look at data for those types of shipments in additional ways that help provide insight to those components of the transportation picture.
Our core items are ton miles, average miles per shipment, and also have those characteristics by commodities shipped, by the modes of transportation, and by origin destination flows and many of our tables are two, three, and even four dimensional tables crossing those characteristics. New for the 2007 CFS are tables that are going to be shown by industry, and in this case the North American industry classification system. I am still used to calling it NACS. We will have those tables for 2007 as well as coming out with the results of the third party logistics provider results.
We do publish the Commodity Flow Survey results at several different levels including the national level, Census regions and divisions for all 50 states and the District of Columbia. We also have it for several metropolitan areas and remainder of state areas. That should be approximately 60 to 65 or so areas. I don't have a complete list here, but if you go to some of the websites that will be highlighting later on, you can see the metropolitan area that is we had for the 2002 survey, and they will be pretty much the same with few additional gateways we added into the sample selection process for 2007.
I mentioned that we do have additional tables and in fact a special report on hazardous material shipments as well as exports as part of our publications severs. I want to talk a little about the improvements we made over the 2007 Commodity Flow Survey. One of the biggest ones of course was an increase in our sample size from 50,000 in 2002 to over 100,000 establishments in 2007, and we hope this will help a great deal with improving the quality of the data that we're going to be able to provide. In addition we conducted an advance survey in February of 2006. We had two targets. First we had approximately 45,000 auxiliary locations which is almost all of them in our universe because we want to better identify those engaged in shipping and include only those establishments in our sampling frame. This improved the sampling frame quite a bit and made the sample 67 more efficient.
In addition, we also mailed out to 40,000 certainty establishments, those are our largest establishments and contributed or to the CFS and want to make sure we had the proper mailing address and the best contact information available for those and one of the things we found and conducted in Commodity Flow Survey is a key to getting the best data possible is find the best contact in the Company to respond to this particular survey, and a lot of other government surveys goes to just a financial area or accounting area with our survey, best if it is goes to the freight transportation experts.
Another was cognitive testing. We visited approximately 70 companies over several months, got their ideas and thoughts on improvements to the questionnaire, instruction guide, quality coding manual and so on and think overall we made a lot of improvements and the form a lot easier to go through for the respondents. Also, we have a new disclosure avoidance methodology. In the past we relied upon what we referred to as cell suppression which hasn't had a big impact on the Commodity Flow Survey in the past but the potential impact was always there and if it was going to happen, it was probably going to impact the publication cycle, so along with several other programs in the economic Census, the 2007 Commodity Flow Survey is using noise infusion methodology to be able to provide more cells to the data users.
Lastly, our data are going to come out primarily on the American Fact Finder which is a web-based data dissemination tool. Census and our 2002 data was already put out there, but this is really the first time we used the American fact finder as the release vehicle for our Commodity Flow Survey, so we hope everybody will look into that and give us any comments back on how they find that environment as far as the data dissemination goes. We will continue some of the print reports that we've had in the past for Commodity Flow Survey, including the U.S. term shipments and export shipment reports. And we're very pleased to report that yesterday we released the 2007 preliminary report on the American fact finder and the tables are available. Joy is going to go over some of the results we published as well as provide some of the direct links on Census side and BTS side where you can access the data, and now we'll work towards the release of the final series of data reports, and that's scheduled for December of 2009 and we'll also take place on American fact finder.
That pretty much sums up what I had to say today. Here is a site we can look up a little bit more information on the Commodity Flow Survey, and my contact information in case there are questions you think of after today's presentation. Thank you very much.
J. Symoun:
Thank you, John. We do have one question and we'll get to it at the end. If anybody thinks of additional questions for John, feel free to type them in. We're now going to move onto Joy Sharp of the Bureau of Transportation Statistics.
Joy Sharp:
Hi, John provided a nice overview of the background of the Commodity Flow Survey and the methods used to collect the data. I am going to focus more on the data itself, especially the preliminary release we released yesterday as well as providing a brief overview of the purpose and uses.
The objective of the Commodity Flow Survey is to provide a comprehensive, multi-modal picture of national freight movement, and this includes producing data products at the national, state, and metropolitan area levels, as well as producing origin and destination flows. A very important aspect of the Commodity Flow Survey is that it represents the only available source of data for the highway or trucking mode, and we know this mode dominates both in terms of value and tonnage of the freight that's been transported.
The next slide just provides you with a sense of the type of data produced in the Commodity Flow Survey, and these results were taken from the 2002 iteration. Essentially what we learned is that in 2002, on a typical day, approximately 43 million-tons of goods valued at nearly $30 billion move nearly 12 billion-ton miles on our nation's transportation network. The second bullet again emphasizes what was said on the previous slide in that trucking dominates the movement of freight. And we learned in 2002 that about 74% or three quarters of the value of freight, and 67% or two-thirds of the weight, were moved by the highway mode. Another important use of the Commodity Flow Survey is that it also provides one of the only available sources of hazardous materials shipped by the highway mode. And from the 2002 survey, we learned that about 20% or one fifth of all goods by weight moved were hazardous materials.
Analysts and researchers use the Commodity Flow Survey data for a variety of purposes. The next slide just gives you a sense of what some of those uses are. It allows us to do trend analysis, to look at the movement of goods over time; it also allows us to conduct economic analysis at a variety of levels. The CFS data are also used for developing models and analytic tools used for policy analysis and investment decisions, and it allow us to do forecasting for future demands for goods movements, and also for infrastructure and equipment needs, and so forth. The Commodity Flow Survey data is used by itself on a number of occasions, but it is also used through a number of value-added or derivative products. For example, the Commodity Flow Survey serves as the core data component for Federal Highway's Freight Analysis Framework and is also used in a number of other private sector products.
John already highlighted a number of differences or improvements made to the 2007 survey, so I will just briefly go through or emphasize some additional ones. First of all, we expanded the geographic coverage in 2007 to include major freight gateways. These included seven port areas, for example, Charleston and Savannah, and also included two border crossing areas. We also included the coverage of certain industries that have previously been excluded, and examples are select retail establishments and service establishments. We also expanded the coverage of those industries shipping hazardous materials recognizing that it is getting more and more important for safety and security reasons. We redesigned the forms based on cognitive interviewing as John mentioned. We also attempted to capture some additional information such as intermodal shipments and on third party logistics use. We doubled the sample size to assist with the reliability of the estimates, and also to cut down on some of the suppression required in the data. We also expanded the computer routines in our processing, and very importantly we introduced the new mileage calculation software. This software has some changes to the mileage calculation algorithms, and I will touch on a couple of these changes as we go through the tables. Finally, the data products have also changed for the 2007 CFS. As John mentioned we're moving more towards an electronic system and kind of getting away from the paper products for the 2007 survey. Census will be publishing the results on their American Fact Finder, their own web tool, and BTS will also continue to produce electronic products on its website in a variety of formats.
I want to add - for the preliminary results of the release yesterday - there will not be a paper product. Instead, the six tables released will be available on both the BTS and Census website.
Again, the preliminary release consists of national level tables. There are basically six transportation characteristics or attributes for each of these tables including the ones that those of you familiar with using the CFS data will recognize - those being mode, total modal activity, distance, weight, and commodity. And, again, as John mentioned the new table that we're releasing this time around is the industry or NAIIS series of tables -one in the preliminary, and there will be a whole set or series released for the final data. Each of the tables consist of various shipment characteristics: value; tons; ton-miles, which is simply a metric the consists of multiplying the weight and distance; and average miles.
Just as a cautionary note, I wanted to mention these data that we released yesterday are preliminary. For the next nine months to a year, we'll be continuing to process the data and make some minor corrections to some of the anomalies we see. And, probably more importantly, for the final data, we also weight or include another adjustment factor based on the information that's collected in the Economic Census.
The tables that we produce for the preliminary report tend to be those that are highly aggregated at the national level and fairly stable, so while these numbers in the tables will change, they're going to remain relatively stable as compared to those that get down to the finer levels on both geography, below the national level, or as you cut and slice the commodity or industry data down to finer goods ot to lower levels of specificity.
For, the next several slides I just wanted to walk through the tables that were released yesterday. This is the first table, table 1, which is the mode of transportation. As you will see, the basic layout of the tables are the same in that down the left-hand column we have the categories of the transportation characteristics, this being the mode of transportation or what the establishment reported in way of mode or modal combinations of how the commodity was shipped. And, you will see the shipment characteristics going across the heading row where we show the value in millions of dollars, the tons in thousands of tons, the ton miles in millions, the average mile per shipment, and so forth. You will also see four new columns added to the right-hand side of the table. In previous CFS publications we have provided measures of sampling error, in this case called coefficients of variation or relative standard error, but again these were previously produced in separate tables appearing at the back of the publications or in separate tables on the website. So this time around - to give the data user a better sense of the accuracy or reliability of the estimate -we're actually providing these in the columns to the right. And as a general rule of thumb, the lower the CV or coefficient of variation, the more reliable the estimate. We do have a rule in the Commodity Flow Survey where we do not show estimates that have a CV of greater than 50%,. This is because if you were to construct a 95% confidence interval for that estimate, it could be as low as zero or as much as twice. So in view of the data quality issues, we suppress it. Just a couple take aways on this table, and again these were just released yesterday. We haven't had a lot of opportunity to do any of our own analysis as a result into some of the trends of what we see. But, just quickly comparing to the same data in 2002, we can see in terms of value there has been an increase of about 41%, in terms of tons about 12%, and in terms of ton-miles 11%. And, this suggests to us that the nation's infrastructure is continuing to be increasingly used for the movement of freight, and it also suggests that we're probably seeing some changes in the way that freight is being moved.
Table 2 is the total modal activity table, and this differs from the previous table in that for the multi-mode components shown in the first table, we actually break them down into their single mode components. So in this table we only see the single mode. For example, for truck in Table 2, this includes the truck ton-miles for those commodities moved only by truck as well as the truck component for multi-modal shipments.
In the next tables, 3 and 4, we show the shipment characteristics by ranges of distance shipped, and then in table 4 we show the shipment characteristics by the ranges of shipment weights. In this particular table, not too surprisingly, if you look at average miles per shipment, the lower weights tend to go the further distances with the exception of the very last category of over 100,000 pounds.
Table 5 is the new table that we talked about, the industry table, and we'll see down the left-hand column the NAICS industry code. Just a note of caution to those of you who aren't used to using NACS, this is a hierarchal industry classification system. There will be industry levels at the two digit, three digit, and four digit level. Those within the three digit comprise the two digit and so forth. So this is just a note to be careful that when you're using these data, realize that some of the industries are part of a broader industry grouping. This table actually goes on for a few pages. This is just the first page of the table from the web.
Then finally table 6 is the commodities, and again from the national level we produce commodity information for two-digit levels of commodity.
John mentioned we'll have a much more comprehensive final data release in December of 2009. The next year we'll be working on completing the processing and the weighting of the data for the final release. We'll have data produced not only at the national level, but state and metropolitan areas, as well as commodity origin and destination flow tables. We'll also produce as we have in the past, a special report on hazardous materials and exports. Based on the information we collected this time on third party logistics provision, we'll produce data on the extent these services are being used by these establishments. And we'll also be putting out some print publications. However, unlike the past where we printed paper publications for national, state, and metropolitan areas, this time we'll only be doing three publications at the national level, and this is for U.S. data, hazardous materials data and the export data.
Finally, for those of you interested in going onto the BTS and/or Census website to obtain the tables or look for more information on the survey, these are the two links that you can visit. The Census website is...that's your link to the American fact finder. Finally, if you have any questions or want to contact me after the webinar, my contact information is provided on the last slide. Thank you.
J. Symoun:
Okay. Thank you, Joy, and I know everyone is rushing to get the web addresses. You can download the presentations and I will try to bring them back up at the end. We're going to move onto our third presentation given by Johanna Zmud, Mia Zmud, and Chris Simek of NuStats.
Johanna Zmud:
Thank you. Thank you, everybody, for participating in this. NuStats is presenting the state and local experience on establishment surveys.
At these levels freight surveys are done for three main purposes, policy decision making, improved understanding of freight movements, and for use in freight travel demand forecast modeling. Some studies have multiple purposes, different purposes lead to different sampling units and to the collection of varying data elements.
Freight movements are considerably more complex than person travel at the local level, and change far more quickly than passenger movements, so there are significant challenges we need to address at the survey design stage. While there are many, I am highlighting three, sampling, instrument design, and data collection method. Sampling challenges begin whether we try to define the commercial vehicle universe for a particular vehicle, goods only, services only, goods and services, flow of goods. In order to define and limit a specific target population we typically end up with a slice of the freight universe. The typical available sampling frames tend to have limitations that often lead to under coverage and over coverage errors. Units are often missing from the frame because the rate at which the database has been updated may not be known or turnover in the area or the universe is simply unknown. Also units may be erroneously included in the frame or included more than once. This not only causes biases in the survey estimates but also is costly to survey execution in terms of budget and schedule as wrongfully included units are identified and started.
Calculating sample sizes for freight surveys is inherently more complicated than passenger travel surveys because establishment or vehicle attributes, trip rates, and other factors are more heterogeneous than for households. This causes complex compilations of sample size requirements and stratification approaches.
While there is more to discuss in terms of sample willing, I need to move onto instrument development. The complexity of goods and services movements means there is an extremely large amount of information required to fully describe every aspect. In order to keep questionnaires short and mitigate errors it is almost impossible to collect all of the needed information in a single survey. Many studies we talk about two examples later in this presentation use a combination of two or more approaches such as a driver survey together with a shipper survey to meet the research purpose. Complicating questionnaire design is the fact that a multi-police at this of agents influence freight movements including the shipper, receiver, customer, carrier, and distributor. Each are all is a potential respondent, but almost no potential respondent has all of the required answers to our survey questions. Most have limited or incomplete knowledge, and in our experience in making the first contact you get a lot of passing the buck before you find a respondent who is willing to participate and qualified enough to accurately answer the question. More data collection challenges focus on the surveying method. While this session is on establishment surveys, it is rent positive to note local and statewide utilize different approaches. Sometimes due to the specific research objectives and other times to factors in the local survey context that point to one survey method over another. For example, most local surveys are constrained by the requirement to design, field, and prepare a database in a matter of months. We have commonly used three approaches to meet compliant needs. First, establishment surveys. Here we typically survey shippers rather than receivers because shippers tend to be few inner number, easy to classify and characterize and easier to identify and locate. Second are carrier surveys that gather data directly from operators or drivers often as a road site, weigh station or rest stop survey, and last our distributor surveys. These are surveys of intermediaries such as warehouses, transportation terminals, or distribution centers. The latter are more complicated and costly than the former and our experience tends to focus on establishment and carrier surveys. We now want to present three case studies to discuss how we addressed these challenges and practice. First, I will discuss a value of time survey of shippers conducted for policy making purposes. My colleague, Mia, will present a commercial vehicle travel diary survey that captured data to update a travel demand forecast model and Chris will present a commercial vehicle driver survey that was conducted to understand freight movements on New York state highways.
Mia Zmud:
This value of time survey of shippers was done for the Georgia DOT you under subcontract to HNTB. The purpose was two fold, first to assess the continues of decision maker as companies with respect for the need for Truck Only Tolling or TOTs and second to calculate time for shippers as an important factor in designing the potential pricing structure. This survey was one of two done for the GDOT. At the same time as the shipper survey we also conducted a truck operator survey, and the in the interest of time I will discuss the tripper survey. The universe for shipper survey was all trucking companies with vehicles having four or more axles operating on the specific segment of I-75. Because we were concerned about under coverage of our survey universe, we used a dual sample approach. First, we obtained an abstracts of trucking companies operating in Georgia and surrounding states from a database of all commercial trucking companies in the U.S. maintained by the motor vehicle carrier administration. Second, we obtained other sample records via the truck operator survey we were doing at the same time. That survey was done at weigh stations. Our surveyors took a digital photograph of the door panel of vehicles stopped us about -- the employment database did not have information on fleet ownership and could not get access to the DMB records for the region. Survey process was similar to a travel study diary study in that we mailed an advanced letter to the firm followed with an improvement call to screen for eligibility, and locate the person responsible for fleet management, eligible firms were then recruited, but rather than sending the did diaries to them for completion, we opted for a more personable approach in which we delivered them in person with the intention to reduce the opportunity for them to opt out of the survey and to give us an opportunity to review the process for completing the did diary directly with the operators or the truck drivers. We also picked up and retrieved the completed diaries so we could determine onsite incomplete data by talking directly to the operators of the fleet managers. The diaries covered information including truck informing, driver information, and identified individual trips on the day of travel. Now we'll review a few of the issues and challenges that were associated with the survey. First, the incidence was lower than we assumed it would be, and the employer database there were fewer firms meeting our universe definition. The majority had two axle, four-wheel trucks instead of the two-axle six-wheel trucks. This assumed they made smaller deliveries or were service trucks.
If the firm did have vehicle that is met our definition the number of vehicles per firm were much fewer than we anticipated. Recognizing that the employment database would want provide us sufficient number of firms to meet our target, number of diaries that we were hoping to complete and finding that the firms were that would qualify was like looking for a needle in a hey stack among our sample, we quickly looked to supplemental sample frames, and we identified three. One was fleet seek which provided about 647 additional sample items, the ATA fleet directory provided about 330 sample items, and U.S. Data Corp yielded 357 sample items. The sample frames helped us targeted firms with trucks we new would be eligible.
Another issue we faced, however, and which slowed recruitment was a high amount of noncontact rates. Our rule was to make at least 15 calls before retiring a sample item, but to improve the noncontact rates we researched firms with repeated no answer or disconnects to get new numbers, companies that is went out of business and we did get new numbers that way. We made in person visits if we repeatedly got answering machines meaning that we would do recruitment in field, go to their place of business to find someone there instead of using the telephone. We also used in-field recruitment in that we as we delivered diaries when we saw a neighboring company that had qualifying trucks, if it was in our sample frame we would also recruit them while in field. These all stepped with recruitment but also added to our time for recruitment and data collection and did have an impact on costs.
Overall we extended recruitment and data collection from four to eight weeks, and despite receiving fewer number of did diaries per participating firm, we did collect sufficient data for the modeling effort. Our response rate was 21% eligibility 13%, and we did have a high refusal rate overall of 66%. Now I will hand it over to Chris who will share his experience with the final study.
Chris Simek:
Thanks, Mia. The New York State Department of Transportation commercial vehicle intercept survey being conducted on behalf of the New York State Department of Transportation and the New York State Thruway Authority as strategic planning study data is currently being connected by Canada as U.S. Canadian border crossings. It will be used for a myriad of purposes including but not limited to facility location, assessment of commercial vehicle parking, commercial vehicle routing, and traffic count locations. Data collection was conducted in two distinct phases. The first occurred in April of 2008, and the second in September of 2008. Eligible respond he wants included drivers of commercial vehicles matching FHWA vehicle classes A through 13. To a large extent it was based on the -- personal computers were used to collect the data in the field. The technology we used allowed to us geo code route origins and destinations in realtime and compute and verify the route used by the driver at the time the survey was conducted. Data elements collected included but are not limited to vehicle facility and freight characteristics and driver attitudes and opinions regarding parking availability, route choice, and facility amenities. Over 1,000 surveys were conducted with commercial vehicle drivers at New York State Department of transportation rest areas, New York stated through way plaza and private truck stops. A total of 30 facilities were sampled statewide with two days of data collection occurring at each facility. The next particular slide actually describes a couple of graphics that we see here. The one in the upper left-hand side shows a screen shot of the actual survey instruments, actually the introductory screen as was seen by the interviewers, and on the bottom right-hand graphic is a screen shot of what the interviewer would see after they have collected a number of different origin destination locations from the commercial vehicle driver in the field at a rest area travel plaza or truck stop. As I said, the technology we used allowed us to compute a route in the field during the time we were surveying the interviewer, and that is the screen that was actually showed to the commercial vehicle driver, kind of looks like what you expect to see on Mapquest or Google maps once you identified a route. It allowed us to verify the route in the field and move on.
Probably the most challenging aspect of the survey was logistics. 30 facilities were spread out across the state, and we had to go to where the truck drivers were multiple times. As such there were a lot of travel costs involved. Further more the coordination required for the effort was massive. With the need for constant communication between interviewers, law enforcement, NYSDOT and NYST A personnel, facility operators and the traffic count contractor as well. Survey participation was approximately 70%. Given the high response rate was being a good listener. During the time of the survey truck drivers were under the crunch of high gas and diesel prices and seemed to over shadow the curses at many times. It made it difficult to keep the survey length down. Average survey length was actually approximately 16 minutes but some instances when we had a talkative truck driver that wanted to vent about gas prices or another topic the survey ranged as high as 30 minutes in length.
Properly trained and motivated staff was also key to the success of the project. Training occurred over a two-day period including classroom and mock interviewing facility. I will mention that nothing motivated surveyors like a paycheck, and we did pay surveyors for more the effort than other field effort we're currently involved. In addition to paying an hourly rate we provided a per diem rate, sometimes if necessary lodging and mileage reimbursement if there was driving involved from the home location to the interview facility site. Lastly regarding refinement of methods.
In closing I believe we have seen there is some overlap in the challenges faced in the state and national level concerning the collection of commercial vehicle survey date amply there is no single method of sampling or mode of data collection best suited for commercial vehicle surveys and the aggregate. Rather the specific goals and data needs much each survey should drive how they're conducted as there is a wide variety of factors to affect response rates. The last slide here simply has Johanna and me and my contact information and should any of you have questions in the near future regarding what we talked about today.
J. Symoun:
Thank you. We're going to move to the question and answer session now. We have a bunch of questions about the Commodity Flow Survey. I do see a question I believe about the case study Mia was presenting, so I will start with that. If you continue to think of questions for Johanna, Mia or Chris, type those in but the question for Mia is what is the MAG employer database and is it publicly available?
M. Zmud:
It is not necessarily the MAG employment database, it was the employment database for the MAG area I guess put out by Phoenix, and it was a publicly accessible database.
J. Symoun:
I am going to move to some questions for Joy and John right now. If you think of questions for the others we'll move back to those. Most of these questions about the Commodity Flow Survey I will put out to both of you and let either one of you jump in and answer. The first question was why were imports not included? That may have been about the reports that were going to be printed, I think.
J. Sharp
Imports aren't included in the scope of Commodity Flow Survey as say U.S.-based establishment survey. We don't have the authority appropriate list, staff, or whatever to sample from those establishments and other countries. What we do get in terms of imports are those domestic legs, so as an import or good comes into the country, and let's say it comes through a port and moves to a distribution center outside the port area, that's where we would pick up the good, but in terms of sampling companies and other countries obviously we don't have the authority, sufficient list, et cetera, to do that. I hope that that's the question Eric was asking.
J. Fowler:
I think that covers everything pretty well. One of the factors is we really don't have a universe of importers we could really blend in there and would probably be a completely different industry coverage as far as that goes.
J. Symoun:
The next question, have you thought of adding a question on vehicle type? It would be helpful to link the type of freight transported by a specific vehicle class.
J. Fowler
That hasn't really come up. I suspect there would be some issues in trying to add that to the commodity flow collection instrument. Right now we break out truck, for example, into private truck and for hire truck which granted isn't that all that complex break out, and yet we have a sufficient number of problems trying to ensure the quality of the minor breakout right there, and along with the fact that each shipment could be not just truck but rail or what we call parcel, waterway shipments, things like that would be trying to target a very specific subset thereby adding that in that additional information, but it hasn't been really one of our priorities so far.
J. Sharp:
I agree with John in that I think it is the type of information, the level of detail that most respondents just don't have, and it is kind of the problem we ran into with trying to add the intermodal question this time, is not only that the respondent didn't really understand what intermodal was, but in some cases they just don't know. They may know a truck of some type shows up at the loading dock, dock, puts the shipment or shipments in the truck, but they don't know what happens after that, and it could go from truck to rail to some other mode, but sometimes they do, but it is kind of surprising how much of the time they don't know that sort of basic information about how their product is being transported.
Mike Sprung:
I think to add to that, this is Mike Sprung from the FHWA Office of Freight Management and Operations, that a lot of times the respondents for the Commodity Flow Survey are filling this out after the fact, most cases filling it out after the fact and going through a list of records from the week that they're being surveyed for, and to ask a question like that at that point you really have a hard time getting any level of accurate data from the respondents.
J. Symoun:
Thank you. Are there movements by day of the week by state available?
J. Sharp/J. Fowler
No, there are not. That's very specific for a sample survey of this type even though we do identify when the shipments took place, the design is such that we're aiming to come up with annual estimate of the shipments and not day by day.
J. Symoun:
I think you may have answered this in your presentation, but if you could just mention again, what CFS '07 printed reports will be created?
J. Fowler:
Just the three national level reports, the U.S. report, the hazardous materials report, and the export report. Unlike what we have done in the past, we won't be doing print publications for any of the state or metropolitan areas.
J. Symoun:
How are companies like UPS and FedEx handled?
J. Sharp:
I think again I hope I am answering your question, Elaine. The CFS again is shipper-based, so the establishments we have in sample are shippers. They're not transportation companies and when it is about outbound shipments not shipments coming in bound. We do capture UPS and Fed Ex shipments if the shipping establishment and sample is using UPS and Fed Ex as their transportation provider, but in terms of sampling the Company of Fed Ex or UPS, we do not do that. Again, UPS and FedEx shipments are included in our parcel shipment estimate.
J. Symoun:
When will the state data be posted on AFF?
J. Fowler:
State level data will be out with the final report series in December of 2009.
J. Symoun:
Are all MSAs included?
J. Sharp:
No. Essentially the MSAs are typically based on those larger metropolitan statistical areas that were used in sampling and where we have sufficient enough data to publish, and I believe in 2002 we had around 64.
I think we have a few more this time around for 2007. John, you have some information on where you can go to find which MAs will be publishing data on?
J. Fowler:
If they want to, they can go to the 2002 Commodity Flow Survey sites under either AFF or under our Census.gov/AFS sites for 2002, and the MSA level reports there would give you a pretty good idea of the level of MSA that we cover. As Joy mentioned there is a few more we added in for 2007. Mostly they're what we would call gateways, so maybe sheer not large in population size, but we think they have been increasing in terms of economic and transportation activity so places like Beaumont, Port Arthur, Texas, Corpus Christi, Kingsville, Texas, El Paso and so on are broken out and included in the 2007 as separate MSAs we would plan to publish for. Plus if they want a complete list and have trouble finding it anywhere on our site, they can always e-mail me and I can provide them that information.
J. Sharp:
We also have a listing of the metropolitan area reports from 2002 on the BTS website.
J. Symoun:
Do you release any information on single shipments?
J. Fowler:
No. That's a quick one. No. The reason for that besides just wouldn't be statistically significant is that we have strict confidentiality rules on that.
And in the past we have gotten a lot of requests for micro data or data at the shipment level, but at least at this time in planning for the 2007 given the confidentiality restrictions that we're under there is just to way that we can release that level of data.
J. Symoun:
What was the response rate and what incentives would you use to maximize responses?
J. Fowler:
Depends on what you want to call response rate. In fact, we're looking into the various measures and coming up with the best measure of a true response rate when you take into account the weighted components of the CFS and how that comes into play. In terms of a straight return rate, mail back type of situation, I would say we approached 70%, very close to 70% each of the quarters on that. What we did to try to increase that response rate was the survey as part of the economic sense success a mandatory survey. Response is required by law, and we do in fact take note of the confidentiality aspects of the survey. That's something that Census takes very seriously and we think helps with our response rates across the board in all our programs. As well we did a mail follow up to all the respondents that were late in each quarter reminding them it was mandatory and gave them a complete new package of forms and instruction guides and so on. As well Midway through the processing cycle if we had establishments and special larger ones that had not reported in either of the first two quarters, we made calls to them, reminded them, see if they had any questions, needed special reporting arrangements and so on. Some of those folks took us up on our limited option this time around for electronic reporting capability, and many of them were able to report based on just some secure Excel sheet reporting we provided to them and we have plans to for a 2012 CFS we would certainly make electronic reporting a key component of our data collection base because more and more establishments are expecting that option.
J. Symoun:
Okay. Will the CFS origin and destination table be just total flow state to state or by state and industry?
J. Sharp/J. Fowler:
Currently we are planning to produce flow tables by state and industry because the industry tables are new this time around we're going to have to look at the data and make sure that states are sufficient quality to produce but at the time this is our plan to do so.
We would still continue the data sets that we had before such as commodity by mode, by origin, by destination.
J. Symoun:
Is any public data source is available containing information on each shipment?
J. Sharp/J. Fowler:
There really isn't only in the aggregate form that we publish. The only way to get the individual shipment information or micro level data is through our center for economic studies at Census bureau, and there is a process outlined on their website that could direct people to it if they're interested, but under certain conditions we do allow researchers to become sworn special status folks and perform certain projects that would be a benefit to Census and the researchers to use the micro data and of course anything would be taken out of the center would have to pass that review as well.
J. Symoun:
Going back to the vehicle class question do you think it would be effective if establishment survey included a picture of the vehicle class type or if vehicle class information could be obtained would it be useful to others?
J. Sharp/J. Fowler:
I would say yes and no. Again, for the commodity survey I am not too sure what the benefit would be of a picture given some of the reasons we had already stated, person filling the form a lot of times just doesn't know. They don't see the truck. As Mike point out it is done retro spec actively after the fact. Do I think it would be useful, it is more for the users to weigh in, but I could certainly see some potential for that data.
I would think if it ever did become an item on the Commodity Flow Survey at the very least in the instruction guides and the web links and all we could include examples and photos, silhouettes of the types of vehicles, sure.
J. Symoun:
I will go back to a question for Chris now based on the case study you presented. Can you please expand your comment about no single type such as establishment, operator, or distributor or mode such as intercept, telephone or web meets needs at the local level?
C. Simek:
What I was trying to get at there is the fact that there is no single one method of data collection by itself or sampling methodology in and of itself that is suited for commercial surveys across the board at general level, rather what I was trying to say is that definitely when coming up with research methods for commercial vehicle surveys, one should really take a close look at the population that they're trying to capture data from as well as the environment, the survey environment so to speak, where you will be conducting the survey, the pros and cons of each mode of data collection regarding response rate and potential costs, and then taking that approach in identifying the proper survey method to conduct your survey.
J. Symoun:
Going back to the CFS survey are the survey questions publicly available?
J. Fowler:
The questionnaire is currently on the Census web site. If you go to the www.Census.gov/CFS, you can obtain a PDF of the questionnaire directly from that site.
J. Symoun:
Okay. Mia actually sent me an explanation of the MAG employer database and I will copy that in for everybody to see. I don't see anything else typed in but we can open the phone lines now to see if anybody has questions over the phone. If the operator could give instructions on how to ask a question over the phone.
Marilee Manalo:
Our DOT sends out a team to each business establishment in communities of 5000 or more in population. The business contact is already being made to collect data for a socioeconomic data for modeling. Are you aware of any State DOTs that are including freight data surveys with the establishment survey? The idea that I was wondering about is having like a two-page freight data survey that would be added to the already developed establishment surveys, and they collect social economic data, they collect it for a town, and they broke it down by how many students, how many people, how many housing units there are, et cetera, et cetera, but they actually visit the businesses, and in along with that I thought it would be useful to then connect freight data to that business asking certain things such as what's the weight, how often is your shipment, when is the shipment. I mean, I have a list of different things that could some of you already used those same types of data in your surveys, but have it like on a two-pager they could finish, and we've already got them there and just have them fill that out, find out the weight, maybe the vehicle type, and then we could use that information and we would know at least what's going on within our state, and we would know which shipments are local versus which ones are passing through, which we do know some of that but it would give us more of the local information. Does that explain what I was missing?
J. Sharp:
I personally am not aware of any state DOTs who do that, but that's really not my background or where I would necessarily be aware. One thing I would mention is there is always a danger when designing survey or data collection instruments, and that because you do have the contact sample and a limited pot of funds to collect data, there is this propensity sometimes to want to do it all with one thing, and a lot of times that can back fire on you and that either you increase the burden on respond end so much that they're not willing to participate at all or especially in the case of an establishment survey it is hard enough to get to the right person that can answer a limited or more focused set of questions, so in terms of whether or not other states are doing this, I really don't know, but just in terms of trying to design something that might be multi-purpose, you know, again I would just caution be careful because your response rate and your quality of data could suffer as a result.
J. Fowler:
I would echo that. What kind of rang a bell was back before the Commodity Flow Survey we had a commodity transportation survey, and I believe they tried to change the methodology on that to make it less burden some and sort of more summary level type of approach as opposed to shipment by shipment we have now and at least in the 783 version that far we really weren't able to publish anything because of reliability and quality issues on that, so Joy is right. There are real dangers in trying to come up with a quick summary version of types of freight, might be able to supplement something with a quick question or two and still make it worthwhile depending on your specific goal, but just have some caveats, I think.
J. Symoun:
I would say if we do have any state DOTs the call and have any thoughts, anything like that, feel free to type it into the chat area or get in touch with Marilee and discuss that more or the LISTSERV is another area you can send the information over.
I think we're going to go ahead and close out for today. I would like to thank all of our presenters for today's webinar. We had a really good turnout today and I think it was a good topic and good presentations. The next seminar will be held on January 21st. We're still in the process of finalizing the topic for that seminar, so it is not available for registration yet, but more information will be sent out through the LISTSERV as soon as is becomes available. With that we're going to end for today, and thank you, everybody, and have a great holiday and happy New Year.
