Skip to contentUnited States Department of Transportation - Federal Highway AdministrationSearch FHWAFeedback
Highways for LIFE

ArrowOverview: Market Research for Highway Technologies

<< PreviousContents

Overview: Market Research for Highway Technologies

Mia Zmud
NuStats
Austin, Texas

July 22, 2010

PowerPoint Version (898 kb)

Slide 1

Objectives

  • Determining the need and target population
  • Matching the method to the need
  • Using research findings
  • Guiding Principles

I will lead you through some of the basics of conducting market research.

First, we'll discuss one of the most important steps in the market research process: determining the need for research or defining the research problem. We'll also cover how you'll use this to determine your research population. Then, I'll provide a overview of research methods and discuss how to match the method to your research need.

We'll talk briefly about analysis and then I'll review a few guiding principles that are drawn from the NCHRP report (Putting Customer Research into Action).

Our focus today is on Primary Data—Information that has been gathered specifically for your immediate research purpose. This is in contrast to Secondary Data which represents information was gathered for some purpose other than the research at hand. Secondary Data may include previously conducted Primary datasets, published articles and reports, and other already existing information. So, for today we are focusing on the process for collecting new information or data.

Slide 2

Determining the Research Need

  • Situation analysis
    • Assesses the current situation/describe the problem
    • Defines research need and research objectives
  • Tools
    • Program or product analyses
    • Primary data collected by others (traffic counts/VMT; NHTS; National surveys) Literature search (TRIS online, NTIS, agency library)

That said, we are going to start off with the first step in any research effort—to define your research problem.

This typically entails a thorough analysis of the transportation issue or product innovation which is the focus of your research. This is so that you have a complete picture or the full background on the research topic to inform your design. Otherwise, you'll spend time formulating a research plan without an end goal—you may end up with data that doesn't meet your needs.

So, a tool for this is conducting a situation analysis—and sometimes it is referred to as an environmental scan.

Tools that you may use include:

1.Internal knowledge and understanding about a program or product innovation from the perspective of colleagues in your agency, program or team and you may also draw from historical data your program has previously collected. You would also want to include discussions with management, key decision-makers in your agency and even stakeholders and industry experts to get their perspectives on the research definition. 2. Another resource is mining primary data collected by other agencies for other purposes; for example, data such as traffic counts, VMT, highway crash and fatality numbers…these are drawn from existing databases. 3.Finally, you might conduct a literature search or product review using TRIS Online (Transportation Research Information Service), NTIS (National Technical Information Service), your agency library, conference proceedings and published articles, and product reviews.

Using this information, you should be able define the context of the research topic from the perspective and needs of your program or agency—You'll have defined, the research topic and what are the issues or problems, and how is research intended to resolve the issues or problems or inform decisions?

Slide 3

Driving Design by the Research Need

  • Who is the audience that can provide the information you need?
  • How can you reach them?
  • What data do you need? How will the research results be used?

Research Objective

Once you have your Research Need defined, the overall research framework or design call fall into place.

Having a clear research objective drives your research design.

For instance,

1.You can next hone in on your target audience by asking from whom can you expect to collect the information your need? With your study population defined, you can then ask… 2.How can you best collect the information from the population and what data do you need? This enables you to select the research method and develop the research questions to match your research objective. 3.Finally, in the study design, you need to consider how you plan on using the data to inform your research goals? This helps you determine what kind of analysis you'll need to conduct—will you need simple frequencies of target population support of the product innovation demonstrate the need? Or higher level analytics requiring detailed segmentation analysis or modeling? These are all important elements of your research plan. Now, we'll look a closer look at each.
Slide 4

Defining the Target Population

  • Considerations:
    • Behavior (licensed drivers; highway usage patterns; attitudes)
    • Demography (age; ethnicity; vehicle ownership)
    • Geographic (regions; cities; suburban/rural)

Sampling is one of the components of research design…we are not going to discuss much about sampling today, but we will take a look at defining the target population—those who possess the information needed. The study population must be defined precisely or else the research effort would be ineffective and the results may be misleading.

Defining the target population depends on the study objective and whether there is a need to hone in on characteristics of the population such as behaviors, demographics and geography, for instance.

Let's look at this more closely with a few examples on the next slide.

Slide 5
Defining Customer Segments Diagram

This slide illustrates how the research objective—the overall research question—can help define the customer population and even further define customer segments within the study population.

On the graphic here, let's look at the center example:

The research purpose is to measure how customers perceive road conditions on state highways.

Therefore, it makes sense that the research population would be state highway users—or drivers. You could stop there.

But, if you wanted to differentiate between different types of drivers, you might further segment your study population by those who drive trucks, state residents who drive vehicles, and visitors to the state.

I'll share another example not on the slide--

For a study we conducted for TxDOT with the research objective of measuring the effectiveness of the Don't Mess with Texas anti-litter campaign. We needed to collect data from persons who were exposed to the campaign in the markets in which the campaign aired. We not only needed to measure their awareness of the campaign, but also to document their propensity to litter as well. The attitudes and littering behaviors were tracked in subsequent surveys, every two years, to measure shifts in attitude and behaviors as a result of the campaign.

For this study, our study population was adults 18 and over in Texas households—further segmented the population to target only those residing in the markets in which the campaign aired.

So, in defining your study population you need to refer to the research objective. In the example I just gave you, with the DMWT campaign, if we had surveyed adult Texans not exposed to the campaign, we would have ended up with misleading data.
Slide 6

Selecting a Research Method

Qualitative Quantitative
Objective Gain directional understanding of underlying reasons/motivations Quantify information and generalize the results from the sample to the pop of interest
Sample Small number of cases; 6-10 per focus group; at least two groups on a topic. Large number of representative cases; Statistically valid sizes—industry standard is 400 provides a margin of error of +/-5%.
Question type Unstructured; Open-ended, open-response Structured; Closed-ended, response options; scales and ratings
Interpretation of Data Directional and reflective of only the persons interviewed; Exploratory Can draw inferences to the population as a whole; Confirmative
Costs Incentives Required; $65-$125
Low-end $3,000; Mod-end $5,000; high end $10-12,000 (hard to reach)
Highly variable; example
Mn/DOT Telephone, N=400, $35,000-45,000

Moving on to selecting a research method, we'll cover two forms of PRIMARY DATA collection:

1.Collecting data from participants through observation studies and focus groups (qualitative) and 2.Collecting data using methods such as telephone, mail, on-line, and face-to-face (quantitative).

Qualitative research provides insights and understanding of the research problem. The outcome is usually an initial or better understanding about the topic. Examples of qualitative research include focus groups, structured or depth interviews which are typically conducted as one-on-one in person or telephone interviews.

Because the sample is generally small (for instance, 6-10 typically participate on a single focus group) qualitative research is rarely if ever regarded as conclusive. Because the sample size is so small, qualitative research is also NOT used to make generalizations to the population of interest. In regards to cost, low end could be $3,000 (not in facility; small group of senior citizens in a community center) and in contrast, one with a hard-to-reach population (such as physicians or specialists held in a focus group facility) could be as high as $12,000 per group. Costs may seem lower than quantitative research, but the industry standard is NOT to only conduct one focus group.

Quantitative research, on the other hand, aims to quantify the information that is collected and therefore typically applies some form of statistical analysis. The outcome is usually a recommended course of action or a hard measure of opinion, attitude, or behavior. Sample sizes are typically significantly larger and because the research participants are generally drawn using a probability-based sampling approach (one in which everyone in the sample frame has an equal chance of being invited to participating), the research data are considered representative of the population of interest. The major difference between quantitative and qualitative research is the statistical validity of the data collected—this in turn affects the ability to draw inferences to a population as a whole. (Remember, with qualitative research, you can't do this.) For instance, the industry standard for a statistically valid survey is about N=400 which provides a margin of error of +/-5%. Deciding upon the N or number of completed interviews is dependent upon your research needs and trade-offs between how much margin of error you are willing to accept and your budget. As the N grows larger, the margin of error gets smaller—but the cost of collecting the data may increase.

So, the costs of quantitative are highly variable and are dependent upon your research method, the complexity of sampling, the survey length, and your budget available. Chris M—who will speak a little later shared that for a telephone survey of 400 residents in the State, the survey cost is about $35,000 - $45,000.

Slide 7

Selecting a Research Method

  • Qualitative—directional insights; exploratory
    • Test ideas, designs/prototypes, branding
    • Understand target population opinions/attitudes about programs/products
    • Pretest survey instruments
    • Explore survey findings
  • Quantitative—confirmatory
    • Point-in-time measure of attitudes, behaviors, opinions
    • Establish baselines
    • Measure shifts over time (trending)

In deciding whether you need qualitative or quantitative research, you need to go back to your research objective. Remember that qualitative provides directional insights and is generally exploratory in nature.

  • Use it as part of your situation analysis or a predecessor to quantitative research to help formulate survey questions or even to pretest survey instrument. Or, you can use it to explore survey findings.
  • You can use it to test campaign or product prototypes/concepts—brands, draft PSAs, informational brochures—before you spend money to finalize a logo, print thousands of brochures, or finalize a PSA.
  • You can use it to understand a population’s attitudes and opinions on issues.

We’ve conducted focus groups for all these purposes. A couple of years ago we tested a brand—the look and feel—for a campaign to support a major highway construction project—the campaign would inform drivers about road closures, delays due to construction, how to access businesses along the construction zones. The PR firm had selected a brand for the campaign drew on NASCAR or a racing theme, complete with racing flags as the focal point of the campaign design. The prototype branding failed miserably in the focus group tests with the highway users….they felt the campaign branding conveyed that it was alright to speed in construction zones.

Next, you’ll want to consider quantitative research for research that needs hard numbers or a point in time measure of something—customer satisfaction, approval or support of a new program, perception of the importance and need for a program or product innovation. You can also use it to define behaviors—the extent to which the public wearing seatbelts, helmets when they bike or drive motorcycles. Quantitative research can measure campaign effectiveness or evaluate programs—and measure shifts in attitudes or behaviors over time.

You’ll want to match the research approach to your research need.

Slide 8

Focus Group—Trade offs

  • Strengths
    • Produce deep insights and wide range of ideas
    • Group setting is comfortable and participants are likely to be highly engaged
    • Allows observation by viewers and sessions can be recorded
    • Can be conducted quickly
  • Limitations
    • Results are exploratory, not conclusive
    • Cannot generalize to the population as a whole
    • Susceptible to client/researcher bias
    • Requires professional moderator
    • Unstructured nature of data makes analysis time consuming
Slide 9

Telephone—Trade offs

  • Strengths
    • Reach majority of Households (cell phone)
    • Fast turnaround
    • Higher response rates
    • Less question nonresponse
    • Interviewers can ask for clarification
  • Limitations
    • Generally, higher cost
    • Respondents may "screen"
    • Interrupts Respondents free time
    • Not including cell phone sample may cause sampling bias
Slide 10

Mail—Trade offs

  • Strengths
    • Relatively inexpensive
    • Reach households with address-based sample
  • Limitations
    • No probing/clarification
    • Lower response rates
    • Less control over respondents
    • Slower turnaround Length can affect completion rates (lower)
Slide 11

Online—Trade offs

  • Strengths
    • Very low cost
    • Can be very fast
    • Flexibility to respondent
    • Provide more detail on open-ended questions
  • Limitations
    • May not reflect population as a whole (not statistically valid if conducted as a population survey)
    • Requires multiple reminder emails
    • Unintended respondents may reply if passcodes not provided Lower completion rates on longer surveys
Slide 12

Putting Data to Use

  • Processing and weighting
  • Coding (qualitative and quantitative)
  • Basic analytics (frequencies, cross-tabulations, tests of significance)
  • Higher level analytics (customer segmentation, stated preference, conjoint analysis, and models)

Finally, putting data to use. The analysis plan should be part of your research plan before you actually conduct the research. Basic analytics range from coding of open-ended questions from surveys—or the information gathered in a qualitative effort. This requires grouping thoughts and ideas into a structured format so that themes and insights can be drawn out in a meaningfully.

For surveys, before you conduct any analysis, the data will need to be processed or “cleaned” and may or may not need to be weighted. Basic analytics may include frequencies of data counts and percentages and cross tabulations. Tests of significance may also be considered basic analytics. You can also consider higher level analytics such as customer segmentation, and deeper analysis requiring special skills which I won’t go into now.

The main message here is to put the data to use. If you’ve defined the research objective, matched the research method to the need (and conducted high-quality data collection), you should end up with data that will be useful and meaningful. The worst thing is to have research sit on a shelf and get dusty! Use it.

Slide 13

Diagram: 6 Guiding Principles

Finally, I am going to quickly end with the set of guiding principles that are outlined in NCHRP 20-07, Putting Customer Research into Action. These were drawn from a review of hundreds of surveys conducted by State DOTs over the past ten years and from case studies demonstrating best practices in market research—including the Mn/DOT program.

1.First, have a clear research objective. Fully define why you are conducting the research and the issue/problem that needs informing. Most importantly, know at the onset how you will use the data to inform the issue/problem or research question. 2.Second, if you aren’t a market research expert, don’t worry. There are research experts in your agency (like Kathy and other programs that routinely conduct primary research), other agencies and external to your agency including consultants. Partner with them for help. 3.Next, we learned that successful research efforts have a research champion—someone external to the research team (a decision-maker, a manager) who supports the research. This may help secure the funding for the research and having the champion can help at the end of the research to convey the findings and see that they are adopted. 4.Fourth, match the method to your need. We spent quite a bit of time on this, already. 5.Fifth, consider customer segmentation—digging deeper into your data to better understand your study population or market on levels that are meaningful to your research need. 6.Finally, apply the research. Put it to good use!
Slide 14

Relevant Resources

  • NCHRP 20-07, Task 260, Putting Customer Research Into Action (2009)
  • NCHRP 20-78, Communicating the Value of Research (2009)
  • NCHRP 08-36, Task 74, Customer Research Practices & Applications: A Guidebook for Practitioners (2007)
  • NCHRP Report 511, Guide for Customer Driven Benchmarking of Maintenance Activities (2004) NCHRP Report 487, Using Customer Needs to Drive Transportation Decisions (2003)

Last but not least, here are some resources I think you’ll find particularly useful for conducting research.

Slide 15

Relevant Resources

Sceen Shot: Office of Management and Budget, White House website

For those of you working in Federal Agencies, you’ll also want to know about OMB guidelines for conducting research and preparing Information Collection Requests, etc.

Slide 16
Slide 17

More Information

Events

Contact

Kathleen Bergeron
Highways for LIFE
202-366-5508
kathleen.bergeron@dot.gov

PowerPoint files can be viewed with the PowerPoint Viewer
Updated: 04/04/2011

FHWA
United States Department of Transportation - Federal Highway Administration