Skip to contentUnited States Department of Transportation - Federal Highway AdministrationSearch FHWAFeedback
Highways for LIFE

ArrowPutting Market Research to Work!

<< PreviousContents Next >>

Putting Market Research to Work!

Chris McMahon
Mn/DOT
St. Paul, MN

July 22, 2010

PowerPoint Version (1.3 mb)

I am honored to share with you today some of the projects that have led to high impact decisions, made possible by adding end-user input to the table with engineers, planners and other strategic thinkers.

Slide 1

Intro/Overview – Getting to Know Mn/DOT Market Researchers

  • Our function began in 1994 and I came aboard shortly thereafter, have been here since.
  • Champions- Yes! And will tell you other necessary elements for successful integration.
  • What else will I share, on UTILITY of the data?
    • Depth and Breadth of Offices Using
    • Decisions We Have Informed
    • Where We Are Going…

Very quickly, my plan today includes sharing with you:

  • The long road we have been on, here at Mn/DOT – when the first in-house mkt. res. consultant was brought on board from the private sector in '94. Within two years, there were 4 of us, all coming from the supplier/consultant side in the industry. This came from demand by many offices and champions!
  • So, yes- will talk about champions, briefly, and then move on to:
  • Other factors that have made our function succeed
  • A quick glance at the many early adopters whom are still heavy users of our consulting/ project mgmt services
  • Then move to the meat of my presentation: how we have helped to inform decisions – note: many projects will be listed; some I may choose to zip through a graph, because I want to get to the project decisions that will most interest you. However, you can request whole presentation for further review at later time, and contact us at Mn/DOT for full reports or Exec. Summ's
  • Last and not least: our newest way of 'harvesting' timely feedback via an Online Community, and how we are using that on a weekly basis
Slide 2

Champions are "the Beginnings"…

  • Funding is essential in beginning & often when newer methods evolve
  • Education is the key to continuous usage; consulting is greater part of our work
  • Top-down Support is another Driver
  • (why it is good to have Champions at Top)
  • Mia told us earlier that a critical component for successful integration of consumer input, is to seek out Champions –we have had many, and they helped us to get started; kept promoting us by telling how they are using the info, in turn: helping to secure funding across the org. Early dollars set aside for the whole agency to use was critical to us in the beginning, while we were teaching what this decision-making tool IS.
  • Our continued success is in the consulting and re-education we are doing annually…. For new grad engineers, new leaders that have not been exposed, etc. (Took many years- up until last, and we re-teach continuously the diff. between customers and stakeholders.) –Mia mentioned doing segmentation studies, and in 2000 we did that. The greatest outcome was confirming that nearly all customer segments think the same (Farmers stand Alone!) –But the best thing we were able to clarify was that stakeholders are our partners…those whom we work with to provide services to end-users. We continue to teach this with each new Admin.
  • We are (HAPPILY!) exploited by the decisions that are made or influenced with our information…. And this is the key outcome of the consulting we do, using the historical and current data we have.
  • Lastly, we succeeded at Mn/DOT because some of our champions were at the top… it's critical to get buy-in from leaders, and since Tom Sorel came aboard, this has been a boon to our visibility and progress..
Slide 3
Diagram
  • Mia told us earlier that a critical component for successful integration of consumer input, is to seek out Champions –we have had many, and they helped us to get started; kept promoting us by telling how they are using the info, in turn: helping to secure funding across the org. Early dollars set aside for the whole agency to use was critical to us in the beginning, while we were teaching what this decision-making tool IS.
  • Our continued success is in the consulting and re-education we are doing annually…. For new grad engineers, new leaders that have not been exposed, etc. (Took many years- up until last, and we re-teach continuously the diff. between customers and stakeholders.) –Mia mentioned doing segmentation studies, and in 2000 we did that. The greatest outcome was confirming that nearly all customer segments think the same (Farmers stand Alone!) –But the best thing we were able to clarify was that stakeholders are our partners…those whom we work with to provide services to end-users. We continue to teach this with each new Admin.
  • We are (HAPPILY!) exploited by the decisions that are made or influenced with our information…. And this is the key outcome of the consulting we do, using the historical and current data we have.
  • Lastly, we succeeded at Mn/DOT because some of our champions were at the top… it's critical to get buy-in from leaders, and since Tom Sorel came aboard, this has been a boon to our visibility and progress..
Slide 4

Early Adopters & "Heavy Users"

  • Traffic Data Gatherers (Omnibus, since "87)
  • Maintenance (Maintenance Business Planning "94)
  • Construction
    • WZ Safety/Training
    • OCIC- Construction and Innovative Contracting
  • Traffic Engineers
    • Congestion Mgmt
    • Safety Engineers
  • Upper Mgmt; Architectural Design; Planners
  • A quick "Breadth" look at those offices who readily wanted to do research among our customers/ the taxpayers who are using the transportation system.
  • Just a note at the top pointing out that Mn/DOT was doing MR before we came, via an Inter-Agency Omnibus Study thru the U of M. (Will talk about that more in a second)
  • The order here, of those who most USE the information is not necessarily by weight… all of these heavy users come to us because of the success they have had with customer data being part of their decision making.
Slide 5

Ongoing Tracking Studies…

  • Omnibus - - - Annual since "87, (minus one year)
  • Maintenance Business Plan/Products and Services Study "94-"05 (4)
  • Perception (of Cngstn Mgmt Tools)Tracking – 9 Waves, "96-"07
  • Congestion Tolerance – Every 3 years : "98 -"07
  • Snow and Ice/ Bare Pavement – "99 and "07
  • Community Construction Evaluation – "05 and "08
  • Speed/HEAT Evaluation – "05-"06 and "09-"11
  • We have many tracking studies that have developed over 16 yr….. The Omnibus being the only one in existence when we came. And the only one we do annually. For those unfamiliar, an Omnibus=
  • annual "pulse –taking" of public; and
  • another way to touch on new issues in a quantitative "quick read;"
  • It also houses a combination of performance measures that allow us to understand behaviors and level of

service. IT IS ALSO VERY INEXPENSIVE FOR EACH CONTRIBUTING OFFICE TO AFFORD – THEY SHARE THE COST OF SET-UP AND PAY LESS FOR A FEW QUESTIONS.

  • We are very careful to let people know that an Omnibus is not a replacement for in-depth research on any given topic…. Those issues we see raised in the Omnibus warrant further/deeper research via single studies.
  • As I go forward, I will be "trolling the waters" here, giving you the barest of the backgrounds that you need, to fully relay the utility of the data. We will not be talking about all of these tracking studies, but for the ones we do touch on,
  • If you are a "deep sea fisher," wanting to know more of the why and wherefores,- again, you will have contact info at end and if I go quickly enough we will have time for Q & A when I am done!
Slide 6

Other Impacting, Single Study Efforts

  • Merge Patterns in Work Zones "01
  • TH 36 Closure, Pre and Post Studies "07-08
  • Mileage Based User Fee (MBUF) "07-"09 Quality of Life Pilot – "09

We are going to talk about how we have used info from these studies and maybe 1 or 2 others that ended in pivotal decisions. However, I will address the last right here and not come back to it.

  • I know that at the Fed. Level you are interested in this topic and I believe you are callling it "Liveability" ? Our "QOL" project is a 3-phased one, and the pilot was simply a few focus groups to help us nail down language and get our brains around how to approach the topic, get the public focused and then take it out to the whole state in a second phase of 20-30 groups.
  • What we will end up with is verbiage to apply to a quant –study: One that will tell us how transportation INTERSECTS with quality of life. So that is the only way we are using the Pilot findings. I don't want to get hung up on this one, so to move on…
Slide 7

A "Sampling" of Impactful Decisions Made

Maintenance Business Plan, "94

Bare Pavement/Snow & Ice, "99 & "07

In case I have not mentioned it yet: our std of operation in the design/investigation stage of a project is to ask "what decisions will this information inform?" – This has contributed to our success in utility, because we deter the projects that are "nice to know" rather than "need to know"

  • I am quickly going to show you two major resource-saving decisions that Maint.Office has made, and how we went back to ensure they were the right decisions
  • First, know that the '94 project was underway when our unit was initiated! The goal was to understand importance and satisfaction for the many services that maint. provides.
  • This project also included a cost-allocation exercise so that we could have the consltnt create quadrant opportunity windows…will explain as I show….
  • Before I get to that, will say quickly that the other study, on snow plowing, was also intended to determine if we were over-performing and could resources be saved.
Slide 8

After 4 waves of research, spanning 11 years, the conditions of the sides of the hwys and frwys stayed in lower right quadrant.

1994

Slide 9 - Table 1

2005

Slide 10 - Table 2

Excuse the kitty-wompus look! Scanning is great technology for those days before our reports were all electronic. But I am not a whiz at importing cropped charts! And, I TRIED!

  • These are quadrant opportunity windows…one on the left from '94 and on the right from the 4th wave of this study. This is a great visual way to show where importance and satisfaction intersect- the Upper Left is where we are doing well…."leave it alone" resource-wise.
  • Notice that the lower right quadrant shows where the public thinks things that are less important may be irritants (the consultant's language) – IN the context that their cost allocations resulted in their telling us to spend the least amt. of $$ on the roadsides.
  • Net result: in '94 we cut back on mowing…. Saved money and worked on planting native grasses and wildlowers to take the place of overgrown grass.
Slide 9

Since "07, phone calls began trickling in,

With consumers and perhaps some local officials complaining about mowing on the roadsides. The implications were that Mn/DOT was not doing enough…..

..and the Maintenance engineers asked if we could confirm or deny...

There were no problems in the mowing cutback for 11 years, and the repeat waves of that study kept confirming that the decision was right…

  • However, in '07 all of the districts were getting a few calls, and they increased in '08-
  • So the Maint. Off. Asked us to help confirm that mowing was okay…. Were they hearing from the vocal few?
  • We used the Omnibus to do this… and the next slide shows how we have been tracking satisfaction for Maint's CORE services
  • We rotate 1 or 2 different services in and out, - so we asked Mowing specifically for the first time in '09 Omnibus
Slide 10

Maintenance Performance (2009 Omnibus)

I want to know HOW WELL you think Mn/DOT is doing in each area. Use any number from 1 to 10; a "10" means they are doing an "Extremely good" job in this area, and "1" means they are doing an

"Extremely poor" job in this area.

Diagram: Maintenance Performance (2009 Omnibus)

Not the easiest slide to read, but gives you a sense of how Maint. has been using this Omnibus study to track for Performance measures.

  • Note that mowing's mean score on a 1-10 scale came out just above the target for performance; (the jagged line between 06 and 08 just indicates we skipped a year for the Omnibus.)
  • More importantly, we asked all of those who rated mowing a 7 or less, if Mn/DOT was doing just the right amount, not enough or too much. Less than 10% thought we should mow more: letting us tell Maint. that YES…the calls WERE coming from the 'vocal few' - - nearly all in the 6 and below range thought we were mowing just enough on state hwy's and frwy's
Slide 11

Moving on –Bare Pavement 2007

  • 1999 Definition of bare pavement changed from "fully bare" to "bare between the wheel paths" Was this still true 8 years later?

With the success of saving resources on mowing, Maint. came along and asked how we might test the waters for Snow removal.

  • Using a 'Central Location Test' method, we showed large groups different variations of snow amounts, on different types of roads in '99, using video clips that we taped here at Mn/DOT (Consultant did the rest!)
  • The study was large enough and reliable enough to change our way of plowing because the public told us that "Full Bare" was not necessary at 6 AM after 3" of snow…we could change the standard to "bare between the wheelpaths" for six hours after a heavy snowfall…
  • Because we wanted to know 7 years later if this was still acceptable, we repeated the study…and learned we were "OKAY"! Another highly valued service could cut some corners without public outcry.
Slide 12

A "Sampling" of Impactful Decisions Made

Merging in Work Zones

Second H.E.A.T. Campaign Eval.

I have decided to just quickly tell you that on these two topics we are currently informing decisions being made today, even though some of the data goes back to '01. The Merging study was a study done so well that we dragged it out several times, mostly in this construction season, to try to move the 90% of "Minn-Nice" drivers, to engage in later merging.

Am leaving detailed notes on this page for those who want to know more… Same is true for our HEAT (heavy enforcement of aggressive travel) campaign. We are helping DPS to use consumer language in their press releases, to indicate what is truly threatening to the public, other than speeders. AGAIN: More detail on each slide…but want to get at the projects you might be most interested in….

2001: Mn/DOT WZ safety folks began trying to understand Merging issues; it has remained an on-again/off-again topic ever since. Early learnings and changes via dynamic message signs in the '02 construction period showed that

  • It would be hard to move "MN-Nice" drivers toward the "Zipper effect"
  • Attempts were tried with some success using dynamic message signs, yet it didn't stick in subsequent years without more aggressive signing
  • Last year became a bit "WARMER ISSUE", and This year: With the massive amount of WZ's around, an aggressive media campaign, 'Hard Signage' (orange) and a few good editorials got the ball rolling and we used our Online Community to further flesh out perceptions and get public talking about how difficult cultural shift is.
  • I bring this topic up because it is a clear example of how we can do a single study that will aid decisions and yet what we do isn't always quite enough. For this project, it has taken 7 years to continue educating and using our new research tools to 'dig a little deeper"
  • The change in behaviors we are seeing this year are due to heavy media campaigns and continual improvement of signage; all based to mkt res done in '01 and '10
  • Some of you in DC are likely aware of heavy enforcement of speed campaigns, being promoted by the Dept's of Public Safety and State Hwy Patrols. Minnesota is in the midst of it second campaign since '05.
  • This time we are surveying more, to dig deeper into Other Aggressive behaviors in order to help our own DPS and SHP to hear the public attitude toward what is most threatening to them
Slide 13
While working primarily to gain awareness and attitudes regarding heightened speed ticketing, we are learning more about what else is threatening and the public would like to see enforced.

Following chart asks:
"People feel differently about how safe or dangerous different types of driving behaviors are. In your opinion, how much of a threat is it to the personal safety of you and/or your family if OTHER DRIVERS do the following?" Using the scale of 0 to 6, with 0 meaning Not a threat and 6 means a Major threat to the personal safety of you and your family, How much of a threat is …

What we are working towards is a more rounded view of what drivers and passengers think are behaviors that most need citing- the next chart will show some surprising data from Wave 1 of this campaign and I will tell you that the next chart is revealing…..

Slide 14

Heat 2009 Wave 1
Presentation to MSP: 5/12/10

Base = 800 Total unless noted

[The red arrows indicate significant differences]

Most important here: Both Distracted and Aggressive behaviours (as they self-defined:Tailgating/weaving), though in the middle, are seen as much more threatening than speeding either over or under limit by 10 mph (far right).

the good news: the 2 most threatening, are the least seen observationally; THE BETTER News is that Mn-STATE HP AND D.P.S. ARE ADDRESSING THESE BEHAVIORS IN THEIR MEDIA RELEASES in association with enhanced Speed enforcement.

Slide 15

A "Sampling" of Impactful Decisions Made

Community Construction Eval.

TH36 Closure pre and Post waves

Now for some topics we have researched that may be closer to your hearts…

  • The first began with one district wanting help…. '05
  • Turned into a first-ever cost collaborative effort between 5 Dx's
  • Funds were matched by OCIC (our Construction Office) – out of interest by O.D., Gary Thompson in learning 'the tipping point' in attitude shift among public (ie, negative speculation toward positive acceptance)
  • Most were highly conspicuous and contested projects that had been completed in '05
  • The 2006 study fielded when all but one project was done, and provided insights that have helped all project mgr's to be more proactive with communication strategies before, during and after const. projects
  • Later became internally known (in OCIC) as a 'Best Practice' for most projects that are sizable and lengthy
  • BECAME PIVOTAL TO OUR FAMOUS (ty, KB!) TH36 Closure study and subsequent assistance in funding from HfL
Slide 16

Overall total of n=1000; 200 per Dx

In the notes below (which I am not going to cover as you will get via the whole presentation) are the explanations of the colors, put in by my report at the time- she managed this project. I wanted you to see this slide so that you could see where information rose to the top as indicated by the gold shading.

This chart shows a clear pattern of response –

Gold – rating above average

Rust – rating below average

This corresponds to the results of the overall opinion question, in which D8 had the highest percentage of negative responses and D4 & D6 had the highest percentages of Very Pos

Slide 17

Pre-Closure Study, TH 36: 2006

A concept test with 2 scenarios read in rotating order to 3 respondent types:

"Highway 36 from Century Avenue to White Bear Avenue would be completely closed for a 5-month period in 2007 from April to September. It would then be open to one lane in either direction from September to November 2007, when it would again be opened to two lanes in each direction. MnKnight Road would also be one lane in either direction from April to September of 2007. Margaret Street would be closed and a vehicle bridge completed in the second year of construction, 2008. A pedestrian bridge will be built near First Street and is expected to be completed in the fall of 2007. Project cost for this construction scenario is 15% lower than the other scenario."

Scenario II is as follows

"In this scenario, Highway 36 from Century Avenue to White Bear Avenue would remain open to traffic with constant lane closures and delays during non-peak hours for 2007 and 2008. McKnight Road would be one lane in either direction from April 2007 through November 2007, and Margaret Street would be closed, with a vehicle bridge completed in the second year. A pedestrian bridge near First Street is expected to be completed in the second year as well. The McKnight signal would remain until July 2008 when it will be removed. Project cost for this construction scenario is 15% higher than the other scenario."

Now for Hwy 36 and its closure: Just note that I wanted you to have the exact way we presented the concepts for people to choose…. We surveyed the community most affected, a smaller 'burb called 'the Town of No. ST. Paul' as they would be most affected;

We also surveyed all the thru-commuters…many many of them coming from Stillwater MN- a highly affluent area that was the receptor site for Wisconsinites on the east, so Hwy 36 was greatly used in the thousands each commuting period, by those from the east, to get to the Metro area

Results and outcomes on next page….

Slide 17

Net result to "06 Pre-Study: 50/50 among all but No.St. Paul residents (3:1 in favor)

Post Closure Study "duped template" of CCE study from"06- - done near end of 36 Project in 2007

Opinion of Decision to Close Road: "07 (project nearly completed)

"How do you feel about this decision to close Highway 36 instead of leaving lanes open for traffic?"

Agreement:

Residents: 92%

Businesses: 84%

Thru-commuters: 89%

(base: 400 residents, 100 businesses, and 400 through-commuters)

Estimated Project Savings: ~ $5M
  • Initial reactions left us with easiest recommendation: Use your tech'l expertise to do the right thing….
  • Closure was re-opened in less than the promised 5 months due to financial incentive
  • We learned that far and away the project could be credited for its frequent, timely and thorough communications efforts, esp. among business owners. Signage was cited as good, but could have been better
  • Ongoing takeaway: This was a unique setting; is not a green flag to close down 'just any highway' for more than 2 months…. But, as we have been monitoring since '95, via a separate study and the Omnibus, proportions increase slightly each few years for those wanting more disruption for a shorter period of time, in the end. In other words, we knew that the potential was there for it being a "ripe time" to close down a major road.
Slide 19

What we are doing today…

  • Learning to be more nimble: Began our online community (OLC) in this calendar year… it is about 6 months old.
  • This allows quicker turnaround for public input that can inform topics in a way that is stronger than most qualitative, and yet still is not an "even exchange" for in-depth quantitative studies. Allows Mn/DOT to "listen in" on interaction from a rep-to-Mn-census group of citizens" dialogues
  • John mentioned two types of Community Blogs….ours is a closed community of randomly chosen people we recruited for one year.
  • This does not allow and 'vocal few' to come in and change the representativeness of the data.
  • You will soon be seeing and hearing more on this, as it is one of Tom Sorel's favorite topics!
Slide 20

Innovation: Mn/DOT"s Online Community 4/19/10

Member Spotlight

Nathan A, District 3

"Mn/DOT makes me think of care and commitment. If Mn/DOT didn't care about Minnesotans, then why would this community exist? Prior to learning about this community, I never would have thought that Mn/DOT would go to such lengths to find out what Minnesotans think. I am pleasantly surprised to discover that Mn/DOT is making such a

commitment to seek feedback and provide an environment where

Minnesotans can interact with each other and members of Mn/DOT, all to hopefully ensure the growth and improvement of transportation in Minnesota for the years to come.Thanks, Mn/DOT!"

This slide….

  • is a piece of a "snapshot"
  • leaves a bit of a wrong impression, as the closed community members do not directly interact with Mn/DOT (but we are glad that some think they do, for we are always in the background, via the online facilitators)
Next slide: is part of a "front page" summary for each survey, which includes snapshots and survey results for that week.
Slide 21

Innovation: Mn/DOT"s Online Community 07/19/10

Mn/DOT Talk Snapshot

Citizens Offer Personality to Mn/DOT Survey: Mn/DOT Personified(N = 181)

Members were presented a list of 34 adjectives and attributes and asked to select those that they felt best exemplify Mn/DOT's personality. Most frequently selected attributes include "Hard Working,""Reliable" and "Dependable." Conversely, the options of "Green," "Open-Book" and "Transparent" rank amongthe least frequently mentioned choices, offering some opportunities for Mn/DOT to investigate.

The true objective in this dialogue was to understand how many words we threw out there dealing with public trust and transparency would rise to the top….. It is great and valuable learning that this is an area in which we need to roll our sleeves up.

Slide 22

SO: how are we utilizing this rich but not cheap information that comes from a world of methodologies, that overlaps qualitative and quantitative? (While being fully neither!)…….

We are informing currently "hot topics" with on-time feedback from more than a few focus groups; Merging was revisited here after construction season started; a few of the other topics include (past 6 months):

  • Transparency in organizations
  • Roundabouts
  • Smooth Roads
  • Snow & Ice at very end of season
  • We are excited to put the Front Page summaries and Snapshots on our internal website, and gearing up for an External site as well.
  • But first and foremost, we are happy to have another tool in our toolbox- a way to get some of the people we may have been missing in phone and mail methodologies.
Slide 23
  • The Messages are Clear for All of Us…. With a customer base that is a moving target, we must go out and meet them on their own turf;
  • We need to make use of every methodological tool;

There is no longer a silver bullet as we had in phone surveys;

  • To Be Nimble, in guiding decisions, we are choosing from a wider array of tools;
  • Each tool is chosen for the amount of precision needed!

This is what we know today!!

Slide 24

Contact Info:

Chris McMahon, Director of Market Research
651.366.3771
chris.mcmahon@state.mn.us

Karla Rains, Statewide Mgr., Market Research
651.366.3172
karla.rains@state.mn.us

Please don't hesitate to contact either me or my supervisor, who was the first to be an in-house consultant, left for 5 years to revisit the private sector, and brought back with her the cutting-edge OLC knowledge, about a year ago.

Hopefully we have time for Questions!

Slide 25

More Information

Events

Contact

Kathleen Bergeron
Highways for LIFE
202-366-5508
kathleen.bergeron@dot.gov

PowerPoint files can be viewed with the PowerPoint Viewer
Updated: 01/30/2013

FHWA
United States Department of Transportation - Federal Highway Administration