U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
REPORT
This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-14-069    Date:  October 2014
Publication Number: FHWA-HRT-14-069
Date: October 2014

 

Collecting and Analyzing Stakeholder Feedback for Signing At Complex Interchanges

CHAPTER 2. METHODS

This section of the report describes the methods used to conduct the stakeholder activities and analyze the data.

STAKEHOLDERS

A key element to the success of this project was the participation of stakeholders. The recruitment methods used and general characteristics of those who participated are described below.

Stakeholder Recruitment

The target stakeholders were transportation professionals, primarily roadway designers and engineers, who work for State transportation departments and have responsibilities related to interchange planning, design, and/or maintenance. Stakeholders were recruited using existing contacts from our Federal Highway Administration (FHWA) complex interchange project panel, through networking and recruitment by FHWA contacts, and at two in-person events: a local Washington Institute of Transportation Engineers (ITE) chapter meeting and the Traffic Control Devices Consortium Pooled Fund Study Annual Meeting. The recruitment flyer, provided in appendix A, was prepared and distributed in-person and electronically to provide background information on the project to stakeholders.

Three criteria were used for recruiting stakeholders to participate in the interview process. The first was based on the recommendations of the members of the complex interchange project panel. The project panel has varied experience with aspects of interchange design and is therefore familiar with States or regions that have a number of complex interchanges. The second criterion was to obtain a geographic dispersion of individuals. The goal was to obtain feedback from a distributed group of States or regions to allow us to gain information about any environmental, geographic, or organizational differences across the country. The third criterion was to gather a variety of professional perspectives from individuals with varied backgrounds, including those who have experience with signing, marking, and geometric design. If the initial contact in a State was with an individual who did not have experience with interchanges or who felt another individual in his or her group or department would be more suited for the interview, contact information for this other individual was provided and they were contacted about the project.

For the follow-up survey, the link to the questions was emailed out to the entire original stakeholder contact list, even those who did not respond to the original request for participation. Participants in the interviews were also emailed, if they were not included in the original list. Those who were emailed were invited to forward the link to anyone else within their organization who would be well-suited to respond.

Participating Organizations

Figure 1 illustrates the geographic distribution of stakeholders who were interviewed, participated in the follow-up survey, or both ("dual participant"). Each region indicated as an interview participant took part in a single interview, with the exception of Washington. Two interviews were conducted with the Washington State Department of Transportation, with two individuals on the first interview and one individual on the second interview. Note that on figure 1, Alaska, Hawaii, and Puerto Rico are shown beneath the contiguous United States.

This illustration is a map of the United States and Puerto Rico. The legend of the map states that States who participated in the interviews have no color fill, States who participated in the follow-up Web activity have a light gray fill, and States who participated in both activities have a dark gray fill.
©Battelle
Figure 1 . Illustration. Map of State and regional transportation departments that participated in the interviews, follow-up survey, or both.

Table 1 provides the number of participants on the phone for each interview and the number of participants in the follow-up survey. Each interview involved one to five individuals representing their organization. Interviews were conducted with 17 State transportation departments, representing 28 individuals. The survey was completed by 66 individuals (or groups), from 32 regions, including some individuals from FHWA. Note that the number of individual participants does not necessarily correspond to the number of people whose opinions are represented. Some individuals who participated in the interview did not contribute comments, while some survey responses may include the opinions of multiple individuals within a group.

Table 1 . Number of participants in each activity by State/region.

State/Region

Interview

Survey

Arkansas

1

California

1

6

Connecticut

1

1

District of Columbia

 

1

FHWA*

5

Florida

1

2

Georgia

1

1

Hawaii

1

Idaho

1

Illinois

1

1

Iowa

5

3

Kansas

2

Louisiana

2

Maryland

1

1

Massachusetts

2

2

Minnesota

2

Mississippi

1

Missouri

1

1

Nebraska

1

Nevada

1

New Jersey

2

New Mexico

1

1

New York

3

3

North Carolina

1

 

Oregon

3

Pennsylvania

1

2

Puerto Rico

1

Tennessee

2

Texas

1

 

Vermont

1

Virginia

1

6

Washington

3

3

West Virginia

1

Wisconsin

3

3

Wyoming

2

Total

28

66

*5 FHWA employees also completed the survey.

Stakeholder Characteristics

Figure 2 is a histogram showing the distribution of years of experience across stakeholders. This figure shows that the full range of years of experience was represented by the participants in each activity. In particular, a significant number of the participants had many years of experience in their technical domain.

This histogram shows the two different sets of stakeholders' years of experience. The x-axis is labeled with two different colors for the two sets of stakeholders: interview (dark gray) and Web activity (light gray). The y-axis is labeled as number of stakeholders. The ranges (in years) and number of stakeholders presented on the histogram are 1¬-5 (4 interview participants and 8 Web activity participants), 6-10 (4 interview participants and 9 Web activity participants), 11-15 (6 interview participants and 10 Web activity participants), 16-20 (4 interview participants and 7 Web activity participants), 21-25 (14 interview participants and 14 Web activity participants), 26-30 (5 interview participants and 15 Web activity participants), and 31+ (0 interview participants and 7 Web activity participants).
Figure 2 . Histogram. Distribution of years of experience across stakeholders.

Stakeholders had experience with the signing, markings, and geometry of interchanges and held jobs at a variety of levels. Table 2 shows the number of stakeholders associated with each combination of job category and primary technical domain. Many participants fell into the Engineer job category. Note that there was substantial overlap between these categories based on the stakeholder descriptions of their job responsibilities, but table 2 provides a high-level overview of the breadth of experience and domain-knowledge of the stakeholders. The table values in brackets represent the interview participants, while the values outside of brackets represent the follow-up survey participants. Since the survey was anonymous, it is unknown if some of the interview participants also completed the survey.

Table 2 . Distribution of job category and primary technical domain of stakeholders for the survey and interviews.

Job Category

Example Titles

Signs and Markings

Geometry

Planning and Development

Policy/ Compliance

Safety*

Manager/
Director

Statewide Director of Design; Manager of Signs, Standards and Specification

3

3

3 [1]

2 [5]

0

Project
Manager

Project Manager/Designer; Program Manager for Highway Construction Program

1

4

2 [2]

0

1

Engineer

State Traffic Engineer; State Sign & Marking Engineer; Policy and Standards Engineer

17 [9]

12 [1]

7 [6]

5 [3]

10

*The Safety category was not given as an option in the survey response set but was written in by many participants.

Although we originally intended to include consultants in the stakeholder group, many stakeholders said that they rarely use outside consultants for human factors or driver-related questions.

PROCEDURES AND MATERIALS

The phone interview was the primary method used to identify research needs and stakeholder concerns. This approach yielded a large and diverse set of research needs. A follow-up survey was then conducted with a broader group of stakeholders to gather information that could be used to prioritize the initial set of research needs.

Phone Interview Protocol

Interview participants received a handout before the interview via email. This provided them with background information on previous complex interchange projects and a list of topics that would be discussed during the interview.(2) Stakeholders were also asked to prepare one or two specific examples of challenges they had previously addressed involving complex interchanges.

Phone interviews were conducted by two researchers, one serving as the interviewer and the other serving as a note-taker. Notes were entered into an electronic version of the interview guide during the interview. All interviews were recorded to allow the notes to be checked afterwards for accuracy. After the notes were verified, the recordings were deleted. The electronic versions of the notes were later compiled by question across interviews, prior to data analysis.

Most interviews lasted 1 h. The interview closely followed the interview guide question sequence and stakeholders were given as much time as they needed to answer each question. As appropriate, follow-up questions were asked to either gain more information about the question topic or to better understand the answer provided.

A presentation provided via web conferencing was used to show examples of complex interchanges to stakeholders (see appendix B). This helped explain different topics such as destination grouping, visual perspective, and information overload, which can affect how drivers interpret the sign information or their understanding of how to complete the movements they want to make. The examples were intended to help the stakeholders come up with examples of their experience with complex interchanges in case they had trouble coming up with examples on their own. In addition to the presentation, there were instances in which the stakeholder shared his/her screen to show the example interchange on a map. If the stakeholder verbally provided the roads that make up the interchange (e.g., I-85N at I-985), this location was entered into Google Earthâ„¢ and shared on the web conference to help the stakeholders explain the example, and to help the researchers understand it.

Interview Guide

The interview moderator guide is provided in appendix C. The first set of questions in the interview guide asked about basic stakeholder background information including job title, job responsibilities, number of years in their current position, and how their role fits into the design or analysis processes for interchanges. If multiple individuals participated in the interview, this section was covered for each individual. The purpose was to obtain information about the individual's background and information about their design process to help understand their experience with interchanges.

The second set of questions was asked to gather information about an example of a time in which stakeholders identified and resolved or attempted to resolve issues with complex interchanges. These questions asked them for details about the example, such as how they knew what the problem was, how they tried to fix it, and what design information would have been useful, if it had been available. If the stakeholder had more than one example, the same questions were asked about each example. The purpose of this set of questions was to obtain specific information about interchange issues and relevant background information on stakeholder processes, such as their approaches for fixing issues, monitoring the success of those fixes, and filling information gaps.

The third set of questions asked about their experience with complex interchanges. Specifically, what they think makes certain interchanges more difficult for drivers to navigate, how often they do projects related to complex interchanges, and based on their definition of complex, approximately how many complex interchanges are in their area. The purpose of the third set of questions was to obtain general information on complex interchanges, their perspective on complex interchanges as an issue, and the pervasiveness of these interchanges.

The fourth set of questions asked about how they design for drivers, such as if they make any assumptions about what drivers need or what drivers know, how they identify driver information needs, what design fundamentals they use, and how they go about designing signing or markings for unique interchanges where there is not sufficient guidance in the Manual on Uniform Traffic Control Devices (MUTCD) or other sources. The purpose of this set of questions was to get an idea of how the stakeholders think about drivers as users of the interchanges and to help us provide information in future projects from a perspective that they are able to relate to.

The fifth set of questions asked about research needs in the area of interchange design. Specifically, what research would help them resolve interchange design issues, if there is any design information missing, and where the collective knowledge gaps are in the area of interchange design. The purpose of the fifth set of questions was to give stakeholders an opportunity to provide more general feedback regarding research needs.

Follow-up Survey Protocol

The follow-up survey was conducted using a web-based survey tool. The link to the survey was emailed out to the stakeholders, who were invited to distribute the link to others. The survey was comprised of 17 questions, which took approximately 10-15 min to complete.

A copy of the questions that were included in the follow-up survey is provided in appendix D. The main purpose of conducting the follow-up survey was to prioritize the research topics that were identified in the phone interviews. This prioritization is important information to guide the selection and development of the research plans in the following task of this study.

The first set of questions asked about participant background information including their job title category, primary technical domain, years of experience in that domain, and State transportation department or other place of employment. The purpose of this set of questions was to help categorize and look for trends among responses.

The second set of questions asked participants to consider 12 research topic ideas. Each topic was included with a scale to indicate the priority of conducting research on the topic, from "low priority" to "high priority," and also including an option to indicate that the idea was "not a perceived problem." Space was provided to include comments on the topic or ideas for clarifying the topic. Participants were asked to consider each topic independently of the others (i.e., not to worry about balance between high and low priority ratings).

The third and final set of questions asked participants to describe any other general human factors research areas, geometric challenges, etc., that they face that also need research.

DATA ANALYSIS

The data gathered during the stakeholder interviews was aggregated by question for later analysis. Prior to beginning the analysis, the data was sorted by topic. Data was then reviewed for trends across stakeholders and across topics.

The primary data gathered during the follow-up survey was the priority ratings and comments for the research topic ideas. One concern with the ratings was that regions that provided many responses may sway the ratings towards research ideas that would only be useful to their region. Therefore, before the ratings were analyzed, an average rating was calculated for each region that provided responses. Then, one rating per region was used in the analysis. (Upon investigation, however, it does not appear that this method substantially affected the ranking of the research topics.) The ratings given by FHWA were not considered in the ranking of the research topics, since their needs and role in design are different than that of the other stakeholders. All of the comments provided by all of the individuals were considered, including those provided by FHWA.

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101