Intersection Conflict Warning System Human Factors: Final Report

PART 1

After each of six videos, participants were asked the following two questions: “What would you do in response to the warning sign(s)?” and “What do the warning sign(s) mean?” The researchers typed the participants’ verbal responses, (displayed on the same screen as the videos) and asked the participants to verify that the transcribed response was correct.

During data reduction, the responses to the two questions were classified by the analysts as correct or incorrect, and the open-ended responses were reduced to up to three categories that best reflected the response. On the minor road, responses such as “I would slow down” were scored as incorrect. It may be that many participants did not stop for stop signs because many of those who said they would slow down then gave a correct interpretation of what the sign meant. This was reflected in the overall findings where the correct meaning was given around 86 percent of the time, whereas the correct action was described far less often. Some responses were scored as incorrect because the participants did not answer the question but rather editorialized about the signs being unnecessary or duplicative and then, even upon further prompting, would go no further.

Another challenge in interpreting the data from part 1 is that with six videos, it was not possible to conduct a factorial analysis to examine sign placement effects, placard use, and blank-out versus static warning signs. For this reason, those three factors were analyzed individually, and possible interactions (e.g., placard on blank-out compared with placard on static) were not examined.

The statistical analyses reported in this section were performed using generalized estimating equation models with binomial response distributions and logit link functions.

Effect of “WHEN FLASHING” Placard on Warning Comprehension and Stated Behavior

Comprehension of the conflict warning signs was not affected by the presence or absence of the “WHEN FLASHING” placard (χ² (1) = 0.0, p = 1.0). The number of responses judged correct as a function of placard presence is shown in table 8.

Table 8. Percent of correct responses to “What does the warning sign mean?” as a function of presence or absence of a “WHEN FLASHING” placard.

Although 86.4 percent of responses were judged correct when asked what the sign(s) meant, the percentage of correct responses to the “what would you do” question was lower (73.4 percent) both with and without the placard. Again, the presence or absence of the “WHEN FLASHING” placard did not have a significant effect (χ² (1) = 0.0, p > 1.0). Table 9 shows the number of correct and incorrect responses to the question “What would you do in response to the warning signs?”

Table 9. Percent of correct responses to “What would you do in response to the warning sign?” as a function of the presence or absence of a “WHEN FLASHING” placard.

The most frequent incorrect reactions were ignoring the sign and making irrelevant comments (e.g., “sign is unnecessary”).

The comprehension of the signs with and without the “WHEN FLASHING” placard did not vary as a function of either major or minor approach (χ² (1) = 1.3, p = 0.2) or as a function of road type on the major road (i.e., two-lane, four-lane undivided, or 4-lane divided) (χ² (2) = 1.3, p = 0.5). The stated response to the “WHEN FLASHING” placard did not interact with approach type (χ² (1) = 0.43, p = 0.5) or major road type (χ² (2) = 2.9, p = 0.3).

In summary, no evidence was found that the presence or absence of the “WHEN FLASHING” placard affected either driver understanding of the ICWS meaning or what drivers say they would do in response to the ICWS signs.

Effect of Blank-Out Versus Static Sign on Comprehension and Stated Behavior

Comprehension of the meaning of the warning signs was not affected by whether the warning was depicted on a standard sign or a blank-out sign (χ² (1) = 0.01, p = 0.9). The percentage of correct responses as to the meaning of the warnings as a function of sign type is shown in table 10.

Table 10. Percent of correct responses to “What does the warning sign mean?” as a function of sign type.

What participants said they would do in response to the signs varied significantly as a function of sign type. As can be seen in table 11, participants were more likely to indicate a correct action in response to the static signs than the blank-out signs (χ² (1) = 6.3, p < 0.05). Whether this small difference is of practical importance is dubious because the proportion of irrelevant responses was substantially greater with the blank-out sign (7 percent with blank-out versus 3.5 percent with static), and correct identification of meaning was not significantly different.

Table 11. Percent of correct responses to “What would you do in response to the warning sign?” as a function of sign type.

In summary, static and blank-out signs were found to be about equally effective in terms of comprehension, but static signs were more likely to result in a correct action.

Effect of Sign Placement on Comprehension and Stated Behavior

The location of the minor-road conflict significantly affected both comprehension (χ² (1) = 5.6, p < 0.02) and what drivers said they would do in response (χ² (2) = 15.5, p < 0.001). The percentage of correct comprehension and reaction responses are shown in table 12.

Table 12. Percent of correct responses to “What does the warning sign mean?” (comprehension) and “What would you do in response to the warning sign?” (reaction) as a function of sign location.

In summary, location mattered. When the minor road ICWS sign faced the drivers as they approached the stop sign, both comprehension and response were better than when they needed to look in the direction of the approaching traffic to see the sign.

PART 2

The perception rating results are shown in table 13. Complete data were available for 187 participants. Recall that not all participants saw the same messages; there were six different signs. Multivariate analysis of variance with statement rating (where 1 represented “strongly agree” and 7 represented “strongly disagree”) as the dependent measure and the six signs as the independent variable suggested that there was no effect of different signs on agreement with the statements F (105, 792) = 1.0, p > 0.05, where the F statistic is an approximation from Wilks’ Lambda.

Table 13. Perception ratings for individual statements.

Given that there wasno reason to suspect an effect of the six sign messages seen by different groups of participants, the 187 ratings on 21 statements were submitted to a maximum likelihood factor analysis. To minimize the number of factors with negative loadings, the polarity of the ratings was changed for statements 5, 10, and 11 by subtracting the original ratings from 8.

The promax rotated factor pattern that resulted from the factor analysis is shown in table 14. The promax rotation allows factors to be correlated. The three-factor correlation matrix is shown in table 15. The three factors accounted for 74, 19, and 7 percent of the variance in the ratings, respectively. The first factor appeared to be associated with how well the messages were understood and was labeled “comprehension.” The second factor appeared to be related to safety. The third factor appeared to be related to feelings of how necessary or distracting the signs might be and was labelled “affinity.” This analysis suggests that when thinking about the warning sign messages, the participants first considered how understandable the signs were, then the safety implications of the signs, and finally how likeable or pleasing the messages were.

Table 14. Factor loadings for three-factor solution.

Note: The criterion used for determining that a variable loaded on a factor was 0.40. Loadings whose absolute value exceeded 0.40 are indicated by bold type.

Table 15. Correlations between factors.

Figure 25 shows the factor pattern loadings for the comprehension and safety factors plotted on orthogonal axes and does not represent the 0.18 correlation between these factors.

Figure 25. Scatter plot. Comprehension and safety factors.

Figure 26 shows the factor pattern loadings for the safety and affinity factors on orthogonal axes and does not represent the 0.48 correlation between these factors.

Figure 26. Scatter plot. Safety and affinity factors.

PART 3

As described in chapter 3, participants were asked to rank 10 messages for the major road approach and 9 messages for the minor road approach. For each approach, the rankings were done twice, once with the “WHEN FLASHING” placard present and once without.

There are many ways to evaluate ranking data. The methods differ in their underlying assumptions, such as ordinal versus interval scaling, and transitive versus intransitive orders. Because it is not clear which assumptions were most appropriate for the present data, the following three different analysis methods were used: the Condorcet method, the Borda method, and the general linear model approach.

Condorcet Method

The Condorcet method finds a group consensus by translating individual rankings into pairwise comparisons where the winning candidate is the one that beats all other candidates a majority of the time. The result is a matrix for each participant where each row and column represents one of the signs. The matrices of all participants were added together, and the sign with the highest row total became the winner. Table 16 shows the Condorcet results for the minor road messages when the “WHEN FLASHING” placard was present.

Table 16. Condorcet results for minor road messages accompanied by the “WHEN FLASHING” placard.

Borda Method

The Borda method assigns point values inversely related to rank order (i.e., the top-ranked sign gets the most points). The winning sign is the one with the most points when summed across all participants.

General Linear Model Method

This method estimates means and standard deviations for each sign’s ranking. It assumes a normal distribution of means and homogeneity of variance. This method enables an estimate of which differences between signs are statistically reliable if the assumptions of the model are met.

Ranking Results

Table 17 shows the top three messages by each method of analysis for each of the ranking scenarios.

Table 17. Top three messages using three methods of analyzing the rank data.

For the minor road approach with a “WHEN FLASHING” placard, the top two message choices were the same for all three methods of analysis. “CROSS TRAFFIC” was the preferred choice with “EXPECT CROSS TRAFFIC” as the runner up. The top two preferred messages were changed when the “WHEN FLASHING” placard was not present. With no placard, the top minor road choice was “EXPECT CROSS TRAFFIC.” The runner up was “TRAFFIC APPROACHING WHEN FLASHING.” With no placard, “CROSS TRAFFIC” was the third most preferred choice.

On the major road, “CROSS TRAFFIC AHEAD” was the top choice regardless of analysis method and whether or not a “WHEN FLASHING” placard was present. “CROSSING TRAFFIC” was the runner up choice. “CROSSING TRAFFIC” and “VEHICLE CROSSING” exchanged second and third places depending on whether the placard was present.

Participants’ Explanations for Their Rankings

After ranking a set of alternative messages, participants were asked to indicate why they ranked the signs the way they did. After all participants had completed the study, analysts reduced the participants’ open-ended responses to a limited number of categories intended to capture the essence of the explanations. Each participant’s responses were reduced to 21 response architypes, with each participant having up to 3 categorized explanations.

The top five categorized explanations for the major road rankings are shown in table 18. The two most frequent comments indicated that the reason “CROSS TRAFFIC” and “EXPECT CROSS TRAFFIC” were favored was that the letters were large, which was possible because the messages were short, which allowed for the use of larger letters. The explanation “wordy signs more clear” refers to a preference for the message “TRAFFIC APPROACHING WHEN FLASHING” which, in the absence of a placard, was judged more complete and clear by many participants and was also favored by those participants who do not like placards.

Table 18. Five most frequent explanations for major road ranking choices.

Note: Bold text indicates the explanation with the highest results.

The explanations for the minor road approach ratings were similar to those for the major road. These are shown in table 19.

Table 19. Five most frequent explanations for message minor road ranking choices.

Note: Bold text indicates the explanation with the highest results.

PART 4

Interpretation of “WHEN FLASHING” Placard When the Beacons Were On

The first question in part 4 asked participants to agree or disagree with the statement that the two signs shown in figure 13 meant the same thing. The majority of participants (70 percent) agreed that the signs did mean the same thing. Because participants were not asked to explain their responses in part 4, it is not clear how the 30 percent who thought the signs did not mean the same thing interpreted the signs. From responses in part 1, it is likely that some participants thought that the sign without the placard always flashed, whereas the sign with the placard indicated that there actually was an entering vehicle.

If there was a “WHEN FLASHING” placard, the preferred wording for the minor approach ICWS was “CROSS TRAFFIC.” When there was no placard, there was a slight preference for “EXPECT CROSS TRAFFIC.” However, more than a few participants commented that “you should always expect cross traffic.” If “EXPECT” was used with “CROSS TRAFFIC,” it should only be when there was no placard.

Although some participants preferred having “WHEN FLASHING” on the diamond sign, this necessarily resulted in lettering that was smaller, which made the signs hard to read from a distance and took longer to comprehend when it became readable. For instance, on a 48-inch diamond sign, “CROSS TRAFFIC” can be applied with 8-inch FHWA Type C lettering, whereas “WATCH FOR ENTERING VEHICLE WHEN FLASHING” limits the size of the letters to 4 inches.

Interpretation of the “WHEN FLASHING” Placard When the Beacons Were Off

The second question in part 4 asked participants to agree or disagree with the statement that when not flashing, the two signs shown in figure 14 meant the same thing. Participants were nearly evenly split on this: 53 percent agreed that the signs mean the same thing, and 47 percent disagreed. In part 3, many participants indicated that the placard clarified the meaning of the signs and inferred that cross traffic was not present when the beacons were not on. Without the placard, those subjects often suggested that it was unclear what they should do in response to the message.

The third question in part 4 (see figure 27) asked participants to agree or disagree with the statement that “when the beacons aren’t flashing, it is not necessary to look for approaching traffic.” This statement drew agreement from 28 percent of the participants.

Figure 27. Photo. Sign accompanied by the statement “When the beacons aren’t flashing, it is not necessary to look for approaching traffic.”

Interpretation of Blank-Out Signs

A series of five questions in part 4 were intended to clarify how participants interpreted blank- out signs in their various blank-out states. The first statement in the blank-out series, “Cross traffic is present,” was accompanied by a picture of a blank-out sign with the beacons on and the message “EXPECT CROSS TRAFFIC.” The majority of participants (89 percent) agreed with this statement. The second statement in the blank-out series was accompanied by the same sign as the first question except that the beacons were off. The statement with this picture, “Cross traffic is not present,” was agreed with by 30 percent of the participants.

The third statement in this series, “Today, there is no traffic approaching,” showed a blank blank-out sign (see figure 19). A surprising 73 percent of participants agreed with the statement that asserted the blank sign meant there was no traffic approaching. This was despite fact that only minutes before they had been given the following explanation for blank-out signs: “Some agencies use blank-out signs. The messages on these signs are formed with illuminated lights so that when the power is off, there is no message.”

Perhaps the message should have been “in case of a power outage” because this statement could be interpreted as meaning power to the whole sign is off when no traffic is present.

The fourth statement in the blank-out series was “No traffic is coming on the crossroad.” The accompanying picture (see figure 20) showed the blank-out sign with the “EXPECT CROSS TRAFFIC” message and the words “not flashing” next to the picture. The beacons in the picture were off. One-third of participants agreed with the statement.

The fifth statement in the blank-out series was “No need to look for traffic when the beacons are not flashing.” The accompanying picture (see figure 21) was the same as the previous one. Only 13 percent of participants agreed with the statement.

The final questions in part 4 asked participants to judge which sign location for the ICWS warnings would be more effective (see figure 24). Only 13 percent of participants selected the upstream location currently used in Missouri and Iowa, and 87 percent chose the location across the intersection from the stop line.