U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

 
REPORT
This report is an archived publication and may contain dated technical, contact, and link information
Back to Publication List        
Publication Number:  FHWA-HRT-15-027    Date:  November 2015
Publication Number: FHWA-HRT-15-027
Date: November 2015

 

Information As A Source of Distraction

 

Chapter 4. CMS Field Test

Introduction

The relevant properties of CMSs were reviewed in chapter 1. Chapter 4 describes testing of the effects of those properties on sign comprehension or message-recognition distance while drivers are engaged in navigating a shallow slalom course.

Methods

The data were collected on a closed course while participants drove an instrumented vehicle that was equipped with a dashboard-mounted eye-tracking system.

Participants

Useable data were obtained from nine participants (six males and three females). The mean age of participants was 39 years old (minimum = 19, maximum = 61). Participants were paid $40/h for 3 to 5h of participation. All drivers were licensed in Virginia.

Before they were scheduled for testing, participants provided signed permission for the investigators to obtain their records from the Virginia Department of Motor Vehicles. Individuals with more than one moving violation or police-reported crash in the preceding year or a driving while intoxicated violation in the preceding 3 years were excluded from participation. Upon arrival at the test facility, participants read and signed an inform consent form.

Test Facility

Testing was conducted on a 30-ft- (9-m-) wide drag strip. A CMS was placed on the left side of the drag strip 1,250 ft (376 m) from the start line. Traffic cones were placed on the track to form a 12-ft- (3.6-m-) wide lane that curved first from the left to the right side of the track, then back to the left, and ended in the middle of the track at the CMS location. The arrangement of the traffic cones is shown in figure 21. The purpose of the curved path was to require participants to attend to lane keeping in addition to viewing messages displayed on the sign.

Changeable Message Sign

The CMS was the same one used in the laboratory study (see chapter 2) and is shown in
figure 22.

Figure 21. Photo. Layout of course on drag strip.

Figure 21. Photo. Layout of course on drag strip.

 

Figure 22. Photo. The CMS used in the study.

Figure 22. Photo. The CMS used in the study.

The brightness of the sign was set at 100 percent because the tests were conducted in daylight between 9 a.m. and 1 p.m. in fall 2012. At the 100-percent brightness setting, a white stimulus 12inches (30 cm) in diameter measured between 3,065 and 3,502 fl (10,500 and 12,000cd/m2) depending on which location on the display was measured. A red stimulus measured a mean of 1,051 fl (3,600 cd/m2) and an amber stimulus a mean of 3,094 fl (10,600cd/m2). Laboratory testing, reported elsewhere, showed that the display was compliant with NEMA standards for LED color displays intended for highway applications.(23)

Research Vehicle

Participants drove a 5-year-old full-sized sports utility vehicle equipped with a dashboard-mounted, three-camera eye-tracking system.(49) The system sampled eye vectors and head position at 60 Hz. Three scene cameras mounted on the roof of the vehicle, directly over the driver’s head, recorded about 80 degrees (horizontal) of the driver’s view of the road ahead. In post processing, the scene camera view was merged with the eye-tracking vectors. In addition to the eye-tracking data, direction measuring equipment, Global Positioning System, and accelerometer data were recorded and synchronized.

The sound recording capability of the eye-tracking system was not functional at the time these tests were conducted. Therefore, to capture participants’ verbal responses, the experimenter operated a handheld voice recorder during each trial. Recordings were started as the participant maneuvered the research vehicle to the start line and stopped after the participant correctly spoke the message on the sign.

Message Types

Message Length

Current guidance suggests that CMS messages should be limited to three units of information. The unit of information concept is probably appropriate for most drivers, at least in the case where the posted information matches drivers’ expectations. If the posted information does not match the typical questions a motorist might have (e.g., What? Where? Action?), then the number of words in the message would be a better metric for message length. In this experiment, the number of words on the CMS was varied from one through seven. Reading time and glance behavior were assessed for all participants with all message lengths. Because of the small size of the CMS that was used, the smallest available font size was used (7 pixels high by 6 pixels wide). The stroke width was 2 pixels and letter height was about 5.5 inches (140 mm). With allowance for some blooming of pixels, the letter height would be equivalent to 6-inch (152-mm) letters on a static sign. Only the first letter of each word was capitalized. Preliminary testing with stationary observers (see chapter 2) showed that these words became legible at 180 ft (55 m) for observers with 20/20 vision. All words displayed were appropriate for traffic-related messages, but the order of words was randomized so that each word represented one information unit. Each word was presented centered on a line. The following is the complete list of words used:

For any particular message, words were selected from the list and ordered randomly. Each participant received a different set of randomized words. To illustrate the font used for all messages, figure 23 shows one of the three-word messages that was used.

Figure 23. Photo. Example of a three-word message.

Figure 23. Photo. Example of a three-word message.

Flashing

The effects of flashing text were evaluated by displaying the same messages three different ways: (1) static messages (no flashing), (2) first of three lines flashing, or (3) entire message flashing. The flash rate was 1 Hz (on 0.05 s, off 0.5s). The following messages were used with slashes indicating line breaks:

The fonts were all 14- by 10-pixel approximations of FHWA type E font. All messages were white (i.e., each illuminated pixel consisted of a fully illuminated red, green, and blue LEDs). Individual participants saw individual messages only once, with the messages randomly assigned to flash mode with the restriction that there was an equal number of trials in each mode.

Symbols Versus Text

This test compared gaze behavior and reading time between pairs of symbol images of traffic signs from the MUTCD and the equivalent text messages for the same signs. Except for the interstate shield text equivalent, all text messages used a 14- by 10-pixel approximation of the FHWA type E font. The text messages were all displayed with positive contrast. The text color was appropriate for the type of message: yellow for warnings, white for the interstate designation, and orange for the work zone sign. The symbol signs and text equivalents are listed in table 10.

Table 10. Signs used for comparisons of symbol- and text-sign reading time and glance behavior.

MUTCD Symbol Sign Designation
Text Alternative
W2-1
intersection ahead
M1-2
interstate shield (I-95)
W2-4R
right lane ends
W3-5
speed reduction (45 mi/h)
W2-6
roundabout ahead
W3-1
stop ahead
W3-3
signal ahead
W21-1a
workers
1 mi/h = 1.6 km/h

 

Abbreviations

To test the effect of abbreviations on gaze behavior and reading time, MUTCD-approved abbreviations for use on portable message signs were used. The abbreviations used and the phrases in which they were used are shown in table 11.

Table 11. Vocabulary for abbreviations test.

Word
Abbreviation
Message Context
Road
RD
ROAD CLOSED
Avenue
AVE
PARK AVENUE
Center
CNTR
CENTER LANE CLOSED
Normal
NORM
NORMAL TRAFFIC
Feet
FT
100 FEET
South
S
I-495 SOUTH
Mile
MI
NO EXIT NEXT 60 MILES
1 ft = 0.305 m
1 mi = 1.6 km

 

With one exception, three fonts were used in the abbreviation test: (1) an approximation of FHWA series E that was 14 pixels high by 10 pixels wide, with a stroke width of 2 pixels; (2) an approximation of FHWA series B that was also 14 pixels by 10 pixels with a 2-pixel stroke width; and (3) a manufacturer-supplied 14- by 8-pixel font with a 2-pixel stroke width. Thus, all three fonts were 11 inches high (or 12 inches if a blooming effect is assumed), and all had a 2‑pixel stroke width. The choice of font for particular messages was based on the need to use only whole words on a line and the limitation to three lines of text with adequate spacing between lines. The one exception was the abbreviation for south. To approximate the FHWA interstate shield as closely as possible, “I-495 S” and “I-495 SOUTH” were displayed using twofonts: I-495 was in a 16- by 10-pixel font with a 3-pixel stroke width, and SOUTH and S were displayed in a 16- by 8-pixel font with a 2-pixel stroke width.

Phasing or Paging

When a message is too long to fit on one screen, an option is to display the message in a sequence of screens. This is referred to as phasing. The MUTCD limits phasing to two screens or pages.(2) Each screen must be interpretable by itself, and the order in which the two screens are read should not matter. The manual indicates that the legibility distance of messages should be greater (larger text should be used) when messages are phased because it will take drivers more time to read the message. However, the manual provides no guidance on how much larger the text should be and indicates that letter heights greater than 18 inches (46 cm) will not improve legibility distance.

The two-phase messages used in this study were compliant with the manual’s regulations and guidance. Only two- and three-line messages were displayed in any single phase. Two phases were used, with each phase displayed for 3 and 0.3 s of blank screen between phases. Each phase was intended to be understandable without reference to the other phase. Only one unit of information was displayed on a line. The messages used in the phasing trials are shown in table 12. Rather than compare the same eight messages in two phases against the same messages presented in one phase, the static messages from the flashing set of trials were used as comparison items. Although not a perfect match, static messages contained a similar number of words and information units (see the bulleted list in the previous section Flashing).

Table 12. Eight two-phase messages used for phasing trials.

Phase Message Used
Phase 1
USE
EXIT 27
CRASH
SR 123
KEEP
RIGHT
SLOW
TO
25 MPH
Phase 2
CIRCUS
TRAFFIC
USE
SR 267
DEBRIS
IN ROAD
JAM
AHEAD
Phase 1
POLICE ACTION
PREPARE TO STOP
ROAD CLOSED
SIGNAL OUT
Phase 2
DO NOT ENTER
ROAD BLOCK
USE EXIT 2
4-WAY STOP

 

Table 13. Comparison of static message length to phased message length.

Static Message Number of Words Information Units Phased Message Number of Words Information Units
Crash Merge Left
3
2
Circus Traffic
Use Exit 27
5
2
Detour Exit 67 Closed
4
3
Crash SR 123
Use SR 267
6
3
Right Lane Blocked
3
1
Keep Right
Debris in Road
5
2
Road Work Ahead
3
2
Slow to 25 mph
Jam Ahead
6
2
Time to Reston 36 Minutes
5
2
Police Action
Do Not Enter
5
2
Traffic Jam Be Calm
4
2
Prepare to Stop
Road Block
5
2
Stopped Traffic Ahead
3
2
Road Closed
Use Exit 2
5
2
Crash Use Alt Route
4
2
Signal Out
4-Way Stop
4
2
Mean
3.625
2
Mean
5.125
2.125
1 mi/h = 1.6 km/h

 

Font

When viewed from an appropriately long distance, full-matrix CMSs are capable of emulating FHWA typefaces that include lowercase text. In search tasks, such as looking for a street name on navigation signs, uppercase/lowercase text can result in longer recognition distances, presumably because the shape of words with ascending and descending characters can be recognized before the individual letters become identifiable. Unfortunately, the number of fonts with true ascenders and descenders was limited with the equipment provided for this test. The messages used in the font comparison trials are listed in table 14. Only the uppercase/lowercase “left lane block” and “normal traffic” messages contained true ascenders. None of the messages contained descenders (e.g., the letter “p” in open did not descend below the line of the other characters). Thus, the current test examined the effects of uppercase versus pseudo mixed-case messages.

Table 14. Messages used for comparison of recognition performance between all uppercase messages and messages in pseudo uppercase/lowercase.

Uppercase
Lowercase
HOT LANE OPEN
HOT Lane Open
LEFT LANE BLOCK
Left Lane Block
ROAD WORK NEXT
Road Work Next
SLOW SPEED 1 MI
Slow Speed 1 Mi
NO EXIT NEXT 60 MILES
No Exit Next 60 Miles
NORMAL TRAFFIC
Normal Traffic

Participants were exposed to both uppercase and mixed case versions of each message. The order of mixed case and all uppercase versions was varied across participants, and the different versions of the same message were separated by at least five trials with unrelated messages, which included messages related to other issues (e.g., phasing or message length).

Test Procedures

A total of 146 messages were prepared. Individual participants were each shown about 60 of the messages (range 55 to 70) such that each sign was shown approximately the same number of times across all participants. The order in which signs were shown was randomized for each participant, subject to the constraint that the same message in a different format (e.g., font, case, and abbreviation) did not occur more than once in any series of five messages.

Participants were instructed to begin each trial at the approach to the left start line. There they waited for an experimenter in the backseat to signal that the data recording equipment was ready. At that point, participants were to briskly accelerate to 25 mi/h (40 km/h) and to maintain that speed while they attempted to read the CMS message. Participants were instructed to read the message aloud as soon as practicable while still maintaining 25 mi/h (40 km/h) and staying within the marked lane.

Results

The data recorded by the eye-tracking system were analyzed using software that related regions of interest (ROIs) marked by an analyst on video records of the forward view to gaze vectors determined by the eye-tracking software. For this study, one ROI was marked for each trial. This ROI covered a rectangular area about the sign that subtended approximately 2 degrees of visual angle regardless of the distance from the sign. A representative ROI captured at three locations within a trial is shown in figure 24 through figure 26. Figure 24 is at the beginning of the trial, approximately 1,250 ft (381 m) from the sign. Figure 26 shows the same ROI just before the sign is passed. The 2-degree size of the ROIs was chosen for two reasons: (1) it represents what is generally considered the area subtended by the fovea, the region of the retina used to capture scene details; and (2) it represents the approximate radial accuracy of the dashboard-mounted eye-tracking system.

Figure 24. Screen capture. Two-degree ROI for the CMS at a distance of approximately 1,250 ft (381 m).

Figure 24. Screen capture. Two-degree ROI for the CMS at a distance of approximately 1,250 ft (381 m).

 

Figure 25. Screen capture. Two-degree ROI for the CMS approximately halfway down the slalom course.

Figure 25. Screen capture. Two-degree ROI for the CMS approximately halfway down the slalom course.

 

Figure 26. Screen capture. Two-degree ROI for the CMS toward the end of a run.

Figure 26. Screen capture. Two-degree ROI for the CMS toward the end of a run.

The primary measure of eye-movement behavior was the look. A look was defined as any accumulation of four or more hits on an ROI within a series of six frames (one frame equals 0.017 s; six frames equals 102 ms). A look began when this criterion was first met and terminated when the number of hits within the preceding six frames dropped below four. The number of looks and the duration of looks were analyzed.

In addition to eye-tracking measurements, two measures were extracted from the voice recordings of participants: (1) trial duration—time from the beginning of a trial until the participant completed correctly reading aloud the message on the sign and (2) response duration—the time between when the participant began repeating the message aloud and when the participant completed reading the message. All measures were to the nearest second. Response durations of zero length could result when responses were rounded to the nearest second. Trials always began with the vehicle at a full stop. The beginning of trials were marked when the sound of the engine revving was noted in the recordings.

All statistical tests employed General Estimating Equation (GEE) models. Analyses of frequency data assumed a Poisson distribution. Analyses of look durations assumed a gamma distribution. All analyses of vocal response assumed a Poisson distribution.

Note that in evaluating time differences, the vehicle traveled at about 25 mi/h (40 km/h) so each second that elapsed equated to about 37 ft (11 m) traveled.

All error bars shown in the charts below represent 95-percent confidence limits about the expected means.

Message Length

Neither the mean number of looks to the sign nor the average duration of each look varied significantly as a function of message length. As can be seen in table 15, the mean number of looks was surprisingly large, and the mean look duration was brief.

Table 15. Number of looks to messages and look duration as a function of the number of words in the messages.

Looks
Message Length (Number of Words)
1
2
3
4
5
6
7
Number of looks
30
38
35
28
37
39
51
Mean duration of looks (s)
0.120
0.104
0.123
0.116
0.118
0.135
0.135

Given the large number of looks—far more than would be necessary to read a seven-word message—it appears that many of the looks were made for the purpose of determining whether the sign was near enough to be read. Standard road signs typically get far fewer looks, probably because drivers have far more experience reading standard signs and thus know when they should become legible, but also because drivers are not typically tasked to read standard signs as soon as possible, which was the instruction for reading the CMS.(50) The relatively short mean durations are not concealing single long glances. Across all participants and all message lengths, the longest recorded look was 0.46s to a six-word message.

There was a significant effect of message length on trial duration, χ2(6) = 92.5, p < 0.001. This effect can be seen in figure 27. Response duration showed a similar significant trend, χ2(6) = 123.9, p < 0.001. Response durations are shown in figure 28. These two measures share the same end-of-response time and are correlated to the extent that the start of reading time is independent of message length. It appears the measures are highly correlated.

Figure 27. Graph. Expected mean trial duration as a function of the number of words in the CMS message.

Figure 27. Graph. Expected mean trial duration as a function of the number of words in the CMS message.

 

Figure 28. Graph. Expected mean response duration as a function of the number of words in the CMS message.

Figure 28. Graph. Expected mean response duration as a function of the number of words in the CMS message.

Flashing

The duration of looks and number of looks did not vary significantly at the p < 0.05 level. However, given the small sample size in the study, it may be worth reporting a trend favoring static and all-lines-flashing messages over messages with the first line flashing. The duration of looks to static and all-lines-flashing messages were nearly the same, 0.13 s. The expected mean duration of looks in which the first line flashed and the second and third lines were static was 0.17 s. The test for differences between the means approached significance, χ2(2) = 5.48, p= 0.06. There was also a non-significant trend ( χ2(2) = 3.78, p =0.15) for first-line-flashing messages to receive more looks. The expected mean number of looks is shown in table 16 as a function of messagetype.

Table 16. Expected mean number looks to messages in the flashing text trials.

Message Type
Expected Mean Number of Looks
Lower .95 Confidence Interval Limit
Upper .95 Confidence Interval Limit
Static
31
24
40
First Line Flashing
36
29
44
All Lines Flashing
27
20
35

The voice response main effects for both trial duration ( χ2(2) = 13.2, p = 0.001) and response duration ( χ2(2) = 8.5, p = 0.015) were significant, and support the contention that static or all flashing messages result in more efficient message transmission than first-line-flashing messages. Figure 29 shows the expected mean trial duration, and the expected mean response duration. From the start of the trial, responses were completed about 1.5 s later when all lines were flashing than when all lines were static or only the first line flashed. Responses were longest when only the first line flashed.

Figure 29. Chart. Trial duration and response duration as a function of flashing mode.

Figure 29. Chart. Trial duration and response duration as a function of flashing mode.

Symbols Versus Text

There were no significant differences in look durations or number of looks between symbol signs and their text equivalents.

Participants responded about 3 s sooner (i.e., trial duration was 3 s shorter) to the symbol signs (expected mean trial duration, 12.1 s ±3.8) than they did to the text-based equivalents of those signs (expected mean, 15.4 s ±3.25), χ2(1) = 6.4, p = 0.012.

Although the symbol signs’ meanings were generally recognized sooner, and thus yielded shorter trial durations than the text equivalents, the workers symbol sign was an exception. As can be seen in figure 30, the mean trial duration for the workers symbol sign was more than twice that for the text based version, χ2(1) = 16.4, p < 0.001.

The total number of looks at the workers symbol sign was significantly greater than to its text equivalent, χ2(1) = 13.5, p < 0.001.

Although the workers symbol sign was probably legible from as great or greater distance than the ROAD WORK sign, several participants struggled to interpret the sign and made comments such as “what is that?”, and then made several incorrect guesses before generating a plausibly correct response such as “person sweeping.” This suggests that the symbol is not easily comprehended.

Figure 30. Chart. Comparison of trial durations for workers symbol signs with other text and symbol signs.

Figure 30. Chart. Comparison of trial durations for workers symbol signs with other text and symbol signs.

Abbreviations

There were no significant differences in look durations or number of looks between messages with FHWA-approved abbreviations and complete word messages. Also, there were no significant differences in either trial duration or response duration.

Phasing or Paging

As shown in figure 31, there were significantly more looks at two-phase messages than one-phase messages, χ2(1) = 8.77, p = 0.003. The duration of these looks did not vary significantly (static message expected mean = 0.136 s, phased message expected mean = 0.151 s).

Figure 31. Chart. Expected mean number of looks at CMSs as a function of the number of message phases.

Figure 31. Chart. Expected mean number of looks at CMSs as a function of the number of message phases.

Trial durations were longer by about 3.5 s with phased messages than with static messages, χ2(1) = 12.0, p=0.001.

Case

There were no significant effects observed in the font trials—look duration, number of looks, trial duration, and response duration effects were all non-significant.

Discussion and Conclusions

In these tests, drivers made a large number of looks to the CMS. Because they were instructed to read aloud the messages as soon as possible, the simplest explanation for this is that participants needed to repeatedly look at the sign to determine when they could begin responding. Whether this finding has any practical significance for CMS applications is uncertain. For static signs, far fewer looks are typically observed. However, drivers presumably have a great deal of experience with static signs, so they may have reasonably good expectations regarding when they should attend to them. Also, the messages on static signs are predictable from context and are fairly standard, so a brief look may be enough to confirm a driver’s expectations. CMSs vary more than static signs in legibility distance, partly because the technology used varies and, at least for older CMSs, the legibility distance decreases with the age of the sign. For these reasons, drivers may start looking at CMSs as soon as they are detected. To the extent that CMS messages are less predictable than static sign messages, drivers may need to maximize the distance at which they attend to messages to allow for the extra processing (i.e., reading time) unpredictable messages will require.

On-road eye-tracking observation in which drivers were not instructed to read CMS messages would be needed to clarify whether CMS messages get more looks than static signs. However, in the driving simulation experiments described in the following chapters, large numbers of looks at simulated CMSs were not observed. In fact, there was some evidence that drivers in those studies could read unexpected CMS messages without fixating directly on the signs. In those studies, the messages were in large letters that could be read from at least 800 ft (244 m) away and were positioned over the roadway.

Message Length

The duration of trials increased with the number of words in messages. However, the differences in response duration and trial duration were negligible when messages contained two to fivewords. Six- and seven-word messages required 4 or 5 s more response time than shorter messages. Although the FHWA guidance on limiting the number of units of information on a CMS to three is valid, the findings of this study suggest that the total number of words is also important and should be limited to five or fewer whenever possible.

Flashing

First-line-flashing messages required more looks than static or all-lines-flashing messages. First-line-flashing messages also had longer response durations than the other two modes. The all-lines-flashing mode had the longest delay before participants completed responses (trial duration). Together these findings support the MUTCD prohibition of flashing messages.(2)

Symbols Versus Text

The present findings support the use of symbols on CMSs, at least in the case where the symbols are already familiar to drivers. The advantage of symbols is that they are recognizable from longer distances than are text messages that require similar display area. The finding that the ROAD WORK symbol sign yielded worse performance than its text-based alternative is important. To provide an advantage, symbols must be familiar or easily comprehended. Symbols that are not quickly comprehended by a large percentage of the population may result in driver distraction while drivers contemplate the symbol’s intended meaning. For standard warning signs, placards are sometimes used to familiarize drivers with the meaning of novel symbols. This practice would not be appropriate for symbols on CMSs, because the use of text with symbols would necessitate reducing the size and thus the legibility distance of both text and symbols.

Abbreviations

The present findings support current FHWA policy regarding the use of abbreviations as specified in the MUTCD.(2) Approved abbreviations appear to have no effect on driver performance compared with full-text messages. Abbreviations not approved for use on portable messages signs were not evaluated in the present study, and therefore these findings should not be generalized to abbreviations not in the MUTCD.

Phasing or Paging

Phased messages take longer to read than static messages and should be avoided. In the present tests, all two-phase messages were successfully read. The CMS was within an unobstructed line‑of-sight for 1,250 ft (381 m), and the vehicle was traveling at 37 ft/s (11 m/s) so that the drivers had about 30 s to view the messages, although the messages may not have been legible for the entire distance. In real-world driving, other vehicles, roadway geometry, and other TCDs may limit viewing distance more than observed in these tests. If two-phase messages are used, it should not be assumed that drivers will have time to safely read them. Therefore, safety-critical messages should not be phased.

Case

All performance differences between uppercase and proper case (uppercase/lowercase) fonts were non-significant. With all four metrics, the actual differences favored the uppercase messages. These results provide no reason to change the current policy, as specified in the MUTCD, which favors the use of only uppercase messages except when emulating guide signs.(2)

The CMS used in this study, a full-color, full-matrix, LED display with 0.79-inch (20-mm) pixels, is legible from distances approaching those of static highway signs. The capability to display symbols and provide high-contrast messages in text in most lighting conditions, is expected to greatly increase the effectiveness of CMS messaging compared with earlier CMS technologies.

 

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101