Field Evaluation of Detection-Control System

CHAPTER 2. RESEARCH METHODOLOGY

INTRODUCTION

To meet the three objectives stated in chapter 1, the Texas Transportation Institute (TTI) developed the following four studies:

• Study 1: Performance Monitoring of Dilemma Zone Occupancy (addresses objective 1).

• Study 2: Before-After Crash Data Study (addresses objective 2 in part).

• Study 3: Before-After Crash Surrogate Study (addresses remainder of objective 2).

• Study 4: Upper Limit Study (addresses objective 3).

A discussion of these four studies follows.

Study 1: Performance Monitoring of Dilemma Zone Occupancy

This study evaluates the effectiveness of the D-CS algorithm by detecting and comparing the number of vehicles trapped in the dilemma zone, the number of red light runners, and the frequency of max-outs for comparable time periods before and after activation of the D-CS algorithm. Study 1 Scope:

• The study duration was originally intended to use 1 week of before data and 1 week of after data.

• Duration was altered because of excessive time required during analysis to a determined number of hours before and after.

• The study includes existing D-CS sites.

• Data collected include the number and type of vehicles trapped in dilemma zone.

• Eight sites were needed.

Study 2: Before-After Crash Data Study

This study evaluates the safety effect of the D-CS algorithm. Study 2 scope:

• This study is based on data for a 5-year before period and a 2-year after period, depending on cooperation of the local agency and availability of data.

• Data collected include dilemma zone-related crash data.

• Study 2 includes the same eight data collection sites used for the other studies. It uses comparison sites (for which D-CS is not installed but sites are similar to the treatment sites in traffic volumes, roadway characteristics, weather, etc., and are located within the same jurisdictions as the treatment sites).

Study 3: Before-After Crash Surrogate Study

This study is aimed at supplementing Study 2 results. Study 3 scope:

• Study 3 was originally intended to use 1 week of before data and 1 week of after data.

• Because of excessive time required during analysis, duration was altered to a determined number of hours before and after.

• Study 3 collected the following data:
  
  
  • Number and type of vehicles trapped in dilemma zone.
    
    
  • Red-light violation frequencies.
    
    
  • D-CS phase max-out frequencies.

• Eight sites were needed.

Study 4: Upper Limit Study

TTI hoped to find at least one high-volume site to test the upper limit of the D-CS algorithm to determine whether there are conditions under which it does not provide additional protection compared with more traditional procedures. TTI tested variations of the Max1 setting in the controller to study its effect on D-CS performance. Study 4 scope:

• Data collected were the following:
  
  
  • D-CS phase max-out frequencies.
    
    
  • Number and type of vehicles trapped in dilemma zone.

• One site was needed.

FIELD SITE SELECTION PLAN

TTI developed a site selection plan to conduct the research to evaluate D-CS. The site selection process was limited by the fact that there were only a few sites from which to choose. In fact, it became difficult to reach the original targeted number of eight sites. Some sites that were initially considered as candidate sites were inappropriate because of one or more of the following factors:

• Lack of before crash data.

• Lack of before field data (new signal or signal not installed long enough).

• Lack of support from the operating agency.

• Atypical controller or cabinet, which made installation and/or data collection difficult.

All but two of the selected sites had National Electrical Manufacturers Association (NEMA) TS2 Type I cabinets and Naztec 2070L controllers loaded with the D-CS algorithm and D-CS user interface installed. One of the remaining sites had an Eagle TS2 cabinet, and the final site had a TS1 cabinet. As a result of these differences, the research team made modest changes to the monitoring equipment.

Site Selection Criteria

The following list includes critical items:

• Equipment running the D-CS algorithm installed and fully operational.

• Willingness of the local transportation department to support research data collection (e.g., provide bucket truck).

• Willingness of the local transportation department to continue operation of D-CS during the 2-year after period.

• Sufficient traffic volume (including sufficient numbers of trucks).

• Traffic signal installed a minimum of 2 years prior to D-CS to ensure before data availability.

• Sites acceptable to both the Government and the research team.

The following list includes items that are desirable but non-critical:

• Newly installed or existing cabinet.

• Sufficient space in the cabinet for research equipment.

• Reasonably good sight distance and geometry on high-speed approaches.

• Properly positioned mounting hardware (e.g., poles) for cameras and Wavetronix™ Advance detectors.

• Sites located reasonably close to TTI headquarters to minimize travel costs.

• Cellular data coverage for continuous operation of wireless routers in cabinets.

• Before detection for dilemma-zone protection.

• Cameras already installed in the optimum locations for D-CS research.

Site Selection Process

Site selection involved finding jurisdictions that were already using D-CS and finding local agencies that were willing to support the research activities. TTI contacted the responsible agencies well in advance of the data collection period to determine their willingness to participate. In some cases, the agencies were using or had been using equipment installed in an earlier TxDOT research project where a personal computer (PC) was loaded with the D-CS algorithm.⁽²⁾ In one case, TTI replaced a failed PC with a Naztec controller (with permission from the responsible agency). This swap was accompanied by training and other support from Naztec. Of the sites identified in this process, only one had a high volume of traffic for determining the range of D-CS effectiveness (study 4, Upper Limit Study). See chapter 4 for more information.

TTI initially contacted all responsible agencies representing the available sites identified by the Government. States that chose to participate were Florida (three sites), Louisiana (one site), Illinois (two sites), and Texas (two sites); New York and Iowa chose not to participate. TTI also initiated communications with Naztec to encourage support for the project. Having Naztec involved in the project was helpful, but some State transportation department personnel who had not used the Naztec controller were not comfortable installing it before being trained by Naztec. This training activity required more time than originally anticipated.

METHODOLOGY FOR FIELD DATA COLLECTION

The general methodology used in this research began with using the site selection process and identifying candidate sites. Following the tentative selection of sites, researchers contacted the local transportation department to determine the willingness of key officials to support the required activities. The support involved providing onsite assistance with a bucket truck and technical information such as signal controller settings, construction plans, and related documents, and providing crash histories before and after installation of D-CS. In cases where the local transportation department had not used Naztec equipment prior to the research project, the transportation department needed to make a commitment to a different and sometimes unknown brand of hardware for at least the duration of the after test period (about 2 years).

Once the local transportation department made a commitment to support the activity, TTI scheduled a date to be onsite for installing the monitoring equipment. The data collection plan used the same equipment to monitor traffic for studies 1, 3, and 4 during the 1-week before period as during the 1-week after period. Table 5 summarizes the equipment required to monitor two sites simultaneously. The general goal was to monitor traffic for 7 days during the before period (using whatever detection the local transportation department had used before D-CS) and for 7 days during the after period (with D-CS). In a few cases, the data collection exceeded the planned duration for the before period, the after period, or both. In one case, problems caused by a power outage reduced the after period to 5 days instead of the desired 7 days.

Table 5. Summary of traffic monitoring and data storage equipment.

CCD = charge-coupled device.
DVR = digital video recorder.
I/O = input/output.
PC = personal computer.

Installing the equipment at the selected sites required coordination between researchers, the local transportation department, and, in some cases, installation contractors and equipment vendors such as Naztec and Wavetronix™. The local transportation department or its contractor provided a bucket truck for installing cameras and Wavetronix™ Advance detectors. In some cases, the intersections already had cameras installed, but all sites required re-aiming these cameras, installing new cameras, or both. Each additional camera required pulling wire from the cabinet to the camera for power and communication.

Communication required coaxial cables, and power required typical outdoor cables for three-phase AC power. The Wavetronix™ sensors required a Siamese cable with both power and communication in one bundle. TTI ran almost all of the cable overhead because of lack of space in underground conduits. The installer used zip ties to strap the cables to existing span wires for the short duration of the study. Removal of cables from either overhead or conduits was quicker than installation. During the removal, research personnel checked for cable damage and then respooled the cable for use at the next site.

Remote monitoring of equipment following installation required installing a cellular router in each cabinet. In a few cases, TTI also used a remote reboot system in the cabinet to overcome power outages or other short-term problems. The remote system allowed TTI engineers to do the following:

• Monitor the digital video recorder (DVR) to determine whether it was recording accurately.

• Monitor the PC.

• Download data each day.

It was not feasible to monitor the images being recorded by the DVR in real time, although researchers could transfer small segments of video using the Internet to verify proper operation. Replay of larger segments of video required transferring at least one of the two DVRs from the field to TTI headquarters.

SITES SELECTED FOR ANALYSIS

Table 6 lists the sites selected for data collection, their location, cabinet type, and the controller/equipment used at the site.

Table 6. Sites selected for data collection.

For studies 1, 3, and 4, TTI followed the general sequence of events described above for the field data collection. As soon as the project got under way, TTI ordered the equipment to install and monitor two D-CS sites simultaneously. Chapter 3 provides detailed information on the equipment used. In States that had two or more sites in close proximity to each other, TTI installed two of the sites on the same trip to reduce travel costs. In all cases, the local transportation department had already installed the D-CS inductive loops prior to beginning the before data collection. However, even though TTI had communicated with States weeks in advance to set project requirements, some States had not installed cabinets prior to researchers arriving at the sites. Three reasons for installing inductive loops prior to the research team arriving onsite were as follows:

• To minimize delays following the before data collection and prior to beginning the after data collection.

• To test the (in some cases newly installed) inductive loops for functionality and avoid delays that might otherwise occur when the D-CS was ready for activation and data collection for the after condition.

• To provide a source of vehicle length and speed data to be used as needed during both before and after periods.

Table 7 provides the pertinent dates of signal installation, dates of D-CS installation, and dates when crash data were available from the local agency. In some cases, the operating agency only knew the year. Table 8 lists the days selected for comparison from before to after D-CS installation for all eight sites.

Table 7. Information on sites selected for data collection.

Table 8. Dates for field data collection.

¹Griffin data incomplete on March 26, 2009.
²LA D-CS turned back on about 1 p.m. on June 12, 2009.
³June 17 and June 19, 2009, are partial days because of a power outage on June 17 and removal of equipment on June 19.
⁴Delayed because of training by Naztec for TxDOT personnel and a power outage on July 16, 2009.
⁵City of Waco reconnected first loop on phases 2 and 6 (at 111 ft from stop line) on Tuesday, October 27, 2009. TTI disconnected all existing loops on October 23 but later discovered that detection required clearing the queue before D-CS takes over.

Florida Sites

The first three sites installed by the research team at the beginning of the field data collection were in Fort Lauderdale along U.S. 27 at the intersections of Griffin Boulevard, Pines Avenue, and Johnson Street. The initial trips involved three researchers traveling during the weeks of November 17 and December 8, 2008. During the first trip, the research team made progress at both Griffin and Pines intersections, but at the end of the week, neither intersection was ready to begin collecting data. Most of the time spent during that week involved pulling wire from the cabinet to the camera or Wavetronix™ Advance mounting locations. A contractor for the Florida Department of Transportation (FDOT) provided a bucket truck and operator for both weeks.

Delays with Florida installations came from several sources, including some of the equipment purchased for the research project to monitor selected field measures of effectiveness (MOE). The principal agencies at the local level were FDOT, FDOT’s consultant, the D-CS installation contractor, and Broward County Transit (BCT) (the traffic signal maintaining agency). Two critical initial problems were local personnel not being trained on the D-CS program in the Naztec controllers and cabinets not being installed. The training was later provided by Naztec, and BCT pretested the cabinets prior to field installation. Fortunately, the D-CS inductive loops had been installed and were operational when researchers arrived. Local transportation department personnel (BCT, FDOT, and their contract representatives) began to immediately install and wire the cabinets to minimize delays to the research project.

The contractor installed the first cabinet at Griffin within 24 h of the researchers’ arrival, but lack of training caused delays in completing cabinet wiring. There were also equipment issues that delayed progress. For example, TTI had purchased new CyberResearch™ industrial computers for use on this project. Because one of them had not arrived before the November trip, TTI used one of its own existing Kontron™ industrial computers. The new CyberResearch™ computers were apparently unable to handle the massive amount of data generated by the Wavetronix™ Advance detectors, whereas the older Kontron™ PC worked flawlessly. The purchase specification for the new computers placed them well ahead of the Kontron™ PCs in all other categories, but researchers finally had to remove the CyberResearch™ PCs from their intended use on this project, resorting instead to the Kontron™ PCs for the remainder of the project.

Naztec provided the necessary training to FDOT and BCT personnel in January 2009. Because BCT was not using similar Naztec equipment anywhere else, the county had no spare or replacement parts in case of failures or damage to cabinets. This became a critical issue in May 2009 when a lightning storm damaged two video cards, four bus interface unit (BIU) cards, and the 2070 controller 2N module in the cabinet at Johnson Street.

The research team encountered other delays during the November trip because of uncertainty on conduit runs and space availability within conduits at both intersections. Knowledgeable local personnel provided field support in spotting lines, but the research team still required more time than originally planned to get the monitoring systems installed. One solution was to run more of the cables overhead, and that option was less stressful on cables used for the research compared with pulling the cables through conduits. The final Griffin wiring was all overhead, and all but one stretch of wiring at Pines was overhead. Over the course of the research project, researchers were able to use Wavetronix™ and camera wiring multiple times by allowing extra length along each wire run. This extra length had to be coiled and strapped overhead (to minimize vandalism), although this took additional time. TTI was able to use the Wavetronix™ wiring purchased at the beginning of the project at all eight sites but was unable to reuse the coaxial cable throughout the project.

On the second trip to Florida, the research team finished all the hardware installation for both Griffin and Pines, but the contractor needed more time to set up cameras and intersection wiring (e.g., D-CS loops). Camera setup was important because TTI planned to use existing video imaging camera systems to detect red-light runners (RLR) by simply adding detection zones just past the stop line in each high-speed approach lane. There was a chance that TTI would have to install additional cameras, but the existing cameras would need to be set up first before that determination could be made. Researchers brought their own setup tool on the second trip, but the contractor needed to re-aim the cameras later.

On the third trip to Florida, TTI was able to finish the installations at Griffin and Pines and begin collecting the before data at these two intersections. BCT personnel agreed to reset the controller when it came time to collect the after data, possibly reducing the need for researchers to travel to the sites. There were also other times when Broward County personnel were helpful in traveling to the sites to check a problem or reboot the system. Multiple power failures at Griffin prompted the research team to add an iBoot device to allow researchers to reboot the system from headquarters in Texas. TTI shipped the device to Broward County for installation, and BCT voluntarily added an uninterrupted power supply to help maintain consistent power to the site.

None of the Florida sites had dilemma-zone protection during the before data collection period. BCT had set up all intersections using stop-line detection, the recall feature in the controllers, or both. This lack of before dilemma-zone detection will be important when analyzing data comparing before with after.

Illinois Sites

Illinois data collection followed the Florida collection. The two Illinois sites were located east of Peoria on U.S. 24—one at the intersection with Cummings Lane and the other at the intersection with Main Street in Washington, IL. The two intersections had almost identical geometry, with two through lanes on each high-speed approach and single-lane left-turn bays on each high-speed approach.

TTI made a total of two trips to the two Illinois sites—one trip for installing the monitoring equipment and the second trip to remove the equipment. The Illinois Department of Transportation (IDOT) had supported the installation of eight cameras under a previous D-CS contract, but three of these cameras were damaged and could not be used. Of the remaining cameras that were operational, the research project needed only two cameras per intersection, so there were enough cameras to conduct the before-after data collection. TTI needed to reorient only two of the remaining cameras at each intersection. TTI removed these cameras following completion of the field data collection at these two sites and shipped them to TTI headquarters.

Louisiana Site

The location of the intersection of LA 3235 and LA 3162 is near the small town of Galliano, LA, in the Lafourche Parish, about 60 mi south of New Orleans. For installation support, TTI contacted the nearest district office of the Louisiana Department of Transportation and Development (LaDOTD) in Houma, LA. LA 3235 is a high-speed roadway with speed limits on each approach at 55 mi/h, although observed local traffic speeds were higher. The D-CS approaches have two through lanes, a left-turn lane for the southbound approach, and a right-turn lane for the northbound approach. Three of the intersection legs serve general purpose traffic, and the fourth leg (eastbound) serves a casino and a convenience store. The only before detection at this intersection was detection at the stop line, so D-CS should significantly improve the safety of the intersection.

Texas Sites

U.S. 281 at E. Borgfeld Drive in San Antonio

The location of this site is at an isolated intersection north of the urbanized area surrounding San Antonio. Peak periods at this site indicated that a significant portion of the traffic was commuter traffic, with heavy inbound movement in the morning hours and heavy outbound movement during the late afternoon hours. The traffic at this site was the heaviest of any of the D-CS data collection sites used in this project and was useful for conducting the upper limit study.

An important aspect of this site involved using the Naztec 2070 controller with a non-Naztec (Eagle) cabinet. Six of the eight sites involved in this research had Naztec controllers and cabinets, so this one offered a good opportunity to test the compatibility of this controller with a different cabinet (the other non-Naztec controller was at the Waco site, described below). During the initial installation of the Naztec controller for data collection around May 30, 2009, the intersection went into flash mode when a detector BIU was unplugged. This occurrence could have been coincidental with removal of the BIU, but Naztec replaced the memory management unit (MMU) anyway and was able to restore normal operation following this change.

TTI completed its normal data collection during the selected before/after period, which ended on July 27, 2009, before another problem occurred related to lightning (according to district personnel). The lightning strike again caused the intersection to go into flash mode. The Naztec controller had operated the intersection successfully for the 8-week period between the installation date of the Naztec controller and the lightning strike. As a temporary fix, district personnel replaced the Naztec controller with the original Eagle controller until Naztec could troubleshoot its 2070 controller. TxDOT shipped the controller back to Naztec during the week of August 17, 2009, to allow Naztec to troubleshoot the problem. Naztec found that the central processing unit (CPU) board had been damaged; a technician from Naztec returned the repaired controller to the district on September 31, 2009. TTI reinstalled its monitoring equipment at the U.S. 281/Borgfeld intersection on October 1, 2009, and began collecting data for the Upper Limit Study.

Upper Limit Study: TTI increased the maximum green setting in the D-CS 2070 controller on November 3, 2009, at the intersection of U.S. 281/Borgfeld to test its upper limit. At this intersection, Naztec and TTI set an initial maximum green of 75 s for phases 2 and 6. After some consideration of adding 80 s and 85 s, TTI selected 85 s and 95 s as the desired additional values to test because of the increased statistical significance in the greater spread.

To prepare for the increased maximum green and possible increase in delay, TTI conducted simulations using the Synchro software. The TxDOT San Antonio District traffic operations engineer mandated that TTI run Synchro to determine the impact of the increased main street green time on overall intersection delay. Even though all decisionmakers realized that Synchro could not simulate D-CS, this was still considered a worthwhile activity because it should at least approximate the increase in delay. Table 9 and table 10 indicate the increased delay based on the Synchro runs. The tabulated values of traffic demand came from TTI counts at the upstream D-CS inductive loops, but TTI had to approximate the side street demand (on Borgfeld Drive) based on loop occupancy values. Based on Synchro results and discussions with the TTI principal investigator regarding how the D-CS algorithm searches for a safe time to end the green phase, the TxDOT traffic operations engineer authorized the increased maximum green settings. Chapter 4 provides the analysis and results of the upper limit study.

U.S. 84 and F.M. 2837 in Waco

The Waco site is at the intersection of U.S. 84 and F.M. 2837 (Old Lorena Road). The location is southwest of Waco and outside the urban area. U.S. 84 is a high-speed roadway with a speed limit of 60 mi/h on each approach and a significant number of trucks. The D-CS approaches have two through lanes, single left-turn lanes, and single right-turn lanes. All four of the intersection legs serve general purpose traffic. Detection prior to D-CS installation consisted of a series of inductive loops upstream for dilemma-zone protection. Dilemma zone detectors were 6-ft by 6-ft loops in each lane at 493, 267, and 111 ft from the stop line. Through lanes on U.S. 84 had no stop-line detection, but left-turn bays did.

Table 9. Synchro results for morning peak at U.S. 281/Borgfeld Dr.

EBL = eastbound left-turn.
EBR =eastbound right-turn.
LOS = level of service.
NBL = northbound left-turn.
NBT = northbound through.
SBT = southbound through.
SBR = southbound right-turn.
N/A = not applicable.

Table 10. Synchro results for afternoon peak at U.S. 281/Borgfeld Dr.

EBL = eastbound left-turn.
EBR = eastbound right-turn.
LOS = level of service.
NBL = northbound left-turn.
NBT = northbound through.
SBT = southbound through.
SBR = southbound right-turn.
N/A = not applicable.

The D-CS site in Waco had a NEMA TS1 cabinet, and City of Waco decisionmakers chose not to replace this cabinet with a TS2 cabinet to accommodate the D-CS, even though they were convinced that D-CS had made a significant difference in improving safety. On the first visit to this site, TTI researchers found that the PC running the D-CS algorithm had failed, so they prepared a replacement PC to be installed in the cabinet. Other TTI researchers had already wired the cabinet for a PC system, reducing the effort required to reinstall D-CS.

The TS1 version of D-CS reads the upstream detector amplifiers directly, so it has to react faster than the TS2 version. The operating system (OS) clock speed is critical.The original TTI research project used Windows 2000 because of the 10-ms OS timer (Windows XP’s timer is 15ms, and earlier Windows OS timers were 55 ms), and that time reduced the speed measurement error.Appendix A has more detailed information on the setup of PCs for TS1 cabinets.

An issue that surfaced immediately following the reinstallation of D-CS at this site was due to not having stop-line detectors on through lanes. The City of Waco technicians had disconnected all previously installed dilemma-zone loops so that only the D-CS loops were connected for main street dilemma-zone protection. The D-CS algorithm does not start to search for gaps in the traffic stream until the termination of the minimum green in the controller or until the stop-line queue is served, whichever is greater. In this case, with the formation of long queues extending past the D-CS loops, the algorithm did not detect gaps appropriately, so it terminated the green phase prematurely with each cycle and forced very long queues to form on the U.S. 84 approaches. The solution involved reconnecting the nearest inductive loops (located at 111 ft from the stop line) and allowing the queue to begin clearing and D-CS to function properly. With reconnection of the closest loops to the stop line, D-CS was able to operate properly. At the first gap-out (or at the end of the minimum green setting, which was usually less), D-CS took over and started looking for gaps to safely end the green phase.

ANALYSIS METHODOLOGY

TTI processed data files collected from field equipment and placed the data in Microsoft Excel files for processing. The amount of data collected at each of the eight sites was enormous, far more than the amount that was actually needed to determine the performance of the D-CS. TTI followed the instructions from the sponsor to collect 7 full days of data but could only analyze a few hours of data from each site to actually compare the before period to the after period. TTI started trying to analyze full days of data, but that was far too time-consuming. The decision based on this realization was to analyze full hours of data until tests of the adequacy of the data indicated that the desired statistical significance had been achieved.

All of the hourly data analysis started with a person watching replay of recorded video to determine RLRs and vehicles caught in the dilemma zone (2 to 6 s from the stop line). The person viewing the video filled in a spreadsheet for each and every signal cycle during the hour based on a predetermined Microsoft® Excel template, from which engineers would subsequently determine actual dilemma-zone violators and RLRs.

The next step in the analysis process involved merging information onto a predetermined worksheet template from the person reviewing video, from phase-status files, from RLR files, and from traffic-volume files (traffic counts of each approach). The data from this worksheet then became part of a final analysis of results based on a regression model technique developed by TTI researchers.