U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

California Division

Home / About / Field Offices / California Division / Systems Engineering Guidebook for ITS

Home What's New Systems Engineering Guidebook Views Search Glossary Resources Feedback Site Map

8.5.1 New York City Transit Automated Train Supervision (ATS)

Background

The New York City Transit subway system is one of the largest and most complex in the world. From the original 28 stations built in Manhattan which opened on October 27, 1904, the subway system has grown to 468 stations located in the four boroughs of Manhattan, Brooklyn, Queens, and the Bronx. These stations are connected by over 840 miles of track. Laid end to end, NYC Transit train tracks would stretch from New York City to Chicago.

NYC Transit is composed of three formerly separate transit systems; the Brooklyn Manhattan Transit Corporation (BMT),Interborough Rapid Transit Company (IRT), and the Independent Rapid Transit Railroad (IND).

Currently rail operations are managed as two separate subdivisions. Subdivision A consists of the IRT lines and Subdivision B consists of the BMT and IND lines. The 26 subway routes are interconnected and many lines feature express trains and across-the-platform transfers to local trains.

Subdivision A includes the numbered routes
Subdivision B includes the lettered routes

* There are three shuttle services: Franklin Avenue, Rockaway Park, and 42 Street.

The system has approximately 10,675 track wayside signals, 205 traction power substations, and over 6,200 revenue (paying passengers) and non-revenue (money trains, maintenance, garbage) vehicles in the fleet. It operates 24 hours a day, running over 6,500 scheduled trains (average 10 cars/train) each weekday and moving 4.5 million riders daily; requiring all work, both capital and maintenance to be done between train service or planned shutdowns.

Legacy system

Presently, NYCT manages subway service through a Subway Control Center located in downtown Brooklyn via voice communications, but the actual control of its interlockings (control of signals and switches) is handled in the field at locations called Master Towers. Each Master Tower controls several interlockings for certain lines, or sections of lines. In coordination with train dispatchers in the field, train routes are manually determined and aligned by the Master Tower operators.

Subway Control Center personnel supervise the actions of all field personnel, including the Master Towers operators and the train crew. Although essentially blind to train location, the Subway Control Center globally monitors the system, and is responsible for emergency management, and handling train incidents, especially those which extend beyond one particular Master Tower control territory. Voice communications is provided through separate devices for radio, telephone, intercom, and “6-wire” (an agency-wide party line).

Project Description

The Automatic Train Supervision (ATS) is a $200M+ project that will provide a system that will facilitate service management for NYCT Subdivision A rail territory, except for the #7 line. It will consolidate most of the work currently performed at both the Master Towers and the Subway Control Center in Brooklyn, and provide:

Real-time centralized train traffic control for the A division from the Rail Control Center (RCC) Operating Theater

Real time train tracking

Integrated voice communications with recording capabilities

Automatically developed train routing schemes based on schedule and service conditions

Improved coordination of emergency response activities between operating divisions to expedite solutions

Provide effective, centralized management for better on-time performance and more regular headway (spacing between trains)

Report generation

Provide customers and the general public with real-time service information

Improve safety and overall system efficiency

Major Work Elements

The ATS project is New York City Transit's first attempt to implement and deploy the first phase of a program to allow Department of Subways' Rapid Transit Operations (RTO) to monitor and control train movement from its newly built Rail Control Center (RCC).

The first phase, ATS for the ‘A' Division, includes several major work elements (design, procurement, installation, testing, commissioning):

Architectural and facilities work for the RCC building, which was built under a previous contract. Work under this project includes lighting, acoustical, and ventilation work in the Operating Theatre, UPS system for critical systems, cooling units for the computer rooms, and a building management system (BMS) to integrate the HVAC, and fire alarm systems.

ATS computer-based office system, which includes outfitting equipment, i.e., servers at the RCC for redundant ATS computer rooms. It includes 50 operator consoles for the Operating Theater and various maintenance & support offices in the RCC; a 150 foot large-scale display in the Operating Theater; 30 operator consoles located at approximately 26 remote sites (dispatcher offices and master towers) for terminal operations including crew assignments, schedule adjustments, and train logging.

Redundant programmable logic controllers (PLCs) in approximately 100 equipment rooms called Relay Rooms and Central Instrument Rooms to acquire real-time data of the interlockings to the ATS office system.

Automatic Vehicle Identification (AVI) readers located on the tracks provide train identification and information to the ATS office system.

Interfaces to several NYCT legacy systems to provide data for ATS reporting functions.

An integrated communications switch (ICS) to consolidate telephone, radio and other NYCT legacy systems (train dispatch system and 6-wire) circuits, and allow an ATS operator the ability to communicate to all these systems through one headset and console display. The communications sub-system includes an automated attendant and a voice recording sub-system.

Contract Documentation

The ATS contract documents (drawings and specifications) are divided into various sections or ‘divisions'. Several of the divisions include the information regarding general requirements of the contract such as terms & conditions and special conditions. The remaining divisions include the requirements for each of the various design discipline areas. Each respective design group within NYCT developed their specific design discipline divisions of the contract and their associated drawings. A consulting firm developed the new ATS system functional requirements that became part of a new division of the specifications. Although there was an exhaustive effort to gather input from the end-user, RTO, and to develop system functional requirements for ATS, a documented list of user requirements and a Concept of Operations document were never developed for this project. The user requirements of other key stakeholders, such as network security and several maintenance organizations were also overlooked. In addition, the functional requirements for the voice communications sub-system are vaguely described in the contract documents as required to be “of similar functionality to what is currently available at the Command Center” in Brooklyn. There are no specific functional requirements detailed in the specifications for the voice communications sub-system, which has resulted in several disputes with the contractor during various phases of the project about what is really required to meet the users' needs and expectations.

Project Status

As of February 2006, the ATS project is currently in the last stages of Implementation and Test. There are still several major software variances as well as several critical field issues that are outstanding and resolution of these items is necessary in order to run efficient and reliable service. In September 2005, NYCT moved RTO from the Brooklyn Command Center to the Rail Control Center in an attempt to utilize the voice communications sub-system and have the ATS operators gain experience and confidence with this system component. However, after 11 days in operation, the project experienced a major setback when the communications system failed due to a high volume of calls as a result of a combination of events - an attempted suicide on the tracks and a bomb scare on the Lexington line. RTO is still awaiting a return to the RCC, and there is significant pressure from the NYCT President and the MTA parent organization on the project team to resolve this system performance issue and complete the project.

Contractor Joint Venture Team

The ATS contract was awarded in November 1997 to a US-based signal company familiar with NYCT and its operations. They assumed responsibility for systems integration, the design and implementation of the signaling and communications work, and the Centralized Train Control (CTC) portion of the office software. The remaining software was the responsibility of a subcontractor who had previously performed a smaller computer-based train control project for NYCT on the 63 rd Street connection. An electrical contractor was subcontracted to do the installation work both in the field and at the RCC. The project duration was initially set at 60 months. Despite 18 months of negotiations during the RFP phase, it was determined that the contractor's software development team could not meet major functional requirements in the contract and there were concerns about the scalability of the system. The contractor was defaulted in 1999.

NYCT then contracted with a Joint Venture (JV) team in September 2000 to complete the project. The newly structured JV consisted of the same signal company and electrical installer from the defaulted team. In the new contract agreement the signal company is only responsible for design, installation, and testing of the PLCs in the field. The new JV partner, a European software company assumed the lead in systems integration and overall software development. The selection of this software firm was based entirely on its established software platform used at other rail transit properties and had originally been developed for SCADA type applications. It would require customization for NYCT's unique signal control functionality. The new contract was given a 48-month duration, the remaining time left from the original contract timeframe. This aggressive schedule was based in part on the determination that documentation from the original contract could be re-used. Unfortunately, the contract documents did not include an updated proposal from the new JV company. This led to many vague interpretations of what was expected throughout the project. The new JV lead also had no experience doing work for NYCT and was not familiar with its complex signaling system and Rapid Transit Operations.

The lead JV partner maintains a NY based project management office responsible for system design and integration as well as hardware selection. Its software development team resides overseas, which poses several challenges related to coordination of project management and systems integration. Although from the same parent company, the managers of these two units report to two different principles within their organization.

NYCT Project Team

NYCT had limited experience in managing large systems projects, but made efforts to follow a disciplined approach to reviewing and approving contractor's submittals and Contract Data Requirements List (CDRLs) by forming working group teams based on areas of responsibility. Working groups were established for Systems, Software, Software Process, RCC (for building issues), Signals, Test & Commissioning, and User (prototyping and training). Chairpersons of these working groups were NYCT personnel from various departments, and members of these groups had cross-functional representation. Additionally, NYCT's own staff was augmented by engineering consultants, co-located in the construction manager's office along with and in support of the Software working group lead, managing the requirements and functional testing of the systems division of the specifications.

The construction manager's office is part of a Program Area, which has field inspectors who have traditionally overseen work on conventional signal projects. Because of the complexity of the ATS system, which requires additional expertise in other areas, such as fiber optics and telecommunications work, this team has had difficulty in conducting inspections and are not familiar with the proper operating procedures and protocols required by the associated operations departments for access and protection and working on live equipment. A provision to have a communications engineer on the project or matrixed personnel from the operations department to perform contractor oversight for this work is being contemplated on the next phase of the ATS program. Additionally, there is a recent acknowledgment from NYCT management that NYCT may need to evaluate its current organizational structure for large systems projects. One option that is being proposed is to provide "systems engineers" resident in the construction manager's office to provide technical support to the construction manager in making more informed decisions with a systems perspective.

Systems Engineering (SE) Management

At the outset of this project there was no formal SE process or products considered. A specific Systems Engineering Management Plan (SEMP) was never developed for this project, which would have defined the formal SE project organization. However within the first year of the new contract, working relationships evolved into an informal team approach. Although the project team did not include all the project stakeholders, it consisted of a core group of representatives from the main design discipline areas, such as Signals and Software, and Rapid Transit Operations, the operational end-user of the system under development. On projects with less complexity, the project team members were able to be stove-piped in their approach. However, they learned that this project would require working in a team, to not only provide their specific domain knowledge but to also understand each other's concerns and evaluate their decisions on the entire system as a whole. As a result the project team was able to overcome the lack of formal processes by their strong communications and their ability to reach consensus with their different perspectives in mind. Unfortunately, some decisions were made in haste; the team evaluation approach having been circumvented largely due to schedule pressures.

Another issue was that existing NYCT-internal project management procedures and templates did not fully define a systems engineering approach to planning, designing, and implementing a capital project such as ATS. There are currently no steps within these procedures, for example, to explain the necessity to identify all the stakeholders of the system (including the system and equipment maintainers), how to capture and document a comprehensive set of user requirements, and the importance of developing a Concept of Operations. The procedures were written in the context of standard “brick and mortar” type projects. They do not address the need for interdisciplinary perspectives nor explain how to manage requirements that may affect multiple stakeholders.

Efforts to Communicate NYCT-Specific Domain Knowledge

The ATS contract required the contractor to be familiar with NYCT operations and its signaling system. Although the signal company and electrical installer had individuals who possessed this knowledge, the JV partner responsible for software development did not, and there was no specific contractual agreements between the partners to share these resources. NYCT had not conducted a formal qualifications evaluation of the new JV partner to assess its NYCT-specific domain knowledge. They had relied on their expectation that the required expertise would be provided by the contractor. As part of the contract deliverables, the contractor's Signal Engineer, provided by the signal company, created an Interlocking Rules document that translated the general functionality of the signal system for the developers' understanding and use in their design. However, an addendum to this document explaining the site-specific nuances of each particular interlockings was never developed. The generalized algorithms designed in response to the Interlocking Rules document in most cases were unable to support the varying inputs from the field. To overcome this deficiency, NYCT tried to supplement the information provided in the Interlocking Rules document by providing training to the software developers in NYCT operations, courses similar to those given to its own train operators. But it was evident that this knowledge could not be captured in such a short amount of time. NYCT was forced to provide the services of its own Signal Engineers to provide guidance and consultation to the contractor's software development team in order to move the project forward.

Design Review Activities

During the development of the ATS system, the contractor was responsible for conducting a series of formal reviews at defined development milestones enabling the NYCT technical team to periodically review the contractor's work for compliance with the specification requirements and the overall system design objectives. The technical review process included a Specification Phase review, a Software Requirements review, a Design Phase Review, a Detailed Design Phase Review, an Implementation Phase Review, and a System Test Readiness Review. Formal reviews would cover each ATS sub-system, function, and hardware component. Deliverables included documentation, drawings, and other submittals as specified by the contract. It was NYCT's intention to have this be a gate-managed process with clear entry and exit criteria, not allowing the contractor to proceed with the next phase review without successfully completing the previous phase. Each review updates the material from previous reviews; as well as maturing the design to the next level. Again, in this case, the schedule unfortunately outweighed the importance of the technical reviews and these reviews were not conducted. Open issues were permitted to remain unresolved from one phase to the next. (Some remain open to this day.)

A related problem connected with the review activity was a lack of participation by all responsible maintenance groups. NYCT was slow to recognize the need to develop a maintenance philosophy tailored to the operational concepts of the new system. Appropriate maintenance stakeholder organizations had not been identified until after design completion. Interfaces to legacy systems were outlined in interface requirements specifications developed by NYCT. However, this was an area which no one from the project team took full ownership of. After 9/11, there were new network security concerns raised in the latter part of the contract, and the security of the network was impacted further as a result of on-going changes to the legacy systems by NYCT software engineers. There were decisions by the systems administrator of the legacy/enterprise systems to re-locate the servers of these systems to the RCC, requiring in-house work by NYCT to implement a new network infrastructure in the non-ATS computer room to support these interfaces. These events delayed review of this portion of the design until after the Final Design Phase. Due to a lack of a formal change management process, these new requirements on the project were never properly evaluated for schedule and resource impacts, and resulted in delays to the project. In addition, the building systems (HVAC, etc.) were also affected by this issue resulting in even further additional cost and schedule impacts.

Requirements Evaluation

It was difficult for the NYCT team to follow a whole systems approach to requirements compliance because the contractor managed the requirements traceability matrix in two pieces, one for the systems and software elements and a separate one for hardware. No requirements tracing was performed for the other divisions. A comprehensive evaluation of the requirements, particularly in the area of performance, where requirements are often allocated across software/hardware and field equipment boundaries, was time consuming and has yet to be completed.

 The processes that the contractor followed for system and software requirements traceability would be a familiar one to most engineers who have had exposure to the standard software practices “V” model. (See the figure below.) The contractor used the requirements tracing tool RequisitePro for systems and software requirements decomposition and allocation to their design and test procedures. Through the use of this tool, individual contract requirements were uniquely labeled with Customer Requirement Definition (CRD) requirement numbers. As shown in the figure, requirements tracing passed in two directions; vertically and horizontally.

Shows the NYCT ATS vertical and horizontal tracing.

Figure 8‑2– NYCT ATS Software Requirements Traceability

The vertical tracing connected each ‘CRD' to a Software Functional Requirements Document (SFRD) feature (FEAT). The FEATs then traced to a Design Element (DE) in the design document and then to individual software modules. The horizontal path directly connects CRDs to verification Test Cases. While there was no equivalent formal traceability done for requirements listed in the contract for other divisions, oversight for these areas was fortunately familiar ground to each responsible NYCT organization. NYCT has extensive experience managing infrastructure and signaling type contracts and these projects have been commissioned successfully by relying heavily on resident expertise for inspections and document & drawing reviews. Contract compliance oversight in these cases was continuous but undocumented.

The verification of the hardware section of the specifications followed a process that has elements of both of the above methods. The contractor had developed a manually generated Requirements Traceability Matrix for the ATS office hardware. This was used primarily as a reference by the responsible NYCT systems organization during the project development. Much of the compliance verification in the case of office hardware focused on review of Commercial-Off-The-Shelf (COTS) equipment catalog cuts.

A major challenge faced by the NYCT team regarding requirements verification activities was a clash in the "corporate culture" between the software engineers who were familiar with standard software development processes and those who were more accustomed to the “reliance of domain experts” used in past projects. Management, who generally fell into the latter category, remained unconvinced of the usefulness of what seemed to them an endless review process in the early requirements and design stages. They had the perception that this activity was holding up their job. Oversight trips to the contractor location for process and development audit purposes were not taken partly as an attempt to shorten these development activities and contributed to phase transition reviews becoming virtual milestones rather than real ones.

Prototyping Phase

Prototyping for this project primarily focused on on-screen dialog and on display content and appearance. Dialogs provide the user interface for train control, scheduling, and configuration. Displays allow the user to monitor train and system status. Reports that were traditionally typed were included which would allow for manual semi-automatic (e.g. through selection lists) and automatic (i.e. system database provided) entry. Initial plans were to prototype these screens on actual user console configurations. However, in order to shorten this development phase, NYCT allowed the contractor to submit individual screens shots that were depicted only from power-point presentation images. Usability issues such as navigating through multiple screens were difficult to evaluate using this method. Without the use of a more sophisticated prototyping tool, the dynamic aspects of GUI interactions were difficult to capture.

Once submitted, the prototype team, made up of engineering and user representatives, discussed and provided comments. A second conference was then held with the contractor where agreements were reached on changes. The figure below represents a typical example of a result of the discussions and agreements, as outlined in the Prototype Comment Report (PCR). Each prototype comment was given a unique PCR number, description and status.

PCR no.

Title

Subtitle/info

NYCT Comment

Status

768

Performance Reports

Train Sheet Performance Report

Train Sheet Performance Report: clarify if the report is filterable only by individual lines (services) or also by multiple services.

15may02: Contractor - will PROT2.2.12.2 provide capability to select multiple services for performance reports.

Figure 8‑3 – NYCT ATS Prototype Agreement Example

The final system suffered from the exclusive use of the PowerPoint presentations for Prototyping. Inter-operability and user experience scenarios were untried prior to Factory Acceptance Test. This led to a design that while meeting the contract requirements is sometimes awkward to use. When a concern became serious enough, it forced NYCT to issue Additional Work Orders (AWOs) to the contractor in order to correct the problem, adding to the project's schedule and cost.

Another issue concerning prototyping was with how the contractor handled the tracing of prototype agreements. These were not handled in the same manner as requirements in that they were not traced horizontally directly to test procedures. The figure below, provided by the contractor showing their concept for traceability, illustrates this.

Shows NYCT ATS prototype traceability.  It  is not traced horizontally or vertically.

Figure 8‑4– NYCT ATS Prototype Traceability

The contractor maintained that all prototype agreements would be kept because they would be addressed by the design and the design would then be tested. However this assertion depended entirely on the quality of the design documents. In many cases the design documents did not include sufficient detail to include this prototyping information. The development of test procedures that addressed verification of prototyping agreement specifics was even more unlikely. The records from the initial prototyping agreements developed by the prototyping team ultimately proved to be an invaluable asset in assuring prototype compliance.

Prototyping of the voice communications Graphical User Interface (GUI) was not done. To compensate for the limited detail in the contract specifications, and the contractor's lack of knowledge of RTO's operational needs, NYCT spent several weeks jointly developing the GUI screens with the contractor for the new sub-system.

Testing Activities

NYCT did not feel it was necessary to closely monitor and audit the contractor's software development progress, and only visited their overseas software facility once prior to the Factory Acceptance Test (FAT). Despite suggestions from the contractor that an NYCT engineer reside at their facility during the software development cycle, NYCT management felt that the potential benefit over other methods of technical exchanges was not worth the added cost.

As required by contract, the JV submitted a FAT readiness certification letter attaching their Pre-FAT test results. The letter was received just days before the NYCT test team was to depart for FAT overseas and a quick review of the Pre-FAT results revealed significant remaining functional deficiencies. Additionally, during FAT, it was discovered that the developers did not have the basic understanding of NYCT's signaling system and had misinterpreted how certain functions were to operate. In order to avoid additional schedule slips, following this discovery that the contractor had not progressed as far along as status reports had indicated, NYCT provided full time support of their own signal engineers to the project. Two signal engineers stayed overseas for 6-month duration to assist the contractor in re-design efforts resulting in unanticipated costs to NYCT.

As per the approved software planning documentation, the contractor had an independent test team responsible for the supervision and execution of software test activities. Although ultimately reporting to the same management, this independency was more a distinction of this team's focus when compared to the primary goals of the development team. For FAT and field performance testing, this team developed the test procedures, and documented and tracked the resolution of variances found during tests. Clear Quest was used to monitor testing progress and allowed the developers, testers, and NYCT the ability to evaluate the latest variance status. A strategy of phasing Field Performance Testing (FPT) in parallel with FAT was developed jointly between NYCT and the contractor, and interim software development milestones were defined. For several months, NYCT had a test team at the contractor's software facility performing FAT while another team began FPT in NY. Also, several FAT activities were conducted in NY on the training system. This posed a challenge for configuration management, keeping track of software and database versions for the test system overseas, the training system in NY, and the actual operating system at the RCC.

The contractor was given permission to have a remote connection at the Rail Control Center to provide for easy downloads of software releases from the development team. This line was open and unmonitored and allowed the contractor to easily make updates to both the ATS office software, front-end application logic, and database without first notifying NYCT. Several times during operational tests, problems were encountered because changes were not properly documented which ultimately resulted in delays in train service.

Another issue that arose during field-testing of the ATS office software was the contractor's inability to provide a realistic test schedule. Schedules were always best case, developed with little contingency planning, and were often useless after the first missed event.

Testing of the voice communications sub-system was the responsibility of the communications integrator, with little oversight by the lead JV partner, who had responsibility for systems engineering/integration. Although systems integration testing of ATS and voice communications was required in a factory setting with simulation, this requirement was not strictly enforced under the project's schedule constraints and the contractor was allowed to conduct these tests on site at the RCC. As a result, a myriad of problems that could have been detected easily in the factory took more time to detect and troubleshoot given the conditions of operating with live circuits on an operating railroad.

Contractor Developed Training

The contract required a Training Program for both the ATS operators to learn how to operate the system, and for the maintainers (both software and hardware) to maintain the system. This program was to be developed as a train-the-trainer, and gave the contractor the option to include OEM courses from third party providers.

The contractor utilized its subsidiary unit to develop the ATS dispatcher course. Although the instructor was familiar with the generic SCADA-based software package that formed the basis of the ATS software, the instructor lacked knowledge of the ATS system and did not have experience in NYCT operations. The instructor had not spent anytime overseas and had little contact with the development team. Training documents were developed before the completion of FAT, and were not updated each time the software was revised. The training was a 10-day course and it was difficult to develop a training schedule given the continual delay in FAT completion and the requirement to train the users within a 6-month window prior to the initiation of actual system operation. Since NYCT had not foreseen the importance of having the RTO instructors participate in the FAT, they had difficulty working with the contractor to develop the coursework to train the operators. They could not explain the differences between the current operations and operations conceptualized with the new system. There was no formal mechanism to alert the instructors to changes that were being made on the training system.

During field performance testing, which requires support from 10-20 trained ATS operators, a challenge for the test team was supplying enough ATS operators to perform required testing. Many of the problems encountered during testing were found to be operator-error.

The courses designed for hardware maintenance were a compilation of various OEM courses that did not contain any project specific information about the system that was being implemented. Courses were not tailored to the appropriate maintenance groups, and the large number of courses, many with other courses as prerequisites, was a constant strain on NYCT resources. Coordination of the training courses was not handled by NYCT's Employee Development and Training division, but by the Construction Manager's office that had difficulty ensuring that the right individuals were attending the right courses.

The software and database maintenance courses provided by the JV lead were generic versions utilized on the SCADA projects. The level of detail included in the manuals to explain the customized parts of the software were not what was expected. There were disputes regarding the definition of “software maintenance” and what tasks were associated with this term. After an initial pass of the software maintenance courses with a core NYCT team, it was determined that these courses were not at an appropriate level to expect NYCT to assume responsibility for maintenance. NYCT initiated its option in the contract to negotiate a maintenance contract with the contractor for a 3-year period at additional expense. NYCT was also remiss in staffing personnel who were proficient in software coding at the beginning of the project in order to understand maintenance of the new system post delivery.

Major Process Lessons Learned

The ATS-A NYCT team has jointly developed lessons learned. This activity takes on a greater importance due to the follow-up plans for the second phase of this project on Subdivision ‘B'. Lessons learned related to this case study are listed below.

Table 8‑1 NYCT ATS Lessons Learned

Improvement for ATS-B

Impact to ATS-A

Prototyping should be demonstrated on an actual workstation mock-up

Future users did not have a clear understanding of screen interactions when only evaluating individual presentation screens. Workstation workspace was purposely planned to be tailor-able to an individual's preferences therefore a “typical” screen layout could not evaluated for usability.

Prototype agreements should clearly be identified following the criteria for requirements: unambiguous, unique, and testable. They should be tracked as supplemental contract requirements

ATS-A prototype agreements were embedded in “discussions”. The contractor tracked them to the design documents that lacked the appropriate detail to address these issues. Much effort was spent by NYCT to make sure that agreements were not bypassed during acceptance test.

Independent Contractor Test Team – Confirm adequate coverage in specification to assure this standard development process protocol.

The independent test team (i.e. sole responsibilities are testing and taking/re-checking variances) slowly disintegrated. Only developers remained. In addition to the conflict of issues problem is: Developers seldom know more than their own area well; not the whole system. Developers also typically feel testing is not part of their job. Time spent by developers testing impacts project schedule because variances are not being corrected.

Design Document updates – When design is altered or more detail is added due to prototype, variances, AWOs etc. A new release is not expected, however the working copy of the design document should be modified and available on-line

As implementation and testing progresses, the design documents are further and further from reality. Other SOWs, letters of direction, memos and Emails constitute the actual design. Every change such as this should include an equivalent documentation update; the release letter should identify which documents are affected.

Design Review milestones should be taken seriously and successful completion should be a prerequisite for proceeding to the next review phase.

No time was saved by accepting virtual milestones; rather time was lost in later phases of the project for longer durations.

An NYCT Integrated Project Team should be formed and dedicated to support the Construction Manager

Inappropriate skills and competencies of personnel resulted in the wrong individuals inspecting, testing and accepting equipment/systems; Construction Manager was forced to make decisions without the right technical support.

Change Control Board – Should be instituted early in the process and include NYCT involvement.

The requirements were never truly “base-lined”, which posed difficulty to the project team to assess necessary changes. Without a formal change process through a Change Control Board many decisions were made by management without careful evaluation of the impacts to the entire system, and the associated risks to the project.

Requirements Traceability Tool should be utilized from a “systems” perspective.

Traceability of system performance requirements is time consuming and has yet to be completed.

Need to have a Systems Engineering Management Plan to define roles & responsibilities of the project team

Although working groups were established, discipline engineers still had a tendency to work with a stove-piped approach to design and implementation of their specific sub-systems.

Operator training needs to be conducted early enough in the project to provide available and qualified resources to support testing activities.

Mismatch of qualified testers during critical phases of site-acceptance testing; Delays to project schedule also resulted in operations staff having to be re-trained.

 

Return to top
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000