U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

California Division

Home / About / Field Offices / California Division / Systems Engineering Guidebook for ITS

Home What's New Systems Engineering Guidebook Views Search Glossary Resources Feedback Site Map

3.7.2 System Validation

 

OBJECTIVE:

This chapter describes the activities needed to effectively operate and maintain the system in a day-to-day operational environment.

DESCRIPTION:

Operations & maintenance involves planning for, and executing, activities, such as operating the system, monitoring system performance, making repairs, hiring and training operators, testing the system after any changes are made, and tuning the system. All systems require regular maintenance. Preventive maintenance involves inspection and proactive actions, such as cleaning, replacement of components prior to the end of their rated life, backing up software, storing data, and replacing components that have become obsolete and unsupported. Reactive maintenance involves correcting faults when they occur. Software maintenance involves correcting malfunctions [bugs] when they are discovered, upgrading components that become obsolete and unsupported, and making minor modifications as needed to improve functionality.

CONTEXT OF PROCESS:

Shows the flow for Phase [4] Task 2, Operations and Maintenance Process.  Summaries are described for inputs, constraints, and enablers into the task;  activities of the task; and outputs from the task.  The flow is described in detail in the accompanying text.

OPERATIONS & MAINTENANCE PROCESS

Inputs:

Project goals and objectives were identified in project planning.

Support products such as users' manuals and maintenance guides were obtained during system development.

Concept of Operations describes the operational scenarios for which procedures are needed.

Changes & upgrades provide opportunities to enhance system operation and maintenance.

Control:

Project Plan/Systems Engineering Management Plan [SEMP] defines the overall operations & maintenance plan for the project, including the goals and objectives.

Configuration management will be used to manage the synchronization of any changes that might occur during the maintenance of the system. This would include replacement elements [spare parts, units, and sub-systems] that would need to be documented as part of the physical audit of the system.

Enablers:

Stakeholder involvement is needed to ensure all parties have input and are aware of ongoing activities.

Outputs:

Operations & Maintenance Plan documents the procedures, resources, training, and support needed for operating and maintaining the system.

Improved operation and maintenance will result for the life of the system.

Updated Operation and Maintenance Procedures will be developed as the system changes over time.

Requirements for next evolution are captured when identified by operations & maintenance personnel.

Process Activities:

Plan Operations & Maintenance

During the Concept of Operations phase of the project, two important views of the system are defined, the operations & maintenance views. These views, which envision how the system will operate and be maintained, become the initial planning for the system when it is commissioned into service. Once the system is commissioned into service, these plans are updated to reflect the as-is operational and maintenance environment. The complete Operations & Maintenance Plan should:

  • Identify funding and policies supporting on-going operation & maintenance
  • Identify the aspects of the system needing operation or maintenance
  • Identify the manuals [users, administrators, and maintenance], configuration records, and procedures that are to be used in operation & maintenance
  • Identify the personnel who will be responsible for operations & maintenance
  • Identify initial and on-going personnel training procedures, special skills, tools, and other resources
  • Identify operations & maintenance related data to be collected and how it is to be processed and reported
  • Identify methods to be used to monitor the effectiveness of operations & maintenance

Collect Operations & Maintenance information

Operations & maintenance information should be collected throughout the operational life of the system including: disruption in service of the system, restoration measures undertaken, and system performance. Down time and the mean time to repair should be documented and used to assess the average availability of the system. Repair logs should include vendor notice of obsolescence and notice of design changes that will affect the maintainability of the system elements.

Perform operations & maintenance

Operations & maintenance procedures need to proceed as defined in the Operations & Maintenance Plan. Over time the procedures will need to be refined and updated because the system changes or improved procedures are developed. The Operations & Maintenance Plan needs to be updated as well as the documented procedures, users' manuals, and maintenance manuals.

Illustrates where Operations and Maintenance occur in the Vee Development Model.  Operations and maintenance are defined in the Concept of Operations section.  Operations and maintenance are performed in the Operations and Maintenance and Changes and Upgrades sections.

Where does Operations & Maintenance take place in the project timeline?

Is there a policy or standard that talks about operations & maintenance?

FHWA Final Rule requires that the identification of procedures and resources necessary for operations & maintenance of the system be determined in the systems engineering analysis for ITS projects funded with Federal money from the Highway Trust Fund, including the Mass Transit Account.

Which activities are critical for the system's owner to do?

  • Secure adequate funding and management support for on-going operations & maintenance
  • Identify and recruit appropriate agency stakeholders to participate in operations & maintenance
  • Review and approve the Operations & Maintenance Plan, including any updates
  • Arrange for on-going monitoring of system performance to ensure it is being operated and maintained adequately

How do I fit this step to my project? [Tailoring]

Operations & maintenance are necessary for all systems of any size or complexity. After the ITS system is built, it is made operational and maintained in operational condition for as long as is needed. However, some systems, such as traffic signals, operate autonomously with little routine human input. They need only initial configuration and periodic review and fine-tuning of the settings. Others, such as a closed circuit television system, require hands-on involvement by a human operator as part of normal operation. But a traffic signal system may involve more intensive maintenance than a CCTV system.

The Operations & Maintenance Plan and associated documents, such as manuals, operating procedures, and system configuration records, should record all the information needed for employees to keep the system operating effectively and for managers to plan for future resource needs. Information provided should include what is needed for day-to-day activities, and also what is needed to plan for occasional activities, such as periodic preventive maintenance and system upgrades. The Concept of Operations, System Requirements, and design documents should be consulted as a checklist of all the system elements and operational aspects that may need coverage in operations & maintenance documentation.

What should I track to reduce project risk and to get what is expected? [Metrics]

During system development and implementation, there is no direct measure of the effectiveness of operations & maintenance planning. Once the system is operational, there are ways to monitor its on-going performance.

Although it is often difficult, attempt to measure the on-going operational effectiveness of the system because this is a measure of the success of both operations & maintenance. If feasible, directly measure traveler experiences, such as travel time and safety rates, either continuously or annually.

Otherwise, track indirect performance measures. Have operators record and periodically summarize notable operational successes and failures. Record maintenance actions in a way which enables calculation of descriptive statistics such as, average number of failures per year and mean-time-between failures. Track the number of traveler complaints related to the system. Look for trends that suggest operation or failure rates are deteriorating. Look for ways to make the trend move in the desired direction.

Checklist: Are all the bases covered?

check Is management support in place for on-going operations & maintenance (O&M)?
check Has funding for O&M been identified?
check Has an O&M Plan been developed and approved?
check Were all key stakeholders involved in development of the O&M Plan?
check Are resources and training in place for system start-up?
check Are established procedures for continually monitoring the effectiveness of operations & maintenance developed and approved?
check Is there a plan for long term upgrades?

Are there any other recommendations that can help?

Caution. Stakeholders often underestimate or neglect the cost of operations & maintenance. Consider the cost of configuration management, as well as hands-on operation & maintenance activities.

Remember that most software requires maintenance. This is especially true of software operating on a general-purpose computer. It may be true of embedded software in a specialty device. Even if no defects surface, most software will need to be updated over time to

  • adjust to changes in external interfaces,
  • upgrade and/or replace obsolete versions of third party components
  • be moved to a new computing platform when the original one becomes unserviceable or inadequate
  • make minor modifications to the functionality to address new requirements, or needs, that were overlooked during initial system development

Configuration management [chapter 3.9.6] keeps the documentation synchronized with the functional and physical characteristics of the system.

Any information that may be needed in the future for any aspect of operation, maintenance, retirement, or replacement should be recorded and kept up-to-date. It is not sufficient to rely on the memory of involved personnel for such information.

Beyond documentation, configuration management involves establishing and following rigorous procedures for controlling changes to the system. Change control ensures operations & maintenance personnel do not make inappropriate or undocumented changes to the system. A Control Change Board reviews and approves or rejects all proposed changes. A change can be as simple as changing a configuration setting, to replacing a major system component. The Change Control Board includes representatives of all parties with an interest or involvement in the system to ensure that all potential options and ramifications [including risk] are considered before proceeding. Development and implementation of any significant changes to the system should follow the same systems engineering process used for the original system development.

Use blanket approvals to cover routine maintenance. Routine maintenance procedures can be handled by blanket approvals of routine activities, including, regular review and a requirement to document all changes. Change control procedures should include periodic audits to confirm that procedures are being followed also, that the functionality and physical characteristics of the system match those required by the approved configuration documentation.

Related Template Checklist  

 

Return to top
Page last modified on September 20, 2016
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000