California Division | Federal Highway Administration

8.4.9 Verification Documents Template

Purpose of these Documents

These documents plan, describe, and record the activity of verifying that the system being built meets the specified requirements. Since a complex system may involve a series of verification activities, several sets of these verification documents may be needed. All of these verification documents follow the master plan for verification defined in the Systems Engineering Management Plan.

Usually, for even moderately complex systems, the following three levels of verification documents are prepared:

a plan to initially lay out the specific verification effort
a procedure that is the specific and detailed steps to be followed to perform the test
a report on the results of the testing activity

These three documents are described in this section.

A critical issue is assuring that all requirements are verified by the testing activity. This is best done by first tracing each requirement into a test case then, into a step in the Verification Procedure.

Additional Information is found in IEEE 1012-1998, Software Verification and Validation.

Tailoring these Documents to Your Project

A separate Verification Plan and procedure may not be required for the simplest projects, especially where the system is essentially COTS and does not involve any custom software development, and where the project office personnel have a very clear understanding of the purpose of the system. In some cases, it is possible to take a copy of the Requirements Document, improvise procedures, and annotate the Requirements Document with the results of each test step. This can be a perfectly acceptable way to verify the operations of a system.

However, preparation of these verification documents is strongly advised if:

the system is more complex
there are a number of separate verification activities
multiple deployment sites are involved
more than one or two stakeholders have to be satisfied

There is also the question of how comprehensive to make the verification effort. It is impossible to test everything, that is, all possible combinations of actions under all possible operational situations. A good rule of thumb is: if it was important enough to write down as a requirement, then it should be tested, at least once, as part of a reasonable operational scenario. This may not, for example, test all possible failure mode conditions. If a good job was done in writing the requirements, then the most important and most likely are verified.

Checklist: Critical Information

	Is there a documented Verification Plan for the Project?
	Does the Verification Plan answer all the questions of who, what, where, and when concerning test conduct?
	Does the Verification Plan make clear what needs to happen if a test failure is encountered?
	Does the Verification Plan define the configuration of the hardware, software, and external system needed for each test case?
	Are all applicable requirements traced to a test case in the Verification Plan? Does each test case define a realistic and doable test?
	Are detailed verification procedures documented for the project?
	Is each step in the Verification Procedure traced to a test case and a requirement?
	Are all of the necessary initial conditions and set-up defined for each procedure?
	Has each verification procedure been dry run prior to the formal test? Have the procedures been updated as a result?
	Is there a Verification Report that documents the project verification results?
	Does the Verification Report describe, in detail, the resolution of every test anomaly encountered during testing?

VERIFICATION PLAN TEMPLATE

IEEE 1012-1998 Independent Verification and Validation

section	contents
Title Page	The title page should follow the Transportation Agency procedures or style guide. At a minimum, it should contain the following information: VERIFICATION PLAN FOR THE [insert name of project] AND [insert name of transportation agency] Contract number Date that the document was formally approved The organization responsible for preparing the document Internal document control number, if available Revision version and date issued
1.0 Purpose of Document	This section identifies the type of verification activity to be performed within this Verification Plan. For instance, this activity may verify the entire system, a sub-system, the deployment at a site, a burn-in test, or any other verification activity called for in the Program Plan or in the SEMP.
2.0 Scope of Project	This section gives a brief description of the planned project and the purpose of the system to be built. Special emphasis is placed on the project’s complexities and challenges that must be addressed and verified by the systems engineering efforts. This section also describes the environment in which the project operates. It identifies the organization structures that encompass all stakeholders. It also gives a brief description of the role to be played by each stakeholder. This includes ad hoc and existing management work groups and multi-disciplinary technical teams that should be formed for supporting the project. Such teams are critical to reaching successful system deployment.
3.0 Referenced Documents	This is a list of all documents used in the preparation of this Verification Plan. This almost always includes the Project Plan, the SEMP [if one was written], and the applicable Requirements Documents. However, reference of other documents, such as descriptions of external systems, standards, a Concept of Operations, and manuals may need to be included.
4.0 Test Conduct	This section provides details on how the testing is accomplished. It defines: who does the testing; when and where it is to be done; the responsibilities of each participant before, during, and after each test; the hardware and software to be used [and other systems as well]; and the documents to be prepared as a record of the testing activity. Another very important part of this section defines how testing anomalies are to be handled [that is, what to do when a test fails]. In general, the following information should be included in this section: A description of the participating organizations and personnel and identification of their roles and responsibilities. This may include for example, a test conductor, test recorder, operators, and/or engineering support. Identification of the location of the testing effort, that is, the place, or places, where the testing progress must be observed. The hardware and software configuration for all of the test cases, including hardware and software under test and any supporting test equipment, software, or external systems. Several configurations may be necessary. Identification of the documents to be prepared to support the testing, including Verification Procedures, a Verification Report and descriptions of special test equipment and software. Details on the actual conduct of the testing, including: - Notification of participants - Emphasis on the management role of the test conductor - Procedures for approving last minute changes to the procedures - The processes for handling a test failure, including recording of critical information, determination of whether to stop the testing, restart, or skip a procedure, resolution of the cause of a failure [e.g. fix the software, reset the system, and/or change the requirements], and determination of the retesting activities necessary as a result of the failure.
5.0 Test Identification	This section is the heart, and largest, section of the Verification Plan. Here we identify the specific test cases to be performed. A test case is a logical grouping of functions and performance criteria [all from the Requirements Documents] that is to be tested together. For instance, a specific test case may cover all the control capabilities to be provided for control of a changeable message sign. There may be several individual requirements that define this capability, and they all are verified in one test case. The actual grouping of requirements into a test case is arbitrary. They should be related and easily combined into a reasonable set of test procedure actions. Each test case should contain at least the following information: A description name and a reference number A complete list of the requirements to be verified. For ease of tracing of requirements into the Verification Plan and other documents, the requirements are given numbers. They can be accurately and conveniently referenced without repeating all the words of the requirement A description of the objective of the test case, usually taken from the wording of the requirements, to aide the reader understanding the scope of the test case Any data to be recorded or noted during the test, such as expected results of a test step. Other data, such as a recording of a digital message sent to an external system, may be required to verify the performance of the system. A statement of the pass/fail criteria. Often, this is just a statement that the system operates per the requirements A description of the test configuration. That is a list of the hardware and software items needed for the test and how they should be connected. Often, the same configuration is used for several tests A list of any other important assumptions and constraints necessary for conduct of the test case

VERIFICATION PROCEDURE TEMPLATE

section	contents
Title Page	The title page should follow the Transportation Agency procedures or style guide. At a minimum, it should contain the following information: VERIFICATION PROCEDURE FOR THE [insert name of project] AND [insert name of transportation agency] Contract number Date that the document was formally approved The organization responsible for preparing the document Internal document control number, if available Revision version and date issued
1.0 Purpose of Document	This section identifies the type of verification to be performed. For instance, this activity may verify the entire system, a sub-system, the deployment at a site, a burn-in test, or any other verification activity called for in the Program Plan or in the SEMP.
2.0 Verification Configuration and Software Under Test	This section identifies the equipment and software to be verified. It also identifies all equipment and software necessary for this verification activity that is external to the system / sub-system configuration under test. This may include special test equipment and any external systems with an interface to the configuration under test. For the hardware / software configuration under test, this section identifies: Each hardware item by part number and serial number Each item of commercial-off-the-shelf [COTS] software, by part number and version number Each source code file of custom developed software, by file name and version number For all special test equipment / software, this section identifies: - Each hardware item by part, serial, and version number - Each item of COTS software, by part number and version number - Each source code file of custom developed software by file name and version number For each external system interface, this section identifies: The name and location of the external system
3.0 Verification Setup	This section describes the steps to be taken to set up each verification configuration, including, but not limited to, tuning of the hardware, configuring and starting the software, starting the special test software, and set-up steps at each external system to be used.
4.0 Verification Procedures	This section describes the step-by-step actions to be taken by the verification operator for each verification case. Each step includes: Operator action to be taken. This operator action may be, for example, an entry at a workstation, initiation of a routine in the special test software, or an action at an external system. Expected result to be observed. This too may take several forms, for example, display of certain information at a workstation, a response at an external system, recording of data for subsequent analysis, or an action by a field device. Pass / fail entry space. Here the verification conductor records whether or not the expected result occurred. If the expected results are not observed, then the procedures for dealing with failures contained in the Verification Plan are invoked. A trace of each verification step from a verification case in the applicable Verification Plan and a trace from a requirement in the applicable Requirements Document.

VERIFICATION REPORT TEMPLATE

section	contents
Title Page	The title page should follow the Transportation Agency procedures or style guide. At a minimum, it should contain the following information: VERIFICATION REPORT FOR THE [insert name of project] AND [insert name of transportation agency] Contract number Date that the document was formally approved The organization responsible for preparing the document Internal document control number, if available Revision version and date issued
1.0 Purpose of Document	This section identifies the type of verification performed. For instance, the activity may verify the entire system, a sub-system, the deployment at a site, a burn-in test, or any other verification activity called for in the Program Plan or in the SEMP. This section can be taken from the applicable Verification Procedure.
2.0 Identification of the Configuration under test	This section identifies the equipment and software verified. It also identifies all equipment and software necessary for this verification activity that is external to the system / sub-system configuration under test. This may include special test equipment and any external systems with an interface to the configuration under test. This section can be taken from the applicable Verification Procedure.
3.0 Individual Test Case Report	This section summarizes the purpose and results of each test case performed in the applicable Verification Procedure. Special attention is paid to any test case where a failure occurred and how the failure was resolved. This section covers: Test case overview and results Completed Verification Procedure pages annotated with pass / fail results Description of each failure, if any, from the expected result called for in the Verification Procedure Any back-up data or records related to the field procedure Details of the resolution of each test failure, including procedure modification, software fix, re-testing and results, regression testing and results, and required document changes [including changes to the requirements].

Previous Document section | Next Document section

Return to top

Page last modified on September 20, 2016