3.9.10 Technical Reviews

OBJECTIVE: Technical reviews provide a structured and organized approach to reviewing project products to determine if they are fit for their intended use. This chapter also describes a process to plan and conduct a meeting that can be used for the different types of technical reviews. Technical reviews are used to identify design defects, suggest alternative approaches, communicate status, monitor risk, and coordinate activities within multi-disciplinary teams.

DESCRIPTION: Technical reviews are critical to the success of Intelligent Transportation System projects. Technical reviews provide status and feedback on the products under review and the on on-going activities of a project. A technical review is the primary method for communicating progress, coordinating tasks, monitoring risk, and transferring products and knowledge between the team members of a project. The Institute of Electrical and Electronics Engineers [IEEE] 1028-1998 identifies the following five types of reviews: management reviews [for example, control gates] technical reviews inspections [primarily for identifying errors or deviations from standards and specifications] walk-through [for example, requirement or design walk-through] audits [for example, physical and functional audits used as part of the configuration management process] The process for conducting review meetings should be established in the Project Plan/SEMP and carried out the same way for each review. The differences in reviews would be in the content and level of formality. This formality would be tailored for the type of review and its purpose. This chapter describes a basic meeting procedure including pre-meeting activities, conduct, and post meeting activities.

CONTEXT OF PROCESS:   TECHNICAL REVIEW PROCESS

Inputs: Purpose for the meeting must be clearly established with expected outcomes. Required Review Inputs should be provided. These are products for the phase under technical review. Unresolved action items from the previous reviews are carried over for continuing discussion and/or decisions.

Control: Project Plan/SEMPcontains the process used to perform technical reviews.

Enablers: Stakeholder involvement is needed to participate and to fill the various roles for the technical reviews.

Outputs: Project Review Planwill identify how technical reviews will be carried out for the project. This will be part of the Project Plan/SEMP. Review of decisions includes the documented acceptance; re-work with comments, and deviations and waivers to the phase products by the participants of the technical review. Action items are assigned with completion dates. Critical items are tracked between meetings if necessary. Assignmentsare documented and sent out as part of the feedback to the participants. This feedback should have a definition of the action item and a planned date for completion. Feedback to participants the results of the meeting and provide a record of the meeting for their review and comments. This ensures that decisions, actions, and assignments were accurately documented.

Process Activities: Plan reviews A plan is developed for the technical reviews of a project. This plan includes the schedule for reviews, who [functionally] will be in attendance, the level of formality for each review, the entry criteria [drafts, final products], the process for the review [structured presentation or informal round-table], and the exit criteria [100% consensus agreement, majority agreement, project manager only]. Perform pre-meeting [review] activities Define the purpose, objectives, and the intended outcomes of the meeting. Prepare an agenda, identifying participants and their roles and distributing the agenda and background material. Reserve and inspect the meeting facilities and location to see if all needed equipment is in working order and that the facility meets the needs for the meeting. Example items to look for are space [size and shape], break rooms, rest rooms, lunch facilities, break-out rooms, climate control, lighting, noise levels, appropriate furniture and configuration, equipment, and electrical. Make arrangements, if necessary, for breakfast, lunch, dinner, and/or break refreshments. Perform meeting/review Technical meetings should start and end on time. The purpose of the meeting should be clearly stated, an updated agenda provided to the attendees, and a roster that documents the attendees present with up to date contact information [email, phone number, organization]. Their role [e.g. presenter, chairman, or observer] in the meeting should be placed with the meeting minutes. The ground rules for the meeting should be reviewed prior to discussion, starting with unresolved action items from the previous review. Conclude one agenda item at a time. Manage discussions so that there is focus on the topic. Follow the pre-arranged ground rules. Keep track of the time. Document all decisions, actions, and assignments. At the close of the meeting, summarize all decisions, actions, and assignments, review agenda items, and assignments for the next meeting. Confirm date, time and place of the next meeting. Finally, end on time. Perform post meeting/review activities The meeting should be followed up with a complete set of minutes that include all decisions, actions, and assignments. The minute taker, if needed, should follow up with the attendees to make sure the minutes are as complete as possible. These minutes and any supporting material should distributed back to the attendees promptly for review and comment. Assignments should be completed, and periodic progress checks on critical action items from the meeting. Honor commitments for the next meeting. Carry over unresolved actions with status and recommended resolutions.

Where do Technical Reviews take place in the project timeline?

Is there a policy or standard that talks about Technical Reviews?

FHWA Final Rule does not specifically mention general reviews and meeting practices. IEEE 1028-1988 Standards for Reviews and Audits provides information useful to decision gates.

Which activities are critical for the system's owner to do?

• Lead the definition and documentation of the process in conducting a technical review.
• Gain stakeholder support in the participation of technical reviews
• Lead the participation of technical reviews
• Review decisions, actions, and assignments from the technical review
• Follow-up on critical assignments

How do I fit these activities to my project? [Tailoring]

In this activity, the number of reviews and level of formality is tailored to the size and type of the project. For example, on a small traffic signal control system that is a COTS product, the number of reviews can be minimal [bi-weekly or monthly]. The meetings may be informal with the project manager and/or traffic engineer in a review of progress. The feedback may be just a summary of the meeting minutes.

What should I track in this process step to reduce project risks and get what is expected? [Metrics]

Technical and project management:

Technical reviews are used to identify design defects, suggest alternative approaches, communicate status, monitor risk, and coordinate activities within multi-disciplinary teams. This would be the time and place to monitor, review, and take action on both technical and project management metrics that were set up for the phase currently in progress.

Checklist: Are all the bases covered?

• Was a review plan developed for the project?

  Did the plan contain:

  – The number or frequency of the reviews?

  – The process for carrying out each review?

  – Roles identified for each review?

  – Level of formality identified for each review?

• Was a technical review agenda developed and distributed well ahead of the scheduled meeting date?
• Was all the supporting and background material generated and distributed to the attendees well ahead of the scheduled meeting date [per the plan]?
• Were the attendees and their roles identified or defined [per the plan]?
• Were the time and location identified?
• Were the purpose and outcomes identified?
• Were all unresolved assignments, identified in the previous meeting, brought forward to the upcoming meeting?
• Has the location of the technical review been checked out for size, climate, configuration, equipment, furniture, noise, and lighting?
• Are all the presenters well prepared for the meeting?
• Were the ground rules for the meeting discussed before the start of discussion?
• Did the meeting start on time?
• Were introductions made by all attendees?
• Was an attendance roster created for the meeting with up to date contact information for each attendee?
• Was the purpose of the meeting clearly stated and what are the expected outcomes?
• Was an updated agenda provided for each attendee with the priorities assigned for each agenda item?
• Was each agenda item concluded before discussing the next or other items?
• Were all decisions, assignments, and actions documented as part of the minutes and summarized at the end of the meeting?
• Did the meeting end on time?
• Were the minutes distributed to the attendees?
• Were all critical assignments followed up between meetings?

Are there any other recommendations that can help?

Have ground rules for technical reviews. The following is a recommended set of ground rules that participants observe during the meeting:

• We tell it like it is, but respect, honor, and trust one another
• We work toward consensus, recognizing that disagreements in the meeting are okay. Once we agree, we all support the decision
• We have one conversation at a time; and our silence is consent
• We focus on issues, not on personalities; and we actively listen and question to understand
• We do not attack the messenger
• We start on time, observe time limits, and structure the agenda to end on time

A closer look at the types of technical reviews used throughout the project timeline

Planning review verifies that plans appropriate for the project are identified. Tailoring for each plan is reviewed and updated as needed.

Concept of Operations review ensures that the operation of the system being defined is appropriate, and addresses the needs of the stakeholders. This is a critical review, as the concept of operations will identify the operational needs, needed agreements, candidate external interfaces, and maintenance responsibilities.

Requirements review is used to ensure the system and sub-system requirements and verification plans are appropriate for the system being defined. This review verifies that the requirements are complete and that each meets all the criteria for a good requirement and traces to user needs.

High level Design review ensures that the project level architecture is well formed, balanced, and appropriate for the problem space, and that the functionality and performance of the defined system meet the intended need . This review verifies that the project architecture is consistent with the regional architecture. If necessary, document the differences. This is a major technical review and is sometimes called a PDR [Preliminary Design Review].

Component level detailed design review is used to ensure that the detailed design is ready for implementation. This is a major review since when completed, the detailed design is ready for implementation. This is sometimes called the CDR [Critical Design Review].

Test Readiness review is used to see if the components, sub-systems, and system are ready for verification. For each level of verification, there should be a review prior to the formal verification of the product.

Operational Review is used to ensure that the system is ready for deployment. This review verifies that all training and support for the system is in place and that the operations & maintenance personnel are ready.