U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

California Division

Home / About / Field Offices / California Division / Systems Engineering Guidebook for ITS

Home What's New Systems Engineering Guidebook Views Search Glossary Resources Feedback Site Map

3.9.8 Decision Gates

 

OBJECTIVE:

Process improvement provides a method for the continuous improvement of processes and products over the project life cycle. These process improvements are transferable to future projects [development, operations & maintenance, and retirement/replacement].

DESCRIPTION:

At the completion of each phase of a project life cycle, the processes and the quality of products delivered should be reviewed, assessed, and documented. At the completion of the project, the assessment for each phase should be reviewed and summarized as to its impact on the success or shortfalls which occurred during the project. The scope of the assessments should cover: processes [methods and techniques] used during the performance of each phase of the project, the quality of products produced, and the stakeholders [system's owner, consultants, vendors, and development teams] that were involved. Once documented, the assessment is used to improve the processes in place. Documented lessons learned will capture the "corporate" knowledge gained from the experience of the project. So, the lessons learned can be applied to remaining phases of the existing project and future projects. The assessment has three primary activities: planning, strategy, and performance. This assessment should be an on-going part of each project as part of an overall process improvement effort for the organization, which is described in chapter 7. This can be performed by the system's owner, other stakeholders, or an independent assessment team.

CONTEXT OF PROCESS:

Shows the flow for cross-cutting activity Process Improvement Process. Summaries are described for inputs, constraints, and enablers into the task; activities of the task; and outputs from the task. The flow is described in detail in the accompanying text.

PROCESS IMPROVEMENT PROCESS

Inputs:

Delivered phase products delivered products for the phase of work.

Process used actual process that was used for the phase of work.

Schedulesthat were used for the phase of work including the original schedule and any updates.

Budget/costs developed for the phase of work and any updates.

Control:

Project Plan/SEMP contains the process used to carry out each phase of the work, and to determine the process improvements for the project.

Enablers:

Stakeholder involvement is essential in process improvement. The stakeholders include the system's owner, development team, consultants, and all direct stakeholders who were involved in the phases of work.

Technical Reviews process is used to carry out the workshops and interview stakeholders on possible process improvements.

Outputs:

Lessons Learned Document for each phase of work will identify the strengths and weaknesses of the process and the products of the phase of work.

Updates to documented process will be developed as a result of the assessment, and recommended changes to existing documented processes.

Process Activities:

Plan Method of Assessment

Identify the set of assessment activities needed for the phase of interest. Identify roles and responsibilities, and the timeline for the assessment.

Identify the:

  • purpose for the assessment
  • scope of assessment
  • assessment team qualifications
  • team members who should be interviewed
  • Phase products and processes which need to be assessed.

Perform Assessment

The methods of assessment include interviews, questionnaires, surveys, and workshops. What criteria are established for the documents needing review? For example, an assessment would compare the planned processes to the actual processes, and the planned deliverables with the actual deliverables

Consolidate and validate data

Document the constraints on the consolidation and validation of the data collected. For example, data that was needed but could not be collected because of security, intellectual, or proprietary information]  Collect and consolidate the data from observations of the products and processes in a way that can lead to unbiased conclusions and accurate observations. One technique is to perform two independent reviews and see if the results are the same.

Analyze assessment data

Reach assessment team consensus on findings, ratings, and validated observations from a minimum of two different sources. Identify strengths and weaknesses of the product and process under assessment.

Document lessons learned and updates to process

Document the lessons observed and reported from the assessment. Develop a plan to update the process and to migrate to the new and improved practices. Clearly linking the lessons documented to the upgrade of documented processes.

Illustrates where Process Improvement occurs in the Vee Development Model. Process improvement is developed/tailored in the Systems Engineering Management Plan. Process improvement is performed in the Concept of Operations; System Requirements; High-Level Design (Project Architecture) Subsystem Requirements; Component Level Detailed Design; Software Coding Hardware Fabrication; Unit Testing; Subsystem Verification; System Verification and Initial Deployment; System Validation; Operations and Maintenance; Changes and Upgrades; and Retirement Replacement sections.

Where does the Process Improvement take place in the project timeline?

Is there a policy or standard which includes Process Improvement?

FHWA Final Rule does not specifically mention general process improvement practices to be followed. CMMI contains the information for process improvement and assessment.  See chapter 7 for more information on CMMI and organizational process improvement.

Which activities are critical for the system's owner to do?

  • Lead the development of the framework for process improvement for the project and include it as a SEMP item
  • Participate in the performance of process improvement interviews and workshops
  • Gain stakeholder support for the process improvement activities
  • Lead the updating of the process improvement documentation as appropriate for the system's owner

How do I fit these activities to my project? [Tailoring]

Process improvement for small projects such as a traffic signal system upgrade using a single vendor can be accomplished one time at the completion of the project. For large projects that may involve consultants and development teams for different phases of the work an assessment is recommended after each phase of work. The purpose is to capture any lessons learned as early as possible while the project activities are still fresh in the memories of the stakeholders involved and before the development team leaves the project. Once documentation for each phase is complete, an overall assessment is recommended.  Collection of lessons learned and assessments at the project level support broader organization process improvement objectives.  See chapter 7 for more information.

What should I track in this process step to reduce project risks and get what is expected? [Metrics]

On the technical side:

  • Assess that the correct technical metrics were defined for the project
  • Assess if the correct technical process was used to carry out the work for the phase
  • Assess the difference between the expected results and the actual results

On the project management side:

  • Assess the cost for collecting the technical metrics
  • Assess the cost for using the process to carry out the work
  • Assess the difference between the planned baseline cost/schedule and the actual cost/schedule

Checklist: Are all the bases covered?

check Is there a policy for process improvement within the system owner's organization?
check Is there a defined and managed process improvement process which has been institutionalized within the system owner's organization?
check Have resources been allocated to the process improvement activity?
check Is there training for process improvement within the system owner's organization?
check Was a process for a project assessment developed as part of the project plan/SEMP?
check Were the key stakeholders identified for the project assessment?
check Were the key stakeholders for the phase interviewed as to the strengths and weaknesses in the performance of the phase of work?
check Are the assessments accomplished in an objective manner?
check Were the assessment results documented and used to update existing processes?
check Were meeting minutes and notes kept for the assessment?
check If a key stakeholder leaves the project, were they interviewed on the strengths and weaknesses of the phase of work?
check Is there a plan for on-going process improvement throughout the operations & maintenance phase of the project?

Are there any other recommendations that can help?

Recommendation.Establish a documented system /software project management process. Before process improvement can be made, an initial process must be established as a baseline.

Tools will help establish and assess the system's owner and development team's capabilities. The Software Institute at Carnegie Mellon University has established the Integrated Capabilities Maturity Model [CMMI]. This model identifies the capabilities needed and the assessment tools used to rate an organization on its capability to perform systems/software development and project management. See chapter 7 for more information about CMMI and organizational process improvement.

Checklist  

 

Return to top
Page last modified on September 20, 2016
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000