U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

Construction

 
<< Previous Contents Next >>

Quality Assurance in Materials and Construction

Observations and Recommendations

Observation #4: Division Offices have a lack of understanding of the six elements of QA and application of the elements in evaluating, approving, and monitoring a State's QA program.

Although this effort was not a compliance review, one of the objectives was to assess how Division Offices were evaluating, approving, and monitoring their State's QA program. In doing so, the Team developed its review guidelines to mirror each of the six elements of a QA program 0(contractor quality control, Agency acceptance, independent assurance, dispute resolution, laboratory accreditation and qualification, and personnel qualification/certification) and assess each one independently. During the interviews, this invariably led to discussions about whether or not elements of the program were in compliance with the regulations and/or Technical Advisory T6120.3. The Team did not view this as a negative consequence, in fact, quite the opposite. This often led to a healthy and vigorous exchange of viewpoints and ideas between Team members and Division Office management and staff.

While it is not appropriate to dwell on the specifics, it is appropriate to present some data related to the magnitude of the issue. Of the seven Division Offices we interviewed, the opinion of the Team was that none of these programs were in full compliance with the QA regulation and/or guidance; this does not necessarily mean the programs were not effective. One Division Office stated very plainly that they were aware their program may not have been following the regulations and guidance in their entirety. This Division Office felt their program was very effective but they had no method of measurement or documentation (see Observation #5). In five States, the Independent Assurance Program was either not complete or not properly implemented; of the four States that utilized contractor test results for acceptance, none were validating the contractor data in accordance with the regulation and/or Technical Advisory; four States were not properly using dispute resolution for materials testing discrepancies; two States lacked laboratory qualification requirements; and one State did not require technician recertification.

During the course of our interviews, it became evident that five of the seven Division Offices had not looked at all the elements of their State's QA programs in many years. It was also apparent to Team members that some Division Offices were unaware of what the six elements of a QA program were.

With FHWA moving towards risk-based decision making, it is imperativethat those individuals assessing risk in a given program area have an in-depth understanding of that program. Without a thorough knowledge and understanding of what a QA program is, the results of any subsequent risk assessment are dubious at best. As an example, the Team opines that no Division Office should have a risk assessment rating of low for impact based on the volume of Federal-aid dollars that are impacted by the QA program; yet there are Division Offices that rated this low for impact.

In looking at how to improve the Agency knowledge base, the views of the Division Offices interviewed, and the Team, converged on three primary areas: (1) Agency resources; (2) training resources; and (3) FHWA QA Certificate.

Agency resources

As noted in previous sections of this report, Agency staffing in QA has been reduced in recent years. With the concurrent change in emphasis areas, employee turnover, and a change in how new employees are brought into FHWA, a gap in the level of knowledge in QA at the Division Office level exists. Prior to the mid-1990s, FHWA engineers went through a training program that included significant exposure to QA and related issues. Typically, the first position an engineer held upon completing the training program was an Assistant Area Engineer where skills in QA were further developed. This is no longer the model in use; the current Professional Development Program does not require a new hire to spend time becoming exposed to QA. Also complicating this issue is that a majority of new employees are mid-career hires who are hired into a specific position and may have no exposure to QA. Consequently, newer employees do not have the background in QA that existed in the past.

There were Division Offices and States that felt they lack the ability, understanding, or awareness of the skills needed to evaluate their programs. For example, information provided in Technical Advisory 6120.3 speaks to the intricacies of the use of contractor test results in the acceptance decision. This will be discussed further in Observation #5, but the fact remains that Division Offices and States believe that they are inadequately trained and equipped to handle the mathematical rigor necessary to perform the analysis.

Throughout FHWA there are a variety of staffing models used in Division Offices and some lend themselves more towards maintaining QA expertise than others. Some offices have a single engineer whose primary function is QA and that person serves as a resource and provides guidance to others in the Division Office. Other Division Offices treat QA as a collateral duty that may rotate periodically among Division Office staff. The QA is an area that takes time, passion for the topic, and hands-on work to develop expertise. While it is likely that not many Division Offices will have an individual with expert-level skills, there remains a need for knowledge in QA topics. This knowledge will assist Division Offices in both assessing overall risk in the Federal-aid program and for working with their State DOT to manage the program. Successful QA program management will accomplish the goal of quality projects constructed in accordance with Federal requirements.

There are several ways to augment the skills and resources that exist in the Division Offices. First, developing an up-to-date Manual of QA Practice would provide Division engineers, and others across the Agency, with a single point of reference for information related to QA. This manual would include all current information on QA including the non-regulatory supplement, policy memoranda, technical advisories, website addresses, and would also include references to other sources of information such as NCHRP syntheses and TRB reports. Ideally, this manual would also be available online which would help address the issue of confusing FHWA Web links as discussed in Observation #3.

Another way to maximize the use of existing Agency resources is to establish an identified network of Agency specialists in QA who can serve the Agency in a capacity beyond their existing office. Although the issue of reduced QA staffing is very real, the Agency does have individuals who are experienced in QA. Several of these individuals work in Division Offices which may be seen as desirable from the standpoint of another Division Office seeking guidance on an issue involving working with the State. Establishing a network of this type would also help to address the confusion that exists at the Division level regarding the role of the Resource Center and various Headquarters Offices in QA, as discussed in Observation #3. Once this network was established, and publicized, Division Offices and others in the Agency or at the State level would have a ready reference to use when seeking help with a variety of efforts such as a process review, specification assistance, or resolution of specific issues.

Building off this idea is the concept of sharing existing employees with expertise in QA in a way that specifically improves Agency competence in QA. As FHWA wrestles with this overall concept, the Agency could use the existing peer review model that exists in the research area and apply that to QA. Using the above-mentioned identified pool of talent in the Agency, a designated number of peer reviews could be conducted each year; if desired, these could be done in conjunction with the current reviews of State QA programs being conducted by the Office of Infrastructure. While not specifically discussed with the Division Offices during the interviews, based on the discussions, the Team's opinion is the Division Offices would welcome peer reviews of this type.

Training resources

Historically, FHWA training in "QA" has focused on the technical aspects of the materials and the associated construction techniques. Some examples are the FHWA Materials Course for Field Engineers, Materials Control and Acceptance Quality - Assurance Course, and the six-week materials course "Highway Materials Engineering," currently presented annually at the University of Nevada-Reno. Over the last several years these classes have shown a continuous decline in attendance. The attendance for the Materials Course for Field Engineers was so low that the course is no longer offered. In the last several years, the Office of Pavement Technology has developed and presented training in statistical acceptance concepts. These seminars/workshops are very good but they are limited in scope and do not address QA from a programmatic perspective.

The FHWA, and States, would benefit significantly from a training course that looks at QA more holistically and covers each element of a QA program. All seven Division Offices requested training in the basics of a QA program and its six elements. As an example, the New England Transportation Technician Certification Program (NETTCP) has a QA Technologist course that currently is the closest thing to providing a full programmatic view of quality assurance. This course was designed to certify State and contractor practitioners in QA concepts as they relate to production of highway materials and the subsequent determination of quality level attained.

The NHI is in the process of modifying the NETTCP course to make it appropriate for a national audience. This course is expected to rollout this year. This NHI course could be used as a base to develop a product that meets the needs of the FHWA audience. We envision the final product being a "QA 101" training course that would detail the responsibilities of Division Office specialists, generalist engineers, and potentially to Division Office management as well. The workshop would present QA terminology in an effort to resolve the widespread confusion in terms that exists along with explaining each component of a QA program and presenting some best practices for each. The 23 CFR 637 and its associated supporting guidance could also be discussed in detail.

FHWA QA Certificate

As we discussed the issue of the Agency's knowledge base during the Division Office interviews, the concept of FHWA-certified QA employees was raised or supported by five of seven Division Offices. A certificate program would support several items discussed in this section. Specifically, it would:

  • Provide motivation for employees to enhance their existing QA knowledge;
  • Help identify the existing knowledge base in FHWA and help maintain the integrity of a nationwide database;
  • Identify potential members of QA Peer Review Teams;
  • Provide a pool of potential instructors to draw from for a QA 101 course; and
  • Raise the overall Agency emphasis on QA as called for in Observation #1.

In the September 2006 FHWA all-employee videoconference, Executive Director Bud Wright expressed his support for the establishment of core competencies and baseline knowledge and the verification of these competencies through certification. Given the significance of QA, it is a logical choice to be a topic area for FHWA to explore this approach.

In summary, each of the recommendations listed below will help to raise the overall Agency knowledge and skill level in the QA area. Increased consistency, improved networking, improved communication of best practices, improvement in QA programs nationwide, and, ultimately, better quality projects for the traveling public are all likely to follow.

Recommendation #4a: The FHWA should develop a QA Manual of Practice to be a single point of reference document that would provide guidance and information to Division Offices.

Recommendation #4b: The FHWA should identify and/or develop a national network of QA talent to serve as a resource to others in the Agency.

Recommendation #4c: The FHWA should establish a QA peer review program using the existing Research Program peer reviews as a model.

Recommendation #4d: The FHWA should develop a comprehensive "QA 101" workshop that is built off of existing materials.

Recommendation #4e: The FHWA should implement a QA certificate program to recognize employees who have demonstrated a level of competence in QA and who can serve as Agency resources in the QA area.

Observation #5: Division Offices could not assess the effectiveness of the six elements of the program.

One of the stated objectives of this review was to assess how Division Offices were reviewing, approving, and monitoring their State's QA program. As previously noted, the Team's review guidelines were set up to address this for each of the six elements of QA. For several elements when we asked how the Division Office was evaluating the QA program, they responded by asking Team members how to properly evaluate the elements. All Division Offices stated that tools are needed to provide guidance in evaluating QA programs; they further noted that determining if a program complies with the regulation is different from determining if a program is effective. The effectiveness of parts of the program are not being adequately assessed; this is directly linked to the knowledge gap discussed in Observation #4 and the lack of a tool to assess the effectiveness of the elements of the program.

The regulation requires the Division Administrator to approve the State's QA program. Throughout the review, we were made aware of the significant benefit of continued Division Office monitoring and evaluation of specific elements of the program and how those efforts led to improvements. There are numerous examples to illustrate this including:

  • The Utah Division Office's detailed joint QA review with Utah DOT resulted in reassessment of the overall QA program in Utah and developed an action plan to make long-term improvements to their program.
  • The Alabama Division Office attended the technician certification classes and raised concerns about the course materials. This resulted in a new vendor being selected and the courses being substantially revised;
  • The Texas and Pennsylvania Division Offices encouraged a statistical review to prevent the State from lowering the testing frequency on project produced materials;
  • The Illinois Division Office makes multiple presentations at industry forums to promote compliance with QA programs and make industry aware of changes resulting from the numerous reviews conducted by the Division Office.
  • The Washington, DC Division Office performed a joint review with the Office of Infrastructure, the Resource Center, and the District that resulted in revisions to testing frequency, IA program, and technician qualification program; and
  • The Wyoming Division Office coordinated with the State to develop four standard quality control plans that are included in the specifications.

This topic area is a logical follow-on to the items discussed throughout this report. As performance expectations are set and the awareness and knowledge of QA is raised, FHWA staff will need tools and methods to properly evaluate QA programs from a more holistic viewpoint and will also look for characteristics exhibited by outstanding programs. Some of the existing guidance contains discrete aspects (e.g., the FAPG, dated July 19, 2006, Transmittal 36, describes what characteristics an acceptable technician certification program should contain) but additional tools/evaluation techniques and methods are needed. As FHWA increases its use of risk-based processes, several potential tools lend themselves to a risk-based approach for program comparison and evaluation.

Division Offices mentioned several topics where they felt tools would be especially useful:

  • Evaluating testing frequencies and risk for various materials for use by Division Offices when approving material test frequency guides;
  • Evaluating different quality measures under a variety of "what if" scenarios, i.e. percent within limits versus average absolute deviation versus conformal index and so on;
  • Acceptance of small quantities;
  • Acceptance of manufactured/certified materials;
  • Overall evaluation of buyer's and seller's risk; and
  • Evaluation of contractor quality control programs.

These tools are focused and we see them as parts of an overall package that should be developed to guide Division Offices in their evaluations of QA programs.

As States face the continued pressure from their governing bodies and face continued budget tightening, there is considerable need for Division Offices to stay engaged in their State's QA program. Providing the tools necessary to evaluate the overall effectiveness of their program will help assure that Division Offices serve as an integral part of the QA process.

Recommendation #5a: The FHWA should develop a question-based evaluation tool to evaluate the effectiveness of the six elements of a QA program and the program as a whole.

Recommendation #5b: The FHWA should develop risk-based evaluation tools to:

  • Develop and assess appropriate testing frequencies and associated levels of risk for a variety of materials.
  • Compare quality measures in a variety of "what if" scenarios (Percent Within Limits versus Absolute Average Deviation versus Conformal Index, and so on).
  • Assess variability of material tests conducted by contractor versus owner. (See discussion in Successful Practices section of report.)
<< Previous Contents Next >>
Updated: 06/27/2017
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000