U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content U.S. Department of Transportation/Federal Highway AdministrationU.S. Department of Transportation/Federal Highway Administration

Federal-aid Program Administration

 

FHWA Guidance Compliance Assessment Program (CAP)

April 2017

What Changed from the 2015 Guidance?

  • Deleted Projects of Corporate Interest (PoCI) reflecting the elimination of the PoCI process.
  • Deleted Figure 1.
  • Changed text throughout to active voice.
  • Clarified in the Review Reporting section that a comment is required with every response for each question.
  • Added text in the Review Reporting section that addresses retention of source documentation or its availability.
  • Changed the starting date for the project population window from April 1 to January 1.

Background

The goal of risk-based project stewardship and oversight is to optimize the successful delivery of programs and projects and help ensure compliance with Federal requirements. FHWA Risk-Based Project Stewardship and Oversight involves three main avenues of project involvement: 1) project approval actions; 2) data-driven compliance assurance, i.e., the Compliance Assessment Program (CAP); and 3) risk-based stewardship and oversight involvement in Projects of Division Interest (PoDI).

Compliance Assessment Program

The purpose of the CAP is to help provide reasonable assurance that Federal-aid highway projects comply with key Federal requirements. The CAP assesses a statistically valid sample of projects to inform the FHWA, with an acceptable level of certainty, of the degree of compliance. The approach is objective, defensible, and informs the development of Corporate and Unit risk assessments with statistically valid information and data. The CAP is one element of project stewardship and oversight, an implementation of FHWA's data-driven and risk-based stewardship and oversight.  The program spans a three year cycle that includes levels for both a national and a Division assessment.

CAP Implementation

Project Population

The CAP program will be implemented each year for a pre-determined population of projects. To ensure consistency, the project population will be drawn from the FHWA Financial Management Information System (FMIS) for each Division. The population could change from year to year depending on whether it is a national or Division focus as described below.

Sample Size

A project sample size will be determined from the project population using the established statistical criteria shown below. These criteria have been selected in consultation with FHWA leadership and in consideration of the tradeoff between statistical accuracy and available resources. All criteria fall within acceptable ranges for government program assessment, industry, and academia.

  • 90% Confidence Level - the amount of tolerable uncertainty
  • 10% Margin of Error - the amount of tolerable error
  • 50% Response Distribution - the expected response to the question

Randomly Sampled Projects

Projects are randomly sampled from the project population in an amount equal to the sample size using randomizing software.

Substitution of Randomly Sampled Projects

If a randomly sampled project is not a good candidate for a CAP review, then the Division may request a randomly-sampled substitute project from the Program Management Improvement (PMI) Team. In order to preserve the statistical integrity of the process, substitute projects are for rare circumstances and the decision is not based only on perceived low risk. Examples of projects that could require a substitution include:

  • Projects that should not have been in the project population; for example, projects that were mistakenly authorized for construction.
  • Projects that were initially authorized for construction but will not advance; for example, projects that a State has pulled from letting or will not award.
  • Projects that are currently in litigation or under scrutiny by other oversight agencies and where additional project review could interfere with those proceedings.

Standard CAP Guides

  1. Core Question Guide

    The Core Question Guide includes compliance questions for key regulatory requirements. The guide is updated annually and questions may change depending upon historical compliance trends or emerging emphasis areas. All Division offices will use the Core Question Guide for all randomly sampled projects each year.

  2. Corporate Review Guide

    Division offices will use a Corporate Review Guide when it is determined necessary to conduct a national review of current or potential risks or key emphasis areas. The review topics and questions will depend upon the risks or issues. Instructions for the Corporate Review Guide outline national requirements and will describe application to the sample projects.

  3. Technical Review Guides

    Program and technical specialists developed the Technical Review Guides. These are provided to the Division for optional use in the CAP Program. Divisions are encouraged to take advantage of the statistical sample of projects and strategically use the Technical Guides to assess known risks or validate perceived risks.

CAP Review Reporting

Every response requires a comment to assist with supervisory and quality assurance review.

Retain source documentation or its availability one year after the close of the performance year.

Divisions will conduct CAP reviews for projects that have been randomly sampled from the applicable project population. Divisions may conduct these reviews in any order that they determine appropriate and as aligned with available resources.

Divisions will directly validate the answers to each CAP question with source documentation. Every response requires a comment to assist with supervisory and quality assurance reviews. It is not necessary to retain copies of source documents in individual project files; however, make source documentation readily available at either the Division or State DOT office(s) for verification or quality assurance reviews. Retain source documentation or its availability one year after the close of the performance year.

Enter responses and information from the Core Question Guide and the Corporate Review Guide (as applicable) into the CAP SharePoint Site (internal FHWA site) as reviews are completed. Results are evaluated and summarized each year with results packages issued to FHWA and Division leadership. Use results to inform strategic and unit planning, corporate and unit risk assessment, and stewardship and oversight initiatives. Similarly, evaluate the results from the Technical Guides to inform unit planning, risk assessment, and stewardship and oversight initiatives.

CAP Review Cycle

The CAP project populations will be adjusted on a 3-year cycle for a National assessment in the first year and more targeted Division assessments in the two subsequent years. This 3-year cycle allows time between National assessments to implement response strategies for compliance findings, and gives flexibility to Divisions to focus on specific areas of concern. It also allows the Corporate Review Guide to be used on the National population during the first year or on a more targeted population during the subsequent years.

3-Year Cycle (typical)

  • Year 1: National Assessment (PY18)
  • Year 2: Division or Corporate Assessment (PY19)
  • Year 3: Division Assessment (PY20)

National Assessment

The National assessment validates compliance with core regulatory requirements and assesses national emphasis areas as necessary.

Project Population:

The project population for the national assessment includes all projects authorized for construction or advance construction within the last 12 months and is pulled from FMIS on January 1 of each year. Table 1 shows the CAP sample size with respect to project population using the established criteria.

Division-level Assessment

The Division-level assessment is conducted in Years 2 and Year 3 to validate compliance with core regulatory requirements and to focus on specific Division areas of concern. The Division will have the option to filter its project population to provide this focus. If there is a corporate concern in a particular risk area, a Division might be required to sample a specific project population as part of a corporate assessment. For example, in PY2016 Divisions with locally-administered projects were required to sample their population of locally administered projects and use a specific Review Guide. Other Divisions, not subject to the corporate concern, would have the flexibility to filter their project population at their discretion.

Project Population:

The project population for the Division-level assessment will include all projects authorized for construction or advance construction within the last 12 months and will generally be pulled from FMIS on January 1 of each year. Table 1 shows the CAP sample size with respect to project population using the established criteria.

Filtered Population (optional):

Unless there is a specific FHWA concern that requires a specific population, Divisions have the option to filter the project population using any combination of these standard FMIS fields:

  • Project location - population targeted to projects in specific counties
  • Total cost - population filtered for projects in a specific range of total cost
  • State-administered projects - population filtered for projects that are State administered
  • Locally-administered projects - population filtered for projects that are locally administered
  • Project improvement type 1 - population filtered for projects with specific improvement types
  • Demo projects - population filtered for projects that are earmarked
  • Project type - population filtered for projects with specific project types
  • Functional system classification - population filtered for projects with specific functional classifications

Divisions that elect to filter their population will coordinate with the PMI Team to identify the filtered population, sample size, and randomly sampled projects.

1 There are some limitations to using the project improvement type field because of the FMIS data structure.

  • Confidence Level: 90%
  • Margin of Error: 10%
  • Response Distribution: 50%
Table 1: CAP Sample Size for Project Population Range
Project Population Sample Size
1 to 30 census
31 to 53 30
54 to 56 31
57 to 59 32
60 to 63 33
64 to 67 34
68 to 71 35
72 to 75 36
76 to 80 37
81 to 85 38
86 to 90 39
91 to 96 40
97 to 102 41
103 to 109 42
110 to 116 43
117 to 124 44
125 to 132 45
133 to 141 46
142 to 151 47
152 to 162 48
163 to 175 49
176 to 188 50
189 to 204 51
205 to 221 52
222 to 241 53
242 to 263 54
264 to 289 55
290 to 320 56
321 to 357 57
358 to 400 58
401 to 454 59
455 to 523 60
524 to 612 61
613 to 732 62
733 to 905 63
906 to 1172 64
1173 to 1641 65
1642 to 2684 66
2685 to 6991 67
6992 to up 68
Updated: 06/27/2017
Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000