FHWA Research and Technology Evaluation Program Summary Report Spring 2016

FHWA R&T and the Evaluation Program

The Federal Government has the responsibility to fund and conduct research and technology (R&T) activities of national interest that will lead to solutions to highway transportation issues and significantly advance technology innovation with a clear public benefit when private investment is neither present nor sufficient.⁽¹⁾ Research results— when implemented appropriately—can save millions of dollars, save lives, extend the life of highway infrastructure, reduce congestion, improve travel time, increase productivity, and positively impact the environment. FHWA’s R&T program has developed a research agenda to:⁽⁴⁾

• Communicate the agenda to external customers, stakeholders, and within FHWA.
• Provide means for input and comment across the entire FHWA R&T portfolio.
• Open opportunities for greater R&D coordination and collaboration.
• Support system performance measures.
• Guide the investment of FHWA resources.

The FHWA R&T agenda considers future transportation needs from two perspectives: national-level challenges, and research programs designed to meet those challenges. FHWA research targets six of the Nation’s high-priority highway challenges:⁽¹²⁾

1. Advancing Safety Toward Zero Deaths—a highway system free of fatalities.
2. Improving the Mobility of People and Goods—moving people and goods reliably and safely, to where they need to go.
3. Maintaining Infrastructure Integrity—keeping pavements, bridges, and structures in good condition.
4. Enhancing System Performance—decreasing highway congestion, safety risks, and wear-and-tear on roadways.
5. Promoting Environmental Sustainability—improving public health, enhancing the environment, and conserving natural resources.
6. Preparing for the Future—transforming big ideas into the innovations of tomorrow.

As the Table 1 shows, four of the eight research components take place under the guidance of the Turner-Fairbank Highway Research Center (TFHRC) in Virginia.

Modified from FHWA. “Strategic Plan for the Turner-Fairbank Highway Research Center” ⁽³⁾

Furthering these research components and meeting the Nation’s high-priority highway challenges will require the coopera­tion and collaboration of numerous stakeholders in the public and private sectors, academia, industry, and the international community.⁽¹⁾ It will also require continually reexamining and improving the process of selecting and executing research projects, disseminating findings, supporting user adoption, and assessing impact.

The Role of Evaluation in FHWA R&T

In 2003, FHWA leadership adopted a strategic management framework called the Corporate Master Plan (CMP) for Research and Deployment of Technology & Innovation developed with input from stakeholders.⁽²⁾ The purpose of the plan is to continue to improve the effectiveness and efficiency of R&T, including the end goal of deploying and implementing technologies and innovations that improve the quality, cost-effectiveness, and timeliness of products, procedures, processes, and practices. It lays out guiding principles, commitments, and key actions to strengthen FHWA roles as innovator and leader in national highway R&T. FHWA leadership pursued this strategic management framework to guide the organization as public demands for safety and efficiency grow faster than available resources.

The CMP is a response to Highway Research: Systematic Selection and Evaluation Processes Needed for Research Program, a report by the General Accounting Office (GAO) to Congress, which recommended that FHWA “develop a systematic process for evaluating significant ongoing and completed research that incorporates peer review or other best practices in use at Federal agencies that conduct research.”⁽¹³⁾ Performance evaluation and measurement form a major element of the CMP, both in the guiding principles and the FHWA R&T framework for applied and advanced research, as shown in figure 1.

Figure 1. FHWA R&T framework for applied and advanced research.

Emphasis Added. Source: FHWA, “Corporate Master Plan for Research and Deployment of Technology & Innovation.”⁽²⁾

Guiding Principle #6 is particularly relevant; it states:

Guiding Principle #6—FHWA measures the performance of R&T on the Agency, program, and project levels.

The Agency commits to:

• Developing, defining, and adopting a framework for measuring performance.
• Using merit review for conducting research evaluations and measuring performance.⁽²⁾

TFHRC leadership has adopted a strategic plan to guide the Center’s continued development. The plan sets forth broad principles and long-term goals to be planned and executed over several years. One of the long-term goals is to calibrate the Center’s efforts to promote the adoption of research findings:

Goal 5 of the Strategic Plan: Research activities and outcomes are appropriately advanced through effective alignment of resources, dissemination of knowledge, and technology transition.⁽³⁾

From a highway research perspective, technology transition is the incorporation of technology into operating transportation systems to achieve increased performance (safety, capacity, speed, energy efficiency, and emissions reductions) and/or to reduce costs. The plan recognizes that the dissemination of knowledge and technology transition require engaging external partners and targeting outreach.⁽³⁾ Evaluation can help in the search for effective alignment by revealing the relationships between outreach, dissemination, transition, and increased performance. These relationships are the focus of the FHWA R&T Evaluation Program.

The FHWA R&T Evaluation Program

The FHWA R&T Evaluation Program has been designed to further the transparency, accessibility, and responsiveness of R&T at TFHRC for stakeholders. Although designed to achieve long-term benefits, stakeholders may request that R&T programs show near-term benefits as well. Governmental R&T programs have the added obligation to justify spending public funds. Many highway research and technology stakeholders have come to recognize that the current, decentralized system for planning, conducting, sharing, and evaluating highway research and technology development is not fully meeting the collective needs of the public and national priorities.⁽¹⁴⁾

To support a more coordinated research agenda, FHWA and the Volpe Center have organized, clarified, and communicated FHWA’s R&T mission, vision, goals, and priorities; and past, current, and planned projects. By sharing this information, FHWA hopes to encourage highway researchers to identify and fill research gaps, to reduce unnecessary duplication of research efforts, to stimulate collaborative research efforts, and to accelerate innovation.

Additionally, FHWA and Volpe have initiated an evaluation of FHWA’s R&T program to guide the further development of the FHWA R&T Agenda and to identify and communicate its full range of benefits to the public. The FHWA R&T Evaluation Program seeks to answer these fundamental questions:

• Are we doing the right kinds of research?
• What portfolio mix best supports our strategic objectives?
• What kinds and extent of risk should we take on?
• Have the research results been deployed?
• Is the research having the desired impact?
• What is the public getting for the funds we spend?

In its initial year, the FHWA R&T Evaluation Program worked with 9 FHWA offices to identify 16 projects for evaluation across all program areas. The evaluations represent a mix of retrospective and prospective studies and range in schedule from 6 months to 4 years or more. The table below shows the 16 projects selected for evaluation, arranged by FHWA R&T research component, the type of evaluation—either prospective or retrospective—and whether the project belongs to the first or second wave of start dates.

Evaluation Type: [R] = Retrospective; [P] = Prospective; — = No Data

Program Status and Annual Report

Each evaluation progresses through an ordered series of deliverables, including the task management plan, preliminary evaluation plan, draft evaluation plan, final evaluation plan, draft report, and final report. Some evaluations—especially those spanning several years—also involve a data acceptability memo, periodic data reports, interim technical memos, and interim briefings. Six evaluation teams have completed data collection and finalized a report, and one other has submitted a final phase I report and is in the process of preparing its phase II report. Three evaluation teams have submitted their final evaluation plans (with one recently delivering an interim tech memo) while five other teams have submitted their draft evaluation plans. Of the remaining two, one team has submitted a preliminary evaluation plan and the other is preparing to initiate planning activities. Table 3 shows the status of each evaluation by the most recently completed deliverable.

A full inventory of completed and planned deliverables is available in the appendixes. FHWA R&T Evaluation Program deliverables have been presented at Research and Technology Coordinating Committee (RTCC) meetings, program reviews, and other venues.

The six evaluation teams that have submitted draft reports all relied on document reviews and interviews. Half of these teams (GRS-IBS, Roadside Revegetation, and NHTS) also analyzed FHWA Web page usage statistics. Additionally, the Roadside Revegetation report draws from the results of a survey conducted by the evaluation team.

This annual report provides FHWA’s Office of R&T with an overview of the entire evaluation effort to date and specific findings for each evaluation. The report provides more detail about evaluations closer to completion. Initial findings are available for each of the wave 1 retrospective evaluations. Proposed evaluation areas, methodology, and schedule are available for each of the wave 1 prospective evaluations. Program descriptions, initial scoping ideas, and anticipated schedules are available for each of the wave 2 evaluations. The evaluation summaries are provided in alphabetical order by wave and research design (retrospective and prospective).