Skip to contentUnited States Department of Transportation - Federal Highway Administration FHWA Home
Research Home
Report
This report is an archived publication and may contain dated technical, contact, and link information
Publication Number: FHWA-HRT-05-040
Date: March 2005

Office of Research, Development, and Technology FY 2004 Performance Report

IV. RD&T Performance Management

FHWA's R&T program supports the mission of the Agency to enhance mobility through innovation, leadership and public policy. It contributes to achievement of the USDOT and Agency strategic goals and is stakeholder driven. Stakeholders are engaged throughout the entire R&T process from agenda setting, through the conduct of research, technology and innovation deployment, implementation, and customer feedback.

In measuring performance, the Office of RD&T uses the Agency Corporate Management Strategies (CMS), based on the Baldrige Criteria, as a framework for improving internal business processes and meeting the needs of our customers. The Baldrige Criteria consist of seven functional categories vital to organizational performance: leadership, strategic planning, customer and market focus, information and analysis, human resource focus, process management, and business results. The U.S. Commerce Department's National Institute of Standards and Technology (NIST) manages the Baldrige National Quality Program in close cooperation with the private sector. Congress established the Baldrige Award program in 1987.

In addition to the CMS framework, RD&T utilizes the following criteria established by the OMB for Federal investment in R&D:

  • Relevance. RD&T program managers must be able to articulate why an investment is important, relevant, and appropriate. Our research activities, products, and services outlined in this plan are designed to support the Agency's goals and address customer needs.
  • Quality. RD&T program managers must justify how funds will be allocated to ensure quality research. Programs allocating funds through means other than a competitive, merit-based process must justify these exceptions and document how quality will be maintained. Quality is also assessed periodically through independent lab assessments.
  • Performance. RD&T program managers must be able to monitor and document how well this investment is performing. Program managers track R&T projects to determine whether the projects are on time and within budget, and assess whether to increase or redirect funding. Customer feedback mechanisms and retrospective benefit studies also provide important information on performance.

Performance Management Framework

Our approach to performance management builds on our planning process. The RD&T Performance Management Framework chart below identifies existing performance measures and assessment mechanisms that are utilized by unit managers and integrates them across management functions using the Agency's Baldrige-based CMS. This approach helps us manage, analyze, and integrate information obtained from a variety of sources and mechanisms. It also helps keep in mind the various dimensions for analyzing program results, such as financial performance, customer feedback, and business results.

The R&D investment criteria are designed to help improve R&D program management, inform funding decisions, and increase public understanding of the potential benefits of investment in Federal research.

 

RD&T Performance Management Framework
CMS Definition Related RD&T Performance Measures Methodology
Leadership Leadership focuses on how senior leaders guide the organization. It describes how leaders set direction and high-performance expectations, project a strong customer focus, and communicate clear and visible values to employees.
  • Leadership Effectiveness Inventory (LEI) results.
  • Action items completed.
  • Performance plan items fulfilled.
  • Self-assessment score.
  • 360-degree feedback.
  • Action agenda.
  • Performance plans.
  • Quality self-assessments.
Strategic Planning Strategic planning examines how the organization sets strategic goals and develops key action plans.
  • Action items completed.
  • Self-assessment score.
  • Progress made on goals established.
  • Performance plans and action agenda.
  • Quality self assessment.
  • Lab assessments.
Customer/ Partner Focus Customer and partner focus examines how the organization determines customer and market requirements and expectations.
  • Percent of satisfaction with RD&T products and services.
  • Number of technology facilitation plans in place.
  • Self-assessment score.
  • Lab assessment results. RD&T customer survey results.
  • Customer satisfaction.
  • Technology Facilitation Action Plan (TFAP).
  • Quality self-assessments.
  • Lab assessments.
  • Customer surveys.
Information and Analysis Information and analysis examines the management, effective use, and analysis of data and information to support key organization processes, to include the organization's objectives.
  • Performance measurement framework.
  • Response level and content of feedback mechanism.
  • Self-assessment score.
  • Lab assessment results.
  • Performance measurement framework.
  • Quality self-assessments.
  • Lab assessments.
Human Resource Development Human resource development and management examines how the organization enables its workforce to develop to its full potential and how the workforce is aligned with the organization's objectives.
  • Self-assessment score.
  • Percent employee satisfaction survey rating.
  • Percent payroll spent on training and development.
  • Number of Individual Development Plans (IDPs) in place and in Learning and Development Systems (LADS).
  • Number of "priority 1" training needs met.
  • Number of vacancies filled.
  • Number of days that positions are vacant.
  • Quality self-assessments.
  • Employee satisfaction survey.
  • LADS.
Process Management Process management examines aspects of how key production, delivery, and support processes are designed, managed, and improved.
  • Number of process improvements documented.
  • Lab assessments.
  • Number of contracts on time and on budget.
  • Quality self-assessments.
  • Lab assessments.
  • Project tracking system.
Business Results Business results show the organization's performance and improvement in its key business areas: customer satisfaction, financial and marketplace performance, human resources, supplier and partner performance, and operational performance. The category examines how the organization performs relative to competitors.
  • Percent of project completion.
  • Number of success stories.
  • Research benefits.
  • Implementation of Corporate Master Plan.
  • Track project and services delivery.
  • RD&T success stories.
  • RD&T performance report.

Performance Management Workgroup

The primary purpose of the Performance Management Workgroup is to help the RD&T leadership develop, coordinate, and implement organizational performance improvement efforts. The workgroup also helps RD&T leadership improve the management of research programs and assists in the adoption of effective practices. Performance management efforts currently underway include:

  • Developing performance goals, measures, strategies, and initiatives to improve organizational performance management in RD&T offices and laboratories.
  • Monitoring results of measures and analyzing performance management information.
  • Coordinating the annual self-assessment process and supporting the RD&T Leadership Council in implementing solutions to issues raised during the process.
  • Contributing to special projects related to RD&T performance management.
  • Providing support for agencywide performance improvement initiatives.
  • Reviewing in retrospect whether investments were well directed, efficient, productive, and essential for validating program design and instilling confidence that future investments will be appropriate. Retrospective RD&T benefit studies are conducted periodically to document the benefits of research products.

RD&T Research Benefits Case Studies

RD&T periodically contracts out retrospective benefits studies to collect data on the benefits of RD&T research, glean insight into characteristics of successful R&T projects, and identify potentially useful methodologies for documenting and/or estimating benefits that could be used to evaluate other RD&T projects. RD&T contracted out evaluations of HSIS, QuickZone, and Infrastructure R&D with the understanding that there is no single approach to the issue of research performance measurement and benefits assessment. A synthesis report on these three studies was completed in FY 2004. All three studies indicated that RD&T products are widely used and well regarded by our customers. Products were especially well received when customers were involved in design and testing. Benefits of research conducted in the areas of high performance steel, ground penetrating radar, and geotechnical R&D, just to name a few, have helped transform the state-of-practice and saved millions of dollars in infrastructure construction and maintenance costs. Cost savings at a national level have been estimated to be more than ten times the annual research funding.

Customer Feedback

To determine whether our research products are meeting the needs of our customers, RD&T seeks and obtains customer feedback through such mechanisms as formal surveys, Web links, and focus group meetings. A Customer Survey Workgroup is helping the Leadership Council identify existing mechanisms for receiving customer feedback and to develop and implement an RD&T customer survey.

The questions in the FHWA's State and local partner survey that relate to deploying technology and innovation provide us with important information/customer feedback. In particular, the R&T Leadership Team has adapted a question, which asks how satisfied customers are with the deployment of technolo-gies/innovations that are designed to help improve their program. The question serves as an overall indicator of how well we are meeting customer needs.

Respondents in the FHWA State Customer and Metropolitan Planning Organization (MPO) Partner Survey Wave III and IV indicated that communications on T&I were good overall. Respondents mentioned that FHWA contacts in the T&I arena were knowledgeable, provide information from other States, and share lessons learned. Areas for improvement included the need for more timely and advanced notice of new T&I and the need for guidelines on how to implement them.

RD&T Lab Assessment Process

Beginning in FY 2004, FHWA established a full-scale lab assessment process. The process has been refined over the course of the year and is working well. RD&T laboratory assessments are unbiased, independent expert reviews of the technical and scientific merit of research conducted at TFHRC. The assessment process is designed to:

  • Enhance laboratory performance and quality by providing feedback to laboratory managers.
  • Provide an opportunity to exchange views among technical experts.
  • Create a feedback method that will increase the opportunities for customer and stakeholder input to research activities.
  • Provide a credible, professional, and objective assessment that further improves stakeholder confidence in the value of the work performed and outcomes produced.

A lab assessment handbook was developed to guide lab assessment panel members in examining the relevance, quality, and performance of laboratory research. Assessments focus primarily on the conduct of research, but also examine whether the research activities have high potential value and whether they have achieved stated objectives. The review process is continuous, with each laboratory having the benefit of an assessment every 3 to 4 years. Three labs were assessed in FY 2004: Asphalt, TReL, and Hydraulics.

The advice we have received from the panels has been invaluable. Knowledge of related work at universities, State DOTs, other government labs, or private industry is often shared. In addition, the experience of preparing for the assessment has been a useful exercise for FHWA staff, and we have strengthened ties to other labs and organizations as a result of on-site interactions and follow-on exchanges between panel members and TFHRC staff.

Lab assessments have also served as a valuable opportunity to obtain feedback from customers and stakeholders (who are interviewed by the panel)—and have raised the visibility of the work of the labs with these customers and stakeholders.

Previous | Table of Contents | Next

ResearchFHWA
FHWA
United States Department of Transportation - Federal Highway Administration