U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
PRESENTATION |
This presentation is an archived publication and may contain dated technical, contact, and link information |
|
Publication Number: N/A Date: May 2011 |
Publication Date: May 2011
|
PDF files can be viewed with the Acrobat® Reader®
FHWA's Exploratory Advanced Research (EAR) Program focuses on long–term, high–risk research with a high payoff potential. The program addresses underlying gaps faced by applied highway research programs, anticipates emerging issues with national implications, and reflects broad transportation industry goals and objectives.
Research, however, is difficult to measure. There can be significant time lags between the conduct of research and the return on the investment. Impacts can be diffuse, accruing to unexpected parties who build on the work of others.
The FHWA Exploratory Advanced Research (EAR) Program has been developing a suite of measures to monitor and improve overall Program portfolio performance and predict the potential impact of research.
The EAR Program uses different measurements of performance to provide a balanced scorecard for day-to-day program management and communication of results to internal and public stakeholders.
This poster provides information on EAR Program measures and background on the search for appropriate measurements that could suggest approaches for other transport research programs.
Financial Perspective |
||
FY 2010 Baseline |
Target |
Comment |
94 percent in FY 2010 92 percent of cumulative Program funding for research projects |
At least 80 percent of funding for research projects |
Internal and public measure of efficiency |
For FY 2010 (funds obligated), 27 percent match |
Amount of matching non-federal funds by sector |
Internal indicator of leverage, partnership Percent by solicitation rather than fiscal year more meaningful |
Measurement being refined |
Funds committed, obligated, expended |
Internal indicator of efficient management process Annual amount appears misleading using fiscal year |
Under development |
Time from budget delivery to awards |
Internal and public measure of efficiency Consider median time and max time. |
Customer Perspective |
||
FY 2010 Baseline |
Target |
Comment |
For most recent round 14 percent (but awards not complete yet) For program overall, 17 percent |
Proposals submitted and awards made (yield rate) |
Public indicator Cumulative percent assumes only full proposals (not FY 2007 BAA pre-proposals) and completion of five awards in process |
Under development |
Number of institutions and external experts involved (and the number who are involved for the first time) |
Internal and public indicator of stakeholder engagement and outreach |
Under development |
Full-time equivalent of researchers, graduate students supported |
Internal and public indicator of building capacity but difficult to measure Requires common definition for FTE across proposals |
Under development |
Percent of initial stage topics open over one year |
Internal indicator of efficient process or complex topics |
On hold till new project management system adopted |
Number of projects or funding contributing to Agency Strategic Plan by goal area |
Public indicator of potential impact. Difficult to measure without adoption of new program management support system (PMSS) Measurement by EAR Program focus areas also could be meaningful |
Internal Business Process |
||
FY 2010 Baseline |
Target |
Comment |
Integrated highway system concepts: 4 in FY 2010 and 10 cumulatively (since FY 2006) Nanoscale research: 3 and 6 Human behavior and travel choices: 4 and 15 New technology and advanced policies for energy and resource conservation: 4 and 12 Information sciences: 4 and 7 Breakthrough Concepts in material science: 5 and 8 Technology for assessing performance: 5 and 15 |
Number of initial stage investigations in each EAR Program focus area |
Internal and public indicator of program balance and breadth Based on all active projects in FY 2010 including continuing and new topics There may be some duplication with linked topics that are assigned to different focus areas |
Cumulatively about 20 percent |
Percent initial stage topics that lead to solicitations or intramural research projects |
Internal and public indicator of program risk Does not reflect some topics leading to multiple awards Difficult to assess for FY 2010 since topics still active |
Under development |
Percent of projects that demonstrate use of effective research approaches |
Retrospective indicator as determined by independent reviewers |
Under development |
Percent of awardees who gain follow-on funding from other sources as a result of EAR Program funded work |
Retrospective indicator of building capacity; a possible start towards return on investment |
Under development |
Percent of projects that lead to continued, applied research with the potential for breakthrough advances |
Retrospective indicator of risk and impact |
Innovation and Learning |
||
FY 2010 Baseline |
Target |
Comment |
5 in FY 2010 32 cumulatively |
Number of new FHWA personnel involved in initial stage research |
Internal indicator of building capacity |
10 percent in FY 2010 10 percent cumulatively |
Percent of initial stage research involving multiple offices or results that have been picked up by a different office |
Internal indicator of program breadth |
Under development |
Percent of projects that lead to adoption of new research processes or approaches |
Retrospective indicator of impact |
Under development |
Percent of projects that close persistent knowledge gaps, result in new fundamental data, or significantly change current understanding |
Retrospective indicator of impact |
Under development |
Use of merit review in each stage of process |
Internal but difficult to measure |
Under development |
Publications and presentations of project results |
Internal and public indicator but difficult to monitor |
David Kuehn
Federal Highway Administration
6300 Georgetown Pike
F-220 (HRTM-30)
McLean, Virginia 22101
Phone 202 493 3414
david.kuehn@dot.gov
Poster Presentation for the 4th International Transportation Systems Performance Measurement Conference on May 18, 2011, in Irvine, CA