2. Preparing for the TRL Assessment

Researchers should provide their assessment panel with a clear understanding of the TRL Scale and how it captures the maturity of a technology at different stages of development. Next, the researchers should present panelists with a discussion of several key elements to think about as they consider using the TRL Scale to assess the research project. If TRL Assessments are the right tool for evaluating a project, then proper preparation for the assessment will help to ensure it is accurate. Preparation involves four major components: selecting panelists, establishing clear goals for the assessment, formalizing timing and location, and creating materials, as shown in figure 5.

Selecting Panelists—The Experts and User Community

Convening a well-rounded panel of experts to assess the maturity of a technology is essential to the success of a TRL Assessment. For most research projects, a panel of four to six stakeholders, researchers, and subject matter experts provides an effective balance. Panelists should at a minimum be knowledgeable about the technology being discussed, the potential users of the technology, or the application environment. Panel members must not have conflicts of interest and must not disclose information about intellectual property or sensitive information.

The panel, the Principal Investigator (PI) of the project, a notetaker, and at least one sponsoring-agency representative should attend the TRL Assessment. If possible, a neutral moderator should facilitate the discussion.

Establishing Goals—“Why are we assessing this technology?”

When conducting TRL Assessments, it is important to establish evaluation parameters for the discussion. Projects may include many different components, and the panel needs a clear focus on a distinct technology or set of components to assess the research accurately. Panel members can be involved in and assist with focusing on the technology, and all panel members should agree on the goal of the TRL Assessment before the assessment begins.

Further, defining the operating environment is key for determining whether the technology is a functional tool for the intended user community. Before the TRL Assessment, all panelists should clearly understand the purpose of the TRL Assessment and what aspects of the project they will evaluate.

Formalizing Timing and Location—The Nuts and Bolts

Timing

A TRL Assessment is most effective if held within 4 to 6 months before the end of a research project’s funding cycle. At this point, the majority of the research will have been completed and the TRL Assessment will have maximum relevance. This timing also presents an opportunity for the research team to incorporate the TRL Assessment’s findings into any final reports, complete other assessments, and find potential followup funding sources.

TRL Assessments usually last from 3 to 4 hours. However, planners can schedule longer meetings to accommodate technology demonstrations or laboratory tours or to assess complex systems where the panel may be considering the maturity of multiple components or component integration.

Location

Location options include the research site, a neutral locale, or the funding agency. Typically, a TRL Assessment requires only a small, audiovisual-capable conference room. Panel organizers may also conduct TRL Assessments via webinar or teleconference, although panelists may consider virtual settings more challenging to facilitate an active discussion.

Creating Materials—Getting to the Starting Line

A comprehensive set of materials is an essential foundation for the TRL Assessment panel’s discussion. In advance of the TRL Assessment, panel members should receive documentation of the research project so that they arrive at the TRL Assessment with all of the information they will need to fairly and fully evaluate the maturity of the research. In addition to receiving the TRL Scale, the panel should be provided with a short framing document that details the goals of the TRL Assessment and specifies the key technology, applications, and operating environments to be assessed during the TRL Assessment.

Creating this framing document for the panel often shapes the goals or aims of the TRL Assessment, and typically requires a good deal of thought and synthesis to succinctly describe the research conducted and the critical elements of the technology to be reviewed.

Documentation

Panel organizers provide the panel with—or offer them access to—technical deliverables for a project, in addition to these framing documents. Deliverables may include the research project proposal; technical reports produced by the research team; and any interim and final reports, presentations, or published papers. If possible, the project PI should complete a “self-assessment” of the project using the TRL Scale and share it with the panel in advance of the TRL Assessment. This will help the PI to better understand the types of questions that the panel will be trying to answer during the TRL Assessment.

Review

The panel should be given ample time to review the project materials provided. A minimum of 1 week should be sufficient time for panelists to review and ask any clarifying questions that they may have. Convening a brief preassessment conference call with the panelists approximately 1 week before the TRL Assessment is a good way to ensure that the panelists understand the TRL Assessment process (and to confirm any last-minute meeting logistics). This conference call also is an opportunity to ask the panelists to review the initial TRLs in advance of the panel meeting. Having these prediscussion scores makes the assessment easier and ensures that the panelists have reviewed the project documentation in advance of the meeting. (The preassessment conference call also is an opportunity for panel members to introduce themselves, their expertise, and biases relating to the technology. This also can take place at the start of the panel meeting.)