Based on the case studies presented, the peer exchange described the typical characteristics of expert panels and the ways they are utilized.
As noted earlier, expert panels are utilized with some variation on the standard Delphi Method, whereby a diverse group of experts is convened and charged with generating a specific set of forecasts. Members of the group, each provided with equal access to information, work independently within a structured process to develop analyses in response to the panel's charge.
Characteristics
These individual analyses are then shared with the rest of the group, usually with each individual's anonymity preserved, and all are given an opportunity to ask questions of the various analyses. Panel members then separate again to respond to the questions and repeat the process. This iterative structure continues until members reach agreement on the forecasts they were charged with developing. Based on the cases reviewed during the peer exchange, it is unusual for a panel to go through more than two rounds of question-and-response.
The peer exchange participants observed that panel members are selected for their knowledge of the local market and land use context. Efforts are also made up front to secure the impartiality of panel members. To maximize its credibility, the panel must be perceived has having no current, direct stake in the outcomes that the panel's forecasts will shape.
Regarding the expert panel procedures, one defining characteristic is that the process is highly visible. This contributes to the panel's credibility and to public understanding of the forecasting effort. There is variation in the degree of visibility. In some cases, analysis of information is anonymous and private, whereas in other cases it may be open (in public meetings).
Panels typically report to an oversight committee, which, in the cases presented during the peer exchange, included the appropriate MPO. The MPO in the cases reviewed during the exchange did not initiate or otherwise manage the work of the expert panel. The exchange participants also noted that key stakeholders typically contribute to the composition and procedural design of the expert panel, which helps ensure that those stakeholders will find the panel's results credible.
In the cases reviewed during the peer exchange, the expert panel's work typically began with an initial presentation on regional transportation and land use estimates and how they were derived. Panels typically started from a regional control number, and thus focused on redistribution. Moreover, because MPO forecasts are based on some 'build' alternative, expert panels typically do not include in their comparative forecasts any true 'no build' alternative.
Participants also observed that, in some of the cases reviewed, results generated by the expert panels were used as part of secondary and cumulative environmental impact analysis.
Expert panels represent an alternative to formal land use models and can yield more effective and widely accepted results.
They are likely to have more credibility with stakeholders than models, in part because members of the community affected are represented on the expert panel. Thus expert panels will be an especially attractive method where detailed knowledge of local conditions is required.
Benefits
Expert panels also provide a highly visible response to issues of land use, which is important, as these issues are invariably sensitive because of their consequences for the local community. Where local community values are in conflict, a relatively public and visible method for deriving impact forecasts is much more likely to be accepted and trusted than is a technical method such as a formal model.
Maryland turned to the use of expert panels for analysis of land use impacts of corridor projects after the assumptions underlying land use models-and the forecasts they produced-were questioned by stakeholders. Forecasts subsequently prepared by expert panels have been widely accepted by stakeholders. Panels can also be more flexible and adaptable than land use models. Experts can often account for planning and policy contexts in ways more subtle and complex than models.
Participants in the peer exchange also noted one other increasingly significant benefit of expert panels, derived from their use to generate secondary and cumulative environmental impact analysis: no better method has yet been identified for generating the products needed to meet NEPA requirements. Because NEPA requires public involvement, expert panels are a strong alternative to land use models or other methods.
Another benefit of expert panels is that they can prove a cost-effective alternative to land use models, especially for smaller MPOs for corridor studies. The costs of data, staff time, technical expertise and technology associated with models can be prohibitive for smaller organizations. Moreover, expert panels can also produce results more quickly than models, if the panels are composed, organized and managed appropriately. Expert panels also represent an attractive method for smaller metro areas, not only because of cost issues, but because the analytical demands should be reduced compared to a larger metro area. Panel members should be able to process the requisite amount of data and prepare meaningful forecasts for different scenarios.
One of the most important products of the peer exchange was the identification of issues that may arise regarding the appropriate and effective use of expert panels to forecast the impacts of transportation projects.
One cluster of issues centers on the relationship of the expert panel to other organizations and authorities, especially the relevant planning agency, such as the MPO or DOT. As a primary source of information, the role of the planning agency is likely to be closely scrutinized and it is best if it is not seen as driving the process. The same is true for whatever agency or agencies would be responsible for implementing the transportation project in question. In cases presented at the peer exchange, planning agencies typically served on panels that oversaw the expert panels and helped with selection of panel members. Participants also observed that panel sponsors should be cautious about facilitating the panel's work. A third party, such as a consultant specializing in facilitation, may be preferable.
Issues-Relationship to Other Organizations
Interestingly, the peer exchange concluded that the strengths of an expert panel-visibility, involvement of multiple experts-also create risk for the implementing agency, because the strengths correlate to reduced control over the forecast outcomes.
A second set of issues centers on how the expert panel is defined. This includes a question as fundamental as panel size. In the cases reviewed, panels ranged in size from as few as six (Washington DOT I-5 project) to more than 40 (Longview, TX). For any given context, the panel sponsor must consider the tradeoffs associated with varying panel sizes and the related information and staffing demands. Another fundamental issue observed during the peer exchange is member motivation. In some cases, members are provided a nominal stipend for their participation, whereas in others no compensation of any kind has been provided. Panel sponsors should determine whether some incentives are appropriate or necessary to ensure full and complete participation by the desired range of panel members.
Issues-Panel Definition
Who determines the panel ground rules-and when-can be an issue when using this method. An expert panel must be given clear and detailed direction and kept on task, but it must also be left with some autonomy and flexibility if it is to function effectively and deliver credible results.
Perhaps most important with regard to panel definition is the nature of demands placed upon it. There may be variation in how quantitative the panel process is. In some cases, panels have been asked to provide rather detailed, quantitative forecasts, whereas in others that may have been asked only to provide ranges or fully qualitative analysis. Sponsors should anticipate the use to which the expert panel's product will be put and gauge how quantitative it is accordingly. The nature of the demands should also influence how the group itself is comprised. Similarly, sponsors should consider whether to ask their panels to prepare forecasts for multimodal alternatives. Maryland's experience with the I-270/U.S. 15 corridor indicates that, while a panel may complete its work, the complexity of the task may stress the panel's capacity to complete its work effectively and on time.
A third set of issues focused on the process by which the expert panel does its work. One significant issue is how open the deliberative process will be. In some cases, the expert panel reviewed its individuals' analyses in a meeting open to the general public. This can lead, the peer exchange participants observed, to panel members being influenced by factors other than the analyses before them.
Issues-Panel Work Process
An issue that generated a good deal of discussion during the peer exchange is whether the panel is charged with achieving consensus on its objectives or simply some definition of consent. Consensus, by definition, requires unanimity, whereas consent can be satisfied by any number of decision rules. Given that one defining strength of the expert panel method is the credibility it carries, how the group conveys its conclusions can be very important. At the same time, the panel and its sponsors must be realistic. It may not be possible to achieve consensus on all items.
A final process issue is how to keep the panel focused on its charge. In some cases discussed during the peer exchange, panel members wanted to or found themselves moving from forecasting outcomes to prescribing mitigation measures. Maintaining a firm separation between those two efforts is an important responsibility of the panel facilitator.
The final cluster of issues identified by the peer exchange participants centered on the analytical decisions of the expert panel. Participants observed that panels may face a number of decisions that will affect both how they do their work and what they produce. Anticipating these questions will help panel sponsors make their own decisions about panel composition, charge, and process. Specific questions noted during the exchange include the following:
Issues-Analytical Decisions
Beyond these questions, any expert panel may also encounter questions regarding its analytical leeway. For example, how far can a panel go in questioning the credibility of the sources of data provided to it? Once these questions have been raised, what impact does this have on the panel's product and how it is used? Similarly, what latitude will a panel be given, if any, to modify existing regional control numbers? In the Maryland 301 corridor study, the panel questioned the validity of the land use forecasts baseline provided to it, and incorporated its questions into the analyses generated by the panel.
Based on the cases presented and the subsequent group discussion, participants of the peer exchange formulated a set of lessons learned about the appropriate and effective use of expert panels to analyze transportation and land use alternatives. These lessons may be organized broadly into considerations of panel design and management.
Panel member selection is a critical consideration. Every effort should be made to select members who understand local real estate, land use, and political contexts. It is also essential to screen out members who have preconceived notions or a current conflict of interest. The Maryland 32 corridor study illustrates the difficulties that may arise if some members enter a panel and demonstrate a lack of objectivity in the process. Panel selection is also important as one aspect of the overall process design. Key decision makers and stakeholders must participate in or accept that design before the panel begins its work. Otherwise, their support of the panel's products cannot be ensured. Also, once the process is established, maintaining continuity throughout the life of the panel is important, as it supports perceptions of stability and credibility.
Lessons-Panel Design
One of the defining strengths of the expert panel method is its credibility, which derives in part from the fact that it entails some measure of public involvement and is more visible than formal models or other methods. Panel sponsors should ensure that the process is visible to stakeholders and the public. While the panel members should prepare their analyses independently and those analyses should remain anonymous, the panel's group work should be accessible to interested parties. In the cases reviewed during the peer exchange, this has often meant panel meetings that are open to the public.
Ensuring that adequate technical support is available, especially during the panel's group deliberations, will also strengthen panel design. Panel members are likely to generate questions that require further information or clarification; staff with the expertise to address those questions should be available.
In general, the peer exchange participants emphasized that attending to small details of panel design significantly increases the chances of success. Details may include having a full range of potentially relevant data and information at hand, logistics of scheduling and meeting locations, establishing ground rules for how meetings will be conducted, and more. Anticipating and planning for these details will help minimize unexpected issues, and make those that do occur more manageable.
Finally, the facilitator or panel sponsor should ask members for feedback that will help improve the design of future expert panels.
Panel facilitation is an important consideration. The facilitator must be neutral with regard to the issues under consideration by the panel. Thus, it is certainly best if the implementing agency (where this is applicable) does not handle the facilitation duties. A third party, separate from agencies that provide data or have some role in implementation, is ideal. The facilitator should have very strong interpersonal skills, to keep active groups on task, energize groups that may be flagging, mediate where disputes arise, and so on.
Lessons-Panel Management
The panel must be given a clear, precise charge when it begins work. It must be provided with specific questions and given direction as to the form in which those questions should be answered. This may be particularly important where the group's product will be largely qualitative. When giving the panel its charge, the facilitator should also emphasize (where this is applicable) that the panel's job is not to prescribe, but to estimate or forecast. Otherwise, panel members can easily, if unconsciously, move to specifying their preferences for how their community will evolve.
Panel sponsors and the facilitator can also take several steps to make the panel's work as easy as possible to complete. This includes managing the presentation of information (as in the Longview cases) so that it is accessible to a wide audience of lay people. The tasks themselves can be structured to be manageable. This may mean breaking a large objective into smaller tasks, or moving from quantitative to qualitative objectives. The Maryland 32 case is instructive on this point.
Finally, participants in the exchange noted that one often-overlooked aspect of expert panels is dealing with the press. Precisely because the process involves the community, is relatively visible, and addresses sensitive issues, it is likely to attract the news media's attention. Panel sponsors should provide members with guidelines, and perhaps some training, on how to respond to media inquiries. Sponsors should also anticipate media interest and plan the overall panel process accordingly.
