United States Department of Transportation - Federal Highway AdministrationSkip to contentFHWA HomeFeedback
Catastrophic Hurricane Evacuation Plan Evaluation: A Report to Congress
 Table of Contents | Previous | Next

Chapter 3: Evacuation Plan Assessment Methodology

This chapter describes the U.S. DOT's study methods used to evaluate the local, State, and Federal plans for evacuations related to hurricanes or other catastrophic incidents in the Gulf Coast region. The U.S. DOT, in coordination with the DHS, collected and evaluated information from State and local plans; held discussions with State and local emergency management, transportation, and law enforcement officials; conducted a review of literature and research on mass evacuations; and vetted concepts with representatives of transportation associations. The U.S. DOT also evaluated its own response to Hurricanes Katrina and Rita and incorporated these findings with the results of the State and local assessments to develop lessons learned, best practices, and findings and recommendations for this report.

Section 10204 of the SAFETEA-LU and the FY 2006 DOT Appropriations Act included specific factors that were a part of this evaluation, such as the potential modes of transportation in evacuations, coordination with neighboring jurisdictions, and communications before and during evacuations. Figure 3-1 lists the eight factors identified in the legislation. The U.S. DOT added other factors to develop a comprehensive view of mass evacuation planning in the region. These factors are discussed in more detail below.

The objective of the evaluation was to assess the status of evacuation planning processes in the Gulf Coast region to determine how well they address the various aspects of evacuation planning and operations, the constraints that State and local jurisdictions face in preparing and implementing evacuation plans, and actions that could be taken to improve evacuation capabilities. The purpose of the evaluation was not to grade the individual local and State plans, but rather to assess and gather information from those plans that will help local, State, and Federal agencies improve their evacuation processes in the future.

The study was led by a U.S. DOT team of representatives from the Office of the Secretary (OST), FHWA, FTA, FRA, FAA, and FMCSA, with assistance from a consultant team. The U.S. DOT also sought comments on the criteria from representatives of a number of its key partners and stakeholder agencies and associations. These included AASHTO, APTA, the American Trucking Associations, American Association of Railroads (AAR), American Bus Association (ABA), Amtrak and the National Council on Disability.

Scope of the Plan Evaluation

Hurricane evacuation plans are prepared at various levels of government including State, county, and municipal, and may be prepared by a variety of agencies ranging from a State DOT, to a State emergency management agency, to a municipal public works agency, or even to a State highway patrol agency. For the evaluation of the adequacy of State and local evacuation plans, the U.S. DOT collected and assessed the plans from various agencies within a State, as well as neighboring jurisdictions that were involved, and assessed how the agencies coordinated their plans across geographic and political boundaries.

Figure 3-2: States Included in Assessment

This image contains a map of the southern U.S. that highlights Gulf Coast States (Arkansas, Georgia, Oklahoma, and Tennessee) and states that are Potential Evacuation Destinations (Alabama, Florida, Louisiana, Mississippi, and Texas).

The review included the State and local jurisdictions in the Gulf Coast region that were involved in major hurricane evacuations (i.e., the States of Alabama, Florida, Louisiana, Mississippi, and Texas). The State and local evaluation reviewed plans for moving people away from areas threatened by catastrophic incidents and for accepting evacuees from other areas, including emergency evacuation plans that were available from State, county, and parish emergency management agencies. The U.S. DOT reviewed evacuation plans for coastal counties and the adjacent counties along the Gulf Coast and many of the counties of Florida. In addition, the evaluation included an assessment of sheltering plans for the States of Arkansas, Georgia, Oklahoma, and Tennessee, because these states may be among those to receive large numbers of evacuees from a catastrophic hurricane. Figure 3-2 shows the States included in the assessments.

The U.S. DOT requested copies of the evacuation plans through three means:

  1. The DHS Nationwide Plan Review
  2. The Secretary of Transportation's letter to the Governors (see Appendix A)
  3. Direct contact with State and local emergency managers and transportation officials.

The U.S. DOT also consulted with the State emergency management agencies to validate that the plans provided included all relevant plans available in each State. While the U.S. DOT did not receive evacuation plans from every jurisdiction in time for the study, the Department collected and assessed evacuation plans from 63 jurisdictions in the Gulf Coast region. This included plans from each of the 5 Gulf Coast States and 58 plans from county and parish jurisdictions from Florida and the coastal counties of Alabama, Louisiana, Mississippi, and Texas. The set of collected plans provided a representative base from which to assess the overall adequacy of evacuation planning for those areas in the Gulf Coast region with the greatest threat of catastrophic hurricanes. Appendix B lists the jurisdictions and agencies included in the assessment of the 5 Gulf Coast States.

Coordination with DHS Study

The U.S. DOT and DHS worked together to coordinate the U.S. DOT study of mass evacuation planning for the Gulf Coast region and the DHS nationwide review of catastrophic emergency operations plans. The U.S. DOT and DHS coordinated the methodologies, criteria, and assessment tools for each study. The U.S. DOT and DHS study teams coordinated site visits and discussed feedback and results and findings and recommendations from the site visits.

Although the U.S. DOT and DHS studies are related, the scope and focus of the two are different. The DHS study assessed the overall emergency operations plans for catastrophic incidents for States, territories, and large urban areas nationwide. The U.S. DOT study focused on mass evacuation planning elements for States, counties, and parishes in the Gulf Coast region. Evacuation plans are generally a subset of or an annex to overall emergency operations plans.


To meet the Congressional requirements for this study and assess the range of factors included in effectively planning and executing a mass evacuation, the U.S. DOT developed a systematic, analytical process to evaluate the readiness and adequacy of State and local jurisdictions to move people in the Gulf Coast region away from catastrophic incidents and to safe shelter. The U.S. DOT's methodology included identifying the major components of a comprehensive evacuation planning and implementation program, collecting current practices and information on evacuations, developing criteria to evaluate current plans, conducting on-site discussions with State and local emergency management officials, assessing the plans, identifying lessons learned and best practices from recent evacuations, and developing findings and recommendations to improve mass evacuation planning in the Gulf Coast region and nationwide. Figure 3-3 highlights the study process.

Identify Key Elements of Evacuation Planning and Implementation

The body of the report is been organized by the key elements related to mass evacuation planning and operations for catastrophic incidents. These elements are major categories of activities or functions that local, State, and Federal emergency management agencies would conduct to plan and execute a mass evacuation. Figure 3-4 shows the seven key elements of mass evacuation planning and operations.

Review Current Evacuation Practices

To establish the current framework that guides evacuation planning and implementation, the U.S. DOT collected and reviewed guidance, literature and research, cost data, and other information on evacuations.

Develop Plan Evaluation Criteria

The U.S. DOT developed evaluation criteria that focus on local and State actions necessary to plan for and implement a mass evacuation. These actions include written policy directives; coordination of planning processes; and provisions made to communicate information to evacuees before, during, and after evacuation. The criteria were used for the detailed review of individual written evacuation planning documents from the jurisdictions in the Gulf Coast region. The criteria were also reviewed and validated during the discussions with the State and local officials during the site visits.

The U.S. DOT developed the evaluation criteria from information gathered from current Federal guidance on evacuation planning and operations as well as recent questions raised after the evacuations for the catastrophic hurricanes in 2005. Specifically, questions used to evaluate plans were drawn from:

Table 3-1 lists the 20 general questions developed by the U.S. DOT to evaluate evacuation plans for this study. For each of the questions, a list of attributes (stated as questions) was developed to define and support the parameters of each question. For example, within the category of communicating evacuation considerations, one of the questions is: “Does the plan describe the provisions and methods for alerting citizens that evacuation may be necessary?” Within this general question, several more specific questions were used to evaluate the plan (e.g., Does the plan identify contingency plans to use if normal means of public communications are unavailable? Does the plan include provisions for communicating with special needs evacuees such as hearing, vision, and physically impaired?). Appendix E provides the detailed list of questions used to evaluate State and local evacuation plans.

Each attribute was also characterized into two distinct categories—essential and non-essential. Essential means that if the attribute were not addressed in the plan, executing key aspects of the evacuation would at risk. Non-essential means that failure to address the attribute could impair execution of the plan but would not be expected to cause key aspects of an evacuation to fail. These two categories were used to weigh individual attributes and prioritize activities..

Table 3-1: General Evaluation Questions
Plan Evaluation Questions for Mass Evacuation from Catastrophic Incidents
Decision Making and Management
D1 Does the plan describe direction and control with respect to catastrophic evacuation?
D2 Does the plan describe the provisions needed to execute a large-scale evacuation?
P1 Does the plan address evacuation planning considerations (e.g., decision making, communications, available transportation modes, special needs, and sheltering) with regard to catastrophic hurricanes and other catastrophic events?
P2 Does the plan require organizations to prepare standard operating procedures that contain the detailed instructions that responsible individuals need to follow to accomplish assigned tasks?
P3 Does the plan include provisions for returning evacuees to their homes?
Public Communication and Preparedness
C1 Does the plan describe the provisions and methods for alerting citizens that evacuation may be necessary?
C2 Does the plan identify what will be done to keep evacuees informed during evacuation to reduce their level of mental and physical stress?
C3 Does the plan describe the means the government will use to keep evacuees and the public informed on the specific actions they should take after the evacuation has started?
Evacuation of People with Special Needs
N1 Does the plan describe provisions for evacuating special needs populations including those in assisted living facilities, hospitals, and those living independently (e.g., people with physical, mental, cognitive, and developmental disabilities)?
N2 Does the plan describe provisions for evacuating other special needs populations (e.g., people in schools, day centers, mobile home parks, prisons, and detention centers, as well as, people that do not speak English or who are tourists, seasonal workers, or homeless)?
O1 Does the plan describe the safe and practical transportation modes that will be available to move evacuees that cannot transport themselves (other than special needs populations)?
O2 Does the plan include provisions to ensure availability of public and private transport modes and necessary transport operators?
O3 Does the plan identify evacuation routes?
O4 Does the plan address the use of contraflow measures?
Sheltering Considerations
S1 Does the plan require the establishment of mutual aid agreements with other jurisdictions to formalize access to and use of shelters?
S2 Does the plan include provisions for informing shelter operators and evacuees about the locations of public shelters outside of the evacuation area and their status (e.g., full, accepting evacuees, accepting evacuees with pets, special needs shelters, etc.)?
S3 Does the plan address strategies and responsibilities for shelter operations?
S4 Does the plan include provisions for the care and protection of animals?
Training and Exercises
T1 Does the plan include provisions for training a volunteer cadre to support shelter management operations, transport of evacuees, and first aid stations along the evacuation routes, etc.?
T2 Does the plan require periodic reviews and updates of the plan, exercises and/or drills, and after-action reports as part of the planning process?

Evaluation of Written Plans

Evacuation planning and operations is a complex process that involves many emergency operation functions. Evacuation plans are generally included in a "family of plans." This means that evacuation plans are often components of the all-hazard emergency response plans developed by State and local governments. Requirements relating to evacuation planning are generally addressed in several parts of the plan, rather than only in an evacuation section. Evacuation plans are included in the basic plan, direction and control, evacuation, emergency public information, resource management, and mass care or the ESF annexes in the emergency response plan. In some jurisdictions, stand-alone hurricane evacuation plans have been prepared. Whether a single plan or a family of plans, these are the primary documents that governments rely on to detail what they will do in catastrophic incidents that warrant mass evacuations.

For this study, the U.S. DOT assembled and assessed these families of plans relating to evacuations for the Gulf Coast jurisdictions to determine how they address the evacuation planning and operational questions developed for this evaluation. The study also assessed how the jurisdictions' plans coordinated with their neighboring jurisdictions and then how all of the jurisdictions in the Gulf Coast region coordinate together to prepare for catastrophic hurricanes. This included assessment of the plans of the neighboring “host” states (i.e., Arkansas, Georgia, Oklahoma, and Tennessee) for sheltering evacuees from the Gulf Coast region.

Figure 3-5 illustrates how the study methodology starts with the assessment of individual plans and then assesses and compares the family of plans that include the individual plan. The evaluation then assesses the coordination with neighboring jurisdictions.

Figure 3-5: Study Methodology for Assessment of Individual Plans

Click for alternative text.

The individual plan assessments included three steps:

Step 1: The first step is a detailed assessment of how a given plan addresses individual attributes for each of the 20 general evaluation questions:

Step 2: The second step is to score the degree to which the plan fulfills each general evaluation question.

The weighted scores for all attributes are added and divided by the total of their weights, to determine the average weighted score for each question.

Step 3: The final step is to roll-up the average weighted scores from all of the plans for each general question. Based on the cumulative score, each question is then given a rating, based on the levels described in Table 3-2.

Table 3-2: Evaluation Ratings
Rank Cumulative Score Description Representation
4 1.51 – 2.00 Very Effective Full circle
3 1.01 – 1.50 Effective Three-quarter circle
2 0.51 – 1.00 Partially Effective Half circle
1 0.00 – 0.50 Marginally Effective One quarter circle

Conduct Site Visits

A key step in the U.S. DOT's study methodology was the on-site discussions with key State and local emergency management, transportation, law enforcement, and non-governmental organization officials. In these meetings, the U.S. DOT discussed and validated the initial results from the evaluation of the written evacuation plans. Written State and local plans did not contain all of the information needed to assess key issues pertaining to evacuation plans. For example, plans do not usually include the cost of implementing the plans, constraints on what can be included in the plans, and recommendations for actions to improve State and local evacuation capabilities beyond the scope of the plan.

The U.S. DOT conducted one visit in each State, and coordinated whenever possible with the DHS staff conducting the on-site peer reviews for Phase II of the DHS Nationwide Plan Review. The U.S. DOT was also able to participate in some of the DHS meetings, and its objectives for each meeting included:

The on-site meetings were a valuable source of additional information and provided a more comprehensive picture of the status of mass evacuation planning and operations in the region. The attendees at the State and local meetings were experts in evacuation planning and operations. Many had years of first-hand experience managing evacuations under very difficult conditions. Their knowledge, expertise, and experience were invaluable to this study. The U.S. DOT prepared a summary of each State visit, and a group of separate reports documents the constraints, recommendations, and best practices. Appendix D includes a list of the State and local agencies that participated in the on-site meetings.

Identify Lessons Learned and Best Practices

From the literature review, the U.S. DOT identified a number of lessons learned and best practices. These lessons learned and best practices were discussed with the participants in the State visits and coordinated with the partners and stakeholders to help identify the highest priority issues. Lessons learned identify gaps, mistakes, or problems encountered that can or should be fixed. A best practice is an effective practice that can be replicated in other jurisdictions. It is typically based on an actual event, so that it has been proven effective in the field.

After-action reports are prepared after drills and exercises and major events. The reports summarize the events and identify what did and did not perform efficiently and effectively. Emergency responders use this analysis of their capabilities to continually improve their plans and processes. The U.S. DOT evaluated a number of after-action reports from the 2005 hurricane season to glean lessons learned that would have broad applicability for jurisdictions in the Gulf Coast region.

Exercises test emergency response capabilities and the performance and skills of the agencies and individuals. Sometimes, exercises and drills are designed to severely strain the current capabilities of the emergency response agencies to the point of failure to help identify potential gaps and weaknesses and ways to improve plans, operations, and training. These types of challenging exercises provide valuable lessons learned. Likewise, actual events provide insights into the capabilities and needs of emergency responders.

The identified lessons learned and corrective actions will be shared with the DHS Lessons Learned Information Sharing portal, and Corrective Action Program associated with the DHS National Exercise Program, to ensure that they influence ongoing Federal, State, and local government preparedness programs and planning activities.

In addition, the U.S. DOT's study benefited from several other recent governmental reviews on emergency response to Hurricane Katrina. This included the Final Report of the Select Bipartisan Committee to Investigate the Preparation for and Response to Hurricane Katrina by the U.S. House of Representatives; The Federal Response to Hurricane Katrina: Lessons Learned from the Assistant to the President for Homeland Security and Counterterrorism; the Governors Task Force on Evacuation, Transportation, and Logistics in Texas; and several reports by the GAO. These studies identified a number of lessons learned that were useful for this study.

 Table of Contents | Previous | Next
FHWA Home | Feedback