Exercise Development

Exercise Development

Exercise Development

Basics of Developing a Successful Tabletop Exercise

The Texas Division of Emergency Management (TDEM), Exercise Unit developed this guidance for developing a successful tabletop exercise. The development of a tabletop exercise should follow the guidance set forth in the Federal Emergency Management Agency (FEMA) Homeland Security Exercise and Evaluation Program (HSEEP). For questions about the TDEM Exercise Program or recommendations for improvement, please contact us via e-mail at TDEM.Exercises@tdem.texas.gov.

The Basics
A tabletop exercise (TTX) is a facilitated discussion following a scripted scenario in an informal, stress-free environment. It should be based on current applicable policies, plans, and procedures. The tabletop exercise design process facilitates conceptual understanding, identifies strengths and weaknesses, and/or achieves changes in policies and procedures. These improvements are important for successful responses to disasters in the future. The success of the exercise depends largely on group participation and most importantly in the honest appraisal and identification of problem areas and the willingness to find resolution of those problems.

General Characteristics
The exercise begins with a general setting, which establishes the stage for a hypothetical situation. In your exercise, the facilitator initiates discussion by providing intelligence or situation updates. These updates describe major events that may be directed to individual players or participating departments, agencies, or organizations. Recipients of the updates then discuss the action(s) they might take in response to the situation or incident.

The facilitator utilizes key questions focused on roles (how the players would respond in a real situation), responsibilities, plans, coordination, the effect of decisions on other organizations, and similar concerns to drive the discussion. A tabletop exercise is focused on discussion of roles and responsibilities rather than simulation. Equipment and resources do not deploy during a tabletop exercise.

Application
A tabletop exercise has several important applications: low stress discussion of coordination and policy that establishes a collaborative environment for problem solving; and providing an opportunity for key agencies, organizations, and stakeholders to become acquainted with one another, realizing their interdependencies, and their respective responsibilities.

Leadership
A facilitator leads the exercise discussion, decides who gets a message or problem statement, calls on others to participate, asks questions, and guides the players toward sound decisions.

Participation
Exercise planners should choose players carefully to adequately represent their discipline, agency, or organization. Players ideally should have the authority to speak on behalf of the stakeholders they represent. Whole community participation is desired as these agencies are expected to respond during a real-world incident.

Duration
This exercise should be scheduled over at least three hours; however, the length is ultimately at your discretion. During the exercise, discussion times should be open-ended, and players encouraged to take their time in arriving at in-depth decisions without time pressure. Although the facilitator maintains an awareness of the time allocated for each area of discussion, the group does not have to complete every item in order to meet the objectives or for the exercise to be a success.

Key Steps to a Successful Exercise

The optimal time to plan a tabletop exercise is three months or twelve weeks. All recommended actions in this guide assume that you will follow this recommendation and allow plenty of time for planning before the desired exercise date. This section outlines the key actions that will be taken in the exercise planning process.

Below is a list of documents provided in your tabletop exercise:
Threat and Hazard Identification and Risk Assessment (THIRA) – helps your community understand their risks and what they need to do to address those risks by answering the following questions:

  • What threats and hazards can affect our community?
  • If they occurred, what impacts would those threats and hazards have on our community?
  • Based on those impacts, what capabilities should our community have?

Plans, policies and procedures – Review the planning documents that will be tested in the exercise. Are current plans addressing the threats and hazards identified in the THIRA?

After-Action Reports – give insight into what happened, why it happened, and how it can be done better of previous exercises or real-world incidents.

The exercise planning team (EPT) is vital to the success of any exercise. The planning team is responsible for guiding the development process, obtaining the necessary venue and resources, and should be able to achieve buy-in from their organizations and community leadership for the exercise.

It is recommended that you think carefully about who should be on the planning team and attempt to keep the total number of planning team members manageable. Six to ten members is recommended. Think about the exercise goal, what it is you want to achieve with the exercise, identify those departments and agencies that would be involved in responding to your scenario, and invite those representatives to be members.

Planning team members will be involved in the details of exercise development and therefore should not be players in the exercise. Suggestions for planning team members to consider are:

  • Community leadership and management
  • Emergency first responders (Law enforcement, fire services, emergency medical services)
  • City/County operations and maintenance (Public Works, road department)
  • Spokesperson/Public Information Officer
  • Private business/industry – Key members of your supply chain
  • Information Technology/Communications
  • State/regional/local health departments
  • State/regional/local emergency management agencies
  • State/regional/local homeland security/counterterrorism agencies
  • State/local fusion centers
  • Subject Matter Experts (SMEs) (Railroad, weather, hazmat, agriculture…)
  • Regulating agencies
  • Non-governmental institutions (Churches, school districts, universities, volunteer agencies, Amateur Radio Emergency Service)
  • Federal partners
  • Others

The Concept and Objectives (C&O) Meeting is the formal start to the exercise planning process. It helps planners determine the exercise program priorities to be addressed, design objectives based on those priorities, and identify exercise planning team members. Note: At times the C&O Meeting is combined with the Initial Planning Meeting. Expected outcomes of a C&O Meeting are:

  • Confirmation of exercise planning team members
  • Agreement regarding exercise concept (scope, type, mission area[s], priorities), exercise objectives, and aligned core capabilities
  • Exercise planning timeline, to include target exercise conduct time frame, with milestones
  • List of assigned tasks prior to the next planning meeting, to include reaching out to additional planning team members and developing detailed exercise objectives

Note: The C&O and IPM can be combined to shorten the planning timeline and be less burdensome resource-wise. Should the meetings be run concurrently, the tasks listed for both should be completed.

The IPM serves to identify exercise design requirements, assumptions and artificialities, scenario variables (e.g., time, location, hazard selection), and exercise logistics, such as exercise location, schedule, duration, participants, and other relevant details. Expected outcomes of the IPM are:
Exercise scenario framework
Clearly defined exercise objectives and aligned core capabilities
Format of exercise (see below for discussion)
Finalized exercise planning timeline with exercise conduct logistics
Confirmation of expected level of effort for all participating organizations
List of assigned tasks prior to the next planning meeting

Exercise formats for consideration:

Plenary: In a plenary format, the players organize as a single group without regard for functional area grouping (e.g., owners, management, local representatives; facility security; engineering; law enforcement). This format requires only a single facilitator, as well as one or two evaluator/data collectors; however, a co-facilitator may ease the burden of a single facilitator. This format is generally best for 25-30 players when there are a limited number of people available to fill the roles of facilitator and evaluator/data collector.

Multi-Table: Under a multi-table format, there are multiple individual tables organized by discipline, agency, organization, or functional area. First, a lead facilitator frames the scenario and poses discussion questions to all players. Group discussions occur at the individual tables, ideally facilitated by someone with functional area expertise. If feasible, it is desirable to assign both a facilitator and an evaluator/data collector to each group so that the facilitator can focus on addressing issues related to exercise objectives, while the evaluator/data collector focuses on capturing general discussion issues.

Breakout Rooms: A tabletop exercise with breakout rooms is advisable for very large exercises with many participants. It requires a large facility with a gathering room for all participants and individual rooms for group discussions. The lead facilitator frames the scenario and poses the problem statements to all participants. Groups of participants break out into assigned rooms and solve stated problems and answer questions. Each room is led by a group facilitator. Once discussion objectives are met, groups return to the assembly to report their findings. At least one or two evaluators are needed in each breakout room to capture the exercise discussion.

In this phase, members of the planning team should complete the assignments given during the first two planning meetings and continue to socialize and build support for the exercise within their own organization and community. Actions should include logistics necessary to secure a venue for the exercise date and developing player handouts or situational manuals (SitMan), facilitator guidance, and evaluation criteria with the agreed upon objectives and core capabilities.

Venue Logistics
Make sure the room is large enough to accommodate all participants and observers and is accessible to both internal and external invitees. It would be beneficial if the required space was open the evening prior to the exercise to setup and work through any technical issues. There should also be an area for the facilitator(s) and evaluator(s)/data collector(s) to meet prior to and after the exercise.

The room must also have adequate audio/video capability in order to run your presentation. A room with adjustable lights is necessary for seeing the projector screen(s) and having at least two wireless microphones to pass around the room is recommended. It is always beneficial to book a backup room at another location in case of unforeseen cancellations or other last-minute issues.

Scenario
Reach out to subject matter experts from the National Weather Service or your local meteorologist, hazardous materials specialist, state/federal law enforcement agencies if you plan a terrorism-related exercise and similar experts to help you write a realistic scenario.

Presentation
Identify a person to develop the PowerPoint presentation for the exercise. Working with SMEs, lay out the scenario and questions to be discussed.

The MPM is the opportunity to discuss exercise staffing and logistics, review the developed documents and presentation, the proposed scenario and discussion questions, and determine the exercise invitation process.

Exercise Staffing:

Facilitators. Facilitators provide situation updates and moderate discussions. They also provide additional information or resolve questions as required. Key Exercise Planning Team members may assist with facilitation as subject matter experts during the exercise. The planning team should identify a primary choice for facilitator during this planning meeting and who should be responsible for confirming whether they can attend. The planning team should also identify table facilitators if using a multi-table or breakout room format.

Evaluators/Data Collectors. Evaluators and/or data collectors are assigned to observe and document certain objectives during the exercise. Their primary role is to document player discussions, including how and if those discussions conform to plans, polices, and procedures. The planning team should identify individuals with the skill sets or subject matter expertise to fill these functions. The planning team needs to identify one or more members of the team to collect the input from the evaluators/data collectors following the exercise and put it into a draft AAR/IP.

Exercise Staff. Any exercise should have enough personnel to register participants, manage refreshments, support information technology, etc.

Discussion questions: The discussion questions provided in the SitMan or handout are suggested general subjects you may wish to address as the discussion progresses. These questions are not meant to constitute a definitive list of concerns to be addressed. You should add, delete, or modify any of the discussion questions to most effectively address the objectives of your exercise and the needs of your organization. The final questions should be based upon the objectives for the exercise and included in the SitMan.

When determining what discussion questions to include, be sure to keep in mind the time frame allotted for each module, as well as for the overall exercise. It is also recommended the planning team select half a dozen additional individual questions or sub-questions for the facilitator to address if a module is running ahead of schedule. These additional questions should be included in the facilitator and evaluator guidance in italics but should not be included in the SitMan.

Logistics: At the MPM, the planning team should confirm exercise logistics, such as estimate number of participants, exercise schedule, and venue. It is highly recommended that refreshments be provided. Depending on start and end times, that could include light snacks, breakfast, lunch, or all the above. This will depend on resources, but experience has shown that exercise participants are much more inclined to engage with exercise materials if they are not hungry. The planning team should determine at the MPM what refreshments, if any, will be provided, and who will be responsible for providing them.

Attendees:
Players. Every exercise will have players. They are personnel who discuss their regular roles and responsibilities during the exercise. They describe what their response to the scenario would be, answer questions, and interact with the facilitator and other players. Players should be chosen carefully to adequately represent their discipline, agency, or organization and must have the authority to speak on its behalf.

Observers. Observers do not generally directly participate in the exercise; however, they may ask relevant questions or provide subject matter expertise if called on by the facilitator.

The invitation should come from your organization’s management in the form of either an email or signed/scanned letter. The invitation should include the exact date, time, location, and duration of the exercise; directions to the facility; security/access requirements; and should state whether food/refreshments will be provided.

Be sure to address all staff and facility access requirements and other needs in the invitation letter. For example, the facility used for the exercise might require a “visitor request form.” In this case, you would ensure all external players fill out the form and return it to you or the appropriate office well before the exercise date. If special parking directions are required, you must include that as well. You can explain the process in words or provide a map.

This is also the phase in which all the discussion question modifications should be made. During this period, the documents should be made into as final a version as possible. These documents should be sent to the planning team for review prior to the Final Planning Meeting (FPM).

In addition to modifying the exercise documents, the planning team members should finalize any logistical details and continue to build support for the upcoming exercise. Members of the planning team should also confirm the facilitator(s), evaluators/data collectors, and exercise staff during this period.

The FPM should focus on ensuring that all elements of the exercise are ready for conduct. No major changes to the exercise’s design or scope should take place at or following the FPM. The FPM ensures that all logistical requirements have been met, outstanding issues have been identified and resolved, and exercise products are ready for printing. Be sure to review the discussion question (and back-up questions) sets in the SitMan and facilitator and evaluator guidance to confirm the modifications made earlier in the process. In summary, the following items should be addressed during the FPM:
Conduct a comprehensive, final review of all exercise documents materials
Run through the presentation slides and confirm content
Resolve any open exercise planning issues and identify last-minute concerns; and
Review all exercise logistical activities (e.g., schedule, registration, attire, special needs)

At a minimum, print one handout for each participant and facilitator and evaluator guidance for each facilitator and evaluator/data collector. It is recommended, however, that you print about twenty percent more handouts than the number of participants that you are expecting.

Exercise conduct involves activities such as preparing for exercise play, managing exercise play (presentation, facilitation, and discussion), and conducting immediate exercise wrap-up activities. Members of the exercise planning team assigned to support exercise setup should visit the exercise site at least one day prior to the event to arrange the room, test A/V equipment, and discuss administrative and logistical issues. On the day of the exercise, planning team members should arrive several hours before the start of the exercise to handle setup activities and arrange for registration.

The presentation typically starts with brief remarks by representatives from the exercise planning team or other high-profile individuals in attendance. After the opening remarks, the presentation moves into a brief introductory and explanatory phase led by the lead facilitator. During this phase, attendees will be introduced to any other facilitators, given background on the exercise process, and advised about their individual roles and responsibilities.

The facilitator generally presents the multimedia briefing, which describes the scenario and any relevant background information. The facilitator also leads the discussion, poses questions to the audience, and ensures that the schedule remains on track.

In a plenary format, players are organized as a single group, without regard for functional area grouping (management, local representatives, engineering, law enforcement). The facilitator(s) briefs the modules and moderates the questions for the entire group.

Under a multi-table or breakout room format, there are multiple individual tables/rooms organized by discipline, agency, organization, or functional area. A group facilitator first frames the scenario and poses discussion questions to all players. Group discussions occur at the individual tables/rooms, ideally facilitated by someone with subject matter expertise.

After the breakout sessions take place, the entire group typically reconvenes to address any key issues, cross-disciplinary issues, or conflicting recommendations that were identified during group discussions. A player from each group briefs the key points of their discussions to the group at large. Under all formats, players should discuss their responses based on their knowledge of current plans, procedures, and capabilities.

The lead facilitator is responsible for keeping the discussion focused on the exercise objectives and making sure all issues are explored within the time allotted. A good facilitator should possess:
The ability to keep side conversations to a minimum, keep discussions on track and within established time limits, control group dynamics and strong personalities, and speak competently and confidently about the subject without dominating conversation
Functional area expertise or experience
Awareness of appropriate plans and procedures
The ability to listen well and summarize player discussions

If feasible and/or appropriate, co-facilitators who are knowledgeable about local issues, plans, and procedures may assist the lead facilitator. Also, designating a recorder to take notes allows the facilitator to focus on key discussion issues.

Prior to the exercise, instruct the evaluators/data collectors to keep an accurate written record of what is observed. To be reliable, they should take notes as players discuss actions, make decisions, and discuss their capabilities during the exercise. Collect this information at the conclusion of the exercise as these notes will form the basis of the analysis for the AAR/IP. At the conclusion of the exercise, it is also beneficial for the after-action process to conduct a hot wash involving players. A hot wash allows players to self-assess and discuss their performance in the exercise. The hot wash also provides the evaluators/data collectors with the opportunity to clarify points or collect any missing information from the players before they leave the exercise.

To supplement the information collected during the player hot wash, the evaluation team distributes participant feedback forms to elicit responses from participants regarding the observed strengths and areas for improvement. At a minimum, the questions on this feedback form should solicit the following:
Substantive information on the most pertinent issues discussed and potential corrective actions to address these issues.
Impressions about exercise conduct and logistics.

Once the hot wash is finished, collect all the participant feedback forms. Information collected from feedback forms contribute to the issues, observations, recommendations, and corrective actions in the AAR/IP. After completing the exercise, instruct the evaluators/data collectors to consolidate the data collected during the exercise and transform it into narratives, or exercise write-ups, which address the course of exercise play, demonstrated strengths, and areas for improvement.

The end goal of the exercise is to produce an AAR/IP with recommendations for improving preparedness capabilities for your organization. The Improvement Plan will provide timelines for improvement recommendation implementation and assignment to responsible parties. This plan should be an ongoing effort by your organization and community. For your reference, there is an AAR/IP template available.

The planning team member(s) identified at the MPM to lead the after-action process should collect the notes and exercise write-ups and transform them into a draft AAR/IP. After drafting the AAR/IP, the documents should be circulated to the planning team for review and comment. Distributing these documents for review prior to the meeting helps to ensure that all attendees are familiar with the content; are prepared to discuss exercise results, identified areas for improvement and corrective actions; and have ample opportunity to comment and work toward consensus. The planning team lead on the AAR/IP should then adjudicate the comments and print copies for the planning team to review at the After-Action Review and Improvement Planning Meeting (AAR/IPM).

The AAR/IPM serves as the forum to review the draft After-Action Report and the Improvement Plan. During the meeting, participants should seek to reach final consensus on strengths and areas for improvement, as well as revise and gain consensus on draft corrective actions. Additionally, as appropriate, AAM participants should develop concrete deadlines for implementation of corrective actions and identify specific corrective action owners/assignees. It is recommended that the planning team members who drafted the AAR/IP walk through the document and encourage the planning team to discuss and finalize each item. The planning team should also discuss any sensitivities in the document and determine how the document will be distributed.

Once any final modifications to the AAR/IP determined during the AAR/IPM have been made, the document should be circulated to the planning team for final approval. The AAR/IP is then considered final, and may be distributed to exercise planners, participants, and other preparedness stakeholders as appropriate.