- Research article
- Open Access
Using exercises to improve public health preparedness in Asia, the Middle East and Africa
BMC Research Notesvolume 7, Article number: 474 (2014)
Exercises are increasingly common tools used by the health sector and other sectors to evaluate their preparedness to respond to public health threats. Exercises provide an opportunity for multiple sectors to practice, test and evaluate their response to all types of public health emergencies. The information from these exercises can be used to refine and improve preparedness plans. There is a growing body of literature about the use of exercises among local, state and federal public health agencies in the United States. There is much less information about the use of exercises among public health agencies in other countries and the use of exercises that involve multiple countries.
We developed and conducted 12 exercises (four sub-national, five national, three sub-regional) from August 2006 through December 2008. These 12 exercises included 558 participants (average 47) and 137 observers (average 11) from 14 countries. Participants consistently rated the overall quality of the exercises as very good or excellent. They rated the exercises lowest on their ability to identifying key gaps in performance. The vast majority of participants noted that they would use the information they gained at the exercise to improve their organization’s preparedness to respond to an influenza pandemic. Participants felt the exercises were particularly good at raising awareness and understanding about public health threats, assisting in evaluating plans and identifying priorities for improvement, and building relationships that strengthen preparedness and response across sectors and across countries. Participants left the exercises with specific ideas about the most important actions that they should engage in after the exercise such as improved planning coordination across sectors and countries and better training of health workers and response personnel.
These experiences suggest that exercises can be a valuable, low-burden tool to improve emergency preparedness and response in countries around the world. They also demonstrate that countries can work together to develop and conduct successful exercises designed to improve regional preparedness to public health threats. The development of standardized evaluation methods for exercises may be an additional tool to help focus the actions to be taken as a result of the exercise and to improve future exercises. Exercises show great promise as tools to improve public health preparedness across sectors and countries.
Since 2001, there has been a dramatic increase in the use of disaster preparedness exercises among public health agencies in the United States[1, 2]. These exercises have explored a wide range of topics from foodborne toxoplasmosis outbreaks, chemical disasters, acute blood shortages, bioterrorism, and severe acute respiratory syndrome (SARS). Exercises have been designed to assess and improve a variety of capabilities such as regional disaster preparedness among rural hospitals, knowledge and confidence of legal authorities, resource allocation and risk communications. A large number of these exercises have been focused on the spread of infectious diseases especially the threat of pandemic influenza because the common challenges pandemic influenza shares with other types of public health emergencies[12–16].
Our knowledge of the use of exercises for public health-related disaster preparedness outside the United States is much more limited. A considerable number of these types of exercises in the United States have been published in the academic literature but few findings from exercises that have taken place outside the United States have been published. Some researchers in the United States have tried to solve this gap by publishing the findings of “virtual” internet-based, long distance exercises conducted remotely with international partners[17, 18]. Even less is known from direct experiences with in-country exercises or exercises that span multiple countries in a given region. The results of these types of exercises may be reported directly to exercise participants but often don’t make it to the scientific literature. If the results from exercises are published in any systematic way, they often get published in in-house publications for domestic audiences rather than scientific journals with a more global reach. The incentives, financial or otherwise, for researchers to turn these in-house publications into scientific papers are limited. This is a major loss to our knowledge base because countries around the world are increasingly recognizing the importance of transnational efforts to complement national efforts to detect and respond to public health threats quickly and effectively[19–23]. Exercises provide these countries with a vehicle to collaborate and test their ability to respond to these transnational threats. Exercises also provide these countries with an avenue to build relationships and trust among colleagues across sectors and across borders.
We developed pandemic influenza tabletop exercises that built on the “Day After” methodology developed by Millot, Molander and Wilson and described elsewhere in greater detail. Countries in three different geographic regions participated: Southeast Asia (Cambodia, China, Lao PDR, Myanmar, Thailand and Vietnam), the Middle East (Israel, Jordan and Palestine) and East Africa (Burundi, Kenya, Rwanda, Tanzania and Uganda). Countries that participated in the exercises were included because they were all part of sub-regional disease surveillance networks established in part through funding from the Rockefeller Foundation. Some countries not included in these networks were invited to observe the exercises. Countries ranged in their past experience with preparedness exercises with some countries having extensive past exercise experience (such as Israel, Vietnam, China and Thailand) and other countries having minimal past exercise experience (such as Cambodia, Lao PDR, Myanmar and Uganda). Exercises were developed and conducted by exercise planning teams that included external exercise development experts from the RAND Corporation as well as senior health leaders from each of the respective localities and/or countries represented in the exercise. There were three different levels of exercises: sub-national (e.g., one or more provincial areas), national (e.g., one country) and sub-regional (e.g., multiple countries from one geographic region).
All exercises were multi-sectorial in nature meaning that they involved representatives from more than one sector of government. Examples of sectors included were health, agriculture, defense and environment. Each exercise focused on three to six different broad topic areas such as surveillance and information sharing, disease control, and communications that were identified in previous exercises as important[2, 12]. Because Thailand had considerable previous experience with exercises, it designed and conducted its sub-national exercise with limited involvement from representatives at RAND. Exercise discussions focused on one topic area at a time each lasting from 30 to 90 minutes. Participants of exercises were selected by the exercise planning team and differed from exercise to exercise, but all exercises included representatives from the health sector of the locality and/or country represented. They also included senior leaders from at least one other non-health sector. In addition to participants, exercises also had “observers” who were invited to watch the exercise but did not directly engage in exercise discussions. All exercises were led by one or two exercise “facilitators” who directed the exercise discussion and probed participants for more information. In general, exercise facilitators can represent a range of disciplines from media professionals to health professionals. In these exercises the facilitators were all health officials or health researchers who were trained in the facilitation of exercises and who had extensive experience facilitating past exercises.
Exercises presented participants with a future scenario that involved an unfolding pandemic influenza crisis at different stages. They were required to respond to the scenario with the actions they would take if the scenario were actually occurring. Exercise facilitators were given discussion points and probes to keep the discussion focused and moving forward. Each section of the exercise ended with participants being asked to make concrete decisions for the topic area being discussed before moving on. The exercise concluded with a debriefing in which all participants evaluated their own response in light of what they learned during the exercise.
All exercise participants were asked to complete an evaluation form immediately after the exercise, before leaving the room. Typically, they spent about 15 minutes to respond to these questions. Six of the exercises were rated for their quality through five Likert scale questions: the overall quality of the exercise (1 = poor; 5 = excellent); the quality of the exercise discussions (1 = poor; 5 = excellent); the exercise identified important key gaps in preparedness (1 = strongly disagree; 5 = strongly agree); the exercise helped participants to better understand the roles and responsibilities of agencies and organizations responding to an influenza pandemic (1 = strongly disagree; 5 = strongly agree); and the exercise generated information that participants planned to use (1 = strongly disagree; 5 = strongly agree).
The remaining six exercises asked participants three different qualitative questions: what was the importance of the exercise; what are the most important actions that should be taken based on the exercise; and what suggestions do you have to help improve future exercises. In addition to participant evaluations, detailed After Action Reports (AARs) were developed for each exercise that summarized the exercise discussions and highlighted key aspects of each exercise. In January 2013, health leaders who were involved in the planning of the exercises from a subset of countries participated in brief semi-structured face-to-face interviews to discuss how their country followed up with the exercises and the current state of their exercise program. Health leaders included health officials working for the ministry of health in their respective countries. These health leaders were all directors of departments (such as communicable disease) within their ministry.
Results and discussion
We developed and conducted 12 exercises from August 2006 through December 2008: four sub-national exercises, five national exercises, and three sub-regional exercises (Table 1). Across all of these exercises there were a total of 558 participants and 137 observers from 14 countries. The average number of participants per exercise was 47 and the average number of observers was 11. Participants from the health sector were represented in every exercise. The most commonly represented sectors other than health were agriculture and defense. Four exercises were shorter than one full 8-hour day in length, three exercises were one full day in length, and five exercises were more than one full day in length. The average length of the exercises was 9.75 hours. All exercises covered three to six of the topic areas outlined in Table 2.
Table 3 highlights the participant evaluation from six exercises that used questionnaires with Likert Scale questions. Participants who completed these evaluation forms consistently rated the overall quality of the exercises as high (88-100% rating the exercise as good or excellent) with one exception (Middle East sub-regional exercise 59% rating the exercise as good or excellent). Participants also consistently rated the exercises highly for helping them to understand the roles and responsibilities of organizations and agencies responding to an influenza pandemic (91-94% rating the exercise as good or excellent in this area) with one exception (China sub-regional exercise 76% rated the exercise as good or excellent in this area). Participants differed on what they felt about the quality of the information shared in the exercises (67%-93% rated the information as good or excellent). Participants rated the exercises lowest on their ability to identify key gaps in performance (50%-73% agreeing or strongly agreeing the exercises identified important key gaps). There are a variety of factors that could explain this result. First, tabletop exercises are discussion based and do not directly test operational capabilities. This may limit the ability of tabletop exercises to concretely identify key operational gaps. Second, tabletop exercises are subjective and participants may disagree about what constitutes a key gap. Third, the limited time frame to conduct tabletop exercises and the limited number of topics that can be discussed in that time frame may inhibit the ability of these exercises to identify a significant number of key gaps. A fourth possibility is that cultural sensitivities in some of these countries may have limited participants’ comfort in identifying gaps in their government’s preparedness system. The exercises were most successful at helping participants gain knowledge that they planned to use to improve the preparedness of their organization (82%-100% agreeing or strongly agreeing that they would use what they learned from the exercise).
Table 4 summarizes the qualitative feedback provided by participants in their evaluation forms. Three general themes came out of the participant comments on the most useful aspects of the exercises: the ability of exercises to raise awareness and understanding about public health threats, the ability of the exercises to assist in evaluating plans and identifying priorities for improvement and the ability of the exercises to build relationships and enhance preparedness and response capabilities across sectors and across countries in a geographic region. Participants also left the exercises with specific ideas about the most important follow-up actions that they should take in the near future. Specifically, participants identified better planning, improved planning coordination across sectors and countries and better training of health workers and response personnel. Finally participants provided feedback on the use of tabletop exercises for pandemic influenza preparedness. No participants stated that they felt the exercises involved too many sectors. In fact, many participants reported that they felt more sectors should be involved and that exercises should also involve more private sector partners and more partners from NGOs. Participants also felt that more could be done in the exercises to ground theoretical responses with more practical responses.
Some health leaders who were part of exercise planning teams participated in semi-structured face-to-face interviews in January 2013. Countries that reported having pre-existing exercise programs prior to participating in the exercises described here were much more likely to report conducting exercises at regular intervals over time compared to countries that did not report a pre-existing exercise program. Most countries reported modifying and using some or all of the exercise template materials that were developed for the exercises described here. However, one country that had no prior exercise experience organized and carried out numerous sub-national exercises on their own after participating in the national and sub-regional exercise. Health leaders in this country reported that participating in an exercise helped to motivate them to develop an exercise program and regularly assess different aspects of their public health preparedness. The largest barriers to continued exercising that were reported included lack of financial resources and limited support among leadership to develop and sustain an exercise program.
These experiences suggest that exercises can be a valuable, low-burden tool to improve emergency preparedness and response in countries around the world. They also demonstrate that countries can work together to develop and conduct successful exercises designed to improve regional preparedness to public health threats. Regular participation in exercises is associated with improved overall response to public health threats. Countries that participated in sub-regional exercises together reported that these exercises improved their response to the 2009 H1N1 pandemic[24, 27]. But exercises are not perfect. Research has called into question the ability of exercises to adequately expose operational and logistical gaps. This is consistent with our finding that exercise participants rated the exercises lowest for identifying key gaps. In addition, there is a lack of consensus on what makes exercises effective tools to assess public health preparedness and how the outputs of exercises such as AARs can be used to support and improve public health preparedness efforts. Thus, the development of standardized evaluation methods for exercises may be an additional tool to help focus the actions to be taken as a result of the exercise and to improve future exercises. Despite these flaws, exercises show great promise as tools to build relationships, assess performance and improve collaborative planning for public health threats across multiple sectors and multiple countries over time.
After action report
African Union/International Bureau for Animal Resources
Food and Agriculture Organization
Severe acute respiratory syndrome
United Nations Development Program
United Nations Population Fund
United Nations Children’s Fund
United Nations Office for the Coordination for Humanitarian Affairs
United Nations Relief Works Agency
United States Agency for International Development.
Dausey DJ, Lurie N, Buehler J: Designing and Conducting Tabletop Exercises to Assess Public Health Preparedness for Man Made and Naturally Occurring Biological Threats. BMC Public Health. 2007, 7: 92-10.1186/1471-2458-7-92.
Dausey DJ, Lurie N, Diamond A, Meade B, Molander R, Ricci K, Stoto M, Wasserman J: Tests to Evaluate Public Health Disease Reporting Systems in Local Public Health Agencies. 2005, Santa Monica, CA: RAND Corporation
Morris JG, Greenspan A, Howell K, Gargano LM, Mitchell J, Jones JL, Potter M, Isakov A, Woods C, Hughes JM: Southeastern Center for Emerging Biologic Threats tabletop exercise: foodborne toxoplasmosis outbreak on college campuses. Biosecur Bioterror. 2012, 10 (1): 89-97. 10.1089/bsp.2011.0040.
High EH, Lovelace KA, Gansneder BM, Strack RW, Callahan B, Benson P: Promoting community preparedness: lessons learned from the implementation of a chemical disaster tabletop exercise. Health Promot Pract. 2010, 11 (3): 310-19. 10.1177/1524839908325063.
Galloway MJ, Jane G, Sudlow L, Trattles J, Watson J: A tabletop exercise to assess a hospital emergency blood management contingency plan in a simulated acute blood shortage. Transfus Med. 2008, 18 (5): 302-7. 10.1111/j.1365-3148.2008.00884.x.
Abraham RT, Walls RT, Fischer M, Markovic-Reed S, Solovieva TI, Russell FK, Ducatman AM: Tabletop scenarios for realism in bioterrorism and threat preparedness. W V Med J. 2012, 108 (6): 12-7.
Sarpy SA, Warren CR, Kaplan S, Bradley J, Howe R: Simulating public health response to a severe acute respiratory syndrome (SARS) event: a comprehensive and systematic approach to designing, implementing, and evaluating a tabletop exercise. J Public Health Manag Pract. 2005, Suppl: S75-S82.
Edwards JC, Kang J, Silenas R: Promoting regional disaster preparedness among rural hospitals. J Rural Health. 2008, 24 (3): 321-5. 10.1111/j.1748-0361.2008.00176.x.
Savoia E, Biddinger PD, Fox P, Levin DE, Stone L, Stoto MA: Impact of tabletop exercises on participants' knowledge of and confidence in legal authorities for infectious disease emergencies. Disaster Med Public Health Prep. 2009, 3 (2): 104-10. 10.1097/DMP.0b013e3181a539bc.
Barnett DJ, Taylor HA, Hodge JG, Links JM: Resource allocation on the frontlines of public health preparedness and response: report of a summit on legal and ethical issues. Public Health Rep. 2009, 124 (2): 295-303.
Freimuth VS, Hilyard KM, Barge JK, Sokler LA: Action, not talk: a simulation of risk communication during the first hours of a pandemic. Health Promot Pract. 2008, 4 (Suppl): 35S-44S.
Dausey DJ, Aledort J, Lurie N: Tabletop Exercises for Pandemic Influenza Preparedness in Local Public Health Agencies. 2006, US Department of Health and Human Services Office of the Assistant Secretary for Preparedness and Response: Washington, DC
Macario E, Heyden L, Nakahara N, Macias-Reynolds V: Preparing for pandemic influenza: California confronts the legal implications. Health Promot Pract. 2009, 10 (4): 573-8. 10.1177/1524839907308118.
Araz OM, Jehn M: Lant T. 2010, A New Method of Exercising Pandemic Preparedness Through an Interactive Simulation and Visualization. J Med Syst: Fowler JW
Otto JL, Lipnick RJ, Sanchez JL, DeFraites RF, Barnett DJ: Preparing military installations for pandemic influenza through tabletop exercises. Mil Med. 2010, 175 (1): 7-13. 10.7205/MILMED-D-09-00118.
Lurie N, Dausey DJ, Knighton T, Moore M, Zakowsky S, Deyton L: Community Planning for Pandemic Influenza: Lessons from the VA Health Care System. Disaster Med Public Health Prep. 2008, 2: 251-257. 10.1097/DMP.0b013e31817dd143.
Valesky W, Silverberg M, Gillett B, Roblin P, Adelaine J, Wallis LA, Smith W, Arquilla B: Assessment of hospital disaster preparedness for the 2010 FIFA World Cup using an internet-based, long-distance tabletop drill. Prehosp Disaster Med. 2011, 26 (3): 192-195. 10.1017/S1049023X11006443.
Gillett B, Silverberg M, Roblin P, Adelaine J, Valesky W, Arquilla B: Computer-facilitated assessment of disaster preparedness for remote hospitals in a long-distance, virtual tabletop drill model. Prehosp Disaster Med. 2011, 26 (3): 230-3. 10.1017/S1049023X11006455.
Kimball AM, Moore M, French HM, Arima Y, Ungchusak K, Wibulpolprasert S, Taylor T, Touch S, Leventhal A: Regional infectious disease surveillance networks and their potential to facilitate the implementation of the international health regulations. Med Clin North Am. 2008, 92 (6): 1459-71. 10.1016/j.mcna.2008.06.001.
Andrews RM, Kearns T, Connors C, Parker C, Carville K, Currie BJ, Carapetis JR: A regional initiative to reduce skin infections amongst aboriginal children living in remote communities of the Northern Territory, Australia. PLoS Negl Trop Dis. 2009, 3 (11): e554-10.1371/journal.pntd.0000554.
Birtwhistle R, Keshavjee K, Lambert-Lanning A, Godwin M, Greiver M, Manca D, Lagacé C: Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med. 2009, 22 (4): 412-22. 10.3122/jabfm.2009.04.090081.
Cohen D, Gargouri N, Ramlawi A, Abdeen Z, Belbesi A, Al Hijawi B, Haddadin A, Sheikh Ali S, Al Shuaibi N, Bassal R, Yishai R, Green MS, Leventhal A: A Middle East subregional laboratory-based surveillance network on foodborne diseases established by Jordan, Israel, and the Palestinian Authority. Epidemiol Infect. 2010, 138 (10): 1443-8. 10.1017/S0950268810000270.
Moore M, Dausey DJ, Phommasack B, Tok S, Guoping L, Nyein SL, Ungchusak K, Vung ND, Oo MK: A model for sustaining sub-regional disease surveillance networking: examination of the mekong basin disease surveillance cooperation. Global Health Governance. 2012, 5 (2): 1-43.
Gresham L, Ramlawi A, Briski J, Richardson M, Taylor T: Trust across borders: responding to 2009 H1N1 influenza in the Middle East. Biosecur Bioterror. 2009, 7 (4): 399-404. 10.1089/bsp.2009.0034.
Millot MD, Molander RC, Wilson P: “The Day After…” Study: Nuclear Proliferation in the Post-Cold War World, Volume I. 1993, Summary Report. Santa Monica: RAND Corporation
Agboola F, McCarthy T, Biddinger PD: Impact of emergency preparedness exercise on performance. J Public Health Manag Pract. 2013, 19 (Suppl 2): S77-83.
Moore M, Dausey DJ: Response to the 2009-H1N1 influenza pandemic in the Mekong Basin: surveys of country health leaders. BMC Res Notes. 2011, 4: 361-10.1186/1756-0500-4-361.
Klima DA, Seiler SH, Peterson JB, Christmas AB, Green JM, Fleming G, Thomason MH, Sing RF: Full-scale regional exercises: closing the gaps in disaster preparedness. J Trauma Acute Care Surg. 2012, 73 (3): 592-7. 10.1097/TA.0b013e318265cbb2.
Savoia E, Preston J, Biddinger PD: A consensus process on the use of exercises and after action reports to assess and improve public health emergency preparedness and response. Prehosp Disaster Med. 2013, 28 (3): 305-8. 10.1017/S1049023X13000289.
We would like to express our thanks to all of the health leaders in Southeast Asia, the Middle East and East Africa that helped us design and conduct these exercises. We would also like to acknowledge the support of the Rockefeller Foundation (through grant 2008 DSN 302), the Nuclear Threat Initiative and the US Centers for Disease Control and Prevention.
The authors declare that they have no competing interests.
DJD and MM designed and conducted the study. DJD led the writing and analysis. MM contributed substantially to both the analysis and writing. Both authors read and approved the final manuscript.