Scientific Papers

Assessing public health preparedness and response in the European Union- a review of regional simulation exercises and after action reviews | Globalization and Health


Recurrent recommendations from the SimEx/AAR on EU level can be roughly organised in three areas.

  1. 1)

    Improve information sharing between sectors, countries and between EU agencies and country level. This concerned both SOPs for information sharing and communication tools.

  2. 2)

    Aachieve a certain standardisation of procedures across sectors and countries on how to respond to incidents of varying severity and extent.

  3. 3)

    Improve clarity and awareness of the roles of different EU agencies in crisis management.

About a third of recommendations targeted Member States, EEA countries and Switzerland. Two thirds targeted EU agencies exclusively. They did not focus particularly on low scoring capacities as measured in the IHR SPAR tool. The most addressed IHR core capacities were C2 IHR coordination, C7 Health Emergency Management and C10 Risk communication. This is in line with the WHO review of national level SimEx and AAR globally conducted between 2016 and 2019 [9], which hypothesises that cross-cutting capacities are more likely to be in the foreground of the regional response in a real-life event. We can make the same hypothesis in this review. As some capacities lend themselves more to be trained in an exercise, recommendations will also more frequently address these. Further, the exercises included in our analysis had international, intersectional and EU agency representation, which would equally facilitate evaluation of capacities that involve coordinated actions from several entities and sectors.

SimEx and AAR have several types of added value in the in the IHRMEF. First, preparedness and response at the regional level is not covered by any other part of the IHRMEF. Secondly, it is important to distinguish between “monitoring” and “evaluation” when comparing quantitative tools (SPAR/ JEE) and qualitative tools (SimEx/AAR). Monitoring implies regular data collection to assess progress against a defined target. In the IHRMEF, the SPAR annual reporting constitutes monitoring—its regularity and completeness of reporting enables mapping of structures in place and the continuous assessment of the development of capacities over time. Evaluation assesses the overall functionality and response time. SimEx and AAR constitute context-specific evaluations of the functionality and effectiveness of the structures in place, thus complementing SPAR and JEE. And finally, SimEx and AAR identify important gaps in public health preparedness, demonstrated by comparison with challenges experienced during COVID-19.

We found that eight out of ten of the lessons learned from COVID-19 in the EU Commission’s 2021 report had already been identified in our SimEx/AAR, and action points had been proposed. This indicates on the one hand that the qualitative tools assessed here are topically relevant and pertinent and identify important gaps in the national and international response. An analysis from 2019 reviewing the effectiveness of AAR as a tool supports this finding [10] and concludes that AARs hold considerable promise as a tool to improve public health preparedness and organizational learning.

One the other hand, it reveals a problem with follow-up and implementation. For example, recommendations to EU agencies focused on improving reliability of information exchange and coherence of public communication, increasing regional emergency coordination and achieving more regional solutions to providing medical countermeasures. These challenges persisted in the COVID-19 pandemic. Both the EU Commission’s 2021 report and the ECDCs technical report [11] focus on international cooperation and coordination, and the need for more coordination of messages at EU level as areas for improvement. Joint procurement and stockpiling of medical countermeasures are the main topic addressed in lesson 6 from the commission. Another example is the Commissions lesson 1, highlighting weaknesses in the international alert and surveillance system. This echoes recommendations from five previous SimEx and AAR. As early as 2005, in the SimEx “Common Ground”, it was recommended to establish efficient on-line, real-time data input in a crisis situation, accessible for relevant bodies. Similar recommendations came in 2009, 2013, 2017 and lastly in 2018, when strengthened information sharing and surveillance between EU agencies and national and international partners were underlined in the pandemic flu SimEx “Taranis”. Shortages of healthcare staff and the need for surge capacity are important issues raised in the ECDC report and the Commission’s. This has been pointed out in earlier recommendations underlining that frontline and support staff shortage in a pandemic situation would severely hamper capability and capacity. Lastly, combating misinformation and disinformation, lesson 10 from the Commission, has been repeated in recommendations over the years raising the issue on how to counter the “anti- vaccine lobby”, monitor accuracy of public health messages, and develop a vaccination strategy during a pandemic. These similarities between exercise recommendations and COVID-19 lessons learned indicate that these gaps have persisted over a number of years and remain a challenge in the regional preparedness and response capacity. Could a better implementation of recommendations throughout the years have strengthened the early response to COVID-19 in Europe?

Indeed, one of the main challenges with SimEx and AAR is follow-up and implementation of recommendations. We did not find consistent assignment of responsibility to implement the recommendations. The exercises and resulting reports were commissioned from exercise management teams, who do not have the authority to develop action plans for implementation of the recommendations. There is frequent use of passive language, and some recommendations are too general or vague for it to be possible to assign responsibility for follow up or measure implementation progress. As a result, the output and impact of the exercises themselves are difficult to measure and likely reduced. This finding is consistent with other literature. A review of AAR reports from anthrax bioterror attacks, SARS outbreak, 2009 H1N1 pandemic, and West African Ebola epidemic reveals a similar pattern of repeated weakness and failure [12]. The phenomena are here described as “lesson observed but not lesson learned”. During the AAR/SimEx global consultation in 2019, the implementation of AAR & SimEx recommendations and findings was highlighted as an area that “remains challenging” [13]. One of the outcomes of the consultation was thus a recommendation to draft “an implementation framework integrating recommendations and action plan into operational planning that should be shared with the Member States to enhance accountability and national ownership and develop a 1–2-page strategy for the implementation of AAR & SimEx finding”. This was also one of the main conclusions in the WHO report on Covid-19 Intra Action Reviews, where one recommendation was to “identify a reliable and systematic approach to monitor AAR recommendations to ensure they progress within the proposed timeline and meet the desired outcomes” [14].

There are some initiatives underway to address the challenges associated with following up recommendations from SimEx and AAR. In 2018, the WHO issued a guideline for implementing SimEx and AAR. The guidelines include a detailed guide on how to develop recommendations which are “specific, feasible, time bound, measurable, and adequately translated into an action plan” [15]. Another WHO guideline for COVID-19 IAR from 2020 [16] underline that implementation of proposed activities should be monitored by a designated follow-up team. All the SimEx/AAR in this report are from 2018 or before, but it will be important that organisers of exercises use these manuals in the future. Recent COVID-19 IAR at EU level and in Ireland are examples that shows this guideline is in use [17,18,19]. An AAR registry is another measure proposed to facilitate organizational improvement [10]. To further embed the implementation, another measure could be that when the Commission orders a SimEx or AAR to be carried out, they have already assigned the responsibility to monitor the follow-up of recommendations to a unit in the EU with the necessary authority to develop action plans and ensure accountability. This would mitigate the fact that recommendations so far have been proposed without any authority to implement them.

Over the years, some recommendations have however been implemented, and we believe the pandemic has had a triggering or accelerating effect on many of these processes. Some relevant recent developments are the European Health Union, which aims to improve preparedness and response capacity and resilience of health care systems in the EU. A new regulation on serious cross-border threats replacing decision 1082 [20] has been adopted, which replaced the Decision on cross border health threats and provide a strengthened framework for regional coordination of health emergencies in the EU. The regulation includes “clear provisions for the EU and Member States to adopt similar and interoperable plans at national and local levels. To ensure these plans are actually operable in times of crisis, regular full-scale exercises and carry-out after-action reviews to implement corrective measures will be organised” [21]. The issue of interoperability was also addressed in part by the ECDC as a result of lessons learnt from the 2009 influenza pandemic, in the “Guide to revision of national pandemic influenza preparedness plans” [22]. Further, the establishment of HERA in 2021 was meant to ensure a coordinated approach to ensuring access to medical countermeasures during crises.

These developments in the European Health Union will be highly relevant to address the challenges with interoperability and standardisation, which were raised numerous times in our data. In a crisis, valuable time can be gained from having streamlined structures and strategies in the countries, as it facilitates collaborative efforts across borders. Further, formalising operating procedures is a way of ensuring efficient, predictable, and coherent responses during international crises. The need for clarification and awareness of roles also arises especially in times of crises when swift reactions hinge on a thorough understanding of the roles and responsibilities. However, as many of the new developments in the European region are in initial stages, it will be necessary to monitor if these efforts, in fact, lead to improvements, both at national and regional levels.

There is an increased focus on preparedness planning, assessment and reporting at national levels in Articles 6, 7 and 8 of the updated Regulation, but no specific references to how the use or follow-up of exercises and AAR should be integrated into these activities [20]. Article 5 obligates the Commission to carry out simulation exercises and after-action reviews “as required” and update a Union health crisis and pandemic plan as required. Explicit consideration on how lessons from previous and future exercises and AAR can contribute to strengthening this plan can ideally lead to strengthened preparedness and response capacities in the region. More detailed reporting and regular assessments of preparedness capacity at national level may increase accountability and highlight persisting gaps. Article 7 describes a triennial reporting which is “based on agreed common indicators”, thus a form of monitoring of progress similar to the SPAR. In addition to capturing pre-defined indicators through the templates that will be defined under this Article 7, efforts should also be made to encourage member states to address gaps identified through SimEx and AAR. Article 8 calls for regular “assessments of prevention, preparedness and response planning”, which can be read as evaluations complementing the monitoring described in Article 7. But Article 8 does not specify how these should be done – it is here that SimEx would be relevant to include as assessments. Simultaneously, AAR from the period in question should be taken into account. The Article describes a nine month delay to execute an action plan addressing the proposed recommendations, but only for Member States. The Commission should also consider how the results of multi-country exercises are followed up to address overarching issues which are not solved at State level, and how this should be monitored at a regional level.

Limitations to this study

The review was conducted as a desk exercise and is based on the information found in the reports. We do not have secondary sources of information to triangulate our findings. The timeline of the included SimEx/AAR is long (2005–2018), during which time preparedness within the EU has been also evolving and developing. Numerous EU agencies, organisations, tools, SOPs and guidance documents have been produced or changed which are difficult to capture and contextualise from the outside. To mitigate this, we have invited key persons in the EC and other EU agencies to review the draft report, and have presented and discussed findings with peers in the Joint Action and members of the EC.

Only SimEx and AAR organized by the EC were included in this analysis, which means only a small number of exercises were reviewed. Exercises at national level have been carried out in the same time period, and might produce other types of recommendations. The scope and topics of the exercises and selection of participants. Selecting common events or rare events as exercise scenarios can influence the result as some participants may be more familiar with scenarios that occur relatively frequently than those who do not, which might influence the nature and quality of recommendations and which IHR core capacities addressed. There will be a certain bias in which capacities are addressed in an exercise because some capacities lend themselves better than others to assessment in an exercise setting.

Finally, this report provides a comparative analysis of data from different sources and different times that were not specifically designed to be compared. Recommendations from SimEx and AAR are not conceived according to the IHR core capacities, and there is therefore some subjectivity in the interpretation of the recommendations and classification according to IHR (2005) capacities.



Source link