International comparisons of COVID-19 pandemic management: What can be learned from activity analysis techniques?* Vı́ctor Giménez Diego Prior Department of Business, Universitat Department of Business, Universitat Autònoma de Barcelona, 08193 Autònoma de Barcelona, 08193 Bellaterra (Barcelona), Spain, email: Bellaterra (Barcelona), Spain, email: victor.gimenez@uab.cat diego.prior@uab.cat Claudio Thieme Emili Tortosa-Ausina School of Economics and Business, Department of Economics, IIDL and Universidad Diego Portales, Av. Santa Ivie, Universitat Jaume I, Campus del Clara 797, Huechuraba, Santiago de Riu Sec, 12071 Castelló de la Plana, Chile, Chile, email: Spain, email: tortosa@uji.es claudio.thieme@udp.cl September 11, 2023 Abstract The global spread of COVID-19 since early 2020 has resulted in significant humanitarian costs. The pandemic has affected most countries to varying degrees, and governments have implemented diverse policies to minimise the impact on public health. However, these policies have varied across regions and even within countries. This study proposes a nonparametric activity analysis methodol- ogy to assess how different countries have managed the pandemic. Specifically, we assess the effec- tiveness of 61 countries nine months into the pandemic using a robust directional Benefit of Doubt (BoD) model according to expert opinion and conditional on country contextual factors. We then estimate the marginal impact of structural and discretionary contextual variables on effectiveness using nonparametric regression analysis, which shows that effectiveness is strongly influenced by socioeconomic and cultural factors. The results reveal three main groups of countries according to their level of effectiveness in pandemic management, suggesting that an accurate assessment of coun- tries’ management of the pandemic benefits greatly from operations research methods, as we obtain benchmarks and find out how these benchmarks (or best practices) vary when contextual factors are included in the analysis. Key words and phrases: Data envelopment analysis, COVID-19, composite indicators, assur- ance region, robust conditional convex frontier models. JEL Classifications: C61, H51, I18. * All four authors are grateful to 145 respondents of the survey conducted for the study. We are particularly grateful to two anonymous reviewers, whose comments have contributed to the overall improvement of the paper. Emili Tortosa-Ausina acknowledges financial support by MCIN/AEI/10.13039/501100011033 (Ministerio de Ciencia e Innovación, PID2020-115450GB-I00) and Universitat Jaume I (UJI-B2020-27). Vı́ctor Giménez and Diego Prior acknowledge financial support from AGAUR, reference 2021SGR00719. The usual disclaimer applies. Communications to: Emili Tortosa-Ausina, Departament d’Economia, Universitat Jaume I, Campus del Riu Sec, 12071 Castelló de la Plana, Spain. Tel.: +34 964387168, e-mail: tortosa@uji.es. 2 1. Introduction The spread of the SARS-CoV-2 virus and its associated COVID-19 disease is the largest public health emergency since the 2002-2004 SARS outbreak in China, becoming a global health threat with unprecedented socioeconomic effects around the world. As the World Health Organi- zation (WHO) Emergency Committee on COVID-19 forecasts the pandemic will not be short lived, having evolved from a public health event into a transnational crisis, continued efforts to fend it off at the community, regional, national and global level should be sustained during the years ahead. The unprecedented impact of the pandemic in most countries of the world has triggered rapid responses from multiple tiers of society, including governing bodies (both national and sub-national, depending on each country’s level of decentralisation) and international organisa- tions, in general, as well as professional and other communities. The scientific community has not remained on the sidelines, but the reactions across disciplines are notably divergent—partly due to their varying relationships with the causes and consequences of the pandemic. Some of the scientific disciplines involved in evaluating COVID-19-related issues emerge naturally; epidemiology (prevention), medicine (treatment) and pharmacy (vaccines) stand out for obvious reasons to do with the public health implications of the pandemic. However, the social sciences are also playing a role, since epidemiologists’ recommendations for lockdowns (Goolsbee and Syverson, 2021; Bourdin et al., 2021; Brodeur et al., 2021a; Pei et al., 2022) and stay-at-home orders inevitably wreaked unprecedented economic havoc in many countries, par- ticularly in those Western economies where the toughest stringency policies were implemented (Aum et al., 2021). Despite the substantial controversy over how lasting and restrictive the measures need to be, as well as their effectiveness,1 there is no doubt that their impact on societies is huge and their effects manifold; these effects can be evaluated from the perspective of several social sci- ence disciplines. Among them, economics is at the forefront, due to the unprecedented decline in GDP caused by the imposition of harsh and lasting lockdowns and other social-distance measures (including closures in the hospitality sector, sports facilities, etc.).2 This unprece- dented blow to the world’s economies has resulted in an equally unprecedented response from 1 E.g., see the discussion on mask wearing in Murray (2020). 2 These measures are likely to have other side effects in the medium to long term, including psychological issues among children due to school closures (Yoshikawa et al., 2020). Although these are more difficult to measure, some studies are already quantifying those more directly related to the impact on the economy (Bianchi et al., 2020). 1 economics, both at the practitioner and academic levels.3 The range of economics-related issues studied has been wide, including not only obvious fields such as business (Juergensen et al., 2020), accounting (Cohen et al., 2021) or banking and finance (Andries et al., 2020), but also interactions with other social science disciplines such as education (Agasisti and Soncin, 2021) and geography (Dokhov and Topnikov, 2021; Harris, 2022; Ascani et al., 2021), to name a few—although the links with psychology (Hamer- mesh, 2020), sociology (Connell, 2020) and political science (Bol et al., 2021) are also evident. The literature from this economics perspective is burgeoning, since it now includes not only the working papers and other documents disseminated by prestigious institutions,4 but also publications by respected journals in the field, dealing with issues related to the economic consequences of COVID-19 and government responses.5 It seems clear that a successful response to COVID-19 requires coordinated strategies from both epidemiologists and economists, since their advice might be mutually exclusive—i.e., lock- downs and business closure mandates are at odds with economists’ advice to avoid recessions (Glover et al., 2020; Favero et al., 2020).6 Some efforts have already attempted to understand how these coordinated strategies can help to improve governments’ responses to COVID-19 (Murray, 2020), to ultimately provide joint advice on the benefits and costs of the different policies (see also Avery et al., 2020). Each government, including sub-national levels within a country,7 chose its own mix of social-distancing measures (Chen et al., 2022), which might include restrictions on mobility (on space and time, i.e., weekdays and weekends), curfews, schools and business closures, travel restrictions, compulsory face masks, different testing and contact-tracing strategies, etc., result- ing in very heterogeneous combinations.8 This is, a priori, surprising, since most territories implementing different measures are part of the same trade bloc (such as the European Union) or even the same country. Some explanations for these mixes might be related to health costs 3 Apart from the contributions published in academic journals, see also the “Covid Economics” initiative by the Centre for Economic Policy Research (CEPR, https://cepr.org/content/covid-economics-vetted-and-real-time- papers-0, “created to quickly disseminate fast-rising scholarly work on the Covid-19 epidemic”, which has pub- lished several hundreds of documents on different aspects related to the economics of COVID. 4 See https://cepr.org/content/covid-economics-vetted-and-real-time-papers-0. 5 The number of contributions is increasing so rapidly to the point that a survey of the literature has already been published (Brodeur et al., 2021a). 6 One might also wonder about how much of the collapse resulted from government-imposed restrictions or peo- ple voluntarily choosing to stay at home, and some studies have found legal shutdown orders contribute modestly to changes in consumer behaviour (Goolsbee and Syverson, 2021). 7 In several countries, decentralisation of powers has resulted in several competencies, particularly health and education, being transferred to sub-national layers of government. 8 Enforcing measures also vary greatly from country to country (Kavanagh and Singh, 2020). 2 and temporary unemployment schemes covered by different layers of government, which could advocate opposing measures. We argue that this variety of responses to the COVID-19 pandemic might be resulting in different degrees of success in governments’ handling of the pandemic (Haushofer and Metcalf, 2020). However, it is no easy task to measure governments’ performance in tackling the pandemic. Nonetheless, this is a critical issue, since measuring the different performances with a certain degree of precision would identify some benchmarks or best-practices. In fact, there is a considerable body of literature concerned with benchmarking and best practice: the activity analysis literature (Farrell, 1957) dealing with performance and efficiency evaluation via frontier methods. This literature is closely related to operations research, since many of the methodologies proposed to evaluate the performance of organisations (including government and public policies) and decision-making units (DMUs) in general come from this field. We consider a broader view, with the aim of evaluating countries’ responses to the pan- demic. As indicated above, the approaches adopted by countries and subnational governments within countries have been heterogeneous in terms of timings and extents of lockdowns, pro- cesses to reopen the economy etc., and views about the onset of successive waves have been disparate and confusing (Nikolopoulos et al., 2021). As a consequence, the performances of different countries and regions also varied. Therefore, in order to identify the policies that work best, it is also necessary to identify the best practices or benchmarks, a task in which operations research methodologies have a long tradition of providing appropriate responses. Consistent with this, several studies have analysed the performance of COVID-19 manage- ment by the health systems of different countries using activity analysis techniques. A review of the literature using Web of Science identifies at least 13 articles along these lines. All of them analyse performance from the perspective of efficiency, mainly using data envelopment anal- ysis (DEA) models (Klumpp et al., 2022; Min et al., 2022; Sinimole, 2022; Lupu and Tiganasu, 2022; Lee and Lim, 2022; Doğan et al., 2021; Breitenbach et al., 2021; Su et al., 2021; Ordu et al., 2021; Mourad et al., 2021; Ibrahim et al., 2020), and in specific cases stochastic frontier anal- ysis (SFA) (Ahmad et al., 2020) and weighted stochastic imprecise data envelopment analysis (WSIDEA) (Aydin and Yurdakul, 2020). This review reveals a significant number of studies using models that measure a country’s performance in different phases of the pandemic (Lupu and Tiganasu, 2022; Ordu et al., 2021; Su et al., 2021; Ibrahim et al., 2020). For example, Lupu and Tiganasu (2022) evaluate the per- formance of 31 European countries. They use DEA performed in three stages of the pandemic 3 (first wave, relaxation and second wave), finding high inefficiency in the first phase, mainly in Western countries, which improved significantly afterwards. A second phase analysis shows that the factors influencing performance differ according to the phase of the pandemic in the country. Ordu et al. (2021) use super efficiency DEA methods to assess the performance of 16 countries in each of the first five weeks of the pandemic. Their results evidence a sharp decrease in efficiency over time, and unlike most of the European countries evaluated, China and South Korea show an increase in efficiency. Ibrahim et al. (2020) use DEA to evaluate the performance of 58 countries in two phases of the pandemic (pandemic control and treatment measurement), and consequently use two models. The outputs they used for the pandemic control phase were COVID-19 confirmed cases and for the treatment measurement phase COVID-19 related deaths and COVID-19 recovered cases. Their evaluation shows that 89.6% of the countries are ineffi- cient in pandemic control and 79% in treatment measures. A sensitivity analysis highlights the critical nature of resources. Similarly, two of these studies apply two-stage network DEA models, which use interme- diate variables that act as hinges, being both an output of the first node and an input of the second node (Klumpp et al., 2022; Min et al., 2022). For example, Klumpp et al. (2022) use a two-stage network DEA model to measure the efficiency of the health systems of 19 OECD countries over 12 time periods. Their model relates the number of tests applied, cases and deaths. The results indicate that, on the one hand, factors such as population size, population density and the country’s stage of development did not play an important role in the success of pandemic management. Pre-pandemic health system policies were decisive, however. Health systems with a primary care orientation and a high ratio of primary care physicians to special- ists proved to be more efficient than systems with a medium level of resources that were partly publicly funded and characterised by a high level of access regulation. From an economic per- spective, strategies without general blockages were identified as more efficient than the total blockage strategy. In other aspects, these studies incorporate contextual variables directly as inputs to the evaluation model. The inputs used in many of these studies are non-discretionary contextual variables or short- or medium-term discretionary contextual variables. Examples include the studies by (i) Min et al. (2022), who use population size, per capita gross national income; (ii) Doğan et al. (2021), who include population density, population aged 65+, chronic diseases; (iii) Ordu et al. (2021), who incorporate population, percentage of population aged 70+, me- dian aged; (iv) Aydin and Yurdakul (2020), who include a stringency index, extreme poverty, 4 CVD death rate, diabetes prevalence, female smokers, male smokers, population, GDP; and (v) Ibrahim et al. (2020), who incorporate population density, percentage of population aged 65+. Although we take a different methodological approach to the above studies, they do demonstrate the importance of incorporating this type of variable in order to compare coun- tries adequately. Finally, although many studies have attempted to evaluate country performance in man- aging the pandemic, very few attempt to explain the differences in performance found. In fact, only four of these studies provide a second-stage evaluation in this regard. These in- clude Klumpp et al. (2022), who perform a correlation analysis; and Min et al. (2022), Lupu and Tiganasu (2022) and Mourad et al. (2021), who use Tobit regression as a post hoc analysis. For example, Min et al. (2022) study the performance of 34 OECD member countries, relating health system performance outcomes to cultural aspects of the country. They use two-stage net- work DEA and conclude that a pervasive, less individualistic and uncertainty-avoiding culture positively influenced country performance. In summary, a body of literature has evaluated the performance of health systems in dif- ferent countries during the COVID-19 pandemic from an efficiency perspective. The diversity of variables used to model the phenomenon and evaluate the pandemic in its different stages reflects the diversity of research objectives pursued. However, more studies are still needed to understand the considerable complexity of this phenomenon and the impact of contextual variables on country performance. Similarly, to the best of our knowledge, no previous studies consider conditional models to evaluate health systems’ performance of COVID-19 manage- ment in different countries from an effectiveness perspective. In light of the above, in this paper we use composite indicators to evaluate the effectiveness with which countries manage the pandemic; these indicators are particularly appropriate for assessing complex phenomena that cannot be evaluated with a single variable (Maricic et al., 2019), as in the case we are dealing with. Specifically, we propose a composite indicator com- bining several proposals in the field of nonparametric activity analysis methods: directional distance functions, benefit of the doubt, and robust conditional convex order-m estimations, a design which has been considered up to now in several studies, including Titl and De Witte (2022), Rogge et al. (2017) and Lavigne et al. (2019), among others. In our case, we will also consider the opinion of the experts when choosing the directional vector of the distance func- tions, proposing a new way to determine the corresponding weights using Best-Worst Scaling. The model is refined in a second stage by considering a conditional model in which two types 5 of variables enter the analysis, namely, structural contextual variables and discretionary con- textual variables. The inclusion of these successive stages in the analysis is essential due to the existence of factors lying beyond governments’ control (i.e., they are non-controllable) and that might clearly affect countries’ performance, as well as other factors over which some control can be exercised (i.e., they are controllable). Since these factors differ in nature, they enter the model sequentially rather than simultaneously so as to assess their effect more precisely The paper proceeds as follows. After this introduction, Section 2 presents the methodology and Section 3 describes the data. Section 4 then reports the results, and finally, Section 5 presents the conclusions of the study and implications for public policy. 2. The proposed methodology 2.1. Effectiveness composite indicator: directional BoD model The evaluation of effectiveness uses different methods to create composite indicators (CI). These indicators have become popular and widely used metrics for assessing complex phenomena which cannot be evaluated with a single variable (Maricic et al., 2019) as they globalise a pro- cess composed of multiple stages that generate different sub-indicators. Most of the methods available to build CI require previous information on the sub-indicators in order to determine the weight assigned to each of them (OECD-European Commission Joint Research Centre, 2008). Here we take an alternative approach by using data envelopment analysis methods in order to obtain endogenous weights without the need to require any previous information (Wang, 2015). Specifically, we propose a composite indicator that combines alternative devel- opments from the research on nonparametric activity analysis, namely, benefit of the doubt (BoD)(Cherchye et al., 2007; Melyn and Moesen, 1991; Cherchye et al., 2008; Despotis, 2005; De Witte and Rogge, 2011; Gaaloul and Khalfallah, 2014; Morais and Camanho, 2011; Stum- briene et al., 2019a; Zanella et al., 2013). BoD models are a variant of the nonparametric frontier models used to measure efficiency (Cooper et al., 2007) where only outputs are considered and have been widely applied to construct composite indicators in different contexts. Some examples of applications are the Health System Performance Index (Lauer et al., 2004), the Hu- man Development Index (Despotis, 2005), the Internal Market Index (Cherchye et al., 2007), the Technology Achievement Index (Cherchye et al., 2008), the Students’ Evaluation of Teaching Indicator (De Witte and Rogge, 2011), the Competitiveness Index (Bowen and Moesen, 2011), the Quality of Life Indicator (Morais and Camanho, 2011), the Digital Access Indicator (Gaaloul 6 and Khalfallah, 2014), education system performance (Stumbriene et al., 2019b), Citizen Satis- faction with Local Police Effectiveness (Verschelde and Rogge, 2012), the Life Satisfaction Index (Guardiola and Picazo-Tadeo, 2014), the Sustainable Energy Index (Wang, 2015), the Environ- mental Performance Index (Zanella et al., 2013), the Well-Being Indicator (Peiró-Palomino and Picazo-Tadeo, 2018), spatial directional robust composite indicators in the presence of undesir- able output (Fusco et al., 2020) and the association between public spending effectiveness and the ideological orientation of regional governments (Titl and De Witte, 2022). There are three relevant issues to be considered when applying this type of model to our case. First, the compensability among different sub-indicators. The traditional BoD model assumes absolute freedom in determining the endogenous weights and, therefore, the total compensability/trade-off between sub-indicators. However, this assumption is not applicable when information regarding the preference structure is available from experts, as occurs in our case. For these cases, Fusco (2015) proposes including a “directional” penalty for the indica- tors considered less important according to the experts’ judgement using a directional distance function (DDF) (Chambers et al., 1998; Färe and Grosskopf, 2000). This type of distance func- tion has been widely used to evaluate environmental efficiency where good/bad outputs to be maximised/minimised coexist (Chung et al., 1997; Oh, 2010; Picazo-Tadeo et al., 2005; Zanella et al., 2013). Consequently, DDFs are especially suitable for evaluating a set of indicators that improve by increasing their value simultaneously with others that have a contrary behaviour. This is common in the health field where it is desirable to minimise, for example, the num- ber of deaths. The second issue to consider is the need to incorporate the environmental or contextual characteristics of each country to obtain the fairest evaluation possible. Finally, the risk of possible outliers increases when data from different countries is used. For all these rea- sons, we propose using a BoD model that is directional (incorporating expert opinion), robust (insensitive to the presence of outliers) and conditional (considering the contextual factors of each country). Regarding the impact of context on the evaluation, we also propose estimating the marginal impact of the structural contextual variables (those that, having an impact on the levels of effectiveness, are not discretionary as their levels are controlled by the authorities) and the marginal impact of the discretionary contextual variables (those that, having an impact on the levels of effectiveness, are discretionary because their level depends on decisions taken by the respective governments). Let us assume that, for K countries, we have information on a set of J desirable or good J indicators to maximise, y ∈ R+ , as well as H undesirable or bad indicators to minimise, b ∈ R+ H. 7 The performance of any country can be measured through the following DDF (Chambers et al., 1996; Sueyoshi and Goto, 2010; Oh, 2010):  D (y, b) = max β | y + βgy , b − βgb (1) The above DDF determines respectively the maximum simultaneous increase and decrease J+H (β) for the indicators y and b over the vector g = ( gy , gb ) ∈ R+ which defines the desir- able directions for improvement for both types of indicator. Various methods can be used to calculate D (y, b). In this study we use a nonparametric frontier model based on Oh (2010), al- though,to be coherent with the BoD formulation, inputs have disappeared. D (y, b) is calculated by solving the following linear program9 for the evaluated country o: Max θ (yoj , boh ) = β s.t. ∑kK=1 λk ykj ≥ yoj + βgy j = 1... J, (2) ∑kK=1 λk bkh ≤ boh − βgb h = 1... H, λk ≥ 0 k = 1... K. where ykj represents the indicator j to be maximised for country k; bkh is the indicator h to be minimised for country k; yoj and boh are the observed levels of each indicator, respectively, for the country under assessment. β is the maximum simultaneously achievable increase (decrease) in the indicators to be maximised (minimised). Note that in the case of a country that has managed the pandemic in both dimensions effectively, β = 0. To optimise the linear program (2), the vector g = ( gy , gb ) must be defined. The literature usually follows Chung et al. (1997) by defining gy = (yo1 , yo2 , ..., yoJ ) and gb = (bo1 , bo2 , ..., boH ), assuming compensability among indicators.10 However, when this assumption is not ad- equate because information is available on the importance of the indicators, Fusco (2015) proposes weighing their components by defining gy = (yo1 w1 , yo2 w2 , . . . , yoJ w J ) and gb = (bo1 v1 , bo2 v2 , . . . , boH v H ). In our estimations, we propose using the Shepard output distance for the CI computation (Fusco, 2015; Rogge et al., 2017; D’Inverno and De Witte, 2020), since it is more easily inter- 9 There are other approaches based on the primal formulation of the program. See, for instance, Lavigne et al. (2019). 10 Indeed, one of the main criticisms underlying composite index methodologies is related to the trade-offs be- tween the variables. See Greco et al. (2019). 8 preted: 1 CI (yoj , boh ) = (3) 1 + θ (yoj , boh ) where CI (yoj , boh ) ≤ 1, the higher the value of CI (yoj , boh ), the higher the level of effectiveness. Consequently, the countries with the best performance in managing the pandemic will obtain CI (yoj , boh ) = 1. 2.2. Conditional model Previous models are labelled as unconditional as they do not restrict the comparison of units with significant differences in their environmental conditions. This implies that the value of the CI for inefficient units can be underestimated with regard to those units operating in a more favourable environment. To control for these circumstances, Daraio and Simar (2007a) devel- oped a conditional approach to account for those variables in the same optimisation model. The basic rationale behind conditional models is to compare the unit under analysis with other units that present similar levels in their contextual or environmental factors. These conditional models are particularly relevant when dealing with units facing disparate environmental con- ditions, which is the case of cross-country studies. The conditional approach has become very popular in the literature on performance mea- surement (e.g., Cherchye et al., 2010; De Witte and Kortelainen, 2013).11 To estimate the con- ditional scores, smoothing techniques are needed such that, in the reference samples, obser- vations with comparable z-values have to be chosen. Therefore, this approach relies on the estimation of a nonparametric kernel function to select the appropriate reference partners and a bandwidth parameter b using a method with some bandwidth choice. As we use continuous z variables, we apply the method proposed by Bădin et al. (2010), based on the least squares cross validation procedure (LSCV) developed in Racine and Li (2004), Hall et al. (2007) and Li and Racine (2007). Following De Witte and Kortelainen (2008), we use the Epanechnikov kernel density func- tion Kb̂ (z, zi ) = b−1 K [( Z − zi )/b)] where b is the smoothing parameter, or bandwidth. The conditional estimation implies the restriction of the technology to consider only those units that happen to have their contextual variables in between the two extremes delimited by the respective bandwidth: (z − b) ≤ zo ≤ (z + b) (4) 11 Although the antecedents of conditional estimations had been introduced earlier by Ruggiero (1996). 9 The conditional formulation of program (2) is: Max θ (yoj , boh |zo ) = β ′ s.t. ∑kK′ =1 λk′ yk′ j ≥ yoj + βgy j = 1... J, (5) K′ ∑ k ′ =1 λk′ bk′ h ≤ boh − βgb h = 1... H, λk′ ≥ 0 k′ = 1... K ′ . where k′ only includes those units with contextual variables meeting condition (4), i.e.: (zk′ − b) ≤ zo ≤ (zk′ + b) (6) being the composite indicator: 1 CI (yoj , boh |zo ) = (7) 1 + θ (yoj , boh |zo ) After estimating the unconditional and conditional directional BoD models, it is possible to define the ratio Qz to show the dependence on the contextual variables of the composite indicator by using a smoothed nonparametric regression (Daraio and Simar, 2005) : CI (yoj , boh ) Qz = (8) CI (yoj , boh |zo ) In our empirical application, contextual variables (z) can be classified as structural (or non- discretionary, zs ) and discretionary (zd ). A similar approach can be found in Rogge et al. (2017), who made the distinction between variables which can only be considered as exogenous in the short term and variables which can be considered as exogenous in both the short and the long term. According to our classification, we define a recursive process to determine the marginal importance of each group of variables by estimating model (5) twice. In the first step, we introduce only the structural environmental variables (zs ), which serves to determine the impact on CI due to the presence of the structural context variables: s CI (yoj , boh ) Qz = (9) CI (yoj , boh |zos ) In the second step, and considering the results of (5) including both environmental vari- ables zs and zd [(zs , zd ) ∈ z], we determine the ratio that shows the additional impact of the 10 discretionary (zd ) context variables: d CI (yoj , boh |zos ) Qz = (10) CI (yoj , boh |zo ) 2.3. Providing robustness to the estimation model One of the issues of nonparametric deterministic models is the lack of statistical properties and robustness when extreme values and outliers are present. In this context, Cazals et al. (2002) propose a robust nonparametric estimation of the frontier: the order-m estimator (see, for instance, Cherchye et al., 2019, for a relevant application). The idea behind it is to compare units against a sub-sample of randomly drawn m units, and repeat the process B times in order to determine the average value of the CI that comes from the solution of B models that only integrates one sub-sample in each run. Although this robust approach is based on probabilistic formulation (Daraio and Simar, 2005), this can also be computed by following a Monte-Carlo algorithm (Daraio and Simar, 2007b). The adaptation to our case of the required steps to complete the algorithm is as follows: 1. Select a subsample of m countries, with replacement, among the K countries. 2. Solve programs (2) and (5) and estimate CI (yoj , boh ), CI (yoj , boh |zos ) and CI (yoj , boh |zo ). 3. Repeat steps 1 and 2 B times. ˜ (yoj , boh ), CI 4. Determine the average values CI ˜ (yoj , boh |zos ) and CI ˜ (yoj , boh |zo ). s d 5. Calculate the average ratios of the corresponding composite indicators Q̃z , Q̃z and Q̃z . 3. Data, sample and variables 3.1. Data sources and variables included in the model For this worldwide cross-sectional study, we used data from several sources, all of them open access. We retrieved Covid-19 related data from the website OurWorldInData.org (Roser et al., 2020). This website has compiled data from several important sources, such as the European Centre for Disease Prevention and Control, national government reports, the Oxford COVID-19 Government Response Tracker, the World Health Organization, and the Global Health Obser- vatory Data Repository, among others. It has documented daily Covid-19 case numbers, death 11 numbers, test numbers and stringency indices, and demographic, health and human develop- ment indicators from more than 180 countries.12 We also consider information on several quality of government indicators. Specifically, the perceptions on the quality of government for the countries in our sample as of 2019 correspond to the Worldwide Governance Indicators (WGI), available via www.govindicators.org (for de- tails on the methodological aspects, see Kaufmann et al., 2010). This is the most widely accepted database on quality of government indicators at the country level.13 Finally, to account for the cultural aspects of each country, we used Hofstede’s six dimensions model (Hofstede, 2001; Hofstede and Hofstede, 2005), data from which are available at www.hofstede-insights.com. For the unconditional model, we consider indicators which could reflect the suc- cess of interventions against COVID-19. Given that the nature of the phenomenon in- volves different phases and different key actions, an evaluation from a holistic perspec- tive is required. In this regard, Kennedy et al. (2021), based on the Global Health Se- curity Index (GHSI) developed by the World Health Organization (2020), point out that the good performance of a country in dealing with the COVID-19 pandemic is based on its capacity to detect, contain and treat the infection. From the review of the lit- erature, as well as websites and institutions that systematise the progress of the pan- demic, such as Worldometer (https://www.worldometers.info/coronavirus/) and Statista (https://www.statista.com/), the five indicators of the unconditional model were defined as follows: Detection: Cumulative tests performed per million inhabitants (y1 ). The higher the value, the higher the effectiveness (Kennedy et al., 2021; Klumpp et al., 2022; Breitenbach et al., 2021; Aydin and Yurdakul, 2020). Positivity rate (total confirmed cases/total tests performed) (b1 ). The higher the value, the lower the effectiveness (Kennedy et al., 2021; Ahmad et al., 2020). Containment: Cumulative confirmed COVID-19 cases per million inhabitants (b2 ). The higher the value, the lower the effectiveness (Kennedy et al., 2021; Klumpp et al., 2022; Min et al., 2022; Lupu 12 The data from the OurWorldInData.org database. We were retrieved on January 4, 2020. 13 The Quality of Government Institute (Stockholm, Sweden) provides data for lower territorial units. 12 and Tiganasu, 2022; Lee and Lim, 2022; Doğan et al., 2021; Su et al., 2021; Mourad et al., 2021; Aydin and Yurdakul, 2020; Ibrahim et al., 2020). Cumulative confirmed COVID-19 deaths per million inhabitants (b3 ). The higher the value, the lower the effectiveness (Kennedy et al., 2021; Klumpp et al., 2022; Min et al., 2022; Sinimole, 2022; Lupu and Tiganasu, 2022; Doğan et al., 2021; Breitenbach et al., 2021; Ordu et al., 2021; Mourad et al., 2021; Ibrahim et al., 2020; Ahmad et al., 2020; Aydin and Yurdakul, 2020). Treatment: Case fatality rate (total deaths/total cases) (b4 ). The higher the value, the lower the effective- ness (Kennedy et al., 2021; Fan et al., 2021; Bignami-Van Assche and Ghio, 2022; Cao et al., 2020a). The literature provides three alternatives to account for the start point of the pandemic: absolute (on a certain calendar date); relative (when certain conditions are met, for example reaching a specific number of infections or deaths); and a combination of the above (from a certain date, but the country is only considered in the sample if after a given number of days, they reach a minimum number of infections). All of them have advantages and disadvantages. In our case we use the relative option since we assume that the phenomenon studied (the management of the pandemic) will develop differently in each country, according to its degree of evolution. Hence, to compare the effectiveness of the action between countries under equal conditions, the start of the pandemic was calculated from the date of the first confirmed case per million inhabitants in each country. A start date relative to each country has also been used by Ordu et al. (2021) and Su et al. (2021).14 Thus, in our sample of 61 countries, the evaluation of the pandemic management of each country in this study is evaluated 270 days after the start of the pandemic, which corresponds to information in the range from November 15, 2020 (South Korea) to December 21, 2020 (Zambia). We report descriptive statistics for these indicators in Table 1. [Table 1 about here] Some stylised facts that emerge when examining the data indicate:15 14 We are grateful to one of the reviewers of the paper for pointing out these alternatives. 15 We do not report individual data for space reasons and because they are available from the websites referred to above. 13 • Luxembourg and Denmark are the countries with the highest number of tests per million inhabitants (2,131,356.5 and 1,247,560.9, respectively). In contrast, Nigeria and Senegal are the countries with the lowest number of tests per million inhabitants (4,488.3 and 14,810.7, respectively). • Mexico is the country with the highest positivity rate (total confirmed cases/total tests performed), followed by Argentina (0.433 and 0.411, respectively). In contrast, New Zealand and Australia are the countries with the lowest positivity rate (0.0016 and 0.0028, respectively). • The country in the sample with the highest cumulative confirmed COVID-19 cases per million of inhabitants on day 270 of the start of the pandemic is Luxembourg (50,295.8 cases per million inhabitants). In contrast, Thailand and Nigeria are the countries with the lowest number of confirmed cases per million inhabitants (58.8 and 405.4, respectively). • Regarding the cumulative confirmed COVID-19 deaths per million inhabitants indica- tor, Spain is the country with the worst performance and Thailand and New Zealand are the countries with the best performance (0.86 and 5.20, respectively). • Finally, with respect to the case fatality rate indicator (total deaths/total cases), Mexico and Iran are the countries with the worst performance (0.091 and 0.053, respectively). In contrast, Sri Lanka and Iceland show the best figures for this indicator (0.0047 and 0.0049, respectively). 3.2. Expert opinions As indicated in the previous section, a key element when dealing with composite indicators is to determine the importance for these indicators—or, more precisely, the subindicators that compose them. Unfortunately, there is no single valid process to achieve this objective (Ueda and Amatatsu, 2009). In cases where an applicable theoretical model is unknown and it is required to objectify these limits appropriately, one of the strategies used is expert judgement (Giménez et al., 2022; Waddock and Graves, 1997). However, this strategy commonly involves a reduced number of key informants and has innumerable methodological and validity disad- vantages (Mumpower and Stewart, 1996). For this purpose, and unlike previous studies, in our case we used a survey based on Best- Worst Scaling (BW), which is a survey method to obtain the relative importance that people 14 attach to the various attributes of a product or service (Finn and Louviere, 1992). This method- ology corresponds to an extension of paired comparisons that models the cognitive process by which respondents identify their preferences over a set of attributes, organised into subgroups of three or more items. The advantages are that respondents only have to make discrete choices and that it provides a more discriminating way of measuring the degree of importance respon- dents attach to each item. Since respondents can only choose one most preferred and one least preferred item in each set of choices, they are necessarily required to make trade-offs among the advantages (Cohen, 2003). Second, BW avoids problems of ranking bias, since there is only one way to choose the most and least preferred item, regardless of the respondent’s cultural background (Goodman, 2009). This is interesting, as it makes the BW method a powerful way of conducting cross-national studies on preferences (Auger et al., 2007; Lee et al., 2007). To carry out this study we conducted a survey using the Sawtooth Software platform among 1,228 academics whose output had been published in BMJ Quality & Safety, which ranks first (according to its impact factor) in the Web of Science category Health Care Sciences & Services. We collected 145 responses, of which 128 were considered valid. Respondents included healthcare professionals from all over the world and sub-disciplines of healthcare sciences.16 Once the responses were received, the Sawtooth Software platform, using Hierarchical Bayes techniques, allowed us to estimate the relative importance that each respondent (expert) gave to each of the five indicators evaluated (Johnson, 2000). Descriptive statistics are reported in Table 2. [Figure 1 about here] [Table 2 about here] 3.3. Structural and discretionary contextual variables In addition, and as mentioned earlier in the paper, we consider a total of six variables that affect the management of the pandemic by the governments of each country. In the first place, four variables are associated with demographic, economic and cultural aspects that are long-term structural aspects of the country and therefore cannot be modified discretionally (i.e., they are non-controllable) by governments in the short term. For instance, Lavigne et al. (2019) took a 16 Each respondent was contacted with the following request: “We are interested in assessing the overall perfor- mance of a country in COVID-19 management, nine months after the start of the pandemic. We would like to know your opinion on the relative importance of each of the most commonly used indicators. To do so, we will present six different sets of variables below and ask you to indicate for each of them the indicators that would be most and least important to you in the overall assessment of a country’s COVID-19 management”. Subsequently, they were asked: “If you were to evaluate a country’s performance in managing COVID-19 nine months after it began, please indicate which variables would be the most and the least important to you for this evaluation?”. An example of this choice is reported in Figure 1. 15 similar approach. These variables were selected based on the previous studies on performance evaluation reviewed in the introduction section and studies that show the relationship between contextual variables and COVID-19 evolution. These variables are: Population density: number of inhabitants per square kilometre of the country’s surface area (Lupu and Tiganasu, 2022; Doğan et al., 2021; Su et al., 2021; Ibrahim et al., 2020). Human Development Index (HDI): summary measure of average achievement in key dimen- sions of human development: a long and healthy life, access to quality education, and having a decent standard of living (Zheng et al., 2022; Ameye et al., 2022; Kavuran et al., 2023).17 Individualism: measure of the degree to which a society emphasises the achievement of in- dividual goals over collective goals. It corresponds to one of Hofstede’s (2001) cultural dimensions (Salvador et al., 2020; Feng et al., 2023). Long-term orientation: reflects a society’s perspective of achieving success and gratification in the long-term rather than in the immediate future. It emphasises persistence, persever- ance, and long-term growth (Hofstede, 2001; Salvador et al., 2020; Schlaegel et al., 2023). Secondly, two short and medium-term discretionary variables under government control were used: rule of law and the stringency index. Rule of law: one of the six governance dimensions of the Worldwide Governance Indicators (WGI), it reflects perceptions of the extent to which agents have confidence in and abide by the rules of society, and in particular the quality of contract enforcement, property rights, the police, and the courts, as well as the likelihood of crime and violence (Hodzic et al., 2021; Martı́nez-Córdoba et al., 2021). Mean stringency index: a composite measure of nine of the response metrics: school closures; workplace closures; cancellation of public events; restrictions on public gatherings; clo- sures of public transport; stay-at-home requirements; public information campaigns; re- strictions on internal movements; and international travel controls. For each country, we averaged the daily stringency index across the 270 pandemic days considered in the study (Aydin and Yurdakul, 2020; Cao et al., 2020b). We report descriptive statistics for these indicators in Table 3. 17 See http://hdr.undp.org/en/content/human-development-index-hdi. 16 [Table 3 about here] 4. Results 4.1. Effectiveness The methodology described above involved calculating three models. All of them use the five outputs described in Section 3.1, but with different assumptions about the influence of ˜ (yoj , boh ), and assumes that contextual factors. The first model (unconditional) calculates CI each country’s contextual factors do not affect its performance in managing the COVID-19 ˜ (yoj , boh |zos ). It is a conditional model pandemic. The second model allows us to compute CI which incorporates the impact of the country’s structural contextual factors in the calculation of ˜ (yoj , boh |zo ), and additionally incorporates the effectiveness index. The third model calculates CI the discretionary factors under the government control. Table 4 summarises the contextual variables chosen in each model. The calculation of these three models requires the definition of the directional vectors gy = (yo1 w1 , yo2 w2 , . . . , yoJ w J ) and gb = (bo1 v1 , bo2 v2 , . . . , boH v H ), more specifically the set of the directional vector components weights wy = (w1 , w2 , . . . , w J ) and vb = (v1 , v2 , . . . , v H )18 . Here, in accordance with the results of our survey we considered that an indicator is more important depending on the average weight assigned, and the consensus achieved among the experts. µw µ w2 µw j We therefore defined the directional vector components weights as wy = ( σw 1 , σw2 , . . . , σw j ) and 1 µ µ µv vb = ( σvv1 , σvv2 , . . . , σv h ), where µ is the average importance assigned by the experts to the in- 1 2 h dicator and σ is the standard deviation.19 In order to normalise wy and vb with respect to y1 , σw1 we multiply each component of both vectors by µ w1 . Consequently, and in accordance with the experts opinions, we set wy = (1) and vb = (1.035, 1.692, 2.646, 1.676). Additionally, the calculation of any order-m model using the Monte-Carlo simulation re- quires the definition of the parameters m and B. There are different alternatives to determine the value of m. For instance, Bonaccorsi et al. (2006) suggest choosing m such that the share of super-efficient observations is around 10% (6 units in our case). Simar (2003), based on previous 18 It is important to differentiate the values used for defining the directional vector components from those assigned to each indicator in the primal program (see, for instance, Lavigne et al., 2019). In our case, we orient the directional vector in accordance with the experts opinions, but we do not restrict the weights of the primal program. However, the results are still sensitive to the chosen weights for the components of the directional vector. This is topic to be explored in future research. 19 In finance, there is a widely used indicator (the Sharpe ratio) that has exactly the same definition. See Sharpe (1998). 17 work by Barnett et al. (1994), also suggests that the maximum percentage of outliers/super- √ n efficient units should be n (8 units in our case). We have followed this criterion by setting m = 40020 . For the parameter B, which determines the number of iterations for the bootstrap, Daraio and Simar (2005) state that it is sufficient to set it to 200. Nevertheless, in our case, we have set B = 500. [Table 4 about here] Table 5 exhibits the average results for the effectiveness in managing the COVID-19 pan- demic at both global and continental/subcontinental level. The higher the value of the score ˜ (•), the higher the level of effectiveness. Column CI CI ˜ (yoj , boh ), corresponding to the uncondi- tional model, shows the results considering only the pandemic indicators, without considering ˜ (yoj , boh |zos ) and CI any contextual variables. Columns CI ˜ (yoj , boh |zo ) show the levels of ef- fectiveness when countries that share similar contextual characteristics are compared. At the global level, the average performance is 0.846 when the particular environmental conditions of the countries are not considered, reaching a value of 0.958 when structural factors are con- sidered and 0.969 when discretionary factors are also incorporated. The global impact of the environment is shown in column Q̃z , standing at 0.874 (the lower the value, the greater the d s impact of the environment). Since Q̃z = 0.989 and Q̃z = 0.885, it is confirmed that a greater general impact can be attributed to structural variables. The results at continent/subcontinent level show that the countries in the sample which best managed the pandemic belong to Oceania (the highest value corresponds to the highest performance), and appear efficient in all our estimations. These are countries that adopted restrictive measures from the beginning, with a strong negative impact on the economy. They are followed by countries in Asia and Africa, which show similar levels of effectiveness (0.880 and 0.843 respectively). In the African case, the good results could be due to a potential lower capacity to report the data correctly given the socio-economic characteristics of many of these countries—at least those in our sample. In the case of Asian countries, the results are consis- tent with the speed, forcefulness, duration and severity of the measures many of them adopted. They are followed by European and South American countries in terms of effectiveness in man- aging the pandemic, with an average value of 0.837 and 0.810, respectively. In these regions, the strategy was mostly to live with the virus, with many of them pursuing reactivation of the economy when health conditions improved. In last place are the North American countries 20 Daraio and Simar (2005, 2007b) noted that the order-m frontier converges to the full frontier when m → ∞ 18 with an effectiveness level of 0.782. As expected, differences between continents decrease when considering the contextual variables. The improvement of North and South American countries is especially remarkable, increasing their levels of effectiveness by 17%, from 0.782 to 0.916 and from 0.810 to 0.950, respectively. Indeed, Q̃z shows that North and South American countries were the most impacted by their environment, with a value of 0.846 and 0.850 respectively. This may be due to the more adverse contextual conditions or the strategies applied by many of the countries belonging to these subcontinents and their potential negative effect on virus control. At the opposite extreme, contextual factors showed no impact on the management of the pandemic in the countries of Oceania. In the rest of the areas, the impact of the environ- ment remains at fairly similar levels, oscillating between 0.869 in Africa and 0.887 in Asia. The d highest values of Q̃z for Oceania, Europe and South America mean that in these geographical areas structural factors had a greater impact. [Table 5 about here] At the country level (Table 6), Australia, Denmark, Estonia, Finland, Iceland, Israel, Luxem- bourg, Malaysia, New Zealand, Norway, Sri Lanka, Thailand and Uruguay stand out for their effective management of the pandemic without considering contextual conditions, followed by Nigeria (0.988), Ireland (0.952) and Ghana (0.908). In contrast, Colombia (0.741), Bulgaria (0.741), Chile (0.740), South Africa (0.739), Argentina (0.738), Sweden (0.729), Iran (0.728) and Mexico (0.727) are in the last positions. Sweden perhaps deserves special comment, since it opted for a lax application of measures to achieve rapid biological immunisation of its pop- ulation. Although this merits further scrutiny, Sweden is ranked 59th out of the 61 countries considered, which suggests that results might not have corresponded to the expectations of the country’s authorities. In contrast, other Scandinavian countries, some of which restricted their citizens’ movement in and out of Sweden at some point, have much higher levels of effec- tiveness. This is the case of Norway, Denmark or Finland, all of which are top performers, as mentioned above. The United Kingdom also followed a strategy similar to that of Sweden in the first stage and, although it became stricter over time, it has often been more lax than other countries. The result has not been so positive, ranking 36th , with a value of 0.801. [Table 6 about here] The results per country in Table 6 reveal three large groups of countries. First, a select group of countries shows very high levels of effectiveness in the management of the pan- demic, whether or not their contextual variables are taken into account. These 15 countries are 19 Australia, Denmark, Estonia, Finland, Iceland, Israel, Luxembourg, Malaysia, New Zealand, Nigeria, Norway, Sri Lanka, Uruguay, Thailand and Ireland. The second group includes the countries whose performance is ineffective after an initial evaluation (not considering contextual factors), but when compared with countries with similar characteristics (especially long-term structural factors) their evaluation improves considerably. It is striking that almost half of the countries in this group are European (15/31). The countries in this group are Ghana, Morocco, Senegal, Zambia, Japan, Kazakhstan, Pakistan, Saudi Arabia, South Korea, Turkey, Austria, Bulgaria, Germany, Greece, Hungary, Latvia, Lithuania, Malta, Netherlands, Portugal, Russia, Serbia, Slovakia, Spain, United Kingdom, Canada, Dominican Republic, United States, Chile, Colombia, and Paraguay. Consequently, these are countries whose governments, relative to others with similar levels of these background variables, have better managed the pandemic. The last group corresponds to countries with low levels of effectiveness in a first evalua- tion, harmed by contextual variables and that maintain their low levels of effectiveness, even though they are compared with countries with similar contexts. This accounts for an inade- quate performance of their governments. These 15 countries are: South Africa, Iran, Croatia, Italy, Romania, Slovenia, Ukraine, Argentina, Mexico, Switzerland, Sweden, Poland, the Czech Republic, the Philippines and Namibia. 4.2. Contextual effects The above insights suggest that at least some contextual variables have a significant effect on the effectiveness of COVID-19 pandemic management. It is therefore of particular interest to analyse which variables have a statistically significant effect, and, if so, what kind of impact. To this end, two nonparametric regressions were performed (Daraio and Simar, 2005, 2007b; Rogge et al., 2017) following a similar approach to De Witte and Schiltz (2018) of gradually introducing s the contextual variables.21 In regression 1 (see Table 7), the Qz ratio was used as the dependent variable and the structural environment variables as covariates. In regression 2, the Qz ratio was used as the dependent variable and all the contextual variables as independent variables. The significance test of the variables was estimated via a bootstrap with 1,000 iterations. [Table 7 about here] 21 Our approach also shares the same underpinnings, based on nonparametric regression, of one of the first contributions in the field which made an attempt to deal with similar issues (Balaguer-Coll et al., 2007). 20 Table 7 shows the significance and type of impact of each contextual factor on the dependent variable. In both regressions all contextual variables, except rule of law and population density, were found to be statistically significant at 10%. One of the advantages of this technique is that it graphically shows the gradient of each independent variable for its different values, which allowed us to determine that the marginal effects were different among the contextual vari- ables. The scatter plot of the gradients22 was calculated for the median in order to identify the marginal impact of each contextual variable on effectiveness. In both regressions, the human development index shows a favourable and significant effect. In other words, the countries with the highest income, educational level and life expectancy managed the pandemic more effectively, as was to be expected. Notably, and more directly related to the pandemic, countries with high levels of human development are probably at lower risk of previous pathologies and enjoy better levels of healthcare services. Additionally, very high levels of the index may be associated with high levels of GDP per capita where a higher percentage of jobs are in the formal economy, more financial support was received from governments, it was easier to implement teleworking, and many citizens were not forced to go out to work or to interact socially to obtain the minimum income to survive. For the individualism variable, the generalised behaviour is an inverted U-shape in both regressions. At the individual level, it has a positive effect on effectiveness. However, when a high level is reached, the effect becomes negative. The explanation for this phenomenon may be that higher values of this characteristic are related to higher levels of free-rider behaviour, while at not very high levels it probably contributed to more effective respect for regulations implemented to reduce social interaction in many countries. Long-term orientation has an unfavourable effect for most of its range and a significant impact on effectiveness. One explanation for this behaviour may be that the higher the long-term orientation, the more difficult it may be for a society to achieve the levels of intensity and commitment in the short term that are required to combat the pandemic. Finally, the marginal effect of the discretionary variables with respect to the structural vari- ables is obtained from regression 2. Rule of law has an unfavourable impact, although not significant, on the effectiveness of pandemic management. The severity of the measures taken (stringency) has an unfavourable impact on effectiveness. This result may be contradictory and unexpected. It is important to note that the effect of the measures taken is seen weeks after their application. As there is no consensus on exactly how long this period is, we chose not 22 Available upon request. 21 to introduce any time lag in the variable. Consequently, a different interpretation is needed. In particular, the suggestion in our results that the greater the stringency, the less effective the management should be interpreted as an effect and not so much as a cause. In other words, countries mainly increased the severity of their measures when pandemic incidence indicators showed high levels. Therefore, it seems that the predominant strategy among the countries analysed was to favour coexistence with the virus, in an attempt to achieve the delicate balance between health and the economic situation, with varying degrees of success. 5. Concluding remarks By February 2021, the number of COVID-19 cases had reached nearly 110 million worldwide, with approximately 2.5 million deaths, a human tragedy with enormous collateral damage to the economy and living conditions of the majority of the population. Governments attempted to address the pandemic from various angles, with varying de- grees of success. In this regard, while the literature on the consequences of COVID-19 and government responses from an economic standpoint is growing exponentially (Brodeur et al., 2021b), the number of initiatives from an operations research position, despite their potential, is relatively limited and has focused on measuring point targets from an efficiency perspective, with few contributions analysing the impact of country contextual factors on the effectiveness of pandemic management. We attempt to fill this gap by proposing a novel method based on nonparametric activity analysis techniques developed in several stages. In the first stage, we propose a model of unconditional benefit of doubt (directional BoD), based on the construc- tion of composite indicators that take into account expert judgment. We refine the model by proposing a conditional model operating in two stages, sequentially introducing structural and discretionary contextual variables (conditional directional BoD), since government actions may be strongly conditioned by factors beyond their control, at least in the short and medium term. We adopted this approach because we believe that to properly evaluate countries’ manage- ment of the pandemic, indicators that measure the phenomenon holistically should be used (Rose et al., 2021; Omberg and Tabarrok, 2022). Thus, a country that manages the pandemic ef- fectively is one that reports high results in terms of detection, control and treatment (Kennedy et al., 2021). Similarly, a flexible methodology is required to guide the evaluation in terms of weights, but within certain normative parameters set by experts, and to ensure a fair allocation of responsibilities of managers. 22 Based on the results, we conclude that effective pandemic management by countries is a multifaceted task that involves not only short-term management measures but also the strengthening of long-term governance and structural conditions that favour detection, control and treatment. In this sense, this study establishes that a country’s socioeconomic and cultural structural variables have a greater impact on the effectiveness of pandemic management than discretionary variables in the medium and short term. Indeed, given the sophistication of the models used, the results can be explored from multi- ple perspectives. According to the results of the unconditional model, in which contextual vari- ables are not taken into account, the region (continent) with the best pandemic management is Oceania, followed by Asia. In contrast, North America is in last place, followed by South America. These results are subject to further examination, particularly at the country level, as this assessment cannot be fully generalised. The inclusion of structural (non-controllable) contextual variables is revealing, as the assessment of some regions varies, particularly South America. This shows the negative impact of the disadvantageous situation of South American countries on the unconditional assessment that does not consider this aspect. The same out- come is seen, but less intensely, when the medium- and short-term contextual variables enter into the analysis. For example, there is no significant change in the evaluation for Europe. Even when all the contextual aspects are taken into account, the best evaluations of effectiveness are found in the countries of Oceania, followed by Asian countries. Europe lies in the middle of the table, above South America, but lower than Africa. North America is the worst performing region, considering its contextual restrictions. At the country level, three major groups can be seen according to their performance in pan- demic management: those that are highly effective (15 countries), those that become effective when considering contextual factors (31 countries) and those that maintain low levels of effec- tiveness, even when considering contextual variables (15 countries). This last group reflects the management weaknesses of their respective governments. In contrast, the nonparametric regression analysis corroborates the results obtained by Kavuran et al. (2023) and Zheng et al. (2022) that socioeconomic aspects such as the human development index, which takes into account aspects of life expectancy, education and per capita income, have a positive impact on a country’s performance in pandemic management. The study also concludes that the individualism of a society is not linearly related to the ef- fectiveness of a government, but rather has an inverted-U shape. That is, as a society’s individ- ualism increases, it has a positive effect on effectiveness, probably because it is easier to contain 23 social interaction and there is more respect for the restrictions that many countries introduced in this regard. However, if individualism is high in society, to the extent that it is exacerbated, it can encourage selfish behaviour in certain segments of the population. This finding deepens the understanding of the impact of cultural factors such as individualism/collectivism, in line with, for example, Feng et al. (2023), who concluded that people from more individualistic countries and regions were less likely to follow social distancing norms during the COVID-19 pandemic, or Chen and Biswas (2022), who found that highly individualistic countries were unsuccessful in limiting the number of deaths and confirmed cases of COVID-19 (as people from such cultures prioritise their freedom and privacy). Our findings also show that long-term orientation has a significant unfavourable impact on effectiveness, probably because the higher the long-term orientation, the more difficult it is for a society to achieve the levels of intensity and commitment needed to combat the pandemic in the short term. Along these lines, Chen and Biswas (2022) note that nations with a short-term orientation are more likely to implement hypervigilance and pandemic prevention measures. However, the nonparametric regression analysis performed to assess the importance of each contextual variable indicates that population density and the level of rule of law in a country are not shown to be significant factors in assessing pandemic management. Thus, the implications of our study are manifold. Not only should we highlight the po- tential usefulness of our results, in terms of providing benchmarks of best practices, and the relevance of contextual factors, but also the potential of operations research to address vari- ous problems related to health policy management. In times of crisis, it is clear that govern- ments must act quickly to introduce what can sometimes be unpopular short-term measures. However, this study shows that the effectiveness of a government’s management depends on socioeconomic and cultural aspects of its society and therefore the measures taken must be analysed in terms of how that culture and socioeconomic situation may affect the behaviour of individuals. There are limitations to assessing pandemic preparedness at the country level. The most obvious is the comparability of data at this level, given differences in reporting systems. These systems may vary not only between countries but also within countries, not only in what information is reported but also in its quality, depending on the degree of decentralisation— i.e. sub-national levels of government may report different information. The possible extensions to this study are multiple. One of the most interesting ones would be to reflect the debate on the (possible) trade-off between public health vis-à-vis economic 24 growth. Although several research initiatives have already addressed this issue (e.g., Antràs et al., 2023), the methods considered here would provide refreshing insights into the debate— not only because of the decline in GDP during the pandemic, but also because of the subsequent macroeconomic effects (e.g., high inflation levels or shortages of raw materials, among many others). 25 References Agasisti, T. and Soncin, M. (2021). Higher education in troubled times: on the impact of Covid-19 in Italy. Studies in Higher Education, 46(1):86–95. Ahmad, W. M. A. W., Zainon, W. M. N. W., Nawi, M. A. A., and Ghazali, F. M. M. (2020). Malaysia’s Effi- ciency in Dealing with COVID-19 Outbreaks Compared to Other Asian Countries by Using Stochastic Frontier Analysis (SFA). Annals of King Edward Medical University, 26(2):324–329. Ameye, S. A., Ojo, T. O., Adetunji, T. A., and Awoleye, M. O. (2022). Is there an association between COVID-19 mortality and Human development index? The case study of Nigeria and some selected countries. BMC Research Notes, 15(1):186. Andries, A. M., Ongena, S., and Sprincean, N. (2020). The COVID-19 Pandemic and Sovereign Bond Risk. Technical Report 20-42, Swiss Finance Institute Research Paper, Zurich. Antràs, P., Redding, S. J., and Rossi-Hansberg, E. (2023). Globalization and pandemics. American Eco- nomic Review, 113(4):939–981. Ascani, A., Faggian, A., and Montresor, S. (2021). The geography of COVID-19 and the structure of local economies: the case of Italy. Journal of Regional Science, 61(2):407–441. Auger, P., Devinney, T. M., and Louviere, J. J. (2007). Using best–worst scaling methodology to investigate consumer ethical beliefs across countries. Journal of Business Ethics, 70(3):299–326. Aum, S., Lee, S. Y. T., and Shin, Y. (2021). COVID-19 doesn’t need lockdowns to destroy jobs: The effect of local outbreaks in Korea. Labour Economics, 70:101993. Avery, C., Bossert, W., Clark, A., Ellison, G., and Ellison, S. F. (2020). Policy implications of models of the spread of coronavirus: Perspectives and opportunities for economists. Working Paper 27007, National Bureau of Economic Research. Aydin, N. and Yurdakul, G. (2020). Assessing countries’ performances against COVID-19 via WSIDEA and machine learning algorithms. Applied Soft Computing, 97:106792. Balaguer-Coll, M. T., Prior, D., and Tortosa-Ausina, E. (2007). On the determinants of local government performance: A two-stage nonparametric approach. European Economic Review, 51(2):425–451. Barnett, V., Lewis, T., et al. (1994). Outliers in Statistical Data, volume 3. Wiley, New York. Bianchi, F., Bianchi, G., and Song, D. (2020). The Long-Term Impact of the COVID-19 Unemployment Shock on Life Expectancy and Mortality Rates. Working Paper 28304, National Bureau of Economic Research. 26 Bignami-Van Assche, S. and Ghio, D. (2022). Comparing COVID-19 fatality across countries: a synthetic demographic indicator. Journal of Population Research, 39(4):513–525. Bol, D., Giani, M., Blais, A., and Loewen, P. J. (2021). The effect of COVID-19 lockdowns on political support: Some good news for democracy? European Journal of Political Research, 60(2):497–505. Bonaccorsi, A., Daraio, C., and Simar, L. (2006). Advanced indicators of productivity of universities: An application of robust nonparametric methods to Italian data. Scientometrics, 66(2):389–410. Bourdin, S., Jeanne, L., Nadou, F., and Noiret, G. (2021). Does lockdown work? A spatial analysis of the spread and concentration of Covid-19 in Italy. Regional Studies, 55(7):1182–1193. Bowen, H. P. and Moesen, W. (2011). Composite competitiveness indicators with endogenous versus predetermined weights: An application to the World Economic Forum’s global competitiveness index. Competitiveness Review: An International Business Journal, 21(2):129–151. Breitenbach, M. C., Ngobeni, V., and Aye, G. C. (2021). Global healthcare resource efficiency in the management of COVID-19 death and infection prevalence rates. Frontiers in Public Health, 9:638481. Brodeur, A., Clark, A. E., Fleche, S., and Powdthavee, N. (2021a). COVID-19, lockdowns and well-being: Evidence from Google Trends. Journal of Public Economics, 193:104346. Brodeur, A., Gray, D., Islam, A., and Bhuiyan, S. (2021b). A literature review of the economics of COVID-19. Journal of Economic Surveys, 35(4):1007–1044. Bădin, L., Daraio, C., and Simar, L. (2010). Optimal bandwidth selection for conditional efficiency measures: A data-driven approach. European Journal of Operational Research, 201(2):633–640. Cao, C., Li, N., and Liu, L. (2020a). Do national cultures matter in the containment of COVID-19? International Journal of Sociology and Social Policy, 40(9/10):939–961. Cao, Y., Hiyoshi, A., and Montgomery, S. (2020b). COVID-19 case-fatality rate and demographic and socioeconomic influencers: worldwide spatial regression analysis based on country-level data. BMJ open, 10(11):e043560. Cazals, C., Florens, J.-P., and Simar, L. (2002). Nonparametric frontier estimation: a robust approach. Journal of Econometrics, 106:1–25. Chambers, R. G., Chung, Y., and Färe, R. (1996). Benefit and distance functions. Journal of Economic Theory, 70(2):407–419. Chambers, R. G., Chung, Y., and Färe, R. (1998). Profit, directional distance functions, and Nerlovian efficiency. Journal of Optimization Theory and Applications, 98:351–364. 27 Chen, K., Pun, C. S., and Wong, H. Y. (2022). Efficient social distancing during the COVID-19 pandemic: Integrating economic and public health considerations. European Journal of Operational Research, forth- coming. Chen, Y. and Biswas, M. I. (2022). Impact of national culture on the severity of the COVID-19 pandemic. Current Psychology, pages 1–14. Cherchye, L., De Witte, K., Ooghe, E., and Nicaise, I. (2010). Efficiency and equity in private and public education: A nonparametric comparison. European Journal of Operational Research, 202(2):563–573. Cherchye, L., De Witte, K., and Perelman, S. (2019). A unified productivity-performance approach applied to secondary schools. Journal of the Operational Research Society, 70(9):1522–1537. Cherchye, L., Lovell, C. K., Moesen, W., and Van Puyenbroeck, T. (2007). One market, one number? A composite indicator assessment of EU internal market dynamics. European Economic Review, 51(3):749– 779. Cherchye, L., Moesen, W., Rogge, N., Van Puyenbroeck, T., Saisana, M., Saltelli, A., Liska, R., and Tarantola, S. (2008). Creating composite indicators with dea and robustness analysis: the case of the technology achievement index. Journal of the Operational Research Society, 59(2):239–251. Chung, Y. H., Färe, R., and Grosskopf, S. (1997). Productivity and undesirable outputs: a directional distance function approach. Journal of Environmental Management, 51(3):229–240. Cohen, S. (2003). Maximum difference scaling: improved measures of importance and preference for segmentation. In Sawtooth Software Conference Proceedings, pages 61–74, Fir St., Sequim, WA. Sawtooth Software, Inc. Cohen, S., Rossi, F. M., Caperchione, E., and Brusca, I. (2021). Debate: If not now, then when? Covid- 19 as an accelerator for public sector accrual accounting in Europe. Public Money & Management, 41(1):10–12. Connell, R. (2020). Covid-19/sociology. Journal of Sociology, 56(4):745–751. Cooper, W., Seiford, L., and Tone, K. (2007). Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software. Springer US, New York, 2 edition. Daraio, C. and Simar, L. (2005). Introducing environmental variables in nonparametric frontier models: A probabilistic approach. Journal of Productivity Analysis, 24(1):93–121. Daraio, C. and Simar, L. (2007a). Advanced Robust and Nonparametric Methods in Efficiency Analysis. Method- ology and Applications. Studies in Productivity and Efficiency. Springer, New York. Daraio, C. and Simar, L. (2007b). Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach. Journal of Productivity Analysis, 28(1):13–32. 28 De Witte, K. and Kortelainen, M. (2008). Blaming the exogenous environment? Conditional efficiency estimation with continuous and discrete environmental variables. Discussion papers 833, Center for Economic Studies, Leuven, Belgium. De Witte, K. and Kortelainen, M. (2013). What explains the performance of students in a heterogeneous environment? conditional efficiency estimation with continuous and discrete environmental variables. Applied Economics, 45(17):2401–2412. De Witte, K. and Rogge, N. (2011). Accounting for exogenous influences in performance evaluations of teachers. Economics of Education Review, 30(4):641–653. De Witte, K. and Schiltz, F. (2018). Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional benefit-of-the-doubt approach. European Journal of Operational Research, 267(3):1172–1181. Despotis, D. K. (2005). A reassessment of the human development index via data envelopment analysis. Journal of the Operational Research Society, 56(8):969–980. D’Inverno, G. and De Witte, K. (2020). Service level provision in municipalities: A flexible directional distance composite indicator. European Journal of Operational Research, 286(3):1129–1141. Doğan, M. İ., Özsoy, V. S., and Örkcü, H. H. (2021). Performance management of OECD countries on Covid-19 pandemic: A criticism using data envelopment analysis models. Journal of Facilities Management, 19(4):479–499. Dokhov, R. and Topnikov, M. (2021). Everyday mobility as a vulnerability marker: The uneven reaction to coronavirus lockdown in Russia. Environment and Planning A: Economy and Space, 53(4):612–615. Fan, G., Yang, Z., Lin, Q., Zhao, S., Yang, L., and He, D. (2021). Decreased case fatality rate of COVID-19 in the second wave: a study in 53 countries or regions. Transboundary and Emerging Diseases, 68(2):213– 215. Färe, R. and Grosskopf, S. (2000). Theory and application of directional distance functions. Journal of Productivity Analysis, 13(2):93–103. Farrell, M. J. (1957). The measurement of productive efficiency. Journal of the Royal Statistical Society, Ser.A,120:253–281. Favero, C. A., Ichino, A., and Rustichini, A. (2020). Restarting the economy while saving lives under Covid-19. Discussion Paper DP14664, CEPR, London. Feng, Z., Zou, K., and Savani, K. (2023). Cultural antecedents of virus transmission: Individualism is associated with lower compliance with social distancing rules during the COVID-19 pandemic. Journal of Personality and Social Psychology, 124(3):461. 29 Finn, A. and Louviere, J. J. (1992). Determining the appropriate response to evidence of public concern: The case of food safety. Journal of Public Policy & Marketing, 11(2):12–25. Fusco, E. (2015). Enhancing non-compensatory composite indicators: A directional proposal. European Journal of Operational Research, 242(2):620–630. Fusco, E., Vidoli, F., and Rogge, N. (2020). Spatial directional robust Benefit of the Doubt approach in presence of undesirable output: An application to Italian waste sector. Omega, 94:102053. Gaaloul, H. and Khalfallah, S. (2014). Application of the “benefit-of-the-doubt” approach for the con- struction of a digital access indicator: A revaluation of the “digital access index”. Social Indicators Research, 118(1):45–56. Giménez, V., Thieme, C., Prior, D., and Tortosa-Ausina, E. (2022). Evaluation and determinants of preschool effectiveness in Chile. Socio-Economic Planning Sciences, 81:100934. Glover, A., Heathcote, J., Krueger, D., and Rı́os-Rull, J.-V. (2020). Health versus Wealth: On the Dis- tributional Effects of Controlling a Pandemic. Working Paper 27046, National Bureau of Economic Research. Goodman, S. (2009). An international comparison of retail consumer wine choice. International Journal of Wine Business Research, 21(1):41–49. Goolsbee, A. and Syverson, C. (2021). Fear, lockdown, and diversion: Comparing drivers of pandemic economic decline 2020. Journal of Public Economics, 193:104311. Greco, S., Ishizaka, A., Tasiou, M., and Torrisi, G. (2019). On the methodological framework of composite indices: A review of the issues of weighting, aggregation, and robustness. Social Indicators Research, 141:61–94. Guardiola, J. and Picazo-Tadeo, A. J. (2014). Building weighted-domain composite indices of life satis- faction with data envelopment analysis. Social Indicators Research, 117(1):257–274. Hall, P., Li, Q., and Racine, J. S. (2007). Nonparametric estimation of regression functions in the presence of irrelevant regressors. The Review of Economics and Statistics, 89(4):784–789. Hamermesh, D. S. (2020). Life satisfaction, loneliness and togetherness, with an application to Covid-19 lock-downs. Review of Economics of the Household, 18(4):983–1000. Harris, J. E. (2022). Geospatial analysis of a COVID-19 outbreak at the University of Wisconsin–Madison: potential role of a cluster of local bars. Epidemiology & Infection, 150. Haushofer, J. and Metcalf, C. J. E. (2020). Which interventions work best in a pandemic? Science, 368(6495):1063–1065. 30 Hodzic, S., Ravselj, D., and Alibegovic, D. J. (2021). E-Government Effectiveness and Efficiency in EU-28 and COVID-19. Central European Public Administration Review, 19:159–180. Hofstede, G. (2001). Culture’s consequences. Sage, Thousand Oaks, CA, second edition. Hofstede, G. and Hofstede, G. J. (2005). Cultures and Organizations. Software of the Mind. McGraw-Hill, New York, second edition. Ibrahim, M. D., Binofai, F. A., and Mm Alshamsi, R. (2020). Pandemic response management framework based on efficiency of COVID-19 control and treatment. Future Virology, 15(12):801–816. Johnson, R. M. (2000). Understanding HB: an intuitive approach. Sawtooth Software Inc, Sequim, WA. Juergensen, J., Guimón, J., and Narula, R. (2020). European SMEs amidst the COVID-19 crisis: assessing impact and policy responses. Journal of Industrial and Business Economics, 47(3):499–510. Kaufmann, D., Kraay, A., and Mastruzzi, M. (2010). The worldwide governance indicators: methodology and analytical issues. Policy Research Working Paper Series 5430, The World Bank. Kavanagh, M. M. and Singh, R. (2020). Democracy, Capacity, and Coercion in Pandemic Response: COVID-19 in Comparative Political Perspective. Journal of Health Politics, Policy and Law, 45(6):997– 1012. Kavuran, G., Gökhan, Ş., and Yeroğlu, C. (2023). COVID-19 and human development: An approach for classification of HDI with deep CNN. Biomedical Signal Processing and Control, 81:104499. Kennedy, D. S., Vu, V., Ritchie, H., Bartlein, R., Rothschild, O., Bausch, D. G., Roser, M., and Seale, A. C. (2021). COVID-19: Identifying countries with indicators of success in responding to the outbreak. Gates Open Research, 4:62. Klumpp, M., Loske, D., and Bicciato, S. (2022). COVID-19 health policy evaluation: integrating health and economic perspectives with a data envelopment analysis approach. The European Journal of Health Economics, forthcoming. Lauer, J. A., Lovell, C. K., Murray, C. J., and Evans, D. B. (2004). World health system performance revisited: the impact of varying the relative importance of health system goals. BMC Health Services Research, 4(1):19. Lavigne, C., De Jaeger, S., and Rogge, N. (2019). Identifying the most relevant peers for benchmarking waste management performance: A conditional directional distance Benefit-of-the-Doubt approach. Waste Management, 89:418–429. Lee, J. A., Soutar, G. N., and Louviere, J. (2007). Measuring values using best-worst scaling: The LOV example. Psychology & Marketing, 24(12):1043–1058. 31 Lee, Y. H. and Lim, S. Y. R. (2022). Efficacy in COVID-19 management: the case of ASEAN. Transforming Government: People, Process and Policy, forthcoming. Li, Q. and Racine, J. S. (2007). Nonparametric Econometrics: Theory and Practice. Princeton University Press, Princeton and Oxford. Lupu, D. and Tiganasu, R. (2022). COVID-19 and the efficiency of health systems in Europe. Health Economics Review, 12(1):1–15. Maricic, M., Egea, J. A., and Jeremic, V. (2019). A hybrid enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) optimization approach for determining weights within composite indicators. Social Indicators Research, 144(2):497–537. Martı́nez-Córdoba, P.-J., Benito, B., and Garcı́a-Sánchez, I.-M. (2021). Efficiency in the governance of the Covid-19 pandemic: political and territorial factors. Globalization and Health, 17:1–13. Melyn, W. and Moesen, W. (1991). Towards a synthetic indicator of macroeconomic performance: un- equal weighting when limited information is available. Public Economics Research Papers, pages 1–24. Min, H., Lee, C. C., and Joo, S.-J. (2022). Assessing the efficiency of the Covid-19 control measures and public health policy in OECD countries from cultural perspectives. Benchmarking: An International Journal, 29(6):1781–1796. Morais, P. and Camanho, A. S. (2011). Evaluation of performance of European cities with the aim to promote quality of life improvements. Omega, 39(4):398–409. Mourad, N., Habib, A., and Tharwat, A. (2021). Appraising healthcare systems’ efficiency in facing covid-19 through data envelopment analysis. Decision Science Letters, 10(3):301–310. Mumpower, J. L. and Stewart, T. R. (1996). Expert judgement and expert disagreement. Thinking & Reasoning, 2(2–3):191–212. Murray, E. J. (2020). Epidemiology’s Time of Need: COVID-19 Calls for Epidemic-Related Economics. Journal of Economic Perspectives, 34(4):105–20. Nikolopoulos, K., Punia, S., Schäfers, A., Tsinopoulos, C., and Vasilakis, C. (2021). Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions. European Journal of Operational Research, 290(1):99–115. OECD-European Commission Joint Research Centre (2008). Handbook on Constructing Composite Indica- tors. Methodology and User Guide. OECD publishing, Paris. Oh, D.-h. (2010). A metafrontier approach for measuring an environmentally sensitive productivity growth index. Energy Economics, 32(1):146–157. 32 Omberg, R. T. and Tabarrok, A. (2022). Is it possible to prepare for a pandemic? Oxford Review of Economic Policy, 38(4):851–875. Ordu, M., Kirli Akin, H., and Demir, E. (2021). Healthcare systems and Covid19: lessons to be learnt from efficient countries. The International Journal of Health Planning and Management, 36(5):1476–1485. Pei, J., de Vries, G., and Zhang, M. (2022). International trade and Covid-19: City-level evidence from China’s lockdown policy. Journal of Regional Science, 62:670–695. Peiró-Palomino, J. and Picazo-Tadeo, A. J. (2018). OECD: One or many? Ranking countries with a composite well-being indicator. Social Indicators Research, 139(3):847–869. Picazo-Tadeo, A. J., Reig-Martinez, E., and Hernandez-Sancho, F. (2005). Directional distance functions and environmental regulation. Resource and Energy Economics, 27(2):131–142. Racine, J. S. and Li, Q. (2004). Nonparametric estimation of regression functions with both categorical and continuous data. Journal of Econometrics, 119(1):99–130. Rogge, N., De Jaeger, S., and Lavigne, C. (2017). Waste performance of NUTS 2-regions in the EU: A conditional directional distance benefit-of-the-doubt model. Ecological Economics, 139:19–32. Rose, S. M., Paterra, M., Isaac, C., Bell, J., Stucke, A., Hagens, A., Tyrrell, S., Guterbock, M., and Nuzzo, J. B. (2021). Analysing COVID-19 outcomes in the context of the 2019 Global Health Security (GHS) Index. BMJ Global Health, 6(12):e007581. Roser, M., Ritchie, H., Ortiz-Ospina, E., and Hasell, J. (2020). Coronavirus Pandemic (COVID-19). OurWorldInData.org, Published online at OurWorldInData.org. Retrieved from https://ourworldindata.org/coronavirus. Ruggiero, J. (1996). On the measurement of technical efficiency in the public sector. European Journal of Operational Research, 90(3):553–565. Salvador, C. E., Berg, M. K., Yu, Q., San Martin, A., and Kitayama, S. (2020). Relational mobility predicts faster spread of COVID-19: A 39-country study. Psychological Science, 31(10):1236–1244. Schlaegel, C., Gunkel, M., and Taras, V. (2023). COVID-19 and individual performance in global virtual teams: The role of self-regulation and individual cultural value orientations. Journal of Organizational Behavior, 44(1):102–131. Sharpe, W. F. (1998). The sharpe ratio. Streetwise–the Best of the Journal of Portfolio Management, 3:169–185. Simar, L. (2003). Detecting outliers in frontier models: A simple approach. Journal of Productivity Analysis, 20:391–424. 33 Sinimole, K. R. (2022). Healthcare efficiency and the best practices of health systems across the world during COVID-19 pandemic. International Journal of Healthcare Management, forthcoming. Stumbriene, D., Camanho, A. S., and Jakaitiene, A. (2019a). The performance of education systems in the light of Europe 2020 strategy. Annals of Operations Research, 288(2):577–608. Stumbriene, D., Camanho, A. S., and Jakaitiene, A. (2019b). The performance of education systems in the light of Europe 2020 strategy. Annals of Operations Research, pages 1–32. Su, E. C.-Y., Hsiao, C.-H., Chen, Y.-T., and Yu, S.-H. (2021). An examination of COVID-19 mitigation efficiency among 23 countries. Healthcare, 9(6):755. Sueyoshi, T. and Goto, M. (2010). Should the US clean air act include CO2 emission control? Examination by data envelopment analysis. Energy Policy, 38(10):5902–5911. Titl, V. and De Witte, K. (2022). How politics influence public good provision. Socio-Economic Planning Sciences, 81:101000. Ueda, T. and Amatatsu, H. (2009). Determination of bounds in DEA assurance region method: Its application to evaluation of baseball players and chemical companies. Journal of the Operations Research Society of Japan, 52(4):453–467. Verschelde, M. and Rogge, N. (2012). An environment-adjusted evaluation of citizen satisfaction with local police effectiveness: Evidence from a conditional Data Envelopment Analysis approach. European Journal of Operational Research, 223(1):214–225. Waddock, S. A. and Graves, S. B. (1997). The corporate social performance–financial performance link. Strategic Management Journal, 18(4):303–319. Wang, H. (2015). A generalized MCDA–DEA (multi-criterion decision analysis–data envelopment anal- ysis) approach to construct slacks-based composite indicator. Energy, 80:114–122. Yoshikawa, H., Wuermli, A. J., Britto, P. R., Dreyer, B., Leckman, J. F., Lye, S. J., Ponguta, L. A., Richter, L. M., and Stein, A. (2020). Effects of the global coronavirus disease-2019 pandemic on early childhood development: short-and long-term risks and mitigating program and policy actions. The Journal of Pediatrics, 223:188–193. Zanella, A., Camanho, A., and Dias, T. G. (2013). Benchmarking countries’ environmental performance. Journal of the Operational Research Society, 64(3):426–438. Zheng, T., Zhang, C., Shi, Y., Chen, D., and Liu, S. (2022). Influencing Factors and Clustering Character- istics of COVID-19: A Global Analysis. Big Data Mining and Analytics, 5(4):318–338. 34 Table 1: Descriptive statistics Mean Std.dev. Min. 25% percentile Median 75% percentile Max. Cumulative performed testsa 288,330.24 332,886.63 4,488.29 69,236.06 247,180.24 354,273.79 2,131,356.47 Positivity rate (confirmed cases/tests performed) 0.09 0.09 0.00 0.03 0.07 0.12 0.43 Cumulative confirmed COVID-19 casesa 17,298.99 13,687.85 58.84 4,088.28 14,108.62 27,512.41 50,295.86 Cumulative confirmed COVID-19 deathsa 339.40 315.37 0.86 70.03 268.35 550.11 1,409.80 Case fatality rate (total deaths/total cases) 0.02 0.01 0.00 0.01 0.02 0.02 0.09 a Per million inhabitants. Table 2: Results of the survey, descriptive statistics Mean Std.dev Min. Median Max. Lower 95.0% CL (median) Upper 95.0% CL (median) Cumulative performed testsa 10.87 12.60 0.25 5.11 46.68 3.26 8.33 Positivity rate (total confirmed cases/total performed tests) 14.51 16.24 0.15 7.03 54.57 5.29 10.63 35 Cumulative confirmed COVID-19 casesa 19.39 13.28 1.26 16.11 46.15 14.21 20.15 Cumulative confirmed COVID-19 deathsa 32.13 14.07 1.79 37.76 51.99 32.68 40.21 Case fatality rate (total deaths/total cases) 23.10 15.97 0.46 21.48 51.06 17.06 26.63 a Per million inhabitants. Table 3: Contextual factors, descriptive statistics Mean Std. Dev. Min. 25% percentile Median 75% Percentile Max. Population density 145.90 209.75 3.08 24.72 83.48 209.59 1454.04 Human development index 0.82 0.11 0.51 0.77 0.85 0.91 0.95 Individualism 47.32 22.18 12.00 30.00 46.00 67.00 91.00 Long-term orientation 46.22 23.38 3.53 27.96 42.90 63.98 100.00 Rule of law 0.62 0.90 -0.90 -0.19 0.56 1.46 2.02 Stringency mean 59.96 11.19 36.21 52.51 58.49 67.23 88.10 Table 4: Effectiveness models Context variable Type ˜ (yoj , boh ) CI ˜ (yoj , boh |zos ) CI ˜ (yoj , boh |zo ) CI Population density Structural X X Human development index Structural X X Individualism Structural X X Long-term orientation Structural X X Rule of law Discretionary X Stringency Discretionary X Table 5: Effectiveness results by continent and subcontinent s d Continent ˜ (yoj , boh ) CI ˜ (yoj , boh |zos ) CI ˜ (yoj , boh |zo ) CI Q̃z Q̃z Q̃z Africa 0.843 0.939 0.974 0.902 0.965 0.869 Asia 0.880 0.963 0.991 0.916 0.970 0.887 Europe 0.837 0.964 0.963 0.870 1.000 0.870 North America 0.782 0.916 0.935 0.862 0.981 0.846 Oceania 1.000 1.000 1.000 1.000 1.000 1.000 South America 0.810 0.950 0.955 0.856 0.994 0.850 Total general 0.846 0.958 0.969 0.885 0.989 0.874 36 Table 6: Effectiveness results by country s d Continent Continent/subcontinent ˜ (yoj , boh ) CI ˜ (yoj , boh |zos ) CI ˜ (yoj , boh |zo ) CI Q̃z Q̃z Q̃z Argentina South America 0.738 0.787 0.810 0.937 0.972 0.911 Australia Oceania 1.000 1.000 1.000 1.000 1.000 1.000 Austria Europe 0.785 1.000 0.999 0.785 1.001 0.786 Bulgaria Europe 0.741 1.005 1.034 0.736 0.972 0.716 Canada North America 0.811 0.969 0.998 0.837 0.971 0.812 Chile South America 0.740 0.964 0.964 0.768 1.000 0.768 Colombia South America 0.741 1.000 1.000 0.741 1.000 0.741 Croatia Europe 0.810 0.928 0.940 0.872 0.987 0.861 Czech Rep. Europe 0.746 0.790 0.790 0.945 1.000 0.945 Denmark Europe 1.001 1.000 1.000 1.001 1.000 1.001 Dominican Rep. North America 0.806 0.966 1.000 0.834 0.966 0.806 Estonia Europe 1.000 1.000 1.000 1.000 1.000 1.000 Finland Europe 1.001 1.000 1.000 1.001 1.000 1.001 Germany Europe 0.828 0.940 0.956 0.880 0.984 0.866 Ghana Africa 0.908 1.000 1.000 0.908 1.000 0.908 Greece Europe 0.814 1.000 1.000 0.814 1.000 0.814 Hungary Europe 0.746 0.991 0.987 0.752 1.004 0.755 Iceland Europe 1.001 1.000 1.000 1.001 1.000 1.001 Iran Asia 0.728 0.765 0.905 0.951 0.846 0.804 Ireland Europe 0.952 0.952 0.952 1.000 1.000 1.000 Israel Asia 1.000 1.000 1.000 1.000 1.000 1.000 Italy Europe 0.791 0.936 0.946 0.845 0.989 0.835 Japan Asia 0.882 1.000 1.000 0.882 1.000 0.882 Kazakhstan Asia 0.839 1.000 1.000 0.839 1.000 0.839 Latvia Europe 0.879 1.000 0.999 0.879 1.001 0.880 Lithuania Europe 0.804 1.000 1.000 0.804 1.000 0.804 Luxembourg Europe 1.001 1.000 1.000 1.001 1.000 1.001 Malaysia Asia 1.000 1.000 1.000 1.000 1.000 1.000 Malta Europe 0.869 1.000 1.000 0.869 1.000 0.869 Mexico North America 0.727 0.731 0.742 0.995 0.986 0.980 Morocco Africa 0.798 0.999 1.000 0.799 0.999 0.798 Namibia Africa 0.857 0.857 0.857 1.000 1.000 1.000 Netherlands Europe 0.773 0.996 0.998 0.776 0.998 0.775 New Zealand Oceania 1.000 1.000 1.000 1.000 1.000 1.000 Nigeria Africa 0.988 1.000 1.000 0.988 1.000 0.988 Norway Europe 1.001 1.000 1.000 1.001 1.000 1.001 Pakistan Asia 0.810 1.000 1.000 0.810 1.000 0.810 Paraguay South America 0.771 1.000 1.000 0.771 1.000 0.771 Philippines Asia 0.763 0.791 0.990 0.964 0.799 0.770 Poland Europe 0.765 0.779 0.824 0.983 0.945 0.929 Portugal Europe 0.783 1.000 1.000 0.783 1.000 0.783 Romania Europe 0.744 0.904 0.894 0.823 1.011 0.832 Russia Europe 0.870 1.000 1.000 0.870 1.000 0.870 Saudi Arabia Asia 0.807 1.000 1.000 0.807 1.000 0.807 Senegal Africa 0.805 0.948 0.961 0.848 0.987 0.838 Serbia Europe 0.791 1.000 1.000 0.791 1.000 0.791 Slovakia Europe 0.870 1.055 1.030 0.825 1.024 0.845 Slovenia Europe 0.748 0.931 0.927 0.804 1.004 0.807 South Africa Africa 0.739 0.769 1.000 0.961 0.769 0.739 South Korea Asia 0.877 1.000 1.000 0.877 1.000 0.877 Spain Europe 0.754 1.000 1.000 0.754 1.000 0.754 Sri Lanka Asia 1.000 1.000 1.000 1.000 1.000 1.000 Sweden Europe 0.729 0.889 0.799 0.820 1.113 0.912 Switzerland Europe 0.758 0.874 0.890 0.867 0.983 0.852 Thailand Asia 1.000 1.000 1.000 1.000 1.000 1.000 Turkey Asia 0.860 1.000 1.000 0.860 1.000 0.860 Ukraine Europe 0.784 0.898 0.898 0.873 1.000 0.873 United Kingdom Europe 0.801 1.000 1.000 0.801 1.000 0.801 United States North America 0.784 1.000 1.000 0.784 1.000 0.784 Zambia Africa 0.810 1.000 1.000 0.810 1.000 0.810 Uruguay South America 1.062 1.000 1.000 1.062 1.000 1.062 Mean 0.846 0.958 0.969 0.885 0.989 0.874 Std Dev 0.099 0.076 0.064 0.091 0.047 0.093 Max 1.062 1.055 1.034 1.062 1.113 1.062 Min 0.727 0.731 0.742 0.736 0.769 0.716 # super-efficient 8 2 2 37 Table 7: Nonparametric regression results s Regression 1 (Q̃z ) Regression 2 (Q̃z ) Variable p-valuea Impact p-valuea Impact Population density 0.122 (-) 0.112 (-) Human development index 0.098 Favourable 0.095 Favourable Individualism 0.008 Inverted U-shape 0.033 Inverted U-shape Long-term orientation 0.035 Unfavourable <2e-16 Unfavourable Rule of law 0.282 (-) Stringency 0.006 Unfavourable R2 0.308 0.448 a p -values in bold indicate significance at least at the 10% level. 38 Figure 1: Example of one of the survey questions 39