Eur. Phys. J. C (2019) 79:970 https://doi.org/10.1140/epjc/s10052-019-7450-8 Regular Article - Experimental Physics ATLAS b-jet identification performance and efficiency √ measurement with t t̄ events in pp collisions at s = 13 TeV ATLAS Collaboration CERN, 1211 Geneva 23, Switzerland Received: 12 July 2019 / Accepted: 31 October 2019 / Published online: 25 November 2019 © CERN for the benefit of the ATLAS collaboration 2019 Abstract The algorithms used by the ATLAS Collabora- 6 Event selection and classification . . . . . . . . . . 10 tion during Run 2 of the Large Hadron Collider to identify 6.1 Lepton object definition . . . . . . . . . . . . 10 jets containing b-hadrons are presented. The performance of 6.2 Event selection . . . . . . . . . . . . . . . . 10 the algorithms is evaluated in the simulation and the effi- 6.3 Event classification . . . . . . . . . . . . . . 12 ciency with which these algorithms identify jets containing 7 Extraction of b-jet tagging efficiency . . . . . . . . 12 b-hadrons is measured in collision data. The measurement 8 Uncertainties . . . . . . . . . . . . . . . . . . . . 14 uses a likelihood-based method in a sample highly enriched 9 Results . . . . . . . . . . . . . . . . . . . . . . . 16 in t t¯ events. The topology of the t → W b decays is exploited 10 Usage in ATLAS analysis . . . . . . . . . . . . . . 17 to simultaneously measure both the jet flavour composition 10.1 Smoothing . . . . . . . . . . . . . . . . . . . 19 of the sample and the efficiency in a transverse momentum 10.2 Extrapolation to high- pT jets . . . . . . . . . 19 range from 20 to 600 GeV. The efficiency measurement is 10.3 Generator dependence . . . . . . . . . . . . . 20 subsequently compared with that predicted by the simula- 10.4 Reduction of systematic uncertainties . . . . 20 tion. The data used in this measurement, corresponding to 10.5 Application to physics analyses . . . . . . . . 20 a total integrated luminosity of 80.5 fb−1 , were collected 11 Conclusion . . . . . . . . . . . . . . . . . . . . . 21 in proton–proton collisions during the years 2015–2017 at References . . . . . . . . . . . . . . . . . . . . . . . . 22 √ a centre-of-mass energy s = 13 TeV. By simultaneously extracting both the efficiency and jet flavour composition, this measurement significantly improves the precision com- 1 Introduction pared to previous results, with uncertainties ranging from 1 to 8% depending on the jet transverse momentum. The identification of jets containing b-hadrons (b-jets) against the large jet background containing c-hadrons but Contents no b-hadron (c-jets) or containing neither b- or c-hadrons (light-flavour jets) is of major importance in many areas of 1 Introduction . . . . . . . . . . . . . . . . . . . . . 1 the physics programme of the ATLAS experiment [1] at the 2 ATLAS detector . . . . . . . . . . . . . . . . . . . 2 Large Hadron Collider (LHC) [2]. It has been decisive in the 3 Key ingredients for b-jet identification . . . . . . . 3 recent observations of the Higgs boson decay into bottom 4 Algorithms for b-jet identification . . . . . . . . . 4 quarks [3] and of its production in association with a top- 4.1 Training and tuning samples . . . . . . . . . 4 quark pair [4], and plays a crucial role in a large number of 4.2 Low-level b-tagging algorithms . . . . . . . . 4 Standard Model (SM) precision measurements, studies of the 4.2.1 Algorithms based on impact parameters 4 Higgs boson properties, and searches for new phenomena. 4.2.2 Secondary vertex finding algorithm . . 5 The ATLAS Collaboration uses various algorithms to 4.2.3 Topological multi-vertex finding algo- identify b-jets [5], referred to as b-tagging algorithms, when rithm . . . . . . . . . . . . . . . . . . 5 analysing data recorded during Run 2 of the LHC (2015– 4.3 High-level tagging algorithms . . . . . . . . 5 2018). These algorithms exploit the long lifetime, high mass 4.3.1 MV2 . . . . . . . . . . . . . . . . . . 5 and high decay multiplicity of b-hadrons as well as the prop- 4.3.2 DL1 . . . . . . . . . . . . . . . . . . 6 erties of the b-quark fragmentation. Given a lifetime of the 4.4 Algorithm performance . . . . . . . . . . . . 7 order of 1.5 ps (< cτ >≈ 450 µm), measurable b-hadrons 5 Data and simulated samples . . . . . . . . . . . . 9 have a significant mean flight length, < l >= βγ cτ , in the  e-mail: atlas.publications@cern.ch detector before decaying, generally leading to at least one 123 970 Page 2 of 36 Eur. Phys. J. C (2019) 79:970 vertex displaced from the hard-scatter collision point. The are described in Sect. 5. The event selection and classifi- strategy developed by the ATLAS Collaboration is based on cation performed for the measurement of the b-jet tagging a two-stage approach. Firstly, low-level algorithms recon- efficiency are summarised in Sect. 6. The measurement tech- struct the characteristic features of the b-jets via two com- nique is presented in Sect. 7 and the sources of uncertainties plementary approaches, one that uses the individual prop- are described in Sect. 8. The results and their usage within erties of charged-particle tracks, later referred to as tracks, the ATLAS Collaboration are discussed in Sects. 9 and 10, associated with a hadronic jet, and a second which com- respectively. bines the tracks to explicitly reconstruct displaced vertices. These algorithms, first introduced during Run 1 [5], have been improved and retuned for Run 2. Secondly, in order 2 ATLAS detector to maximise the b-tagging performance, the results of the low-level b-tagging algorithms are combined in high-level The ATLAS detector [1] at the LHC covers nearly the entire algorithms consisting of multivariate classifiers. The perfor- solid angle around the collision point. It consists of an mance of a b-tagging algorithm is characterised by the prob- inner tracking detector (ID) surrounded by a superconduct- ability of tagging a b-jet (b-jet tagging efficiency, εb ) and the ing solenoid, electromagnetic and hadronic calorimeters and probability of mistakenly identifying a c-jet or a light-flavour a muon spectrometer incorporating three large superconduct- jet as a b-jet, labelled εc (εl ). In this paper, the performance of ing toroid magnets. the algorithms is quantified in terms of c-jet and light-flavour The ID consists in a high-granularity silicon pixel detector jet rejections, defined as 1/εc and 1/εl , respectively. which covers the vertex region and typically provides four The imperfect description of the detector response and measurements per track. The innermost layer, known as the physics modelling effects in Monte Carlo (MC) simula- insertable B-layer (IBL) [9], was added in 2014 and provides tions necessitates the measurement of the performance of the high-resolution hits at small radius to improve the tracking b-tagging algorithms with collision data [6–8]. In this paper, performance. For a fixed b-jet efficiency, the incorporation the measurement of the b-jet tagging efficiency of the high- of the IBL improves the light-flavour jet rejection of the b- level b-tagging algorithms used in proton–proton ( pp) colli- tagging algorithms by up to a factor of four [10]. The sili- √ sion data recorded during Run 2 of the LHC at s = 13 TeV con pixel detector is followed by a silicon microstrip tracker is presented. The corresponding measurements for c-jets and (SCT) that typically provides eight measurements from four light-flavour jets, used in the measurement of the b-jet tag- strip double layers. These silicon detectors are complemented ging efficiency to correct the simulation such that the overall by a transition radiation tracker (TRT), which enables radi- tagging efficiency of c-jets and light-flavour jets match that ally extended track reconstruction up to the pseudorapidity1 of the data, are described elsewhere [7,8]. The production of |η| = 2.0. The TRT also provides electron identification t t¯ pairs at the LHC provides an abundant source of b-jets by information based on the fraction of hits (typically 33 in the virtue of the high cross-section and the t → W b branching barrel and up to an average of 38 in the endcaps) above a ratio being close to 100%. A very pure sample of t t¯ events is higher energy-deposit threshold corresponding to transition selected by requiring that both W bosons decay leptonically, radiation. The ID is immersed in a 2 T axial magnetic field referred to as dileptonic t t¯ decays in the following. A com- and provides charged-particle tracking in the pseudorapidity binatorial likelihood approach is used to measure the b-jet range |η| < 2.5. tagging efficiency of the high-level b-tagging algorithms as The calorimeter system covers the pseudorapidity range a function of the jet transverse momentum ( pT ). This version |η| < 4.9. Within the region |η| < 3.2, electromag- of the analysis builds upon the approach used previously by netic calorimetry is provided by barrel and endcap high- the ATLAS Collaboration [6], extending the method to derive granularity lead/liquid-argon (LAr) sampling calorimeters, additional constraints on the flavour composition of the sam- with an additional thin LAr presampler covering |η| < 1.8 to ple, which reduces the uncertainties by up to a factor of two correct for energy loss in material upstream of the calorime- relative to previous publication. ters. Hadronic calorimetry is provided by a steel/scintillator- The paper is organised as follows. In Sect. 2, the ATLAS tile calorimeter, segmented into three barrel structures within detector is described. Section 3 contains a description of the objects reconstructed in the detector which are key ingre- 1 ATLAS uses a right-handed coordinate system with its origin at the dients for b-tagging algorithms, while Sect. 4 describes the nominal interaction point (IP) in the centre of the detector and the z- b-tagging algorithms and the evaluation of their performance axis along the beam pipe. The x-axis points from the IP to the centre of in the simulation. The second part of the paper focuses on the LHC ring, and the y-axis points upwards. Cylindrical coordinates (r, φ) are used in the transverse plane, φ being the azimuthal angle the b-jet tagging efficiency measurement carried out in col- around the z-axis. The pseudorapidity is defined in terms of the polar lision data and the application of these results in ATLAS angle θas η = − ln tan(θ/2). Angular distance is measured in units of analyses. The data and simulated samples used in this work R ≡ ( η)2 + ( φ)2 . 123 Eur. Phys. J. C (2019) 79:970 Page 3 of 36 970 |η| < 1.7, and two copper/LAr hadronic endcap calorime- with the PV that has the highest sum of squared trans- ters. The solid angle coverage is completed with forward cop- verse momenta of contributing tracks selected as the pri- per/LAr and tungsten/LAr calorimeter modules optimised for mary interaction point. Displaced charged-particle tracks electromagnetic and hadronic measurements, respectively. originating from b-hadron decays can then be selected The muon spectrometer comprises separate trigger and by requiring large transverse and longitudinal impact high-precision tracking chambers measuring the deflection of parameters, IPr φ = |d0 | and IPz = |z 0 sin θ |, respec- muons in a magnetic field generated by superconducting air- tively, where d0 (z 0 ) represents the transverse (longitudi- core toroids. The precision chamber system covers the region nal) perigee parameter defined at the point of the closest |η| < 2.7 with three layers of monitored drift tubes, com- approach of the trajectory to the z-axis [16]. Upper limits plemented by cathode-strip chambers in the forward region. on the values of these parameters are used to reduce the The muon trigger system covers the range |η| < 2.4 with contamination from pile-up, secondary and fake tracks. resistive-plate chambers in the barrel and thin-gap chambers Hadronic jets are built from topological clusters of energy in the endcap regions. in the calorimeter [17], calibrated to the electromagnetic A two-level trigger system [11] is used to select interest- scale, using the anti-kt algorithm with radius parameter ing events. The first level of the trigger is implemented in R = 0.4 [18]. Jet transverse momenta are further cor- hardware and uses a subset of detector information to reduce rected to the corresponding particle-level jet pT , based the event rate to a design value of at most 100 kHz. It is fol- on the simulation [19]. Remaining differences between lowed by a software-based trigger that reduces the event rate data and simulated events are evaluated and corrected to a maximum of around 1 kHz for offline storage. for using in situ techniques, which exploit the trans- verse momentum balance between a jet and a reference object such as a photon, Z boson, or multi-jet system 3 Key ingredients for b-jet identification in data. After these calibrations, all jets in the event with pT ≥ 20 GeV and |η| < 4.5 must satisfy a set The identification of b-jets is based on several objects recon- of loose jet-quality requirements [20], else the event is structed in the ATLAS detector which are described here. discarded. These requirements are designed to reject fake jets originating from sporadic bursts, large coherent noise Tracks are reconstructed in the ID [12]. The b-tagging algo- or isolated pathological cells in the calorimeter system, rithms only consider tracks with pT larger than 500 MeV, hardware issues, beam-induced background or cosmic- with further selection criteria applied to reject fake and ray muons. Jets with pT < 20 GeV or |η| ≥ 2.5 are then poorly measured tracks [13]. The combined efficiency discarded. In order to reduce the number of jets with large of the track reconstruction and selection criteria, evalu- energy fractions from pile-up collision vertices, the Jet ated by using minimum-bias simulated events (in which Vertex Tagger (JVT) algorithm is used [21]. The JVT more than 98% of charged-particles are pions), ranges procedure builds a multivariate discriminant for each jet from 91% in the central (|η| < 0.1) region to 73% in the within |η| < 2.5 based on the ID tracks ghost-associated forward (2.3 ≤ |η| < 2.5) region of the detector. Addi- with the jet [22]; in particular, jets with a large fraction tional selections on reconstructed tracks are applied in of high-momentum tracks from pile-up vertices are less the low-level b-tagging algorithms described in Sect. 4.2 likely to pass the JVT requirement. The rate of pile-up to specifically select b- and c-hadron decay track candi- jets with pT ≥ 120 GeV is sufficiently small that the JVT dates and improve the rejection of tracks originating from requirement is removed above this threshold. The JVT pile-up.2 efficiency for jets originating from the hard pp scatter- Primary vertex (PV) reconstruction [14,15] on an event-by- ing is 92% in the simulation. Scale factors of order unity event basis is particularly important for b-tagging since are applied to account for efficiency differences relative it defines the reference point from which track and ver- to collision data. tex displacements are computed. A longitudinal vertex Track–jet matching is performed using the angular sep- position resolution of about 30 µm is achieved for events aration R between the track momenta, defined at the with a high multiplicity of reconstructed tracks, while the perigee, and the jet axis, defined as the vectorial sum of transverse resolution ranges from 10 to 12 µm, depend- the four-momenta of the clusters associated with the jet. ing on the LHC running conditions that determine the Given that the decay products from higher- pT b-hadrons beam spot size. At least one PV is required in each event, are more collimated, the R requirement varies as a func- tion of jet pT , being wider for low- pT jets (0.45 for jet 2 pT = 20 GeV) and narrower for high- pT jets (0.26 for jet Pile-up interactions correspond to additional pp collisions accompa- nying the hard-scatter pp interaction in proton bunch collisions at the pT = 150 GeV). If more than one jet fulfils the matching LHC. criteria, the closest jet is preferred. The jet axis is also 123 970 Page 4 of 36 Eur. Phys. J. C (2019) 79:970 used to assign signed impact parameters to tracks, where tribution of the jets produced in the hadronic decays up to a the sign is defined as positive if the track intersects the jet jet pT of 1.5 TeV. The branching fractions of the decays are axis in the transverse plane in front of the primary vertex, set to be one-third each for the bb, cc and light-flavour quark and as negative if the intersection lies behind the primary pairs to give a pT spectrum uniformly populated by jets of all vertex. flavours. The hybrid sample is obtained by selecting b-jets Jet flavour labels are attributed to the jets in the simulation. from t t¯ events if the corresponding b-hadron pT is below Jets are labelled as b-jets if they are matched to at least one 250 GeV and from the Z  sample if above, with a similar weakly decaying b-hadron having pT ≥ 5 GeV within a strategy applied for c-jets and light-flavour jets. More details cone of size R = 0.3 around the jet axis. If no b-hadrons about the production of the t t¯ simulated sample, referred to are found, c-hadrons and then τ -leptons are searched for, as the baseline t t¯ sample in the following, and the Z  simu- based on the same selection criteria. The jets matched to lated sample are given in Sect. 5. Events with at least one jet a c-hadron (τ -lepton) are labelled as c-jets (τ -jets). The are selected, excluding the jets overlapping with a generator- remaining jets are labelled as light-flavour jets. level electron originating from a W - or Z -boson decay. 4.2 Low-level b-tagging algorithms 4 Algorithms for b-jet identification 4.2.1 Algorithms based on impact parameters This section describes the different algorithms used for b-jet identification and the evaluation of their performance in sim- There are two complementary impact parameter-based algo- ulation. Low-level b-tagging algorithms fall into two broad rithms, IP2D and IP3D [23]. The IP2D tagger makes use categories. A first approach, implemented in the IP2D and of the signed transverse impact parameter significance of IP3D algorithms [23] and described in Sect. 4.2.1, is inclu- tracks to construct a discriminating variable, whereas IP3D sive and based on exploiting the large impact parameters of uses both the track signed transverse and the longitudinal the tracks originating from the b-hadron decay. The second impact parameter significance in a two-dimensional template approach explicitly reconstructs displaced vertices. The SV1 to account for their correlation. Probability density func- algorithm [24], discussed in Sect. 4.2.2, attempts to recon- tions (pdf) obtained from reference histograms of the signed struct an inclusive secondary vertex, while the JetFitter transverse and longitudinal impact parameter significances of algorithm [25], presented in Sect. 4.2.3, aims to reconstruct tracks associated with b-jets, c-jets and light-flavour jets are the full b- to c-hadron decay chain. These algorithms, first derived from MC simulation. The pdfs are computed in exclu- introduced during Run 1 [5], benefit from improvements and sive categories that depend on the hit pattern of the tracks to a new tuning for Run 2. To maximise the b-tagging per- increase the discriminating power. The pdfs are used to calcu- formance, low-level algorithm results are combined using late ratios of the b-jet, c-jet and light-flavour jet probabilities multivariate classifiers. To this end, two high-level tagging on a per-track basis. Log-likelihood ratio (LLR) discrimi- algorithms have been developed. The first one, MV2 [23], is nants are then defined as the sum of the per-track  N probability based on a boosted decision tree (BDT) discriminant, while ratios for each jet-flavour hypothesis, e.g. i=1 log ( pb / pu ) the second one, DL1 [23], is based on a deep feed-forward for the b-jet and light-flavour jet hypotheses, where N is the neural network (NN). These two algorithms are presented in number of tracks associated with the jet and pb ( pu ) is the Sects. 4.3.1 and 4.3.2, respectively. template pdf for the b-jet (light-flavour jet) hypothesis. The flavour probabilities of the different tracks contributing to the 4.1 Training and tuning samples sum are assumed to be independent of each other. In addition to the LLR separating b-jets from light-flavour jets, two extra The new tuning and training strategies of the b-tagging algo- LLR functions are defined to separate b-jets from c-jets and rithms for Run 2 are based on the use of a hybrid sample c-jets from light-flavour jets, respectively. These three like- composed of t t¯ and Z  simulated events. Only t t¯ decays with lihood discriminants for both the IP2D and IP3D algorithms at least one lepton from a subsequent W -boson decay are are used as inputs to the high-level taggers. considered in order to ensure a sufficiently large fraction of Both the IP2D and IP3D algorithms benefited from a com- c-jets in the event whilst maintaining a jet multiplicity pro- plete retuning prior to the 2017–2018 ATLAS data taking file similar to that in most analyses. A dedicated sample of period [23]. In particular, a reoptimisation of the track cat- Z  decaying into hadronic jet pairs is included to optimise egory definitions was performed, allowing the IBL hit pat- the b-tagging performance at high jet pT . The cross-section tern expectations and the next innermost layer information of the hard-scattering process is modified by applying an to be fully exploited. The rejection of tracks originating from event-by-event weighting factor to broaden the natural width photon conversions, long-lived particles decays (K s , ) and of the resonance and widen the transverse momentum dis- interactions with detector material by the secondary vertex 123 Eur. Phys. J. C (2019) 79:970 Page 5 of 36 970 algorithms has also been improved. An additional set of new built from three SV1 output variables: the vertex mass, the template pdfs was also produced using a 50%/50% mixture energy fraction and the number of two-track vertices. of t t¯ and Z  simulated events for extra track-categories with no hits in the first two layers, which are populated by long- 4.2.3 Topological multi-vertex finding algorithm lived b-hadrons traversing the first layers before they decay. The t t¯ sample is used to populate all remaining categories. The topological multi-vertex algorithm, JetFitter [25], exploits the topological structure of weak b- and c-hadron 4.2.2 Secondary vertex finding algorithm decays inside the jet and tries to reconstruct the full b-hadron decay chain. A modified Kalman filter [26] is used to find The secondary vertex tagging algorithm, SV1 [24], recon- a common line on which the primary, bottom and charm structs a single displaced secondary vertex in a jet. The recon- vertices lie, approximating the b-hadron flight path as well struction starts from identifying the possible two-track ver- as the vertex positions. With this approach, it is possible to tices built with all tracks associated with the jet, while reject- resolve the b- and c-hadron vertices even when a single track ing tracks that are compatible with the decay of long-lived is attached to them. particles (K s or ), photon conversions or hadronic inter- Several improvements [25], prior to the 2017-2018 ATLAS actions with the detector material. The SV1 algorithm runs data taking period, have been introduced in the current ver- iteratively on all tracks contributing to the cleaned two-tack sion of the JetFitter algorithm. These include, a reoptimi- vertices, trying to fit one secondary vertex. In each iteration, sation of the track selection to better mitigate the effect of the track-to-vertex association is evaluated using a χ 2 test. pile-up tracks, an improvement in the rejection of material The track with the largest χ 2 is removed and the vertex fit interactions, and the introduction of a vertex-mass dependent is repeated until an acceptable vertex χ 2 and a vertex invari- selection during the decay chain fit to increase the efficiency ant mass less than 6 GeV are obtained. With this approach, for tertiary vertex reconstruction. Eight discriminating vari- the decay products from b- and c-hadrons are assigned to a ables, including the track multiplicity at the JetFitter dis- single common secondary vertex. placed vertices, the invariant mass of tracks associated with Several refinements in the track and vertex selection were these vertices, their energy fraction and their average three- made prior to the 2016–2017 ATLAS data taking period to dimensional decay length significance, are used as inputs improve the performance of the algorithm, resulting in an to the high-level taggers. The b-tagging performance of the increased pile-up rejection and an overall enhancement of the JetFitter algorithm is evaluated using a LLR discriminant performance at high jet pT [24]. Among the various algorithm based on likelihood functions combining pdfs extracted from improvements, additional track-cleaning requirements are some of the JetFitter output variables (vertex mass, energy applied for jets in the high-pseudorapidity region (|η| ≥ 1.5) fraction and decay length significance) and parameterised for to mitigate the negative influence of the increasing amount of each of the three jet flavours. detector material on the secondary vertex finding efficiency. The fake-vertex rate is also better controlled by limiting the 4.3 High-level tagging algorithms algorithm to only consider the 25 highest- pT tracks in the jets, which preserves all reconstructed tracks from b-hadron 4.3.1 MV2 decays, whilst limiting the influence of additional tracks in the jet. The selection of two-track vertex candidates prior The MV2 algorithm [23] consists of a boosted decision to the χ 2 –fit was also reoptimised. Extra candidate-cleaning tree (BDT) algorithm that combines the outputs of the low- requirements were introduced to further reduce the number of level tagging algorithms described in Sect. 4.2 and listed fake vertices and material interactions, such as the rejection in Table 1. The BDT algorithm is trained using the ROOT of two-track vertex candidates with an invariant mass greater Toolkit for Multivariate Data Analysis (TMVA) [27] on the than 6 GeV, which are not likely to originate from b- and c- hybrid t t¯ + Z  sample. The kinematic properties of the jets, hadron decays. Eight discriminating variables, including the namely pT and |η|, are included in the training in order to number of tracks associated with the SV1 vertex, the invariant take advantage of the correlations with the other input vari- mass of the secondary vertex, its energy fraction (defined as ables. However, to avoid differences in the kinematic distri- the total energy of all the tracks associated with the secondary butions of signal (b-jets) and background (c-jets and light- vertex divided by the energy of all the tracks associated with flavour jets) being used to discriminate between the different the jet), and the three-dimensional decay length significance jet flavours, the b-jets and c-jets are reweighted in pT and are used as inputs to the high-level taggers. The b-tagging per- |η| to match the spectrum of the light-flavour jets. No kine- formance of the SV1 algorithm is evaluated using a LLR dis- matic reweighting is applied at the evaluation stage of the criminant based on pdfs for the b-jet, c-jet and light-flavour multivariate classifier. For training, the c-jet fraction in the jet hypotheses computed from three-dimensional histograms background sample is set to 7%, with the remainder com- 123 970 Page 6 of 36 Eur. Phys. J. C (2019) 79:970 Table 1 Input variables used by the MV2 and the DL1 algorithms. The JetFitter c-tagging variables are only used by the DL1 algorithm Input Variable Description Kinematics pT Jet pT η Jet |η| IP2D/IP3D log(Pb /Plight ) Likelihood ratio between the b-jet and light-flavour jet hypotheses log(Pb /Pc ) Likelihood ratio between the b- and c-jet hypotheses log(Pc /Plight ) Likelihood ratio between the c-jet and light-flavour jet hypotheses SV1 m(SV) Invariant mass of tracks at the secondary vertex assuming pion mass f E (SV) Energy fraction of the tracks associated with the secondary vertex NTrkAtVtx (SV) Number of tracks used in the secondary vertex N2TrkVtx (SV) Number of two-track vertex candidates L x y (SV) Transverse distance between the primary and secondary vertex L x yz (SV) Distance between the primary and the secondary vertex Sx yz (SV) Distance between the primary and the secondary vertex divided by its uncertainty R( pjet , pvtx )(SV) R between the jet axis and the direction of the secondary vertex relative to the primary vertex JetFitter m(JF) Invariant mass of tracks from displaced vertices f E (JF) Energy fraction of the tracks associated with the displaced vertices R( pjet , pvtx )(JF) R between the jet axis and the vectorial sum of momenta of all tracks attached to displaced vertices Sx yz (JF) Significance of the average distance between PV and displaced vertices NTrkAtVtx (JF) Number of tracks from multi-prong displaced vertices N2TrkVtx (JF) Number of two-track vertex candidates (prior to decay chain fit) N1-trk vertices (JF) Number of single-prong displaced vertices N≥2-trk vertices (JF) Number of multi-prong displaced vertices JetFitter c-tagging L x yz (2nd/3rdvtx)(JF) Distance of 2nd or 3rd vertex from PV L x y (2nd/3rdvtx)(JF) Transverse displacement of the 2nd or 3rd vertex m Trk (2nd/3rdvtx)(JF) Invariant mass of tracks associated with 2nd or 3rd vertex E Trk (2nd/3rdvtx)(JF) Energy fraction of the tracks associated with 2nd or 3rd vertex f E (2nd/3rdvtx)(JF) Fraction of charged jet energy in 2nd or 3rd vertex NTrkAtVtx (2nd/3rdvtx)(JF) Number of tracks associated with 2nd or 3rd vertex min , Y max , Y avg Ytrk trk trk (2nd/3rdvtx)(JF) Min., max. and avg. track rapidity of tracks at 2nd or 3rd vertex posed of light-flavour jets. This allows the charm rejection to Table 2 List of optimised hyperparameters used in the MV2 tagging be enhanced whilst preserving a high light-flavour jet rejec- algorithm tion. The BDT training hyperparameters of the MV2 tagging Hyperparameter Value algorithm are listed in Table 2. They have been optimised Number of trees 1000 to provide the best separation power between the signal and Depth 30 the background. The output discriminant of the MV2 algo- rithm for b-jets, c-jets and light-flavour jets evaluated with Minimum node size 0.05% the baseline t t¯ simulated events are shown in Fig. 1a. Cuts 200 Boosting type Gradient boost Shrinkage 0.1 4.3.2 DL1 Bagged sample fraction 0.5 The second high-level b-tagging algorithm, DL1 [23], is based on a deep feed-forward neural network (NN) trained jet or a light-flavour jet. The topology of the output consists using Keras [28] with the Theano [29] backend and the Adam of a mixture of fully connected hidden and Maxout layers optimiser [30]. The DL1 NN has a multidimensional output [31]. The input variables to DL1 consist of those used for the corresponding to the probabilities for a jet to be a b-jet, a c- MV2 algorithm with the addition of the JetFitter c-tagging 123 Eur. Phys. J. C (2019) 79:970 Page 7 of 36 970 Fraction of jets / 0.05 Fraction of jets / 0.50 10 ATLAS Simulation 10 ATLAS Simulation s = 13 TeV, tt b-jets 1 s = 13 TeV, tt b-jets c-jets c-jets 1 Light-flavour jets 10−1 Light-flavour jets 10−2 10−1 10−3 10−2 10−4 10−5 10−3 10−6 10−4−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 10−7−8 −6 −4 −2 0 2 4 6 8 10 12 DMV2 DDL1 (a) (b) Fig. 1 Distribution of the output discriminant of the (a) MV2 and (b) DL1 b-tagging algorithms for b-jets, c-jets and light-flavour jets in the baseline t t¯ simulated events Table 3 List of optimised hyperparameters used in the DL1 tagging vides the DL1 algorithm with a smaller memory footprint algorithm than BDT-based algorithms. The final DL1 b-tagging dis- Hyperparameter Value criminant is defined as:   pb Number of input variables 28 DDL1 = ln , Number of hidden layers 8 f c · pc + (1 − f c ) · plight Number of nodes [per layer] [78, 66, 57, 48, where pb , pc , plight and f c represent respectively the b-jet, 36, 24, 12, 6] c-jet and light-flavour jet probabilities, and the effective c- Number of Maxout layers [position] 3 [1, 2, 6] jet fraction in the background training sample. Using this Number of parallel layers per Maxout layer 25 approach, the c-jet fraction in the background can be cho- Number of training epochs 240 sen a posteriori in order to optimise the performance of the Learning rate 0.0005 algorithm. An optimised c-jet fraction of 8% is used to eval- Training minibatch size 500 uate the performance of the DL1 b-tagging algorithm in this paper. The output discriminants of the DL1 b-tagging algorithms variables listed in Table 1. The latter relate to the dedicated for b-jets, c-jets and light-flavour jets in the baseline t t¯ sim- properties of the secondary and tertiary vertices (distance ulated events are shown in Fig. 1b. to the primary vertex, invariant mass and number of tracks, energy, energy fraction, and rapidity of the tracks associated 4.4 Algorithm performance with the secondary and tertiary vertices). A jet pT and |η| reweighting similar to the one used for MV2 is performed. The evaluation of the performance of the algorithms is car- The DL1 algorithm parameters, listed in Table 3, include ried out using b-jet tagging single-cut operating points (OPs). the architecture of the NN, the number of training epochs, These are based on a fixed selection requirement on the the learning rates and training batch size. All of these are b-tagging algorithm discriminant distribution ensuring a spe- optimised in order to maximise the b-tagging performance. cific b-jet tagging efficiency, εb , for the b-jets present in Batch normalisation [32] is added by default since it is found the baseline t t¯ simulated sample. The selections used to to improve the performance. define the single-cut OPs of the MV2 and the DL1 algo- Training with multiple output nodes offers additional flex- rithms, as well as the corresponding c-jet, τ -jet and light- ibility when constructing the final output discriminant by flavour jet rejections, are shown in Table 4. The MV2 combining the b-jet, c-jet and light-flavour jet probabilities. and the DL1 discriminant distributions are also divided Since all flavours are treated equally during training, the into five ‘pseudo-continuous’ bins, (O k )k=1...5 , delimited trained network can be used for both b-jet and c-jet tagging. In by the selections used to define the b-jet tagging single-cut addition, the use of a multi-class network architecture pro- OPs for 85%, 77%, 70% and 60% efficiency, and bounded 123 970 Page 8 of 36 Eur. Phys. J. C (2019) 79:970 Table 4 Selection and c-jet, τ -jet and light-flavour jet rejections corresponding to the different b-jet tagging efficiency single-cut operating points for the MV2 and the DL1 b-tagging algorithms, evaluated on the baseline t t¯ events b MV2 DL1 Selection Rejection Selection Rejection c-jet τ -jet Light-flavour jet c-jet τ -jet Light-flavour jet 60% > 0.94 23 140 1200 > 2.74 27 220 1300 70% > 0.83 8.9 36 300 > 2.02 9.4 43 390 77% > 0.64 4.9 15 110 > 1.45 4.9 14 130 85% > 0.11 2.7 6.1 25 > 0.46 2.6 3.9 29 (a) (b) Fig. 2 The (a) light-flavour jet and (b) c-jet rejections versus the b-jet tagging efficiency for the IP3D, SV1, JetFitter, MV2 and DL1 b-tagging algorithms evaluated on the baseline t t¯ events by the trivial 100% and 0% selections. The value of the thereby their complementarity in the multivariate combina- pdf in each bin is called the b-jet tagging probability and tions, with the performance of the DL1 and MV2 discrim- labelled Pb in the following. The b-jet tagging efficiency inants found to be similar. The two algorithms tag a highly of the εb = X % single-cut OP can then be defined as correlated sample of b-jets, where the relative fraction of the sum of the b-jet tagging probabilities in the range jet exclusively tagged by each algorithm is around 3% at [X %,0%]. the εb = 70% single-cut OP. The relative fractions of light- The light-flavour jet and c-jet rejections as a function of flavour jets exclusively mis-tagged by the MV2 or the DL1 the b-jet tagging efficiency are shown in Fig. 2 for the vari- algorithms at the εb = 70% single-cut OP reach 0.2% and ous low- and high-level b-tagging algorithms. This demon- 0.1%, respectively. strates the advantage of combining the information provided However, the additional JetFitter c-tagging variables by the low-level taggers, where improvements in the light- used by DL1 bring around 30% and 10% improvements in flavour jet and c-jet rejections by factors of around 10 and the light-flavour jet and c-jet rejections, respectively, at the 2.5, respectively, are observed at the b = 70% single-cut b = 70% single-cut OP compared to MV2. OP of the high-level algorithms compared to low-level algo- rithms. This figure also illustrates the different b-jet tagging efficiency range accessible with each low-level algorithm and 123 Eur. Phys. J. C (2019) 79:970 Page 9 of 36 970 5 Data and simulated samples real prompt leptons include single top production in asso- ciation with a W boson (W t), diboson production (W W , In order to use data to evaluate the performance of the high- W Z , Z Z ) where at least two leptons are produced in the level b-tagging algorithms, a sample of events enriched in t t¯ electroweak boson decays, and Z +jets, with Z decaying dileptonic decays is selected. into leptons. The W t single top production was modelled The analysis is performed with a pp collision data sam- using PowhegBox v2 interfaced to Pythia 8.230 using √ ple collected at a centre-of-mass energy of s = 13 TeV the ‘diagram removal’ scheme [45,46] with the A14 set of during the years 2015, 2016 and 2017, corresponding to an tuned parameters and the NNPDF30NNLO (NNPDF2.3LO) integrated luminosity of 80.5 fb−1 and a mean number of [41,42] parton distribution functions in the matrix elements pp interactions per bunch crossing of 31.9. The uncertainty (parton shower). Diboson production with additional jets in the integrated luminosity is 2.0% [33], obtained using the was simulated using Sherpa [47,48] v2.2.1 (for events LUCID-2 detector [34] for the primary luminosity measure- where one boson decays hadronically) or Sherpa v2.2.2 ments. All events used were recorded during periods when (for events where no bosons decay hadronically), using all relevant ATLAS detector components were functioning the PDF set NNPDF30NNLO [41]. This includes the 4, normally. The dataset was collected using triggers requiring ν, νν, ννν, qq and νqq final states, which cover the presence of a single, high- pT electron or muon, with pT W W , W Z and Z Z production including off-shell Z con- thresholds that yield an approximately constant efficiency for tributions. Z +jets production (including both Z → τ τ and leptons passing an offline selection of pT ≥ 28 GeV. Z → ee/μμ) was modelled using Sherpa v2.2.1 with PDF The baseline t t¯ full simulation sample was produced using set NNPDF30NNLO. Processes with one real lepton include PowhegBox v2 [35–38] where the first-gluon-emission cut- t-channel and s-channel single top production [49]. These off scale parameter h damp is set to 1.5m t , with m top = processes were modelled with the same generator and parton 172.5 GeV used for the top-quark mass. PowhegBox was shower combination as the W t channel. W +jets production, interfaced to Pythia 8.230 [39] with the A14 set of tuned with the W boson decaying into eν, μν or τ ν with the τ - parameters [40] and NNPDF30NNLO (NNPDF2.3LO) [41, lepton decaying leptonically, was modelled in a similar way 42] parton distribution functions in the matrix elements (par- to the Z +jets production described above. ton shower). This set-up was found to produce the best mod- Alternative samples of non-t t¯ processes include the W t elling of the multiplicity of additional jets and both the indi- single top production using PowhegBox v2 interfaced to vidual top-quark and t t¯ system pT [43]. Herwig 7.0.4. The effects of ISR and FSR are evaluated Alternative t t¯ simulation samples were generated using by reweighting the baseline single-top events in a manner PowhegBox v2 interfaced to Herwig 7.0.4 [44] with the that either reduces or increases parton shower radiation. H7-UE-MMHT set of tuned parameters. The effects of An additional W t sample using PowhegBox v2 interfaced initial- and final-state radiation (ISR, FSR) are explored by to Pythia 8.230 with the alternative ‘diagram subtraction’ reweighting the baseline t t¯ events in a manner that reduces scheme [45,46] is used to investigate the impact of the inter- (reduces and increases) initial (final) parton shower radi- ference between t t¯ and W t production. Uncertainties in dibo- ation [43] and by using an alternative PowhegBox v2 + son and Z +jets production are estimated by reweighting the Pythia 8.230 sample with h damp set to 3m top and parame- baseline samples, whereas uncertainties in processes with ter variation group Var3 (described in Ref. [43]) increased, one real lepton are evaluated directly from data, as described leading to increased ISR. later in Sect. 8. The majority of events with at least one ‘fake’ lepton in As described in Sect. 4, the new Run 2 b-tagging algo- the selected sample arise from t t¯ production where only one rithm training strategy is based on the use of a hybrid sample of the W bosons, which originated from a top-quark decay, composed of both the baseline t t¯ event sample and a dedi- decays leptonically. These fake leptons come from several cated sample of Z  decaying into hadronic jet pairs. This Z  sources, including non-prompt leptons produced from bot- sample was generated using Pythia 8.2.12 with the A14 set tom or charm hadron decays, electrons arising from a photon of tuned parameters for the underlying event and the leading- conversion, jets misidentified as electrons, or muons pro- order NNPDF2.3LO [42] parton distribution functions. duced from in-flight pion or kaon decays. This background The Evtgen [50] package was used to handle the decay is also modelled using the t t¯ production described above. The of heavy-flavour hadrons for all samples except for those rate of events with two fake leptons is found to be negligible. generated with the Sherpa generator, for which the default Non-t t¯ processes, which are largely subdominant in this Sherpa configuration recommended by the Sherpa authors analysis, can be classified into two types: those with two was used. All MC events have additional overlaid minimum- real prompt leptons from W or Z decays (dominant) and bias interactions generated with Pythia 8.160 with the A3 those where at least one of the reconstructed lepton candi- set of tunes parameters [51] and NNPDF2.3LO parton dis- dates is ‘fake’ (subdominant). Backgrounds containing two tribution functions to simulate pile-up background and are 123 970 Page 10 of 36 Eur. Phys. J. C (2019) 79:970 weighted to reproduce the observed distribution of the aver- muon spectrometer due to very energetic punch-through age number of interactions per bunch crossing of the cor- jets or from badly measured inner detector tracks in jets responding data sample. The nominal MC samples were wrongly matched to muon spectrometer track segments, processed through the full ATLAS detector simulation [52] the whole event is vetoed. A tight muon identification based on GEANT4 [53], but most samples used for system- requirement is applied to the muon candidates to further atic uncertainty evaluation were processed with a faster sim- suppress the background. Isolation selections similar to ulation making use of parameterised showers in the calorime- the ones applied to the electron candidates are imposed to ters [54]. The simulated events were reconstructed using the reject candidates coming from sources other than prompt same algorithms as the data. massive boson decays (hadrons faking a muon signature or heavy-flavour decays). SFs of order unity, similar to those for electrons and derived in Z → μ+ μ− events, 6 Event selection and classification are applied to account for differences in reconstruction, identification and isolation efficiencies between data and A sample of events enriched in t t¯ dileptonic decays is selected simulated events. Muon momenta are calibrated using by requiring exactly two well-identified lepton candidates the Z mass peak. and two jets to be present in each event. Events are further classified on the basis of two topological variables to control If electrons, muons or jets overlap with each other, all processes including non-b-jets. The lepton definition, and the but one object must be removed from the event. The dis- event selection and classification are described in this section. tance metric  used to define overlapping objects is defined as R  = ( φ)2 + ( y)2 where y represents the rapidity 6.1 Lepton object definition difference. To prevent double-counting of electron energy deposits as jets, jets within R  = 0.2 of a reconstructed In addition to the objects reconstructed for b-tagging, electron candidate are removed. If the nearest remaining jet described in Sect. 3, the event selection for the efficiency is within R  = 0.4 of the electron, the electron is discarded. measurement requires electron and muon candidates, defined To reduce the background from muons from heavy-flavour as follows. decays inside jets, muons are required to be separated by R  ≥ 0.4 from the nearest jet. In cases where a muon and a Electron candidates are reconstructed from an isolated jet are reconstructed within R  < 0.4, the muon is removed energy deposit in the electromagnetic calorimeter matched if the jet has at least three associated tracks; the jet is removed to an ID track [55]. Electrons are selected for inclu- otherwise. This avoids an inefficiency for high-energy muons sion in the analysis within the fiducial region of trans- undergoing significant energy loss in the calorimeter. verse energy E T ≥ 28 GeV and |η| < 2.47. Candidates within the transition region between the barrel and end- 6.2 Event selection cap electromagnetic calorimeters, 1.37 ≤ |η| < 1.52, are removed in order to avoid large trigger efficiency uncer- To be considered in this analysis, events must have at least tainties in the turn-on region of the lowest pT trigger. A one lepton identified in the trigger system. This triggered tight likelihood-based electron identification requirement lepton must match an offline electron or muon candidate. is used to further suppress the background from multi- For each applicable trigger, scale factors are applied to the jet production. Isolation criteria are used to reject can- simulation in order to correct for known differences in trigger didates coming from sources other than prompt decays efficiencies between the simulation and collision data [11]. from massive bosons (hadrons faking an electron signa- In order to reject backgrounds with fewer than two prompt ture, heavy-flavour decays or photon conversions). Scale leptons, exactly two reconstructed leptons with opposite factors (SFs), of order unity, derived in Z → e+ e− events charges are required. Contributions from backgrounds with are applied to simulated events to account for differences Z bosons are reduced by requiring that one lepton is an elec- in reconstruction, identification and isolation efficiencies tron and the other is a muon. The residual contribution from between data and simulation. Electron energies are cali- Z → τ τ events, which populate the low mass region, is fur- brated using the Z mass peak. ther reduced by considering only events with m eμ ≥ 50 GeV. Muon candidates are reconstructed by combining tracks The contribution from t t¯ events with light-flavour jets from found in the ID with tracks found in the muon spec- ISR or FSR or from W bosons is reduced by requiring exactly trometer [56]. Muons are selected for inclusion in the two reconstructed jets. analysis within the fiducial region of transverse momen- Since the aim of the study is to measure the b-jet tagging tum pT ≥ 28 GeV and |η| < 2.5. If the event contains efficiency, it is useful to label simulated events according a muon reconstructed from high hit multiplicities in the to the generator-level flavour of the two selected jets, fol- 123 Eur. Phys. J. C (2019) 79:970 Page 11 of 36 970 (a) (b) Fig. 3 Distribution of the jet (a) pT and (b) η in the events passing the selection. Simulated events are split into physics process. The ratio panels show the data-to-simulation ratio as well as the fraction of t t¯ events among the simulated events lowing the definitions introduced in Sect. 3, instead of the ulation. The event classification based on these variables is physics process they originate from. Events with two b-jets described in more detail in the next section. (non-b-jets) are labelled bb (ll). Events with one selected b- According to the simulation, about 85% of the events pass- jet and one non-b-jet are labelled bl events if the b-jet pT ing the selection are dileptonic t t¯ events, about 65% of which is larger than the non-b-jet pT and lb in the opposite case. are bb events. Single top production in association with a W According to the simulation, more than 90% of the non-b-jets boson accounts for 8% of the events, with about 30% of these are light-flavour jets, the rest being composed of c-jets, and events containing two selected b-jets. Diboson and Z +jet more than 95% of the b-jets originate from a top-quark decay. production represent respectively about 5% and 2% of the The fraction of τ -jets is predicted to be negligible. selected events, 85% of these events being ll events. Events In order to create bb, bl, lb and ll-enriched regions in the originating from W +jets production are negligible (< 0.1%). selected sample, each of the two leptons is paired with a jet in The main source of non-b-jets therefore originates from t t¯ an exclusive way to determine whether they originate from bl or lb events, i.e. dileptonic t t¯ events with a high- pT light- the same top-quark decay. The pairing is performed such that flavour jet originating from ISR or FSR. it minimises (m 2j1,i + m 2j2, j ), where j1 ( j2) is the highest- Figure 3 shows the level of agreement between data and pT (second highest- pT ) jet, i, j are the two leptons and m j1, simulation as a function of the pT and η of the selected jets as (m j2, ) is the invariant mass of the system including the high- well as the expected fraction of t t¯ events. The overall level est (second highest) pT jet and its associated lepton. Choos- of agreement between data and simulation is fairly good, ing the pairing that mimimises this quantity relies on the fact although some mismodelling is present at high jet pT , possi- that if the pairs of objects are from the same original parti- bly related to the modelling of the top-quark pT [57], which cles then they are likely to have similar masses. Using the motivates the extraction of the b-jet tagging efficiency in jet minimum of squared masses penalises asymmetric pairings pT bins. The distribution of the discriminant of the MV2 with one high-mass lepton–jet pair, as well as combinations algorithm for events passing the selection is shown in Fig. 4. including two very high invariant masses, which are unlikely Generally, good modelling is observed, indicating similar for those arising from top-quark decay. Events are required b-jet tagging efficiencies in data and simulation. to have m j1, ≥ 20 GeV and m j2, ≥ 20 GeV in order to avoid configurations in which a soft jet and a soft lepton are close to each other, which are not well described by the sim- 123 970 Page 12 of 36 Eur. Phys. J. C (2019) 79:970 • m j1, < 175 GeV, m j2, ≥ 175 GeV, bl control region (CRBL ): high bl purity control region used to constrain the bb, bl, lb and ll fractions in the SR, • m j1, ≥ 175 GeV, m j2, < 175 GeV, lb control region (CRLB ): high lb purity control region used to constrain the bb, bl, lb and ll fractions in the SR. Finally, the events in the SR are further classified as a func- tion of the pseudo-continuous binned b-tagging discriminant of the two jets, denoted w1 and w2 , as defined in Sect. 4.4. These classifications result in a total of 1260 orthogonal categories. A schematic diagram illustrating the event cat- egorisation is shown in Fig. 6. The bb event purity in the signal regions for the different pT,1 , pT,2 bins is shown in Fig. 7. The lowest purity (19%) occurs when both jets have very low pT ; however, the majority of bins have a bb event purity greater than 70% and the highest purity (where both jets have high pT ) reaches 93%. The CRLL , CRBL and CRLB control regions are enriched in their targeted backgrounds relative to the corresponding SR. Their purity in ll, bl and Fig. 4 Distribution of the output discriminant of the MV2 algorithm lb events varies across the pT,1 , pT,2 plane and ranges in the for the jets in events passing the selection. Simulated events are clas- simulation from 30% to 90% (CRLL ), 32% to 79% (CRBL ) sified according to the flavour composition of the two jets, where the and 20% to 74% (CRLB ), respectively. The dominant back- first term in each legend entry represents the flavour of the jet which ground in each SR always benefits from a high-purity (i.e. is plotted (b or l) and the following term in parenthesis represents the flavour composition of the event (bb, bl + lb or ll). The ratio panels ≥ 50%) control region. show the data-to-simulation ratio as well as the fraction of bb events among the simulated events 7 Extraction of b-jet tagging efficiency 6.3 Event classification Once events have been selected and classified, the measure- ment of the b-jet tagging probabilities is performed. The pre- The distributions of the m j1, and m j2, observables are cision of the previous ATLAS measurement [6] was limited shown in Fig. 5. In the case of t t¯ events with two b-jets, both by the uncertainty in the fractions of bb, bl, lb and ll events m j1, and m j2, have an upper limit around m t = 172.5 GeV in the selected sample, which is driven by the modelling of and are usually significantly smaller due to the undetected top-quark pair production. The main novelty of this work neutrino. This is generally not the case for bl, lb and ll events, in comparison to Ref. [6] lies in the measurement method, which result in high m j1, and/or m j2, values more often which uses both signal and control region data to define a at high jet pT . Therefore, the m j1, and m j2, observables joint log-likelihood function allowing the simultaneous esti- discriminate between bb, bl, lb and ll events while being mate of the b-jet tagging probabilities and flavour compo- uncorrelated with the b-tagging discriminants, which do not sitions. This new technique leads to a reduction in the total make use of leptons outside jets. uncertainties by up to a factor of two, as discussed in Sect. 8. The selected events are classified into 45 different bins The general form of an extended binned log-likelihood according to the pT of the two jets, allowing the b-jet tag- function, after dropping terms that do not depend on the ging efficiency to be measured as a function of the jet pT . parameters to be estimated, is provided in Eq. (1): In addition, in each leading jet pT , subleading jet pT bin ( pT,1 , pT,2 ), the events are further classified into four bins    N   according to the m j1, and m j2, values: ˆ = −νtot + log L νtot ,  ˆ , n i log νi νtot ,  (1) i • m j1, , m j2, < 175 GeV, signal region (SR): high bb where νtot is the total expected number of events,  ˆ = purity region used to measure the b-jet tagging efficiency, (1 , . . . , m ) is the list of parameters to be estimated, • m j1, , m j2, ≥ 175 GeV, ll control region (CRLL ): high including the parameters of interest (POI) and nuisance ll purity control region used to constrain the bb, bl, lb parameters, νi (n i ) is the number of expected (observed) and ll fractions in the SR, events in the bin i and N bins are considered in total. In 123 Eur. Phys. J. C (2019) 79:970 Page 13 of 36 970 (a) (b) Fig. 5 Distribution of the (a) m j1, and (b) m j2, observables in events passing the selection. Simulated events are classified in terms of the flavour composition of the two jets. The ratio panels show the data-to-simulation ratio as well as the fraction of bb events among the simulated events Fig. 6 Schematic illustration of the event categorisation. Events are (CR) and signal (SR) regions. The events in the SR are then classified binned according to the pT of both the leading and subleading jet in according to the b-tagging discriminant of the two jets the event. In each of these bins, events are further assigned to control this work, the POIs are the b-jet tagging probabilities, Pb , In each control region, the number of events in a given introduced in Sect. 4.4. They are defined in this measure- pT,1 , pT,2 bin (T m , T n ) is written as the sum of the bb, m,n m,n m,n ment per pT bin, i.e. as the conditional probabilities for a bl, lb and ll yields expected in that bin (νbb , νbl , νlb , m,n b-jet with a transverse momentum falling in one of the nine νll ), corrected by pT,1 , pT,2 dependent correction fac- m,n m,n m,n pT bins (T m )m=1...9 of the measurement to have a b-tagging tors (cbb , cbl , clb , cllm,n ), forming the nuisance param- discriminant w falling in one of the five pseudo-continuous eters: bins (O k )k=1...5 . The b-jet tagging efficiency of the single- m,n m,n m,n m,n m,n m,n cut OP X in that jet pT bin, εb , relates to the POIs as outlined νC R (T m , T n ) = cbb νC R,bb + cbl νC R,bl + clb νC R,lb below: +cllm,n νCm,n R,ll .  εb (X |T m ) = Pb (O k |T m ). In each signal region, the events are further binned accord- Ok ⊂ X ing to the b-tagging discriminants of the two jets, w1 , w2 . The 123 970 Page 14 of 36 Eur. Phys. J. C (2019) 79:970 Fig. 7 Fraction of bb events in 90.4 90 Fraction of bb events [%] Subleading jet p [GeV] 250-600 the signal region as predicted by the simulation as a function of ATLAS Simulation ± 1.0 93.1 91.7 80 the leading and subleading jet 175-250 s = 13 TeV ± 0.4 ± 0.6 T pT . Uncertainties are statistical 91.3 90.8 89.6 70 only 140-175 tt POWHEG + PYTHIA 8 ± 0.4 ± 0.3 ± 0.9 89.3 89.6 89.1 85.2 60 110-140 ± 0.3 ± 0.3 ± 0.4 ± 1.0 86.1 87.2 86.8 85.7 82.7 50 85-110 ± 0.2 ± 0.2 ± 0.3 ± 0.4 ± 1.1 80.4 82.1 82.1 81.3 80.1 77.2 40 60-85 ± 0.2 ± 0.2 ± 0.2 ± 0.3 ± 0.4 ± 1.2 69.2 74.6 75.2 73.6 71.9 69.7 67.8 30 40-60 ± 0.2 ± 0.1 ± 0.2 ± 0.2 ± 0.4 ± 0.6 ± 1.3 47.3 57.4 62.7 62.5 61.3 59.8 56.9 55.1 20 30-40 ± 0.6 ± 0.2 ± 0.2 ± 0.3 ± 0.3 ± 0.5 ± 0.7 ± 1.8 10 20-30 19.1 ± 0.4 30.5 ± 0.4 38.3 ± 0.3 41.7 ± 0.2 43.6 ± 0.3 43.7 ± 0.3 43.5 ± 0.4 39.0 ± 0.6 35.2 ± 1.6 20-3 30-4 40-6 60-8 85-1 1 1 1 2 0 0 0 0 5 10 10-140 40-175 75-250 50-600 Leading jet p [GeV] T number of events expected in a given pT,1 , pT,2 , w1 , w2 bin the expected data statistical uncertainty (0.6–3.7%), and to (T m , T n , O k , O p ) is thus written: the sum in quadrature of the expected data statistical uncer- tainty, the MC statistical uncertainty and the physics mod- ν S R (T m , T n , O k , O p ) elling uncertainties quoted for the final measurement (0.9– m,n m,n 5.4%). The non-closure effects are found to be within uncer- = cbb ν S R,bb · Pb (O k |T m ) · Pb (O p |T n ) tainties in each jet pT bin such that no additional uncertainty m,n m,n + cbl ν S R,bl · Pb (O k |T m ) · Pl (O p |T n ) related to the signal extraction method is considered. m,n m,n + clb ν S R,lb · Pl (O k |T m ) · Pb (O p |T n ) + cllm,n ν Sm,n R,ll · Pl (O |T ) · Pl (O |T ), k m p n 8 Uncertainties where Pl is the effective b-jet tagging probability of the Uncertainties affecting the measurement which originate mix of c-jets and light-flavour jets predicted by the simulation from statistical sources are considered together with sys- in each pT,1 , pT,2 bin. The POIs and correction factors are tematic uncertainties related to the detector calibration and estimated by minimising the negative log-likelihood func- physics modelling. tion defined above with the MINUIT algorithm [58]. Both The data statistical uncertainty in the b-jet tagging prob- the POIs and correction factors are free parameters during abilities, and their bin-to-bin correlations, are obtained from the minimisation procedure. Signal and control region data the error matrix returned by MINUIT [58] and propagated are provided as input as well as the Pl conditional probabil- to the b-jet tagging efficiencies via a basis transformation. ities, which are estimated from the MC simulation corrected The data statistical uncertainty reaches about 4% (2%) for to match data (see Sect. 1). The simulation is also used to jets within 20 ≤ pT < 30 GeV (30 ≤ pT < 40 GeV), ranges determine the bb, bl, lb and ll yield fractions according to from 1% to 3% for jet pT ≥ 140 GeV and is below 1% the type of region (SR, CRs), as the correction factors are elsewhere. defined as a function of pT,1 and pT,2 only. The bootstrap resampling technique [59] is used to assess The extraction method is validated using pseudo-data gen- the MC statistical uncertainty by creating an ensemble of erated with a known flavour composition. This is created statistically equivalent measurements in which the weight by combining events from either nominal or alternative MC of each simulated event used in the nominal measurement simulation fluctuated according to the statistical uncertainty is multiplied by an additional term, randomly chosen for expected from the actual dataset. The input parameters of each event from a Poisson distribution with a mean of one. the minimisation procedure are taken from the nominal MC The standard deviation of the distribution of these measure- simulation in all cases. The size of the non-closure effects ments is taken as the MC statistical uncertainty. This method observed when using pseudo-data based on the nominal and allows all correlations to be preserved and the uncertainty alternative MC simulation are compared respectively with in the value of any parameter to be extracted. One hundred 123 Eur. Phys. J. C (2019) 79:970 Page 15 of 36 970 bootstrap replicas of each simulated sample are used for this contribution. The single-top uncertainty reaches about 3% evaluation. The MC statistical uncertainty in the b-jet tagging for jet pT < 30 GeV due to the parton shower and hadro- efficiencies is found to be non-negligible only for jet pT ≤ nisation model variation, and 1% for jet pT ≥ 250 GeV, 40 GeV, where it reaches about 2% and 1% for jets within where the uncertainty in the interference with t t¯ events is the 20 GeV ≤ pT < 30 GeV and 30 GeV ≤ pT < 40 GeV, dominant contribution. It is below 1% elsewhere. The uncer- respectively. tainties associated with the modelling of top-quark events The systematic uncertainties are derived by varying a are reduced by up to a factor of two relative to the previous parameter in the simulated events, repeating the complete ATLAS analysis [6] due to the new b-jet tagging efficiency analysis with this varied parameter and taking the difference extraction method, which allows the bb event yield to be between the updated measurement of the b-jet tagging effi- determined at a precision of a few percent in each pT,1 , pT,2 ciency or probability and the nominal measurement as the bin. (bin-wise correlated) uncertainty. For b-jet tagging efficien- The uncertainty in the modelling of diboson and Z +jet cies, the bootstrap replicas of simulated events are then used production [61,62] is evaluated by varying the total cross- to evaluate the MC statistical uncertainty in each systematic section and the factorisation and renormalisation scales for variation. Variations of the b-jet tagging efficiency that are these processes, as well as propagating the uncertainty from not statistically significant undergo a bin-merging procedure the PDF. The total cross-section is kept constant when per- over an increasing number of pT bins to improve their signifi- forming the scale and PDF variations such that only the cance. Following this procedure, only statistically significant shapes of the kinematic distributions are impacted. The total variations are considered as systematic uncertainties. cross-section is varied by ± 6% (± 5%) for Z +jets (diboson) Uncertainty sources related to the energy scale and res- production. The scale uncertainties are estimated simultane- olution of hadronic jets [19] encompass both the modelling ously by varying the nominal values by a factor of two up of the detector response and the analysis techniques used and down and taking the largest deviations from the nominal to derive the calibration. The impact of the jet energy scale predictions in each direction as uncertainties. PDF uncertain- uncertainty reaches from 4 to 5% for jet pT ≤ 30 GeV, 1% ties are evaluated using the 100 bootstrap replicas provided for 30 GeV ≤ pT < 40 GeV, and is negligible elsewhere. It by the NNPDF30NNLO [41] using the same method as out- is dominated by the prediction of the quark/gluon origin of lined for the MC statistical uncertainty earlier in this section. the light-flavour jets and by the difference in their energy The final diboson and Z +jets uncertainties are found to be response, as well as the difference in the calorimeter energy negligible in the entire range covered by the analysis. response as a function of η. The uncertainty originating from The number of events with a selected muon not originating the jet energy resolution is negligible. Uncertainty sources from a Z - or W -boson decay is predicted by the simulation to related to the performance of the JVT algorithm [21], the b- be negligible after the event selection. This is due to the tight tagging performance for light-flavour jets [8] and c-jets [7] muon identification and isolation criteria applied. The num- as well as the modelling of pile-up interactions were inves- ber of events with a selected electron not originating from a tigated and found to be negligible, as were lepton-related Z - or W -boson decay passing the event selection (1NPel, for uncertainties, including energy/momentum scale and resolu- 1 non-prompt electron) is also predicted by the simulation to tion, identification, isolation, trigger and track–vertex asso- be very small but one order of magnitude higher, reaching ciation efficiency. about 0.3% of the total event yield after selection. An uncer- The uncertainty in the physics modelling of top-quark tainty in this yield is derived by comparing the number of data events is evaluated by changing the parton shower and hadro- and MC events in an alternative region defined by requiring nisation model from Pythia 8 to Herwig 7 and increasing or two same-sign (SS) leptons instead of opposite-sign (OS). decreasing the amount of ISR and FSR within Pythia 8 [43]. The SS region is predicted by the simulation to have a com- The uncertainties originating from parton distribution func- position that is 12% 1NPel events, with the remaining 88% of tions (PDF) are quantified following the PDF4LHC recom- the sample coming from non-1NPel events, which is dom- mendations [60]. An additional source of uncertainty orig- inated by diboson production. This is estimated from sim- inates from the mismodelling of the interference between ulation and subtracted from the data. The remaining data single top W t and t t¯ production. It is evaluated by switch- events are then compared with MC predictions in bins of ing the nominal single-top simulation sample, based on the electron pT . The data-to-simulation ratio ranges from val- ‘diagram removal’ scheme, to the one based on the ‘diagram ues close to 3 for pT < 120 GeV to values close to 1 for subtraction’ scheme [45]. The final t t¯ modelling uncertainty pT ≥ 300 GeV. These values are used as simulation-to-data reaches 3% (2%) for jet pT < 30 GeV (30 ≤ pT < 40 GeV) scale factors to correct the yield of simulated 1NPel events in and about 1% for pT ≥ 40 GeV. It is dominated at low pT by the OS region in order to estimate an uncertainty in the fake- PDF and ISR/FSR variations whereas at higher pT the choice lepton modelling. The b-jet tagging efficiency measurement of parton shower and hadronisation model is the dominant is then repeated with these scale factors applied and com- 123 970 Page 16 of 36 Eur. Phys. J. C (2019) 79:970 1.15 b-jet tagging efficiency SF b-jet tagging efficiency 0.9 ATLAS s = 13 TeV, 80.5 fb-1 ATLAS s = 13 TeV, 80.5 fb-1 1.1 MV2, εb = 70% single-cut OP MV2, εb = 70% single-cut OP 0.8 1.05 1 0.7 0.95 0.6 0.9 Data (stat. unc.) Data (total unc.) Scale factor (stat. unc.) 0.85 Scale factor (total unc.) 0.5 tt MC 0.8 30 40 102 2×102 30 40 102 2×102 Jet p [GeV] Jet p [GeV] T T (a) (b) Fig. 8 The (a) b-jet tagging efficiency and (b) b-jet tagging efficiency passing the signal region selection. Vertical error bars include data sta- simulation-to-data scale factors for the εb = 70% single-cut OP of tistical uncertainties only while the green bands correspond to the sum the MV2 tagger as a function of jet pT . The efficiency measurement in quadrature of statistical and systematic uncertainties. The dots are is shown together with the efficiency derived from t t¯ simulated events located at the mean of the b-jet pT distribution in each pT bin pared with the nominal measurement. Differences of about between the measured b-jet tagging efficiency to the b-jet 1% to 2% for jet pT < 40 GeV and negligible elsewhere tagging efficiency derived from the simulation, are shown are observed and accounted for as an additional systematic in Fig. 8b. Scale factors are derived for all single-cut OPs uncertainty. and for the DL1 tagger using the same technique, resulting in similar results, as illustrated in Fig. 9. The scale factors have values very close to one and are approximately con- stant throughout the entire pT range, illustrating the good 9 Results modelling of the b-jet tagging performance. The b-jet tag- ging efficiency measurement for the εb = 70% single-cut The goodness-of-fit is evaluated by computing a Pearson’s OP of the MV2 algorithm as a function of jet |η| and the cor- χ 2 and comparing it with the number of degrees of free- responding simulation-to-data scale factors are presented in dom (nd f ) of the fit [63]. This procedure tests the hypoth- Fig. 10. The b-jet tagging probability and efficiency measure- esis that the remaining differences between observed and ment was also repeated considering only data and simulated expected yields post-fit originate only from the limited size events with either less or more than 28 additional pp interac- of the dataset. The χ 2 /nd f value obtained for the nominal tions per bunch crossing, and separately for 2015–2016 and measurement is 0.98, corresponding to a p-value of about 2017 data. In all cases, consistent results were observed. 0.65. This result illustrates the high goodness-of-fit already The uncertainty in the efficiency measurement for the observed before accounting for the other sources of uncer- εb = 70% single-cut OP of the MV2 tagger is sum- tainty discussed in Sect. 8. marised in Table 5. The total uncertainty reaches about 1% The bb, bl, lb and ll yield post-fit correction factors are for 40 GeV ≤ pT < 250 GeV, where it is dominated by the of order unity, compatible with unity within uncertainties, uncertainty in the physics modelling of t t¯ events and the and typically constrained within 2–5% for bb, 5–10% for data statistical uncertainty. At lower pT values (20 GeV ≤ bl, and 7–20% for lb and ll. The central values of the bb pT < 40 GeV), the total uncertainty increases to 8% due to yield correction factors tend to be a few percent below unity, higher uncertainties in the jet energy scale, the modelling of pointing to a slight underestimate of the number of light- t t¯ and single-top-quark events, the limited number of data flavour jets in the nominal simulation. The yield correction and MC events and the modelling of fake leptons. For jet factors deviate more strongly from unity when running on pT ≥ 250 GeV, the uncertainty increases to about 3% due the alternative simulated samples. to the limited number of data events. These observations are The b-jet tagging efficiency measurement for the εb = consistent across single-cut OPs and taggers. 70% single-cut OP of the MV2 algorithm is presented in The measurement of the b-jet tagging probabilities in the Fig. 8a as a function of jet pT together with the effi- MV2 and DL1 algorithm output bins is presented in Fig. 11a, ciency derived from t t¯ simulated events passing the sig- c together with the b-jet tagging probabilities derived from nal region selection. The corresponding b-jet tagging effi- t t¯ simulated events passing the signal region selection. The ciency simulation-to-data scale factors, defined as the ratio 123 Eur. Phys. J. C (2019) 79:970 Page 17 of 36 970 1.15 1.15 b-jet tagging efficiency SF b-jet tagging efficiency SF ATLAS s = 13 TeV, 80.5 fb-1 ATLAS s = 13 TeV, 80.5 fb-1 1.1 1.1 MV2, various single-cut OP DL1, various single-cut OP 1.05 1.05 1 1 0.95 0.95 εb = 85% scale factor (total unc.) εb = 85% scale factor (total unc.) 0.9 εb = 77% scale factor (total unc.) 0.9 εb = 77% scale factor (total unc.) εb = 70% scale factor (total unc.) εb = 70% scale factor (total unc.) 0.85 0.85 εb = 60% scale factor (total unc.) εb = 60% scale factor (total unc.) 0.8 0.8 30 40 102 2×102 30 40 102 2×102 Jet p [GeV] Jet p [GeV] T T (a) (b) Fig. 9 b-jet tagging efficiency simulation-to-data scale factors as a are offset for visual effects but computed in the same jet pT range. function of jet pT . The 60%, 70%, 77% and 85% single-cut OP of the Vertical error bars represent the total uncertainty (a) MV2 and (b) DL1 taggers are shown. The various groups of points 1.15 b-jet tagging efficiency SF b-jet tagging efficiency ATLAS s = 13 TeV, 80.5 fb-1 ATLAS s = 13 TeV, 80.5 fb-1 0.8 1.1 MV2, εb = 70% single-cut OP MV2, εb = 70% single-cut OP 1.05 0.75 1 0.7 0.95 0.65 0.9 Data (stat. unc.) Data (total unc.) Scale factor (stat. unc.) 0.6 tt MC 0.85 Scale factor (total unc.) 0.8 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 Jet |η| Jet |η| (a) (b) Fig. 10 The (a) b-jet tagging efficiency and (b) b-jet tagging efficiency tical uncertainties only while the green bands correspond to the sum simulation-to-data scale factor for εb = 70% single-cut OP of the MV2 in quadrature of statistical and systematic uncertainties. The dots are tagger as a function of jet |η|. Vertical error bars include data statis- located at the mean of the b-jet |η| distribution in each |η| bin probabilities are shown for jets with 110 GeV ≤ pT < 10 Usage in ATLAS analysis 140 GeV, which is located close to the b-jet tagging efficiency maximum. The corresponding b-jet tagging probability scale This section details how the simulation-to-data scale factors factors are shown in Fig. 11b, d. Scale factors are derived for are incorporated into ATLAS physics analyses. Scale factors all single-cut OPs using the same technique, resulting in sim- are smoothed, extrapolated beyond the jet pT range of the ilar results. The scale factors have values close to one and are data measurement and corrected taking into account the gen- about constant throughout the pseudo-continuous bins. erator dependence in the simulation. The number of system- The uncertainty in this measurement is summarised in atic uncertainties is reduced while preserving the bin-by-bin Table 6. The total uncertainty varies from about 9% in the correlations. The scale factors are then applied to ATLAS 100–85% bin to about 1% in the 60–0% bin. It is driven by physics analyses by correcting the b-jet tagging response in the t t¯ modelling uncertainties and data statistics, which is simulation and by applying related uncertainties to the cor- consistent with the result reported for the 70% single-cut OP rection. in this pT range. 123 970 Page 18 of 36 Eur. Phys. J. C (2019) 79:970 Table 5 Breakdown of the systematic uncertainties in the b-jet tagging the uncertainties related to jet energy resolution, electron and muon per- efficiency measurement for the 70% single-cut OP of the MV2 tagger formance, b-tagging performance for light-flavour jets and c-jets, JVT as a function of the jet pT bin. The ‘t t¯ modelling’ and the ‘Single top performance, diboson and Z +jet modelling (including normalisation modelling’ uncertainties correspond to the sum in quadrature of the and shape uncertainties) and pile-up modelling. All systematic uncer- uncertainty in the parton shower, hadronisation model, initial-state and tainties are fully correlated bin-by-bin whereas the statistical uncertainty final-state radiation and PDF for t t¯ and single top-quark production, correlations are evaluated following the procedures described in Sect. 8. respectively. The ‘Single top modelling’ uncertainties include an addi- In the case of correlated systematic uncertainties, the relative sign of tional source originating from the interference between single top and the uncertainty in each bin is taken into account, even if not shown here t t¯ production. ‘Other sources’ corresponds to the sum in quadrature of Source of uncertainty Relative uncertainty on εb [%] per jet pT bin [GeV] 20–30 30–40 40–60 60–85 85–110 110–140 140–175 175–250 250–600 Data statistics 3.7 1.7 0.7 0.6 0.6 0.6 0.8 1.1 2.8 MC statistics 2.2 1.0 0.4 0.2 0.2 0.2 0.2 0.2 0.5 Jet energy scale 4.5 0.8 0.3 0.1 0.1 0.1 0.1 0.2 0.4 t t¯ modelling 3.2 1.5 1.0 0.7 0.7 0.8 1.0 0.8 0.5 Single top modelling 2.5 0.5 0.6 0.6 0.4 0.3 0.3 0.4 1.1 Fake leptons modelling 1.8 1.1 0.1 0.2 < 0.1 < 0.1 0.2 < 0.1 0.2 Other sources 1.4 0.9 0.2 0.3 0.2 0.1 0.1 0.1 0.3 Total 7.7 3.0 1.4 1.1 1.0 1.1 1.3 1.5 3.1 1 1.3 b-jet tagging probability b-jet tagging probability SF ATLAS s = 13 TeV, 80.5 fb-1 ATLAS s = 13 TeV, 80.5 fb-1 0.8 MV2, 110 ≤ jet p < 140 GeV 1.2 MV2, 110 ≤ jet p < 140 GeV T T Data (stat. unc.) 0.6 1.1 Data (total unc.) tt MC 0.4 1 0.2 0.9 Scale factor (stat. unc.) Scale factor (total unc.) 0 0.8 100-85% 85-77% 77-70% 70-60% 60-0% 100-85% 85-77% 77-70% 70-60% 60-0% DMV2 OP DMV2 OP (a) (b) 1 b-jet tagging probability ATLAS s = 13 TeV, 80.5 fb-1 0.8 DL1, 110 ≤ jet p < 140 GeV T Data (stat. unc.) 0.6 Data (total unc.) tt MC 0.4 0.2 0 100-85% 85-77% 77-70% 70-60% 60-0% DDL1 OP (c) (d) Fig. 11 The (a, c) b-jet tagging probability and (b, d) b-jet tagging ing the signal region selection. Vertical error bars include data statis- probability simulation-to-data scale factors for the (a, b) MV2 and tical uncertainties only while the green bands correspond to the sum (c, d) DL1 tagger for jets with 110 ≤ pT < 140 GeV in the var- in quadrature of statistical and systematic uncertainties. The dots are ious pseudo-continuous bins. The probability measurement is shown located at the bin centres together with the probabilities derived from t t¯ simulated events pass- 123 Eur. Phys. J. C (2019) 79:970 Page 19 of 36 970 Table 6 Breakdown of the systematic uncertainties in the b-jet tag- nominal simulation is then taken as an additional systematic ging probability measurement of the MV2 tagger as a function of the uncertainty. ‘pseudo-continuous’ bins for jets satisfying 110 ≤ pT < 140 GeV. The ‘t t¯ modelling’ and the ‘Single top modelling’ uncertainties correspond Four distinct sets of variables, related to the reconstruc- to the sum in quadrature of the uncertainty in the parton shower, hadro- tion of tracks, of jets, the modelling of the b-hadrons and the nisation model, initial-state and final-state radiation and PDF for t t¯ and interaction of long-lived b-hadrons with the detector mate- single-top-quark production, respectively. The ‘Single top modelling’ rial, are considered. Among the uncertainties related to the uncertainties include an additional source originating from the interfer- ence between single top and t t¯ production. ‘Other sources’ corresponds reconstruction of tracks, the ones that are found to most to the sum in quadrature of the uncertainties related to jet energy res- affect the b-tagging performance are those related to the track olution, electron and muon performance, b-tagging performance for impact-parameter resolution, the fraction of fake tracks, the light-flavour jets and c-jets, JVT performance, diboson and Z +jet mod- description of the detector material, and the track multiplicity elling (including normalisation and shape uncertainties) and pile-up modelling. All systematic uncertainties are fully correlated bin-by-bin per jet. The uncertainty in the impact-parameter resolution whereas the statistical uncertainty correlations are evaluated following includes the effects of alignment, dead modules and addi- the procedures described in Sect. 8. In the case of correlated systematic tional material not accurately modelled in the simulation. uncertainties, the relative sign of the uncertainty in each bin is taken The uncertainty is derived from several event topologies, into account, even if not shown here including dijet events where effects due to tracking in dense Source of uncertainty Impact on Pb [%] per environments, such as in the cores of high-energy jets, are pseudo-continuous OP included [12]. No dedicated studies of samples enriched in 100–85 85–77 77–70 70–60 60–0 high-energy b-jets, where collimated tracks from displaced Data statistics 4.2 3.0 3.2 2.6 0.7 decay vertices conspire to create a challenging environment MC statistics 1.4 0.5 0.3 0.3 0.2 for the track reconstruction algorithm, are included at this Jet energy scale 1.0 0.2 0.2 0.2 0.1 stage. The effect of the parton shower simulation and b-quark t t¯ modelling 7.2 0.9 0.8 0.9 0.9 fragmentation function is evaluated by comparing the b-jet tagging efficiency with the one obtained from the alterna- Single top modelling 2.4 0.5 0.3 0.2 0.3 tive t t¯ event simulations described in Sect. 5. In standard Fake leptons modelling 0.1 0.1 < 0.1 < 0.1 < 0.1 ATLAS MC simulations, interactions with detector material Other sources combined 2.1 0.5 0.5 2.2 0.1 are simulated only for the decay products of the b-hadron Total 9.1 3.3 3.3 3.2 1.2 and not for the b-hadron itself. Given that about 5% of the b-hadrons within b-jets with jet pT = 150 GeV decay after the innermost pixel detector layer, differences in the b-jet 10.1 Smoothing tagging efficiency at high pT are expected. In order to eval- uate the size of the effect, the Z  sample described in Sect. 5 The simulation-to-data scale factors for single cut OPs are was enhanced to include the interaction of b-hadrons with smoothed in jet pT using a local polynomial kernel estimator the detector material, and the b-jet tagging efficiency derived with a bandwith parameter of 0.2 following the procedure from this sample is compared with the one obtained from the described in Ref. [6]. This procedure prevents distortions in nominal Z  sample. the variables of interest induced by the application of the These sources of uncertainties are found to have a similar scale factors. impact on the b-jet tagging efficiency of the MV2 and DL1 taggers in the jet pT range 400 GeV to 1 TeV. In this jet pT regime, the modelling uncertainties are dominant, reaching 10.2 Extrapolation to high- pT jets 2% for pT ∼ 400 GeV and growing linearly to ∼ 4% at the TeV scale. The uncertainty due to the interaction with the The analysis described in this paper provides a precise mea- detector material is also important and found to be ∼ 1% at surement of the b-jet tagging efficiency in data and compares pT ∼ 700 GeV, growing to ∼ 2% at ∼ 1 TeV. Other leading it with the one obtained from MC simulation. Since there are uncertainties include the jet energy scale and track impact- currently not many b-jets in data for jet pT above 400 GeV parameter resolution uncertainties, reaching about 2.5% and in di-lepton t t¯ events, an alternative assessment of the uncer- 1% at ∼ 1 TeV, respectively. At the TeV scale, the impact of tainty in the b-jet tagging efficiency for jet pT in this range the extrapolation uncertainty is different for MV2 and DL1, is developed to extend the single cut OP calibration to the due to the differing efficiency profile of the two b-taggers. entire jet pT range inspected by physics analyses in ATLAS. The b-jet tagging efficiency of DL1 falls more steeply at high Underlying quantities that are known to affect the b-tagging pT compared to that of MV2, which is approximately con- performance are varied in the simulation one by one and stant. This results in the jet energy scale uncertainty having a the b-jet tagging efficiency is recomputed in each case. The much larger impact for the DL1 tagger, due to the increased difference from the b-jet tagging efficiency obtained in the impact of the migration of jets between the pT bins. 123 970 Page 20 of 36 Eur. Phys. J. C (2019) 79:970 (a) (b) Fig. 12 b-jet tagging efficiency simulation-to-data scale factors for the εb = 70% single cut OP of the (a) MV2 and (b) DL1 taggers, including the smoothed and extrapolated results. The bin centres are used for the smoothing whereas the dots are located at the mean of the b-jet pT distribution in each pT bin The simulation-to-data scale factor measured in the high- 10.4 Reduction of systematic uncertainties est pT bin considered in the collision data analysis is extrap- olated for pT ≥ 400 GeV. The mean value and uncertain- The individual application in a physics analysis of each inde- ties after smoothing at pT = 400 GeV are assumed to stay pendent systematic uncertainty included in Fig. 12 would valid for higher jet pT . An extrapolation uncertainty is then lead to a large number of variations. A method for reducing constructed as the sum in quadrature of all the uncertainties the total number of uncertainties while preserving the bin- described above, rescaled in proportion to their respective by-bin correlations is provided for use in ATLAS physics values in the highest pT bin of the data measurement, and analyses and is described in Ref. [6]. This is achieved by added in quadrature to the pre-existing uncertainties. The constructing the covariance matrix for each source of uncer- result of the procedures of smoothing and extrapolating the tainty and by summing these matrices together. Bin-by-bin single cut OP scale factors are shown in Fig. 12, where both correlations are kept as non-zero off-diagonal elements. As the b-jet tagging efficiency as directly measured in data and this equates to the total covariance matrix, which is sym- its extrapolation are shown for the εb = 70% single cut OP metric and positive-definite, an eigenvector decomposition of the MV2 and DL1 taggers. is performed. The resulting number of variations equals the number of jet pT bins and is further reduced where eigen- value variations are shown to have a negligible impact on a 10.3 Generator dependence result. The b-jet tagging efficiency in the simulation depends on 10.5 Application to physics analyses several properties, such as the production fractions of the different b-hadron species, the fragmentation function and For each jet where b-jet tagging is applied in ATLAS physics the number of additional charged particles near the b-hadron, analyses, a weight is applied in simulation to match the tag- which are not necessarily identical among the different MC ging rate as measured in data by the calibration analyses. event generators. Simulation-to-simulation scale factors are The weight is jet-flavour dependent. The calibration analy- therefore derived to take into account differences in the b-jet sis described in this paper is the baseline correction for jets tagging efficiency due to the usage of a different fragmenta- labelled as b-jets. If the jet is tagged using a single cut OP in tion model to that used to derive the simulation-to-data scale MC simulation the weight is simply the smoothed simulation- factors. The simulation-to-simulation scale factors are com- to-data scale factor itself: puted as ratios of b-jet tagging efficiencies in the same jet pT wjet = S F( pT ) , (2) bins of the alternative and nominal t t¯ samples. For b-jets, they range from 1 to 3% as a function of jet pT . These scale where S F( pT ) is the smoothed b-jet tagging efficiency factors are used when the b-jet tagging efficiency simulation- scale factor evaluated at a given pT . If the jet is not tagged to-data scale factors are applied to a sample produced with the weight becomes: a showering generator different from the one used for the nominal t t¯ sample used in the denominator of the scale fac- 1− b ( pT ) data 1 − S F( pT ) b ( pT ) MC wjet = = . (3) tor calculation. 1− b ( pT ) MC 1− MC ( p ) b T 123 Eur. Phys. J. C (2019) 79:970 Page 21 of 36 970 The latter form of Eq. (3) is adopted because, in this way, to 8% for single cut OPs. The precision of the measurement and by constructing high-granularity efficiency distributions, is limited at low pT by the uncertainty in the jet energy scale, possible differences in the tagging rate induced by event which is detector-related, and at high pT by the size of the topologies are minimised. These weights are necessary to dataset, which will grow in the future. It was previously lim- ensure that the number of events remains the same after cor- ited by the modelling of top-quark pair production. Further rections. The final event weight is then computed as the prod- procedures relating to the simulation-to-data scale factors are uct of all jet weights. In cases where the physics analysis does undertaken to correct for generator dependences, smooth the not rely on the single cut OP but on the pseudo-continuous shape, extrapolate beyond the range of the data measurement, bins of the discriminant distribution, Eq. (2) becomes depen- reduce the number of nuisance parameters and apply them dent on the pT and pseudo-continuous bins T m , O k , and in ATLAS analyses. S F(T m , O k ) becomes the b-jet tagging probability SF mea- This result demonstrates a significant improvement in the sured in the pseudo-continuous bin O k and pT bin T m while precision of the b-jet tagging efficiency measurement for the Eq. (3) becomes unnecessary. ATLAS experiment, which is typically limited by systematic uncertainties. The total uncertainty is reduced by up to a factor of two relative to the previous ATLAS results. This is 11 Conclusion achieved by the improvement in the measurement method, which simultaneously extracts the b-jet tagging efficiency Several b-tagging algorithms are used to analyse data and jet flavour composition. recorded by the ATLAS experiment during Run 2 of the LHC. Their performance is evaluated in simulation, and the Acknowledgements We thank CERN for the very successful oper- ation of the LHC, as well as the support staff from our institutions b-jet tagging efficiencies of the MV2 and DL1 algorithms without whom ATLAS could not be operated efficiently. We acknowl- are measured in pp collision data. edge the support of ANPCyT, Argentina; YerPhI, Armenia; ARC, The b-jet identification strategy combines the results of Australia; BMWFW and FWF, Austria; ANAS, Azerbaijan; SSTC, low-level algorithms (IP2D, IP3D, SV1, JetFitter) into Belarus; CNPq and FAPESP, Brazil; NSERC, NRC and CFI, Canada; CERN; CONICYT, Chile; CAS, MOST and NSFC, China; COLCIEN- high-level algorithms based on multivariate classifiers (MV2, CIAS, Colombia; MSMT CR, MPO CR and VSC CR, Czech Repub- DL1). The low-level algorithms either exploit the large lic; DNRF and DNSRC, Denmark; IN2P3-CNRS, CEA-DRF/IRFU, impact parameters of the tracks originating from the b-hadron France; SRNSFG, Georgia; BMBF, HGF, and MPG, Germany; GSRT, decay products or attempt to directly reconstruct heavy- Greece; RGC, Hong Kong SAR, China; ISF and Benoziyo Center, Israel; INFN, Italy; MEXT and JSPS, Japan; CNRST, Morocco; NWO, flavour hadron vertices. Large increases in light-flavour jet The Netherlands; RCN, Norway; MNiSW and NCN, Poland; FCT, Por- and c-jet rejection are obtained by the MV2 and DL1 algo- tugal; MNE/IFA, Romania; MES of Russia and NRC KI, Russian Fed- rithms compared to each individual low-level algorithm, eration; JINR; MESTD, Serbia; MSSR, Slovakia; ARRS and MIZŠ, illustrating the high complementarity of the latter and val- Slovenia; DST/NRF, South Africa; MINECO, Spain; SRC and Wal- lenberg Foundation, Sweden; SERI, SNSF and Cantons of Bern and idating the overall strategy followed by the ATLAS Collab- Geneva, Switzerland; MOST, Taiwan; TAEK, Turkey; STFC, United oration. Kingdom; DOE and NSF, USA. In addition, individual groups and mem- The b-jet tagging efficiency of the MV2 and DL1 algo- bers have received support from BCKDF, CANARIE, CRC and Com- rithms are measured in 80.5 fb−1 of proton–proton colli- pute Canada, Canada; COST, ERC, ERDF, Horizon 2020, and Marie Skłodowska-Curie Actions, European Union; Investissements d’ Avenir sion data collected by the ATLAS detector over the period √ Labex and Idex, ANR, France; DFG and AvH Foundation, Germany; 2015–2017 at a centre-of-mass energy s = 13 TeV. A Herakleitos, Thales and Aristeia programmes co-financed by EU-ESF high-purity sample of dileptonic t t¯ events is obtained by and the Greek NSRF, Greece; BSF-NSF and GIF, Israel; CERCA Pro- retaining events with exactly one muon, one electron and two gramme Generalitat de Catalunya, Spain; The Royal Society and Lev- erhulme Trust, UK. The crucial computing support from all WLCG hadronic jets. Events are classified according to the transverse partners is acknowledged gratefully, in particular from CERN, the momentum of each of the two jets as well as the jet–lepton ATLAS Tier-1 facilities at TRIUMF (Canada), NDGF (Denmark, Nor- invariant masses obtained when pairing exclusively each jet way, Sweden), CC-IN2P3 (France), KIT/GridKA (Germany), INFN- with the lepton most likely to originate from the same top- CNAF (Italy), NL-T1 (The Netherlands), PIC (Spain), ASGC (Tai- wan), RAL (UK) and BNL (USA), the Tier-2 facilities worldwide and quark decay. A combinatorial likelihood approach is then large non-WLCG resource providers. Major contributors of computing used to simultaneously extract the jet flavour composition resources are listed in Ref. [64]. of the sample and the b-jet tagging probabilities for jets in a transverse momentum range from 20 to 600 GeV, from which Data Availability Statement This manuscript has no associated data or the data will not be deposited. [Authors’ comment: All ATLAS sci- the b-jet tagging efficiencies for single cut OPs are obtained. entific output is published in journals, and preliminary results are made Simulation-to-data scale factors are computed by comparing available in Conference Notes. All are openly available, without restric- the efficiency measured in collision data with that observed tion on use by external parties beyond copyright law and the standard in the simulation. The measured simulation-to-data scale fac- conditions agreed by CERN. Data associated with journal publications are also made available: tables and data from plots (e.g. cross section tors are close to unity with a total uncertainty ranging from 1 123 970 Page 22 of 36 Eur. Phys. J. C (2019) 79:970 values, likelihood profiles, selection efficiencies, cross section limits, 15. ATLAS Collaboration, √ Vertex Reconstruction Performance of the ...) are stored in appropriate repositories such as HEPDATA (http:// ATLAS Detector at s = 13 TeV, ATL-PHYS-PUB-2015-026, hepdata.cedar.ac.uk/). ATLAS also strives to make additional material 2015. https://cds.cern.ch/record/2037717 related to the paper available that allows a reinterpretation of the data 16. P. Billoir, S. Qian, Fast vertex fitting with a local parametrization in the context of new theoretical models. For example, an extended of tracks. Nucl. Instrum. Methods Phys. Res., A 311, 139 (1992) encapsulation of the analysis is often provided for measurements in the 17. ATLAS Collaboration, Topological cell clustering in the ATLAS framework of RIVET (http://rivet.hepforge.org/). This information is calorimeters and its performance in LHC Run 1, Eur. Phys. J. C taken from the ATLAS Data Access Policy, which is a public docu- 77, 490 (2017). arXiv: 1603.02934 [hep-ex] ment that can be downloaded from http://opendata.cern.ch/record/413 18. M. Cacciari, G.P. Salam, G. Soyez, The anti-kt jet clustering algo- [opendata.cern.ch]. rithm. JHEP 04, 063 (2008). arXiv:0802.1189 [hep-ph] 19. ATLAS Collaboration, Jet energy scale measurements and √ their Open Access This article is distributed under the terms of the Creative systematic uncertainties in proton–proton collisions at s = Commons Attribution 4.0 International License (http://creativecomm 13 TeV with the ATLAS detector. Phys. Rev. D 96, 072002 (2017). ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, arXiv:1703.09665 [hep-ex] and reproduction in any medium, provided you give appropriate credit 20. ATLAS Collaboration, Selection of jets produced in 13 TeV to the original author(s) and the source, provide a link to the Creative proton–proton collisions with the ATLAS detector. ATLAS- Commons license, and indicate if changes were made. CONF-2015-029 (2015). https://cds.cern.ch/record/2037702 Funded by SCOAP3 . 21. ATLAS Collaboration, Tagging and suppression of pileup jets with the ATLAS detector. ATLAS-CONF-2014-018 (2014). https://cds. cern.ch/record/1700870 22. M. Cacciari, G.P. Salam, G. Soyez, The catchment area of jets. JHEP 04, 005 (2008). arXiv:0802.1188 [hep-ph] 23. ATLAS Collaboration, Optimisation and performance studies of References the ATLAS b-tagging algorithms for the 2017-18 LHC run. ATL- PHYS-PUB-2017-013 (2017). https://cds.cern.ch/record/2273281 1. ATLAS Collaboration, The ATLAS Experiment at the CERN Large 24. ATLAS Collaboration, Secondary vertex finding for jet flavour Hadron Collider, JINST 3, S08003 (2008) identification with the ATLAS detector, ATL-PHYS-PUB-2017- 2. L. Evans, P. Bryant, LHC Machine, JINST 3, S08001 (2008) 011 (2017). https://cds.cern.ch/record/2270366 3. ATLAS Collaboration, Observation of H → bb̄ decays and V H 25. ATLAS Collaboration, Topological b-hadron decay reconstruction production with the ATLAS detector, Phys. Lett. B 786, 59 (2018). and identification of b-jets with the JetFitter package in the ATLAS arXiv: 1808.08238 [hep-ex] experiment at the LHC, ATL-PHYS-PUB-2018-025, 2018, URL: 4. ATLAS Collaboration, Observation of Higgs boson production in https://cds.cern.ch/record/2645405 association with a top quark pair at the LHC with the ATLAS 26. R. Frühwirth, Application of Kalman filtering to track and vertex detector, Phys. Lett. B 784, 173 (2018). arXiv: 1806.00425 [hep- fitting. Nucl. Instrum. Meth. A 262, 444 (1987) ex] 27. A. Höcker et al., TMVA—toolkit for multivariate data analysis 5. ATLAS Collaboration, Performance of b-jet identification in the (2007). arXiv:physics/0703039 [physics.data-an] ATLAS experiment, JINST 11, P04008 (2016). arXiv: 1512.01094 28. F. Chollet et al., Keras. https://keras.io (2015) [hep-ex] 29. Theano Development Team, Theano: a Python framework 6. ATLAS Collaboration, Measurements of b-jet √ tagging efficiency for fast computation of mathematical expressions (2016). with the ATLAS detector using t t¯ events at s = 13 TeV, JHEP arXiv:1605.02688 [cs.SC] 08, 089 (2018). arXiv: 1805.01845 [hep-ex] 30. D.P. Kingma, J. Ba, Adam: A method for stochastic optimization 7. ATLAS Collaboration, Calibration of the ATLAS b-tagging algo- (2014). arXiv: 1412.6980 [cs.SC] rithm in t t¯ semileptonic events. ATLAS-CONF-2018-045, 2018. 31. I.J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, Y. Ben- https://cds.cern.ch/record/2638455 gio, Maxout networks (2013). arXiv:1302.4389 [stat.ML] 8. ATLAS Collaboration, Calibration of light-flavour b-jet mistag- √ 32. S. Ioffe, C. Szegedy, Batch normalization: accelerating deep ging rates using ATLAS proton–proton collision data at s = network training by reducing internal covariate shift (2015). 13 TeV, ATLAS-CONF-2018-006, 2018. https://cds.cern.ch/ arXiv:1502.03167 [cs.LG] record/2314418 33. ATLAS √ Collaboration, Luminosity determination in pp collisions 9. ATLAS Collaboration, ATLAS Insertable B-Layer Technical at s = 13 TeV using the ATLAS detector at the LHC. ATLAS- Design Report, ATLAS-TDR-19, 2010, URL: https://cds.cern. CONF-2019-021 (2019). https://cds.cern.ch/record/2677054 ch/record/1291633, Addendum: ATLAS-TDR-19-ADD-1, 2012. 34. G. Avoni et al., The new LUCID-2 detector for luminosity mea- https://cds.cern.ch/record/1451888 surement and monitoring in ATLAS. JINST 13, P07017 (2018) 10. B. Abbott et al., Production and integration of the ATLAS 35. P. Nason, A New method for combining NLO QCD with Insertable B-Layer. JINST 13, T05008 (2018). arXiv:1803.00844 shower Monte Carlo algorithms. JHEP 11, 040 (2004). [physics.ins-det] arXiv:hep-ph/0409146 11. ATLAS Collaboration, Performance of the ATLAS trigger system 36. S. Frixione, P. Nason, C. Oleari, Matching NLO QCD computations in 2015, Eur. Phys. J. C 77, 317 (2017). arXiv:1611.09661 [hep-ex] with parton shower simulations: the POWHEG method. JHEP 11, 12. ATLAS Collaboration, Performance of the ATLAS track recon- 070 (2007). arXiv:0709.2092 [hep-ph] struction algorithms in dense environments in LHC Run 2, Eur. 37. S. Frixione, P. Nason, G. Ridolfi, A Positive-weight next-to- Phys. J. C 77, 673 (2017). arXiv:1704.07983 [hep-ex] leading-order Monte Carlo for heavy flavour hadroproduction. 13. ATLAS Collaboration, √ Early Inner Detector Tracking Performance JHEP 09, 126 (2007). arXiv:0707.3088 [hep-ph] in the 2015 Data at s = 13 TeV, ATL-PHYS-PUB-2015-051, 38. S. Alioli, P. Nason, C. Oleari, E. Re, A general framework for 2015. https://cds.cern.ch/record/2110140 implementing NLO calculations in shower Monte Carlo programs: 14. ATLAS Collaboration, Reconstruction of primary vertices at the the POWHEG BOX. JHEP 06, 043 (2010). arXiv:1002.2581 [hep- ATLAS experiment in Run 1 proton-proton collisions at the LHC, ph] Eur. Phys. J. C 77, 332 (2017). arXiv:1611.10235 [hep-ex] 123 Eur. Phys. J. C (2019) 79:970 Page 23 of 36 970 39. T. Sjöstrand, S. Mrenna, P. Z. Skands, A brief introduction to 53. S. Agostinelli et al., GEANT4—a simulation toolkit. Nucl. Instrum. PYTHIA 8.1, Comput. Phys. Commun. 178 , 852 (2008). arXiv: Meth. A 506, 250 (2003) 0710.3820 [hep-ph] 54. ATLAS Collaboration, The simulation principle and performance 40. ATLAS Collaboration, ATLAS Pythia 8 tunes to 7 TeV data. ATL- of the ATLAS fast calorimeter simulation FastCaloSim. ATL- PHYS-PUB-2014-021 (2014). https://cds.cern.ch/record/1966419 PHYS-PUB-2010-013 (2010). https://cds.cern.ch/record/1300517 41. R.D. Ball et al., Parton distributions for the LHC Run II. JHEP 04, 55. ATLAS Collaboration, Electron and photon performance measure- 040 (2015). arXiv:1410.8849 [hep-ph] ments with the ATLAS detector using the 2015–2017 LHC proton– 42. R.D. Ball et al., Parton distributions with LHC data. Nucl. Phys. B proton collision data (2019). arXiv: 1908.00005 [hep-ex] 867, 244 (2013). arXiv:1207.1303 [hep-ph] 56. ATLAS Collaboration, Muon reconstruction performance √ of the 43. ATLAS Collaboration, Studies on top-quark Monte Carlo mod- ATLAS detector in proton–proton collision data at s = 13 TeV, elling for Top2016. ATL-PHYS-PUB- 2016-020, (2016). https:// Eur. Phys. J. C 76, 292 (2016). arXiv:1603.05598 [hep-ex] cds.cern.ch/record/2216168 57. ATLAS Collaboration, Measurements of differential cross sections 44. J. Bellm et al., Herwig 7.0/Herwig++ 3.0 release note. Eur. Phys. of top quark √ pair production in association with jets in pp colli- J. C 76, 196 (2016) sions at s = 13 TeV using the ATLAS detector. JHEP 10, 159 45. S. Frixione, E. Laenen, P. Motylinski, B.R. Webber, C.D. White, (2018). arXiv:1802.06572 [hep-ex] Single-top hadroproduction in association with a W boson. JHEP 58. F. James, M. Winkler, MINUIT user’s guide (2004). http://seal. 07, 029 (2008). arXiv:0805.3067 [hep-ph] web.cern.ch/seal/documents/minuit/mnusersguide.pdf 46. E. Re, Single-top Wt-channel production matched with parton 59. G. Bohm, G. Zech, Introduction to statistics and data analysis for showers using the POWHEG method. Eur. Phys. J. C 71, 1547 physicists; 3rd revised, Verlag Deutsches Elektronen-Synchrotron (2011). arXiv:1009.2450 [hep-ph] (2017) 488, ISBN: 978-3-945931-13-4. http://bib-pubdb1.desy.de/ 47. T. Gleisberg et al., Event generation with SHERPA 1.1. JHEP 02, record/389738 007 (2009). arXiv:0811.4622 [hep-ph] 60. J. Butterworth et al., PDF4LHC recommendations for LHC Run 48. S. Höche, F. Krauss, S. Schumann, F. Siegert, QCD matrix elements II. J. Phys. G 43, 023001 (2016). arXiv:1510.03865 [hep-ph] and truncated showers. JHEP 05, 053 (2009). arXiv:0903.1219 61. ATLAS Collaboration, Multi-Boson Simulation for 13 TeV [hep-ph] ATLAS analyses. ATL-PHYS-PUB-2017-005 (2017). https://cds. 49. S. Alioli, P. Nason, C. Oleari, E. Re, NLO single-top production cern.ch/record/2261933 matched with shower in POWHEG: s- and t-channel contribu- 62. ATLAS Collaboration, ATLAS simulation of boson plus jets pro- tions. JHEP 09, 111 (2009). [Erratum: JHEP02,011(2010)], arXiv: cesses in Run 2. ATL-PHYS-PUB- 2017-006 (2017). https://cds. 0907.4076 [hep-ph] cern.ch/record/2261937 50. D.J. Lange, The EvtGen particle decay simulation package. Nucl. 63. K. Pearson, On the criterion that a given system of deviations from Instrum. Meth. A 462, 152 (2001) the probable in the case of a correlated system of variables is such 51. ATLAS Collaboration, The Pythia 8 A3 tune description of that it can be reasonably supposed to have arisen from random ATLAS minimum bias and inelastic measurements incorporat- sampling. Lond. Edinburgh Dublin Philos. Mag. J. Sci. 50, 157 ing the Donnachie-Landshoff diffractive model. ATL-PHYS-PUB- (1900) 2016- 017 (2016). https://cds.cern.ch/record/2206965 64. ATLAS Collaboration, ATLAS computing acknowledgements. 52. ATLAS Collaboration, The ATLAS Simulation Infrastructure. Eur. ATL-GEN-PUB-2016-002. https://cds.cern.ch/record/2202407 Phys. J. C 70, 823 (2010). arXiv: 1005.4568 [physics.ins-det] ATLAS Collaboration G. Aad101 , B. Abbott128 , D. C. Abbott102 , A. Abed Abud70a,70b , K. Abeling53 , D. K. Abhayasinghe93 , S. H. Abidi167 , O. S. AbouZeid40 , N. L. Abraham156 , H. Abramowicz161 , H. Abreu160 , Y. Abulaiti6 , B. S. Acharya66a,66b,o , B. Achkar53 , S. Adachi163 , L. Adam99 , C. Adam Bourdarios5 , L. Adamczyk83a , L. Adamek167 , J. Adelman121 , M. Adersberger114 , A. Adiguzel12c,aj , S. Adorni54 , T. Adye144 , A. A. Affolder146 , Y. Afik160 , C. Agapopoulou132 , M. N. Agaras38 , A. Aggarwal119 , C. Agheorghiesei27c , J. A. Aguilar-Saavedra140a,140f,ai , F. Ahmadov79 , W. S. Ahmed103 , X. Ai18 , G. Aielli73a,73b , S. Akatsuka85 , T. P. A. Åkesson96 , E. Akilli54 , A. V. Akimov110 , K. Al Khoury132 , G. L. Alberghi23a,23b , J. Albert176 , M. J. Alconada Verzini161 , S. Alderweireldt36 , M. Aleksa36 , I. N. Aleksandrov79 , C. Alexa27b , D. Alexandre19 , T. Alexopoulos10 , A. Alfonsi120 , M. Alhroob128 , B. Ali142 , G. Alimonti68a , J. Alison37 , S. P. Alkire148 , C. Allaire132 , B. M. M. Allbrooke156 , B. W. Allen131 , P. P. Allport21 , A. Aloisio69a,69b , A. Alonso40 , F. Alonso88 , C. Alpigiani148 , A. A. Alshehri57 , M. Alvarez Estevez98 , D. Álvarez Piqueras174 , M. G. Alviggi69a,69b , Y. Amaral Coutinho80b , 103 135 26 105 A. Ambler , L. Ambroz , C. Amelung , D. Amidei , S. P. Amor Dos Santos 140a , S. Amoroso46 , C. S. Amrouche54 , F. An , C. Anastopoulos , N. Andari , T. Andeen , C. F. Anders , J. K. Anders , A. Andreazza68a,68b , V. Andrei61a , 78 149 145 11 61b 20 C. R. Anelli176 , S. Angelidakis38 , A. Angerami39 , A. V. Anisenkov122a,122b , A. Annovi71a , C. Antel61a , M. T. Anthony149 , M. Antonelli51 , D. J. A. Antrim171 , F. Anulli72a , M. Aoki81 , J. A. Aparisi Pozo174 , L. Aperio Bella36 , G. Arabidze106 , J. P. Araque140a , V. Araujo Ferraz80b , R. Araujo Pereira80b , C. Arcangeletti51 , A. T. H. Arce49 , F. A. Arduh88 , J.- F. Arguin109 , S. Argyropoulos77 , J.-H. Arling46 , A. J. Armbruster36 , A. Armstrong171 , O. Arnaez167 , H. Arnold120 , A. Artamonov111,* , G. Artoni135 , S. Artz99 , S. Asai163 , N. Asbah59 , E. M. Asimakopoulou172 , L. Asquith156 , J. Assahsah35d , K. Assamagan29 , R. Astalos28a , R. J. Atkin33a , M. Atkinson173 , N. B. Atlay19 , H. Atmani132 , K. Augsten142 , G. Avolio36 , R. Avramidou60a , M. K. Ayoub15a , A. M. Azoulay168b , G. Azuelos109,ay , H. Bachacou145 , K. Bachas67a,67b , M. Backes135 , 123 970 Page 24 of 36 Eur. Phys. J. C (2019) 79:970 F. Backman45a,45b , P. Bagnaia72a,72b , M. Bahmani84 , H. Bahrasemani152 , A. J. Bailey174 , V. R. Bailey173 , J. T. Baines144 , M. Bajic40 , C. Bakalis10 , O. K. Baker183 , P. J. Bakker120 , D. Bakshi Gupta8 , S. Balaji157 , E. M. Baldin122a,122b , P. Balek180 , F. Balli145 , W. K. Balunas135 , J. Balz99 , E. Banas84 , A. Bandyopadhyay24 , Sw. Banerjee181,j , A. A. E. Bannoura182 , L. Barak161 , W. M. Barbe38 , E. L. Barberio104 , D. Barberis55a,55b , M. Barbero101 , T. Barillari115 , M.-S. Barisits36 , J. Barkeloo131 , T. Barklow153 , R. Barnea160 , S. L. Barnes60c , B. M. Barnett144 , R. M. Barnett18 , Z. Barnovska-Blenessy60a , A. Baroncelli60a , G. Barone29 , A. J. Barr135 , L. Barranco Navarro45a,45b , F. Barreiro98 , J. Barreiro Guimarães da Costa15a , S. Barsov138 , R. Bartoldus153 , G. Bartolini101 , A. E. Barton89 , P. Bartos28a , A. Basalaev46 , A. Bassalat132,ar , M. J. Basso167 , R. L. Bates57 , S. Batlamous35e , J. R. Batley32 , B. Batool151 , M. Battaglia146 , M. Bauce72a,72b , F. Bauer145 , K. T. Bauer171 , H. S. Bawa31,m , J. B. Beacham49 , T. Beau136 , P. H. Beauchemin170 , F. Becherer52 , P. Bechtle24 , H. C. Beck53 , H. P. Beck20,s , K. Becker52 , M. Becker99 , C. Becot46 , A. Beddall12d , A. J. Beddall12a , V. A. Bednyakov79 , M. Bedognetti120 , C. P. Bee155 , T. A. Beermann76 , M. Begalli80b , M. Begel29 , A. Behera155 , J. K. Behr46 , F. Beisiegel24 , A. S. Bell94 , G. Bella161 , L. Bellagamba23b , A. Bellerive34 , P. Bellos9 , K. Beloborodov122a,122b , K. Belotskiy112 , N. L. Belyaev112 , D. Benchekroun35a , N. Benekos10 , Y. Benhammou161 , D. P. Benjamin6 , M. Benoit54 , J. R. Bensinger26 , S. Bentvelsen120 , L. Beresford135 , M. Beretta51 , D. Berge46 , E. Bergeaas Kuutmann172 , N. Berger5 , B. Bergmann142 , L. J. Bergsten26 , J. Beringer18 , S. Berlendis7 , N. R. Bernard102 , G. Bernardi136 , C. Bernius153 , T. Berry93 , P. Berta99 , C. Bertella15a , I. A. Bertram89 , O. Bessidskaia Bylund182 , N. Besson145 , A. Bethani100 , S. Bethke115 , A. Betti24 , A. J. Bevan92 , J. Beyer115 , R. Bi139 , R. M. Bianchi139 , O. Biebel114 , D. Biedermann19 , R. Bielski36 , K. Bierwagen99 , N. V. Biesuz71a,71b , M. Biglietti74a , T. R. V. Billoud109 , M. Bindi53 , A. Bingul12d , C. Bini72a,72b , S. Biondi23a,23b , M. Birman180 , T. Bisanz53 , J. P. Biswal161 , D. Biswas181,j , A. Bitadze100 , C. Bittrich48 , K. Bjørke134 , K. M. Black25 , T. Blazek28a , I. Bloch46 , C. Blocker26 , A. Blue57 , U. Blumenschein92 , G. J. Bobbink120 , V. S. Bobrovnikov122a,122b , S. S. Bocchetta96 , A. Bocci49 , D. Boerner46 , D. Bogavac14 , A. G. Bogdanchikov122a,122b , C. Bohm45a , V. Boisvert93 , P. Bokan53,172 , T. Bold83a , A. S. Boldyrev113 , A. E. Bolz61b , M. Bomben136 , M. Bona92 , J. S. Bonilla131 , M. Boonekamp145 , H. M. Borecka-Bielska90 , A. Borisov123 , G. Borissov89 , J. Bortfeldt36 , D. Bortoletto135 , V. Bortolotto73a,73b , D. Boscherini23b , M. Bosman14 , J. D. Bossio Sola103 , K. Bouaouda35a , J. Boudreau139 , E. V. Bouhova-Thacker89 , D. Boumediene38 , S. K. Boutle57 , A. Boveia126 , J. Boyd36 , D. Boye33b,as , I. R. Boyko79 , A. J. Bozson93 , J. Bracinik21 , N. Brahimi101 , G. Brandt182 , O. Brandt32 , F. Braren46 , B. Brau102 , J. E. Brau131 , W. D. Breaden Madden57 , K. Brendlinger46 , L. Brenner46 , R. Brenner172 , S. Bressler180 , B. Brickwedde99 , D. L. Briglin21 , D. Britton57 , D. Britzger115 , I. Brock24 , R. Brock106 , G. Brooijmans39 , W. K. Brooks147b , E. Brost121 , J. H. Broughton21 , P. A. Bruckman de Renstrom84 , D. Bruncko28b , A. Bruni23b , G. Bruni23b , L. S. Bruni120 , S. Bruno73a,73b , B. H. Brunt32 , M. Bruschi23b , N. Bruscino139 , P. Bryant37 , L. Bryngemark96 , T. Buanes17 , Q. Buat36 , P. Buchholz151 , A. G. Buckley57 , I. A. Budagov79 , M. K. Bugge134 , F. Bührer52 , O. Bulekov112 , T. J. Burch121 , S. Burdin90 , C. D. Burgard120 , A. M. Burger129 , B. Burghgrave8 , J. T. P. Burr46 , J. C. Burzynski102 , V. Büscher99 , E. Buschmann53 , P. J. Bussey57 , J. M. Butler25 , C. M. Buttar57 , J. M. Butterworth94 , P. Butti36 , W. Buttinger36 , A. Buzatu158 , A. R. Buzykaev122a,122b , G. Cabras23a,23b , S. Cabrera Urbán174 , D. Caforio56 , H. Cai173 , V. M. M. Cairo153 , O. Cakir4a , N. Calace36 , P. Calafiura18 , A. Calandri101 , G. Calderini136 , P. Calfayan65 , G. Callea57 , L. P. Caloba80b , S. Calvente Lopez98 , D. Calvet38 , S. Calvet38 , T. P. Calvet155 , M. Calvetti71a,71b , R. Camacho Toro136 , S. Camarda36 , D. Camarero Munoz98 , P. Camarri73a,73b , D. Cameron134 , R. Caminal Armadans102 , C. Camincher36 , S. Campana36 , M. Campanelli94 , A. Camplani40 , A. Campoverde151 , V. Canale69a,69b , A. Canesse103 , M. Cano Bret60c , J. Cantero129 , T. Cao161 , Y. Cao173 , M. D. M. Capeans Garrido36 , M. Capua41a,41b , R. Cardarelli73a , F. Cardillo149 , G. Carducci41a,41b , I. Carli143 , T. Carli36 , G. Carlino69a , B. T. Carlson139 , L. Carminati68a,68b , R. M. D. Carney45a,45b , S. Caron119 , E. Carquin147b , S. Carrá46 , J. W. S. Carter167 , M. P. Casado14,e , A. F. Casha167 , D. W. Casper171 , R. Castelijn120 , F. L. Castillo174 , V. Castillo Gimenez174 , N. F. Castro140a,140e , A. Catinaccio36 , J. R. Catmore134 , A. Cattai36 , J. Caudron24 , V. Cavaliere29 , E. Cavallaro14 , M. Cavalli-Sforza14 , V. Cavasinni71a,71b , E. Celebi12b , F. Ceradini74a,74b , L. Cerda Alberich174 , K. Cerny130 , A. S. Cerqueira80a , A. Cerri156 , L. Cerrito73a,73b , F. Cerutti18 , A. Cervelli23a,23b , S. A. Cetin12b , Z. Chadi35a , D. Chakraborty121 , S. K. Chan59 , W. S. Chan120 , W. Y. Chan90 , J. D. Chapman32 , B. Chargeishvili159b , D. G. Charlton21 , T. P. Charman92 , C. C. Chau34 , S. Che126 , A. Chegwidden106 , S. Chekanov6 , S. V. Chekulaev168a , G. A. Chelkov79,ax , M. A. Chelstowska36 , B. Chen78 , C. Chen60a , C. H. Chen78 , H. Chen29 , J. Chen60a , J. Chen39 , S. Chen137 , S. J. Chen15c , X. Chen15b,aw , Y. Chen82 , Y-H. Chen46 , H. C. Cheng63a , H. J. Cheng15a,15d , A. Cheplakov79 , E. Cheremushkina123 , R. Cherkaoui El Moursli35e , E. Cheu7 , K. Cheung64 , T. J. A. Chevalérias145 , L. Chevalier145 , V. Chiarella51 , G. Chiarelli71a , G. Chiodini67a , A. S. Chisholm36,21 , A. Chitan27b , I. Chiu163 , Y. H. Chiu176 , M. V. Chizhov79 , K. Choi65 , A. R. Chomont72a,72b , S. Chouridou162 , Y. S. Chow120 , M. C. Chu63a , X. Chu15a , J. Chudoba141 , A. J. Chuinard103 , J. J. Chwastowski84 , L. Chytka130 , D. Cieri115 , K. M. Ciesla84 , D. Cinca47 , V. Cindro91 , I. A. Cioară27b , A. Ciocio18 , F. Cirotto69a,69b , Z. H. Citron180,k , M. Citterio68a , D. A. Ciubotaru27b , B. M. Ciungu167 , A. Clark54 , M. R. Clark39 , P. J. Clark50 , C. Clement45a,45b , 123 Eur. Phys. J. C (2019) 79:970 Page 25 of 36 970 Y. Coadou101 , M. Cobal66a,66c , A. Coccaro55b , J. Cochran78 , H. Cohen161 , A. E. C. Coimbra36 , L. Colasurdo119 , B. Cole39 , A. P. Colijn120 , J. Collot58 , P. Conde Muiño140a,f , E. Coniavitis52 , S. H. Connell33b , I. A. Connelly57 , S. Constantinescu27b , F. Conventi69a,az , A. M. Cooper-Sarkar135 , F. Cormier175 , K. J. R. Cormier167 , L. D. Corpe94 , M. Corradi72a,72b , E. E. Corrigan96 , F. Corriveau103,ae , A. Cortes-Gonzalez36 , M. J. Costa174 , F. Costanza5 , D. Costanzo149 , G. Cowan93 , J. W. Cowley32 , J. Crane100 , K. Cranmer124 , S. J. Crawley57 , R. A. Creager137 , S. Crépé-Renaudin58 , F. Crescioli136 , M. Cristinziani24 , V. Croft120 , G. Crosetti41a,41b , A. Cueto5 , T. Cuhadar Donszelmann149 , A. R. Cukierman153 , S. Czekierda84 , P. Czodrowski36 , M. J. Da Cunha Sargedas De Sousa60b , J. V. Da Fonseca Pinto80b , C. Da Via100 , W. Dabrowski83a , T. Dado28a , S. Dahbi35e , T. Dai105 , C. Dallapiccola102 , M. Dam40 , G. D’amen29 , V. D’Amico74a,74b , J. Damp99 , J. R. Dandoy137 , M. F. Daneri30 , N. P. Dang181,j , N. S. Dann100 , M. Danninger175 , V. Dao36 , G. Darbo55b , O. Dartsi5 , A. Dattagupta131 , T. Daubney46 , S. D’Auria68a,68b , W. Davey24 , C. David46 , T. Davidek143 , D. R. Davis49 , I. Dawson149 , K. De8 , R. De Asmundis69a , M. De Beurs120 , S. De Castro23a,23b , S. De Cecco72a,72b , N. De Groot119 , P. de Jong120 , H. De la Torre106 , A. De Maria15c , D. De Pedis72a , A. De Salvo72a , U. De Sanctis73a,73b , M. De Santis73a,73b , A. De Santo156 , K. De Vasconcelos Corga101 , J. B. De Vivie De Regie132 , C. Debenedetti146 , D. V. Dedovich79 , A. M. Deiana42 , M. Del Gaudio41a,41b , J. Del Peso98 , Y. Delabat Diaz46 , D. Delgove132 , F. Deliot145,r , C. M. Delitzsch7 , M. Della Pietra69a,69b , D. Della Volpe54 , A. Dell’Acqua36 , L. Dell’Asta73a,73b , M. Delmastro5 , C. Delporte132 , P. A. Delsart58 , D. A. DeMarco167 , S. Demers183 , M. Demichev79 , G. Demontigny109 , S. P. Denisov123 , D. Denysiuk120 , L. D’Eramo136 , D. Derendarz84 , J. E. Derkaoui35d , F. Derue136 , P. Dervan90 , K. Desch24 , C. Deterre46 , K. Dette167 , C. Deutsch24 , M. R. Devesa30 , P. O. Deviveiros36 , A. Dewhurst144 , F. A. Di Bello54 , A. Di Ciaccio73a,73b , L. Di Ciaccio5 , W. K. Di Clemente137 , C. Di Donato69a,69b , A. Di Girolamo36 , G. Di Gregorio71a,71b , B. Di Micco74a,74b , R. Di Nardo102 , K. F. Di Petrillo59 , R. Di Sipio167 , D. Di Valentino34 , C. Diaconu101 , F. A. Dias40 , T. Dias Do Vale140a , M. A. Diaz147a , J. Dickinson18 , E. B. Diehl105 , J. Dietrich19 , S. Díez Cornell46 , A. Dimitrievska18 , W. Ding15b , J. Dingfelder24 , F. Dittus36 , F. Djama101 , T. Djobava159b , J. I. Djuvsland17 , M. A. B. Do Vale80c , M. Dobre27b , D. Dodsworth26 , C. Doglioni96 , J. Dolejsi143 , Z. Dolezal143 , M. Donadelli80d , B. Dong60c , J. Donini38 , A. D’onofrio92 , M. D’Onofrio90 , J. Dopke144 , A. Doria69a , M. T. Dova88 , A. T. Doyle57 , E. Drechsler152 , E. Dreyer152 , T. Dreyer53 , A. S. Drobac170 , Y. Duan60b , F. Dubinin110 , M. Dubovsky28a , A. Dubreuil54 , E. Duchovni180 , G. Duckeck114 , A. Ducourthial136 , O. A. Ducu109 , D. Duda115 , A. Dudarev36 , A. C. Dudder99 , E. M. Duffield18 , L. Duflot132 , M. Dührssen36 , C. Dülsen182 , M. Dumancic180 , A. E. Dumitriu27b , A. K. Duncan57 , M. Dunford61a , A. Duperrin101 , H. Duran Yildiz4a , M. Düren56 , A. Durglishvili159b , D. Duschinger48 , B. Dutta46 , D. Duvnjak1 , G. I. Dyckes137 , M. Dyndal36 , S. Dysch100 , B. S. Dziedzic84 , K. M. Ecker115 , R. C. Edgar105 , M. G. Eggleston49 , T. Eifert36 , G. Eigen17 , K. Einsweiler18 , T. Ekelof172 , H. El Jarrari35e , M. El Kacimi35c , R. El Kosseifi101 , V. Ellajosyula172 , M. Ellert172 , F. Ellinghaus182 , A. A. Elliot92 , N. Ellis36 , J. Elmsheuser29 , M. Elsing36 , D. Emeliyanov144 , A. Emerman39 , Y. Enari163 , M. B. Epland49 , J. Erdmann47 , A. Ereditato20 , M. Errenst36 , M. Escalier132 , C. Escobar174 , O. Estrada Pastor174 , E. Etzion161 , H. Evans65 , A. Ezhilov138 , F. Fabbri57 , L. Fabbri23a,23b , V. Fabiani119 , G. Facini94 , R. M. Faisca Rodrigues Pereira140a , R. M. Fakhrutdinov123 , S. Falciano72a , P. J. Falke5 , S. Falke5 , J. Faltova143 , Y. Fang15a , Y. Fang15a , G. Fanourakis44 , M. Fanti68a,68b , M. Faraj66a,66c,u , A. Farbin8 , A. Farilla74a , E. M. Farina70a,70b , T. Farooque106 , S. Farrell18 , S. M. Farrington50 , P. Farthouat36 , F. Fassi35e , P. Fassnacht36 , D. Fassouliotis9 , M. Faucci Giannelli50 , W. J. Fawcett32 , L. Fayard132 , O. L. Fedin138,p , W. Fedorko175 , M. Feickert42 , L. Feligioni101 , A. Fell149 , C. Feng60b , E. J. Feng36 , M. Feng49 , M. J. Fenton57 , A. B. Fenyuk123 , J. Ferrando46 , A. Ferrante173 , A. Ferrari172 , P. Ferrari120 , R. Ferrari70a , D. E. Ferreira de Lima61b , A. Ferrer174 , D. Ferrere54 , C. Ferretti105 , F. Fiedler99 , A. Filipčič91 , F. Filthaut119 , K. D. Finelli25 , M. C. N. Fiolhais140a,140c,a , L. Fiorini174 , F. Fischer114 , W. C. Fisher106 , I. Fleck151 , P. Fleischmann105 , R. R. M. Fletcher137 , T. Flick182 , B. M. Flierl114 , L. Flores137 , L. R. Flores Castillo63a , F. M. Follega75a,75b , N. Fomin17 , J. H. Foo167 , G. T. Forcolin75a,75b , A. Formica145 , F. A. Förster14 , A. C. Forti100 , A. G. Foster21 , M. G. Foti135 , D. Fournier132 , H. Fox89 , P. Francavilla71a,71b , S. Francescato72a,72b , M. Franchini23a,23b , S. Franchino61a , D. Francis36 , L. Franconi20 , M. Franklin59 , A. N. Fray92 , P. M. Freeman21 , B. Freund109 , W. S. Freund80b , E. M. Freundlich47 , D. C. Frizzell128 , D. Froidevaux36 , J. A. Frost135 , C. Fukunaga164 , E. Fullana Torregrosa174 , E. Fumagalli55a,55b , T. Fusayasu116 , J. Fuster174 , A. Gabrielli23a,23b , A. Gabrielli18 , G. P. Gach83a , S. Gadatsch54 , P. Gadow115 , G. Gagliardi55a,55b , L. G. Gagnon109 , C. Galea27b , B. Galhardo140a , G. E. Gallardo135 , E. J. Gallas135 , B. J. Gallop144 , G. Galster40 , R. Gamboa Goni92 , K. K. Gan126 , S. Ganguly180 , J. Gao60a , Y. Gao50 , Y. S. Gao31,m , C. García174 , J. E. García Navarro174 , J. A. García Pascual15a , C. Garcia-Argos52 , M. Garcia-Sciveres18 , R. W. Gardner37 , N. Garelli153 , S. Gargiulo52 , V. Garonne134 , A. Gaudiello55a,55b , G. Gaudio70a , I. L. Gavrilenko110 , A. Gavrilyuk111 , C. Gay175 , G. Gaycken46 , E. N. Gazis10 , A. A. Geanta27b , C. N. P. Gee144 , J. Geisen53 , M. Geisen99 , M. P. Geisler61a , C. Gemme55b , M. H. Genest58 , C. Geng105 , S. Gentile72a,72b , S. George93 , T. Geralis44 , L. O. Gerlach53 , P. Gessinger-Befurt99 , G. Gessner47 , S. Ghasemi151 , M. Ghasemi Bostanabad176 , A. Ghosh132 , A. Ghosh77 , B. Giacobbe23b , S. Giagu72a,72b , N. Giangiacomi23a,23b , P. Giannetti71a , A. Giannini69a,69b , G. Giannini14 , S. M. Gibson93 , 123 970 Page 26 of 36 Eur. Phys. J. C (2019) 79:970 M. Gignac146 , D. Gillberg34 , G. Gilles182 , D. M. Gingrich3,ay , M. P. Giordani66a,66c , F. M. Giorgi23b , P. F. Giraud145 , G. Giugliarelli66a,66c , D. Giugni68a , F. Giuli73a,73b , S. Gkaitatzis162 , I. Gkialas9,h , E. L. Gkougkousis14 , P. Gkountoumis10 , L. K. Gladilin113 , C. Glasman98 , J. Glatzer14 , P. C. F. Glaysher46 , A. Glazov46 , G. R. Gledhill131 , M. Goblirsch-Kolb26 , S. Goldfarb104 , T. Golling54 , D. Golubkov123 , A. Gomes140a,140b , R. Goncalves Gama53 , R. Gonçalo140a,140b , G. Gonella52 , L. Gonella21 , A. Gongadze79 , F. Gonnella21 , J. L. Gonski59 , S. González de la Hoz174 , S. Gonzalez-Sevilla54 , G. R. Gonzalvo Rodriguez174 , L. Goossens36 , P. A. Gorbounov111 , H. A. Gordon29 , B. Gorini36 , E. Gorini67a,67b , A. Gorišek91 , A. T. Goshaw49 , M. I. Gostkin79 , C. A. Gottardo119 , M. Gouighri35b , D. Goujdami35c , A. G. Goussiou148 , N. Govender33b , C. Goy5 , E. Gozani160 , I. Grabowska-Bold83a , E. C. Graham90 , J. Gramling171 , E. Gramstad134 , S. Grancagnolo19 , M. Grandi156 , V. Gratchev138 , P. M. Gravila27f , F. G. Gravili67a,67b , C. Gray57 , H. M. Gray18 , C. Grefe24 , K. Gregersen96 , I. M. Gregor46 , P. Grenier153 , K. Grevtsov46 , C. Grieco14 , N. A. Grieser128 , J. Griffiths8 , A. A. Grillo146 , K. Grimm31,l , S. Grinstein14,z , J.-F. Grivaz132 , S. Groh99 , E. Gross180 , J. Grosse-Knetter53 , Z. J. Grout94 , C. Grud105 , A. Grummer118 , L. Guan105 , W. Guan181 , J. Guenther36 , A. Guerguichon132 , J. G. R. Guerrero Rojas174 , F. Guescini115 , D. Guest171 , R. Gugel52 , T. Guillemin5 , S. Guindon36 , U. Gul57 , J. Guo60c , W. Guo105 , Y. Guo60a,t , Z. Guo101 , R. Gupta46 , S. Gurbuz12c , G. Gustavino128 , P. Gutierrez128 , C. Gutschow94 , C. Guyot145 , C. Gwenlan135 , C. B. Gwilliam90 , A. Haas124 , C. Haber18 , H. K. Hadavand8 , N. Haddad35e , A. Hadef60a , S. Hageböck36 , M. Haleem177 , J. Haley129 , G. Halladjian106 , G. D. Hallewell101 , K. Hamacher182 , P. Hamal130 , K. Hamano176 , H. Hamdaoui35e , G. N. Hamity149 , K. Han60a,al , L. Han60a , S. Han15a,15d , Y. F. Han167 , K. Hanagaki81,x , M. Hance146 , D. M. Handl114 , B. Haney137 , R. Hankache136 , E. Hansen96 , J. B. Hansen40 , J. D. Hansen40 , M. C. Hansen24 , P. H. Hansen40 , E. C. Hanson100 , K. Hara169 , A. S. Hard181 , T. Harenberg182 , S. Harkusha107 , P. F. Harrison178 , N. M. Hartmann114 , Y. Hasegawa150 , A. Hasib50 , S. Hassani145 , S. Haug20 , R. Hauser106 , L. B. Havener39 , M. Havranek142 , C. M. Hawkes21 , R. J. Hawkings36 , D. Hayden106 , C. Hayes155 , R. L. Hayes175 , C. P. Hays135 , J. M. Hays92 , H. S. Hayward90 , S. J. Haywood144 , F. He60a , M. P. Heath50 , V. Hedberg96 , L. Heelan8 , S. Heer24 , K. K. Heidegger52 , W. D. Heidorn78 , J. Heilman34 , S. Heim46 , T. Heim18 , B. Heinemann46,at , J. J. Heinrich131 , L. Heinrich36 , C. Heinz56 , J. Hejbal141 , L. Helary61b , A. Held175 , S. Hellesund134 , C. M. Helling146 , S. Hellman45a,45b , C. Helsens36 , R. C. W. Henderson89 , Y. Heng181 , S. Henkelmann175 , A. M. Henriques Correia36 , G. H. Herbert19 , H. Herde26 , V. Herget177 , Y. Hernández Jiménez33c , H. Herr99 , M. G. Herrmann114 , T. Herrmann48 , G. Herten52 , R. Hertenberger114 , L. Hervas36 , T. C. Herwig137 , G. G. Hesketh94 , N. P. Hessey168a , A. Higashida163 , S. Higashino81 , E. Higón-Rodriguez174 , K. Hildebrand37 , E. Hill176 , J. C. Hill32 , K. K. Hill29 , K. H. Hiller46 , S. J. Hillier21 , M. Hils48 , I. Hinchliffe18 , F. Hinterkeuser24 , M. Hirose133 , S. Hirose52 , D. Hirschbuehl182 , B. Hiti91 , O. Hladik141 , D. R. Hlaluku33c , X. Hoad50 , J. Hobbs155 , N. Hod180 , M. C. Hodgkinson149 , A. Hoecker36 , F. Hoenig114 , D. Hohn52 , D. Hohov132 , T. R. Holmes37 , M. Holzbock114 , L.B.A.H Hommels32 , S. Honda169 , T. M. Hong139 , A. Hönle115 , B. H. Hooberman173 , W. H. Hopkins6 , Y. Horii117 , P. Horn48 , L. A. Horyn37 , S. Hou158 , A. Hoummada35a , J. Howarth100 , J. Hoya88 , M. Hrabovsky130 , J. Hrdinka76 , I. Hristova19 , J. Hrivnac132 , A. Hrynevich108 , T. Hryn’ova5 , P. J. Hsu64 , S.-C. Hsu148 , Q. Hu29 , S. Hu60c , D. P. Huang94 , Y. Huang60a , Y. Huang15a , Z. Hubacek142 , F. Hubaut101 , M. Huebner24 , F. Huegging24 , T. B. Huffman135 , M. Huhtinen36 , R. F. H. Hunter34 , P. Huo155 , A. M. Hupe34 , N. Huseynov79,ag , J. Huston106 , J. Huth59 , R. Hyneman105 , S. Hyrych28a , G. Iacobucci54 , G. Iakovidis29 , I. Ibragimov151 , L. Iconomidou-Fayard132 , Z. Idrissi35e , P. Iengo36 , R. Ignazzi40 , O. Igonkina120,ab,* , R. Iguchi163 , T. Iizawa54 , Y. Ikegami81 , M. Ikeno81 , D. Iliadis162 , N. Ilic119,167,ae , F. Iltzsche48 , G. Introzzi70a,70b , M. Iodice74a , K. Iordanidou168a , V. Ippolito72a,72b , M. F. Isacson172 , M. Ishino163 , W. Islam129 , C. Issever135 , S. Istin160 , F. Ito169 , J. M. Iturbe Ponce63a , R. Iuppa75a,75b , A. Ivina180 , H. Iwasaki81 , J. M. Izen43 , V. Izzo69a , P. Jacka141 , P. Jackson1 , R. M. Jacobs24 , B. P. Jaeger152 , V. Jain2 , G. Jäkel182 , K. B. Jakobi99 , K. Jakobs52 , S. Jakobsen76 , T. Jakoubek141 , J. Jamieson57 , K. W. Janas83a , R. Jansky54 , J. Janssen24 , M. Janus53 , P. A. Janus83a , G. Jarlskog96 , N. Javadov79,ag , T. Javůrek36 , M. Javurkova52 , F. Jeanneau145 , L. Jeanty131 , J. Jejelava159a,ah , A. Jelinskas178 , P. Jenni52,b , J. Jeong46 , N. Jeong46 , S. Jézéquel5 , H. Ji181 , J. Jia155 , H. Jiang78 , Y. Jiang60a , Z. Jiang153,q , S. Jiggins52 , F. A. Jimenez Morales38 , J. Jimenez Pena115 , S. Jin15c , A. Jinaru27b , O. Jinnouchi165 , H. Jivan33c , P. Johansson149 , K. A. Johns7 , C. A. Johnson65 , K. Jon-And45a,45b , R. W. L. Jones89 , S. D. Jones156 , S. Jones7 , T. J. Jones90 , J. Jongmanns61a , P. M. Jorge140a , J. Jovicevic36 , X. Ju18 , J. J. Junggeburth115 , A. Juste Rozas14,z , A. Kaczmarska84 , M. Kado72a,72b , H. Kagan126 , M. Kagan153 , C. Kahra99 , T. Kaji179 , E. Kajomovitz160 , C. W. Kalderon96 , A. Kaluza99 , A. Kamenshchikov123 , L. Kanjir91 , Y. Kano163 , V. A. Kantserov112 , J. Kanzaki81 , L. S. Kaplan181 , D. Kar33c , K. Karava135 , M. J. Kareem168b , S. N. Karpov79 , Z. M. Karpova79 , V. Kartvelishvili89 , A. N. Karyukhin123 , L. Kashif181 , R. D. Kass126 , A. Kastanas45a,45b , C. Kato60c,60d , J. Katzy46 , K. Kawade150 , K. Kawagoe87 , T. Kawaguchi117 , T. Kawamoto163 , G. Kawamura53 , E. F. Kay176 , V. F. Kazanin122a,122b , R. Keeler176 , R. Kehoe42 , J. S. Keller34 , E. Kellermann96 , D. Kelsey156 , J. J. Kempster21 , J. Kendrick21 , O. Kepka141 , S. Kersten182 , B. P. Kerševan91 , S. Ketabchi Haghighat167 , M. Khader173 , F. Khalil-Zada13 , M. Khandoga145 , A. Khanov129 , A. G. Kharlamov122a,122b , T. Kharlamova122a,122b , 123 Eur. Phys. J. C (2019) 79:970 Page 27 of 36 970 E. E. Khoda175 , A. Khodinov166 , T. J. Khoo54 , E. Khramov79 , J. Khubua159b , S. Kido82 , M. Kiehn54 , C. R. Kilby93 , Y. K. Kim37 , N. Kimura94 , O. M. Kind19 , B. T. King90,* , D. Kirchmeier48 , J. Kirk144 , A. E. Kiryunin115 , T. Kishimoto163 , D. P. Kisliuk167 , V. Kitali46 , O. Kivernyk5 , T. Klapdor-Kleingrothaus52 , M. Klassen61a , M. H. Klein105 , M. Klein90 , U. Klein90 , K. Kleinknecht99 , P. Klimek121 , A. Klimentov29 , T. Klingl24 , T. Klioutchnikova36 , F. F. Klitzner114 , P. Kluit120 , S. Kluth115 , E. Kneringer76 , E. B. F. G. Knoops101 , A. Knue52 , D. Kobayashi87 , T. Kobayashi163 , M. Kobel48 , M. Kocian153 , P. Kodys143 , P. T. Koenig24 , T. Koffas34 , N. M. Köhler36 , T. Koi153 , M. Kolb61b , I. Koletsou5 , T. Komarek130 , T. Kondo81 , N. Kondrashova60c , K. Köneke52 , A. C. König119 , T. Kono125 , R. Konoplich124,ao , V. Konstantinides94 , N. Konstantinidis94 , B. Konya96 , R. Kopeliansky65 , S. Koperny83a , K. Korcyl84 , K. Kordas162 , G. Koren161 , A. Korn94 , I. Korolkov14 , E. V. Korolkova149 , N. Korotkova113 , O. Kortner115 , S. Kortner115 , T. Kosek143 , V. V. Kostyukhin166 , A. Kotwal49 , A. Koulouris10 , A. Kourkoumeli-Charalampidi70a,70b , C. Kourkoumelis9 , E. Kourlitis149 , V. Kouskoura29 , A. B. Kowalewska84 , R. Kowalewski176 , C. Kozakai163 , W. Kozanecki145 , A. S. Kozhin123 , V. A. Kramarenko113 , G. Kramberger91 , D. Krasnopevtsev60a , M. W. Krasny136 , A. Krasznahorkay36 , D. Krauss115 , J. A. Kremer83a , J. Kretzschmar90 , P. Krieger167 , F. Krieter114 , A. Krishnan61b , K. Krizka18 , K. Kroeninger47 , H. Kroha115 , J. Kroll141 , J. Kroll137 , J. Krstic16 , U. Kruchonak79 , H. Krüger24 , N. Krumnack78 , M. C. Kruse49 , J. A. Krzysiak84 , T. Kubota104 , O. Kuchinskaia166 , S. Kuday4b , J. T. Kuechler46 , S. Kuehn36 , A. Kugel61a , T. Kuhl46 , V. Kukhtin79 , R. Kukla101 , Y. Kulchitsky107,ak , S. Kuleshov147b , Y. P. Kulinich173 , M. Kuna58 , T. Kunigo85 , A. Kupco141 , T. Kupfer47 , O. Kuprash52 , H. Kurashige82 , L. L. Kurchaninov168a , Y. A. Kurochkin107 , A. Kurova112 , M. G. Kurth15a,15d , E. S. Kuwertz36 , M. Kuze165 , A. K. Kvam148 , J. Kvita130 , T. Kwan103 , A. La Rosa115 , L. La Rotonda41a,41b , F. La Ruffa41a,41b , C. Lacasta174 , F. Lacava72a,72b , D. P. J. Lack100 , H. Lacker19 , D. Lacour136 , E. Ladygin79 , R. Lafaye5 , B. Laforge136 , T. Lagouri33c , S. Lai53 , S. Lammers65 , W. Lampl7 , C. Lampoudis162 , E. Lançon29 , U. Landgraf52 , M. P. J. Landon92 , M. C. Lanfermann54 , V. S. Lang46 , J. C. Lange53 , R. J. Langenberg36 , A. J. Lankford171 , F. Lanni29 , K. Lantzsch24 , A. Lanza70a , A. Lapertosa55a,55b , S. Laplace136 , J. F. Laporte145 , T. Lari68a , F. Lasagni Manghi23a,23b , M. Lassnig36 , T. S. Lau63a , A. Laudrain132 , A. Laurier34 , M. Lavorgna69a,69b , M. Lazzaroni68a,68b , B. Le104 , E. Le Guirriec101 , M. LeBlanc7 , T. LeCompte6 , F. Ledroit-Guillon58 , C. A. Lee29 , G. R. Lee17 , L. Lee59 , S. C. Lee158 , S. J. Lee34 , B. Lefebvre168a , M. Lefebvre176 , F. Legger114 , C. Leggett18 , K. Lehmann152 , N. Lehmann182 , G. Lehmann Miotto36 , W. A. Leight46 , A. Leisos162,y , M. A. L. Leite80d , C. E. Leitgeb114 , R. Leitner143 , D. Lellouch180,* , K. J. C. Leney42 , T. Lenz24 , B. Lenzi36 , R. Leone7 , S. Leone71a , C. Leonidopoulos50 , A. Leopold136 , G. Lerner156 , C. Leroy109 , R. Les167 , C. G. Lester32 , M. Levchenko138 , J. Levêque5 , D. Levin105 , L. J. Levinson180 , D. J. Lewis21 , B. Li15b , B. Li105 , C-Q. Li60a , F. Li60c , H. Li60a , H. Li60b , J. Li60c , K. Li153 , L. Li60c , M. Li15a , Q. Li15a,15d , Q. Y. Li60a , S. Li60c,60d , X. Li46 , Y. Li46 , Z. Li60b , Z. Liang15a , B. Liberti73a , A. Liblong167 , K. Lie63c , S. Liem120 , C. Y. Lin32 , K. Lin106 , T. H. Lin99 , R. A. Linck65 , J. H. Lindon21 , A. L. Lionti54 , E. Lipeles137 , A. Lipniacka17 , M. Lisovyi61b , T. M. Liss173,av , A. Lister175 , A. M. Litke146 , J. D. Little8 , B. Liu78 , B. L. Liu6 , H. B. Liu29 , H. Liu105 , J. B. Liu60a , J. K. K. Liu135 , K. Liu136 , M. Liu60a , P. Liu18 , Y. Liu15a,15d , Y. L. Liu105 , Y. W. Liu60a , M. Livan70a,70b , A. Lleres58 , J. Llorente Merino152 , S. L. Lloyd92 , C. Y. Lo63b , F. Lo Sterzo42 , E. M. Lobodzinska46 , P. Loch7 , S. Loffredo73a,73b , T. Lohse19 , K. Lohwasser149 , M. Lokajicek141 , J. D. Long173 , R. E. Long89 , L. Longo36 , K. A. Looper126 , J. A. Lopez147b , I. Lopez Paz100 , A. Lopez Solis149 , J. Lorenz114 , N. Lorenzo Martinez5 , M. Losada22 , P. J. Lösel114 , A. Lösle52 , X. Lou46 , X. Lou15a , A. Lounis132 , J. Love6 , P. A. Love89 , J. J. Lozano Bahilo174 , M. Lu60a , Y. J. Lu64 , H. J. Lubatti148 , C. Luci72a,72b , A. Lucotte58 , C. Luedtke52 , F. Luehring65 , I. Luise136 , L. Luminari72a , B. Lund-Jensen154 , M. S. Lutz102 , D. Lynn29 , R. Lysak141 , E. Lytken96 , F. Lyu15a , V. Lyubushkin79 , T. Lyubushkina79 , H. Ma29 , L. L. Ma60b , Y. Ma60b , G. Maccarrone51 , A. Macchiolo115 , C. M. Macdonald149 , J. Machado Miguens137 , D. Madaffari174 , R. Madar38 , W. F. Mader48 , N. Madysa48 , J. Maeda82 , S. Maeland17 , T. Maeno29 , M. Maerker48 , A. S. Maevskiy113 , V. Magerl52 , N. Magini78 , D. J. Mahon39 , C. Maidantchik80b , T. Maier114 , A. Maio140a,140b,140d , K. Maj83a , O. Majersky28a , S. Majewski131 , Y. Makida81 , N. Makovec132 , B. Malaescu136 , Pa. Malecki84 , V. P. Maleev138 , F. Malek58 , U. Mallik77 , D. Malon6 , C. Malone32 , S. Maltezos10 , S. Malyukov79 , J. Mamuzic174 , G. Mancini51 , I. Mandić91 , L. Manhaes de Andrade Filho80a , I. M. Maniatis162 , J. Manjarres Ramos48 , K. H. Mankinen96 , A. Mann114 , A. Manousos76 , B. Mansoulie145 , I. Manthos162 , S. Manzoni120 , A. Marantis162 , G. Marceca30 , L. Marchese135 , G. Marchiori136 , M. Marcisovsky141 , C. Marcon96 , C. A. Marin Tobon36 , M. Marjanovic38 , Z. Marshall18 , M.U.F Martensson172 , S. Marti-Garcia174 , C. B. Martin126 , T. A. Martin178 , V. J. Martin50 , B. Martin dit Latour17 , L. Martinelli74a,74b , M. Martinez14,z , V. I. Martinez Outschoorn102 , S. Martin-Haugh144 , V. S. Martoiu27b , A. C. Martyniuk94 , A. Marzin36 , S. R. Maschek115 , L. Masetti99 , T. Mashimo163 , R. Mashinistov110 , J. Masik100 , A. L. Maslennikov122a,122b , L. Massa73a,73b , P. Massarotti69a,69b , P. Mastrandrea71a,71b , A. Mastroberardino41a,41b , T. Masubuchi163 , D. Matakias10 , A. Matic114 , P. Mättig24 , J. Maurer27b , B. Maček91 , D. A. Maximov122a,122b , R. Mazini158 , I. Maznas162 , S. M. Mazza146 , S. P. Mc Kee105 , T. G. McCarthy115 , W. P. McCormack , E. F. McDonald , J. A. Mcfayden , M. A. McKay , K. D. McLean , S. J. McMahon144 , 18 104 36 42 176 123 970 Page 28 of 36 Eur. Phys. J. C (2019) 79:970 P. C. McNamara104 , C. J. McNicol178 , R. A. McPherson176,ae , J. E. Mdhluli33c , Z. A. Meadows102 , S. Meehan36 , T. Megy52 , S. Mehlhase114 , A. Mehta90 , T. Meideck58 , B. Meirose43 , D. Melini174 , B. R. Mellado Garcia33c , J. D. Mellenthin53 , M. Melo28a , F. Meloni46 , A. Melzer24 , S. B. Menary100 , E. D. Mendes Gouveia140a,140e , L. Meng36 , X. T. Meng105 , S. Menke115 , E. Meoni41a,41b , S. Mergelmeyer19 , S. A. M. Merkt139 , C. Merlassino20 , P. Mermod54 , L. Merola69a,69b , C. Meroni68a , O. Meshkov110,113 , J. K. R. Meshreki151 , A. Messina72a,72b , J. Metcalfe6 , A. S. Mete171 , C. Meyer65 , J. Meyer160 , J-P. Meyer145 , H. Meyer Zu Theenhausen61a , F. Miano156 , M. Michetti19 , R. P. Middleton144 , L. Mijović50 , G. Mikenberg180 , M. Mikestikova141 , M. Mikuž91 , H. Mildner149 , M. Milesi104 , A. Milic167 , D. A. Millar92 , D. W. Miller37 , A. Milov180 , D. A. Milstead45a,45b , R. A. Mina153,q , A. A. Minaenko123 , M. Miñano Moya174 , I. A. Minashvili159b , A. I. Mincer124 , B. Mindur83a , M. Mineev79 , Y. Minegishi163 , Y. Ming181 , L. M. Mir14 , A. Mirto67a,67b , K. P. Mistry137 , T. Mitani179 , J. Mitrevski114 , V. A. Mitsou174 , M. Mittal60c , O. Miu167 , A. Miucci20 , P. S. Miyagawa149 , A. Mizukami81 , J. U. Mjörnmark96 , T. Mkrtchyan184 , M. Mlynarikova143 , T. Moa45a,45b , K. Mochizuki109 , P. Mogg52 , S. Mohapatra39 , R. Moles-Valls24 , M. C. Mondragon106 , K. Mönig46 , J. Monk40 , E. Monnier101 , A. Montalbano152 , J. Montejo Berlingen36 , M. Montella94 , F. Monticelli88 , S. Monzani68a , N. Morange132 , D. Moreno22 , M. Moreno Llácer36 , C. Moreno Martinez14 , P. Morettini55b , M. Morgenstern120 , S. Morgenstern48 , D. Mori152 , M. Morii59 , M. Morinaga179 , V. Morisbak134 , A. K. Morley36 , G. Mornacchi36 , A. P. Morris94 , L. Morvaj155 , P. Moschovakos36 , B. Moser120 , M. Mosidze159b , T. Moskalets145 , H. J. Moss149 , J. Moss31,n , E. J. W. Moyse102 , S. Muanza101 , J. Mueller139 , R. S. P. Mueller114 , D. Muenstermann89 , G. A. Mullier96 , J. L. Munoz Martinez14 , F. J. Munoz Sanchez100 , P. Murin28b , W. J. Murray144,178 , A. Murrone68a,68b , M. Muškinja18 , C. Mwewa33a , A. G. Myagkov123,ap , J. Myers131 , M. Myska142 , B. P. Nachman18 , O. Nackenhorst47 , A.Nag Nag48 , K. Nagai135 , K. Nagano81 , Y. Nagasaka62 , M. Nagel52 , E. Nagy101 , A. M. Nairz36 , Y. Nakahama117 , K. Nakamura81 , T. Nakamura163 , I. Nakano127 , H. Nanjo133 , F. Napolitano61a , R. F. Naranjo Garcia46 , R. Narayan42 , I. Naryshkin138 , T. Naumann46 , G. Navarro22 , H. A. Neal105,* , P. Y. Nechaeva110 , F. Nechansky46 , T. J. Neep21 , A. Negri70a,70b , M. Negrini23b , C. Nellist53 , M. E. Nelson135 , S. Nemecek141 , P. Nemethy124 , M. Nessi36,d , M. S. Neubauer173 , M. Neumann182 , P. R. Newman21 , Y. S. Ng19 , Y. W. Y. Ng171 , B. Ngair35e , H. D. N. Nguyen101 , T. Nguyen Manh109 , E. Nibigira38 , R. B. Nickerson135 , R. Nicolaidou145 , D. S. Nielsen40 , J. Nielsen146 , N. Nikiforou11 , V. Nikolaenko123,ap , I. Nikolic-Audit136 , K. Nikolopoulos21 , P. Nilsson29 , H. R. Nindhito54 , Y. Ninomiya81 , A. Nisati72a , N. Nishu60c , R. Nisius115 , I. Nitsche47 , T. Nitta179 , T. Nobe163 , Y. Noguchi85 , I. Nomidis136 , M. A. Nomura29 , M. Nordberg36 , N. Norjoharuddeen135 , T. Novak91 , O. Novgorodova48 , R. Novotny142 , L. Nozka130 , K. Ntekas171 , E. Nurse94 , F. G. Oakham34,ay , H. Oberlack115 , J. Ocariz136 , A. Ochi82 , I. Ochoa39 , J. P. Ochoa-Ricoux147a , K. O’Connor26 , S. Oda87 , S. Odaka81 , S. Oerdek53 , A. Ogrodnik83a , A. Oh100 , S. H. Oh49 , C. C. Ohm154 , H. Oide165 , M. L. Ojeda167 , H. Okawa169 , Y. Okazaki85 , Y. Okumura163 , T. Okuyama81 , A. Olariu27b , L. F. Oleiro Seabra140a , S. A. Olivares Pino147a , D. Oliveira Damazio29 , J. L. Oliver1 , M. J. R. Olsson171 , A. Olszewski84 , J. Olszowska84 , D. C. O’Neil152 , A. P. O’neill135 , A. Onofre140a,140e , P. U. E. Onyisi11 , H. Oppen134 , M. J. Oreglia37 , G. E. Orellana88 , D. Orestano74a,74b , N. Orlando14 , R. S. Orr167 , V. O’Shea57 , R. Ospanov60a , G. Otero y Garzon30 , H. Otono87 , P. S. Ott61a , M. Ouchrif35d , J. Ouellette29 , F. Ould-Saada134 , A. Ouraou145 , Q. Ouyang15a , M. Owen57 , R. E. Owen21 , V. E. Ozcan12c , N. Ozturk8 , J. Pacalt130 , H. A. Pacey32 , K. Pachal49 , A. Pacheco Pages14 , C. Padilla Aranda14 , S. Pagan Griso18 , M. Paganini183 , G. Palacino65 , S. Palazzo50 , S. Palestini36 , M. Palka83b , D. Pallin38 , I. Panagoulias10 , C. E. Pandini36 , J. G. Panduro Vazquez93 , P. Pani46 , G. Panizzo66a,66c , L. Paolozzi54 , C. Papadatos109 , K. Papageorgiou9,h , S. Parajuli43 , A. Paramonov6 , D. Paredes Hernandez63b , S. R. Paredes Saenz135 , B. Parida166 , T. H. Park167 , A. J. Parker89 , M. A. Parker32 , F. Parodi55a,55b , E. W. P. Parrish121 , J. A. Parsons39 , U. Parzefall52 , L. Pascual Dominguez136 , V. R. Pascuzzi167 , J. M. P. Pasner146 , E. Pasqualucci72a , S. Passaggio55b , F. Pastore93 , P. Pasuwan45a,45b , S. Pataraia99 , J. R. Pater100 , A. Pathak181,j , T. Pauly36 , B. Pearson115 , M. Pedersen134 , L. Pedraza Diaz119 , R. Pedro140a , T. Peiffer53 , S. V. Peleganchuk122a,122b , O. Penc141 , H. Peng60a , B. S. Peralva80a , M. M. Perego132 , A. P. Pereira Peixoto140a , D. V. Perepelitsa29 , F. Peri19 , L. Perini68a,68b , H. Pernegger36 , S. Perrella69a,69b , K. Peters46 , R. F. Y. Peters100 , B. A. Petersen36 , T. C. Petersen40 , E. Petit101 , A. Petridis1 , C. Petridou162 , P. Petroff132 , M. Petrov135 , F. Petrucci74a,74b , M. Pettee183 , N. E. Pettersson102 , K. Petukhova143 , A. Peyaud145 , R. Pezoa147b , L. Pezzotti70a,70b , T. Pham104 , F. H. Phillips106 , P. W. Phillips144 , M. W. Phipps173 , G. Piacquadio155 , E. Pianori18 , A. Picazio102 , R. H. Pickles100 , R. Piegaia30 , D. Pietreanu27b , J. E. Pilcher37 , A. D. Pilkington100 , M. Pinamonti73a,73b , J. L. Pinfold3 , M. Pitt161 , L. Pizzimento73a,73b , M.-A. Pleier29 , V. Pleskot143 , E. Plotnikova79 , P. Podberezko122a,122b , R. Poettgen96 , R. Poggi54 , L. Poggioli132 , I. Pogrebnyak106 , D. Pohl24 , I. Pokharel53 , G. Polesello70a , A. Poley18 , A. Policicchio72a,72b , R. Polifka143 , A. Polini23b , C. S. Pollard46 , V. Polychronakos29 , D. Ponomarenko112 , L. Pontecorvo36 , S. Popa27a , G. A. Popeneciu27d , L. Portales5 , D. M. Portillo Quintero58 , S. Pospisil142 , K. Potamianos46 , I. N. Potrap79 , C. J. Potter32 , H. Potti11 , T. Poulsen96 , J. Poveda36 , T. D. Powell149 , G. Pownall46 , M. E. Pozo Astigarraga36 , P. Pralavorio101 , S. Prell78 , D. Price100 , M. Primavera67a , S. Prince103 , M. L. Proffitt148 , N. Proklova112 , K. Prokofiev63c , F. Prokoshin79 , 123 Eur. Phys. J. C (2019) 79:970 Page 29 of 36 970 S. Protopopescu29 , J. Proudfoot6 , M. Przybycien83a , D. Pudzha138 , A. Puri173 , P. Puzo132 , J. Qian105 , Y. Qin100 , A. Quadt53 , M. Queitsch-Maitland46 , A. Qureshi1 , M. Racko28a , P. Rados104 , F. Ragusa68a,68b , G. Rahal97 , J. A. Raine54 , S. Rajagopalan29 , A. Ramirez Morales92 , K. Ran15a,15d , T. Rashid132 , S. Raspopov5 , D. M. Rauch46 , F. Rauscher114 , S. Rave99 , B. Ravina149 , I. Ravinovich180 , J. H. Rawling100 , M. Raymond36 , A. L. Read134 , N. P. Readioff58 , M. Reale67a,67b , D. M. Rebuzzi70a,70b , A. Redelbach177 , G. Redlinger29 , K. Reeves43 , L. Rehnisch19 , J. Reichert137 , D. Reikher161 , A. Reiss99 , A. Rej151 , C. Rembser36 , M. Renda27b , M. Rescigno72a , S. Resconi68a , E. D. Resseguie137 , S. Rettie175 , E. Reynolds21 , O. L. Rezanova122a,122b , P. Reznicek143 , E. Ricci75a,75b , R. Richter115 , S. Richter46 , E. Richter-Was83b , O. Ricken24 , M. Ridel136 , P. Rieck115 , C. J. Riegel182 , O. Rifki46 , M. Rijssenbeek155 , A. Rimoldi70a,70b , M. Rimoldi46 , L. Rinaldi23b , G. Ripellino154 , I. Riu14 , J. C. Rivera Vergara176 , F. Rizatdinova129 , E. Rizvi92 , C. Rizzi36 , R. T. Roberts100 , S. H. Robertson103,ae , M. Robin46 , D. Robinson32 , J. E. M. Robinson46 , C. M. Robles Gajardo147b , A. Robson57 , A. Rocchi73a,73b , E. Rocco99 , C. Roda71a,71b , S. Rodriguez Bosca174 , A. Rodriguez Perez14 , D. Rodriguez Rodriguez174 , A. M. Rodríguez Vera168b , S. Roe36 , O. Røhne134 , R. Röhrig115 , C. P. A. Roland65 , J. Roloff59 , A. Romaniouk112 , M. Romano23a,23b , N. Rompotis90 , M. Ronzani124 , L. Roos136 , S. Rosati72a , K. Rosbach52 , G. Rosin102 , B. J. Rosser137 , E. Rossi46 , E. Rossi74a,74b , E. Rossi69a,69b , L. P. Rossi55b , L. Rossini68a,68b , R. Rosten14 , M. Rotaru27b , J. Rothberg148 , D. Rousseau132 , G. Rovelli70a,70b , A. Roy11 , D. Roy33c , A. Rozanov101 , Y. Rozen160 , X. Ruan33c , F. Rubbo153 , F. Rühr52 , A. Ruiz-Martinez174 , A. Rummler36 , Z. Rurikova52 , N. A. Rusakovich79 , H. L. Russell103 , L. Rustige38,47 , J. P. Rutherfoord7 , E. M. Rüttinger149 , M. Rybar39 , G. Rybkin132 , E. B. Rye134 , A. Ryzhov123 , G. F. Rzehorz53 , P. Sabatini53 , G. Sabato120 , S. Sacerdoti132 , H.F-W. Sadrozinski146 , R. Sadykov79 , F. Safai Tehrani72a , B. Safarzadeh Samani156 , P. Saha121 , S. Saha103 , M. Sahinsoy61a , A. Sahu182 , M. Saimpert46 , M. Saito163 , T. Saito163 , H. Sakamoto163 , A. Sakharov124,ao , D. Salamani54 , G. Salamanna74a,74b , J. E. Salazar Loyola147b , P. H. Sales De Bruin172 , A. Salnikov153 , J. Salt174 , D. Salvatore41a,41b , F. Salvatore156 , A. Salvucci63a,63b,63c , A. Salzburger36 , J. Samarati36 , D. Sammel52 , D. Sampsonidis162 , D. Sampsonidou162 , J. Sánchez174 , A. Sanchez Pineda66a,66c , H. Sandaker134 , C. O. Sander46 , I. G. Sanderswood89 , M. Sandhoff182 , C. Sandoval22 , D. P. C. Sankey144 , M. Sannino55a,55b , Y. Sano117 , A. Sansoni51 , C. Santoni38 , H. Santos140a,140b , S. N. Santpur18 , A. Santra174 , A. Sapronov79 , J. G. Saraiva140a,140d , O. Sasaki81 , K. Sato169 , F. Sauerburger52 , E. Sauvan5 , P. Savard167,ay , N. Savic115 , R. Sawada163 , C. Sawyer144 , L. Sawyer95,am , C. Sbarra23b , A. Sbrizzi23a , T. Scanlon94 , J. Schaarschmidt148 , P. Schacht115 , B. M. Schachtner114 , D. Schaefer37 , L. Schaefer137 , J. Schaeffer99 , S. Schaepe36 , U. Schäfer99 , A. C. Schaffer132 , D. Schaile114 , R. D. Schamberger155 , N. Scharmberg100 , V. A. Schegelsky138 , D. Scheirich143 , F. Schenck19 , M. Schernau171 , C. Schiavi55a,55b , S. Schier146 , L. K. Schildgen24 , Z. M. Schillaci26 , E. J. Schioppa36 , M. Schioppa41a,41b , K. E. Schleicher52 , S. Schlenker36 , K. R. Schmidt-Sommerfeld115 , K. Schmieden36 , C. Schmitt99 , S. Schmitt46 , S. Schmitz99 , J. C. Schmoeckel46 , U. Schnoor52 , L. Schoeffel145 , A. Schoening61b , P. G. Scholer52 , E. Schopf135 , M. Schott99 , J. F. P. Schouwenberg119 , J. Schovancova36 , S. Schramm54 , F. Schroeder182 , A. Schulte99 , H-C. Schultz-Coulon61a , M. Schumacher52 , B. A. Schumm146 , Ph. Schune145 , A. Schwartzman153 , T. A. Schwarz105 , Ph. Schwemling145 , R. Schwienhorst106 , A. Sciandra146 , G. Sciolla26 , M. Scodeggio46 , M. Scornajenghi41a,41b , F. Scuri71a , F. Scutti104 , L. M. Scyboz115 , C. D. Sebastiani72a,72b , P. Seema19 , S. C. Seidel118 , A. Seiden146 , B. D. Seidlitz29 , T. Seiss37 , J. M. Seixas80b , G. Sekhniaidze69a , K. Sekhon105 , S. J. Sekula42 , N. Semprini-Cesari23a,23b , S. Sen49 , S. Senkin38 , C. Serfon76 , L. Serin132 , L. Serkin66a,66b , M. Sessa60a , H. Severini128 , T. Šfiligoj91 , F. Sforza55a,55b , A. Sfyrla54 , E. Shabalina53 , J. D. Shahinian146 , N. W. Shaikh45a,45b , D. Shaked Renous180 , L. Y. Shan15a , R. Shang173 , J. T. Shank25 , M. Shapiro18 , A. Sharma135 , A. S. Sharma1 , P. B. Shatalov111 , K. Shaw156 , S. M. Shaw100 , A. Shcherbakova138 , M. Shehade180 , Y. Shen128 , N. Sherafati34 , A. D. Sherman25 , P. Sherwood94 , L. Shi158,au , S. Shimizu81 , C. O. Shimmin183 , Y. Shimogama179 , M. Shimojima116 , I. P. J. Shipsey135 , S. Shirabe87 , M. Shiyakova79,ac , J. Shlomi180 , A. Shmeleva110 , M. J. Shochet37 , J. Shojaii104 , D. R. Shope128 , S. Shrestha126 , E. M. Shrif33c , E. Shulga180 , P. Sicho141 , A. M. Sickles173 , P. E. Sidebo154 , E. Sideras Haddad33c , O. Sidiropoulou36 , A. Sidoti23a,23b , F. Siegert48 , Dj. Sijacki16 , M. Jr. Silva181 , M. V. Silva Oliveira80a , S. B. Silverstein45a , S. Simion132 , E. Simioni99 , R. Simoniello99 , S. Simsek12b , P. Sinervo167 , V. Sinetckii110,113 , N. B. Sinev131 , M. Sioli23a,23b , I. Siral105 , S. Yu. Sivoklokov113 , J. Sjölin45a,45b , E. Skorda96 , P. Skubic128 , M. Slawinska84 , K. Sliwa170 , R. Slovak143 , V. Smakhtin180 , B. H. Smart144 , J. Smiesko28a , N. Smirnov112 , S. Yu. Smirnov112 , Y. Smirnov112 , L. N. Smirnova113,v , O. Smirnova96 , J. W. Smith53 , M. Smizanska89 , K. Smolek142 , A. Smykiewicz84 , A. A. Snesarev110 , H. L. Snoek120 , I. M. Snyder131 , S. Snyder29 , R. Sobie176,ae , A. Soffer161 , A. Søgaard50 , F. Sohns53 , C. A. Solans Sanchez36 , E. Yu. Soldatov112 , U. Soldevila174 , A. A. Solodkov123 , A. Soloshenko79 , O. V. Solovyanov123 , V. Solovyev138 , P. Sommer149 , H. Son170 , W. Song144 , W. Y. Song168b , A. Sopczak142 , F. Sopkova28b , C. L. Sotiropoulou71a,71b , S. Sottocornola70a,70b , R. Soualah66a,66c,g , A. M. Soukharev122a,122b , D. South46 , S. Spagnolo67a,67b , M. Spalla115 , M. Spangenberg178 , F. Spanò93 , D. Sperlich52 , T. M. Spieker61a , R. Spighi23b , G. Spigo36 , M. Spina156 , D. P. Spiteri57 , M. Spousta143 , A. Stabile68a,68b , B. L. Stamas121 , 123 970 Page 30 of 36 Eur. Phys. J. C (2019) 79:970 R. Stamen61a , M. Stamenkovic120 , E. Stanecka84 , B. Stanislaus135 , M. M. Stanitzki46 , M. Stankaityte135 , B. Stapf120 , E. A. Starchenko123 , G. H. Stark146 , J. Stark58 , S. H. Stark40 , P. Staroba141 , P. Starovoitov61a , S. Stärz103 , R. Staszewski84 , G. Stavropoulos44 , M. Stegler46 , P. Steinberg29 , A. L. Steinhebel131 , B. Stelzer152 , H. J. Stelzer139 , O. Stelzer-Chilton168a , H. Stenzel56 , T. J. Stevenson156 , G. A. Stewart36 , M. C. Stockton36 , G. Stoicea27b , M. Stolarski140a , S. Stonjek115 , A. Straessner48 , J. Strandberg154 , S. Strandberg45a,45b , M. Strauss128 , P. Strizenec28b , R. Ströhmer177 , D. M. Strom131 , R. Stroynowski42 , A. Strubig50 , S. A. Stucci29 , B. Stugu17 , J. Stupak128 , N. A. Styles46 , D. Su153 , S. Suchek61a , V. V. Sulin110 , M. J. Sullivan90 , D. M. S. Sultan54 , S. Sultansoy4c , T. Sumida85 , S. Sun105 , X. Sun3 , K. Suruliz156 , C. J. E. Suster157 , M. R. Sutton156 , S. Suzuki81 , M. Svatos141 , M. Swiatlowski37 , S. P. Swift2 , T. Swirski177 , A. Sydorenko99 , I. Sykora28a , M. Sykora143 , T. Sykora143 , D. Ta99 , K. Tackmann46,aa , J. Taenzer161 , A. Taffard171 , R. Tafirout168a , H. Takai29 , R. Takashima86 , K. Takeda82 , T. Takeshita150 , E. P. Takeva50 , Y. Takubo81 , M. Talby101 , A. A. Talyshev122a,122b , N. M. Tamir161 , J. Tanaka163 , M. Tanaka165 , R. Tanaka132 , S. Tapia Araya173 , S. Tapprogge99 , A. Tarek Abouelfadl Mohamed136 , S. Tarem160 , G. Tarna27b,c , G. F. Tartarelli68a , P. Tas143 , M. Tasevsky141 , T. Tashiro85 , E. Tassi41a,41b , A. Tavares Delgado140a,140b , Y. Tayalati35e , A. J. Taylor50 , G. N. Taylor104 , W. Taylor168b , A. S. Tee89 , R. Teixeira De Lima153 , P. Teixeira-Dias93 , H. Ten Kate36 , J. J. Teoh120 , S. Terada81 , K. Terashi163 , J. Terron98 , S. Terzo14 , M. Testa51 , R. J. Teuscher167,ae , S. J. Thais183 , T. Theveneaux-Pelzer46 , F. Thiele40 , D. W. Thomas93 , J. O. Thomas42 , J. P. Thomas21 , A. S. Thompson57 , P. D. Thompson21 , L. A. Thomsen183 , E. Thomson137 , E. J. Thorpe92 , Y. Tian39 , R. E. Ticse Torres53 , V. O. Tikhomirov110,aq , Yu. A. Tikhonov122a,122b , S. Timoshenko112 , P. Tipton183 , S. Tisserant101 , K. Todome23a,23b , S. Todorova-Nova5 , S. Todt48 , J. Tojo87 , S. Tokár28a , K. Tokushuku81 , E. Tolley126 , K. G. Tomiwa33c , M. Tomoto117 , L. Tompkins153,q , B. Tong59 , P. Tornambe102 , E. Torrence131 , H. Torres48 , E. Torró Pastor148 , C. Tosciri135 , J. Toth101,ad , D. R. Tovey149 , A. Traeet17 , C. J. Treado124 , T. Trefzger177 , F. Tresoldi156 , A. Tricoli29 , I. M. Trigger168a , S. Trincaz-Duvoid136 , W. Trischuk167 , B. Trocmé58 , A. Trofymov145 , C. Troncon68a , M. Trovatelli176 , F. Trovato156 , L. Truong33b , M. Trzebinski84 , A. Trzupek84 , F. Tsai46 , J.C.-L. Tseng135 , P. V. Tsiareshka107,ak , A. Tsirigotis162 , N. Tsirintanis9 , V. Tsiskaridze155 , E. G. Tskhadadze159a , M. Tsopoulou162 , I. I. Tsukerman111 , V. Tsulaia18 , S. Tsuno81 , D. Tsybychev155 , Y. Tu63b , A. Tudorache27b , V. Tudorache27b , T. T. Tulbure27a , A. N. Tuna59 , S. Turchikhin79 , D. Turgeman180 , I. Turk Cakir4b,w , R. J. Turner21 , R. T. Turra68a , P. M. Tuts39 , S. Tzamarias162 , E. Tzovara99 , G. Ucchielli47 , K. Uchida163 , I. Ueda81 , M. Ughetto45a,45b , F. Ukegawa169 , G. Unal36 , A. Undrus29 , G. Unel171 , F. C. Ungaro104 , Y. Unno81 , K. Uno163 , J. Urban28b , P. Urquijo104 , G. Usai8 , Z. Uysal12d , L. Vacavant101 , V. Vacek142 , B. Vachon103 , K. O. H. Vadla134 , A. Vaidya94 , C. Valderanis114 , E. Valdes Santurio45a,45b , M. Valente54 , S. Valentinetti23a,23b , A. Valero174 , L. Valéry46 , R. A. Vallance21 , A. Vallier36 , J. A. Valls Ferrer174 , T. R. Van Daalen14 , P. Van Gemmeren6 , I. Van Vulpen120 , M. Vanadia73a,73b , W. Vandelli36 , A. Vaniachine166 , D. Vannicola72a,72b , R. Vari72a , E. W. Varnes7 , C. Varni55a,55b , T. Varol42 , D. Varouchas132 , K. E. Varvell157 , M. E. Vasile27b , G. A. Vasquez176 , J. G. Vasquez183 , F. Vazeille38 , D. Vazquez Furelos14 , T. Vazquez Schroeder36 , J. Veatch53 , V. Vecchio74a,74b , M. J. Veen120 , L. M. Veloce167 , F. Veloso140a,140c , S. Veneziano72a , A. Ventura67a,67b , N. Venturi36 , A. Verbytskyi115 , V. Vercesi70a , M. Verducci71a,71b , C. M. Vergel Infante78 , C. Vergis24 , W. Verkerke120 , A. T. Vermeulen120 , J. C. Vermeulen120 , M. C. Vetterli152,ay , N. Viaux Maira147b , M. Vicente Barreto Pinto54 , T. Vickey149 , O. E. Vickey Boeriu149 , G. H. A. Viehhauser135 , L. Vigani61b , M. Villa23a,23b , M. Villaplana Perez68a,68b , E. Vilucchi51 , M. G. Vincter34 , G. S. Virdee21 , A. Vishwakarma46 , C. Vittori23a,23b , I. Vivarelli156 , M. Vogel182 , P. Vokac142 , S. E. von Buddenbrock33c , E. Von Toerne24 , V. Vorobel143 , K. Vorobev112 , M. Vos174 , J. H. Vossebeld90 , M. Vozak100 , N. Vranjes16 , M. Vranjes Milosavljevic16 , V. Vrba142 , M. Vreeswijk120 , R. Vuillermet36 , I. Vukotic37 , P. Wagner24 , W. Wagner182 , J. Wagner-Kuhr114 , S. Wahdan182 , H. Wahlberg88 , V. M. Walbrecht115 , J. Walder89 , R. Walker114 , S. D. Walker93 , W. Walkowiak151 , V. Wallangen45a,45b , A. M. Wang59 , C. Wang60c , C. Wang60b , F. Wang181 , H. Wang18 , H. Wang3 , J. Wang157 , J. Wang61b , P. Wang42 , Q. Wang128 , R.-J. Wang99 , R. Wang60a , R. Wang6 , S. M. Wang158 , W. T. Wang60a , W. Wang15c,af , W. X. Wang60a,af , Y. Wang60a,an , Z. Wang60c , C. Wanotayaroj46 , A. Warburton103 , C. P. Ward32 , D. R. Wardrope94 , N. Warrack57 , A. Washbrook50 , A. T. Watson21 , M. F. Watson21 , G. Watts148 , B. M. Waugh94 , A. F. Webb11 , S. Webb99 , C. Weber183 , M. S. Weber20 , S. A. Weber34 , S. M. Weber61a , A. R. Weidberg135 , J. Weingarten47 , M. Weirich99 , C. Weiser52 , P. S. Wells36 , T. Wenaus29 , T. Wengler36 , S. Wenig36 , N. Wermes24 , M. D. Werner78 , M. Wessels61a , T. D. Weston20 , K. Whalen131 , N. L. Whallon148 , A. M. Wharton89 , A. S. White105 , A. White8 , M. J. White1 , D. Whiteson171 , B. W. Whitmore89 , W. Wiedenmann181 , M. Wielers144 , N. Wieseotte99 , C. Wiglesworth40 , L. A. M. Wiik-Fuchs52 , F. Wilk100 , H. G. Wilkens36 , L. J. Wilkins93 , H. H. Williams137 , S. Williams32 , C. Willis106 , S. Willocq102 , J. A. Wilson21 , I. Wingerter-Seez5 , E. Winkels156 , F. Winklmeier131 , O. J. Winston156 , B. T. Winter52 , M. Wittgen153 , M. Wobisch95 , A. Wolf99 , T. M. H. Wolf120 , R. Wolff101 , R. W. Wölker135 , J. Wollrath52 , M. W. Wolter84 , H. Wolters140a,140c , V. W. S. Wong175 , N. L. Woods146 , S. D. Worm21 , B. K. Wosiek84 , K. W. Woźniak84 , K. Wraight57 , S. L. Wu181 , X. Wu54 , Y. Wu60a , T. R. Wyatt100 , B. M. Wynne50 , S. Xella40 , Z. Xi105 , L. Xia178 , X. Xiao105 , 123 Eur. Phys. J. C (2019) 79:970 Page 31 of 36 970 D. Xu15a , H. Xu60a,c , L. Xu29 , T. Xu145 , W. Xu105 , Z. Xu60b , Z. Xu153 , B. Yabsley157 , S. Yacoob33a , K. Yajima133 , D. P. Yallup94 , D. Yamaguchi165 , Y. Yamaguchi165 , A. Yamamoto81 , M. Yamatani163 , T. Yamazaki163 , Y. Yamazaki82 , Z. Yan25 , H. J. Yang60c,60d , H. T. Yang18 , S. Yang77 , X. Yang58,60b , Y. Yang163 , W.-M. Yao18 , Y. C. Yap46 , Y. Yasu81 , E. Yatsenko60c,60d , J. Ye42 , S. Ye29 , I. Yeletskikh79 , M. R. Yexley89 , E. Yigitbasi25 , K. Yorita179 , K. Yoshihara137 , C. J. S. Young36 , C. Young153 , J. Yu78 , R. Yuan60b,i , X. Yue61a , S. P. Y. Yuen24 , B. Zabinski84 , G. Zacharis10 , E. Zaffaroni54 , J. Zahreddine136 , A. M. Zaitsev123,ap , T. Zakareishvili159b , N. Zakharchuk34 , S. Zambito59 , D. Zanzi36 , D. R. Zaripovas57 , S. V. Zeißner47 , C. Zeitnitz182 , G. Zemaityte135 , J. C. Zeng173 , O. Zenin123 , T. Ženiš28a , D. Zerwas132 , M. Zgubič135 , D. F. Zhang15b , F. Zhang181 , G. Zhang15b , H. Zhang15c , J. Zhang6 , L. Zhang15c , L. Zhang60a , M. Zhang173 , R. Zhang24 , X. Zhang60b , Y. Zhang15a,15d , Z. Zhang63a , Z. Zhang132 , P. Zhao49 , Y. Zhao60b , Z. Zhao60a , A. Zhemchugov79 , Z. Zheng105 , D. Zhong173 , B. Zhou105 , C. Zhou181 , M. S. Zhou15a,15d , M. Zhou155 , N. Zhou60c , Y. Zhou7 , C. G. Zhu60b , H. L. Zhu60a , H. Zhu15a , J. Zhu105 , Y. Zhu60a , X. Zhuang15a , K. Zhukov110 , V. Zhulanov122a,122b , D. Zieminska65 , N. I. Zimine79 , S. Zimmermann52 , Z. Zinonos115 , M. Ziolkowski151 , L. Živković16 , G. Zobernig181 , A. Zoccoli23a,23b , K. Zoch53 , T. G. Zorbas149 , R. Zou37 , L. Zwalinski36 1 Department of Physics, University of Adelaide, Adelaide, Australia 2 Physics Department, SUNY Albany, Albany, NY, USA 3 Department of Physics, University of Alberta, Edmonton, AB, Canada 4 (a) Department of Physics, Ankara University, Ankara, Turkey; (b) Istanbul Aydin University, Istanbul, Turkey; (c) Division of Physics, TOBB University of Economics and Technology, Ankara, Turkey 5 LAPP, Université Grenoble Alpes, Université Savoie Mont Blanc, CNRS/IN2P3, Annecy, France 6 High Energy Physics Division, Argonne National Laboratory, Argonne, IL, USA 7 Department of Physics, University of Arizona, Tucson, AZ, USA 8 Department of Physics, University of Texas at Arlington, Arlington, TX, USA 9 Physics Department, National and Kapodistrian University of Athens, Athens, Greece 10 Physics Department, National Technical University of Athens, Zografou, Greece 11 Department of Physics, University of Texas at Austin, Austin, TX, USA 12 (a) Bahcesehir University, Faculty of Engineering and Natural Sciences, Istanbul, Turkey; (b) Istanbul Bilgi University, Faculty of Engineering and Natural Sciences, Istanbul, Turkey; (c) Department of Physics, Bogazici University, Istanbul, Turkey; (d) Department of Physics Engineering, Gaziantep University, Gaziantep, Turkey 13 Institute of Physics, Azerbaijan Academy of Sciences, Baku, Azerbaijan 14 Institut de Física d’Altes Energies (IFAE), Barcelona Institute of Science and Technology, Barcelona, Spain 15 (a) Institute of High Energy Physics, Chinese Academy of Sciences, Beijing, China; (b) Physics Department, Tsinghua University, Beijing, China; (c) Department of Physics, Nanjing University, Nanjing, China; (d) University of Chinese Academy of Science (UCAS), Beijing, China 16 Institute of Physics, University of Belgrade, Belgrade, Serbia 17 Department for Physics and Technology, University of Bergen, Bergen, Norway 18 Physics Division, Lawrence Berkeley National Laboratory and University of California, Berkeley, CA, USA 19 Institut für Physik, Humboldt Universität zu Berlin, Berlin, Germany 20 Albert Einstein Center for Fundamental Physics and Laboratory for High Energy Physics, University of Bern, Bern, Switzerland 21 School of Physics and Astronomy, University of Birmingham, Birmingham, UK 22 Facultad de Ciencias y Centro de Investigaciónes, Universidad Antonio Nariño, Bogota, Colombia 23 (a) Dipartimento di Fisica, INFN Bologna and Universita’ di Bologna, Bologna, Italy; (b) INFN Sezione di Bologna, Bologna, Italy 24 Physikalisches Institut, Universität Bonn, Bonn, Germany 25 Department of Physics, Boston University, Boston, MA, USA 26 Department of Physics, Brandeis University, Waltham, MA, USA 27 (a) Transilvania University of Brasov, Brasov, Romania; (b) Horia Hulubei National Institute of Physics and Nuclear Engineering, Bucharest, Romania; (c) Department of Physics, Alexandru Ioan Cuza University of Iasi, Iasi, Romania; (d) Physics Department, National Institute for Research and Development of Isotopic and Molecular Technologies, Cluj-Napoca, Romania; (e) University Politehnica Bucharest, Bucharest, Romania; (f) West University in Timisoara, Timisoara, Romania 123 970 Page 32 of 36 Eur. Phys. J. C (2019) 79:970 28 (a) Faculty of Mathematics, Physics and Informatics, Comenius University, Bratislava, Slovakia; (b) Department of Subnuclear Physics, Institute of Experimental Physics of the Slovak Academy of Sciences, Kosice, Slovak Republic 29 Physics Department, Brookhaven National Laboratory, Upton, NY, USA 30 Departamento de Física, Universidad de Buenos Aires, Buenos Aires, Argentina 31 California State University, CA, USA 32 Cavendish Laboratory, University of Cambridge, Cambridge, UK 33 (a) Department of Physics, University of Cape Town, Cape Town, South Africa; (b) Department of Mechanical Engineering Science, University of Johannesburg, Johannesburg, South Africa; (c) School of Physics, University of the Witwatersrand, Johannesburg, South Africa 34 Department of Physics, Carleton University, Ottawa, ON, Canada 35 (a) Faculté des Sciences Ain Chock, Réseau Universitaire de Physique des Hautes Energies-Université Hassan II, Casablanca, Morocco; (b) Faculté des Sciences, Université Ibn-Tofail, Kénitra, Morocco; (c) Faculté des Sciences Semlalia, LPHEA-Marrakech, Université Cadi Ayyad, Marrakesh, Morocco; (d) Faculté des Sciences, Université Mohamed Premier and LPTPM, Oujda, Morocco; (e) Faculté des sciences, Université Mohammed V, Rabat, Morocco 36 CERN, Geneva, Switzerland 37 Enrico Fermi Institute, University of Chicago, Chicago, IL, USA 38 LPC, Université Clermont Auvergne, CNRS/IN2P3, Clermont-Ferrand, France 39 Nevis Laboratory, Columbia University, Irvington, NY, USA 40 Niels Bohr Institute, University of Copenhagen, Copenhagen, Denmark 41 (a) Dipartimento di Fisica, Università della Calabria, Rende, Italy; (b) Laboratori Nazionali di Frascati, INFN Gruppo Collegato di Cosenza, Cosenza, Italy 42 Physics Department, Southern Methodist University, Dallas, TX, USA 43 Physics Department, University of Texas at Dallas, Richardson, TX, USA 44 National Centre for Scientific Research “Demokritos”, Agia Paraskevi, Greece 45 (a) Department of Physics, Stockholm University, Stockholm, Sweden; (b) Oskar Klein Centre, Stockholm, Sweden 46 Deutsches Elektronen-Synchrotron DESY, Hamburg and Zeuthen, Germany 47 Lehrstuhl für Experimentelle Physik IV, Technische Universität Dortmund, Dortmund, Germany 48 Institut für Kern- und Teilchenphysik, Technische Universität Dresden, Dresden, Germany 49 Department of Physics, Duke University, Durham, NC, USA 50 SUPA-School of Physics and Astronomy, University of Edinburgh, Edinburgh, UK 51 INFN e Laboratori Nazionali di Frascati, Frascati, Italy 52 Physikalisches Institut, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany 53 II. Physikalisches Institut, Georg-August-Universität Göttingen, Göttingen, Germany 54 Département de Physique Nucléaire et Corpusculaire, Université de Genève, Genève, Switzerland 55 (a) Dipartimento di Fisica, Università di Genova, Genoa, Italy; (b) INFN Sezione di Genova, Genoa, Italy 56 II. Physikalisches Institut, Justus-Liebig-Universität Giessen, Giessen, Germany 57 SUPA-School of Physics and Astronomy, University of Glasgow, Glasgow, UK 58 LPSC, Université Grenoble Alpes, CNRS/IN2P3, Grenoble INP, Grenoble, France 59 Laboratory for Particle Physics and Cosmology, Harvard University, Cambridge, MA, USA 60 (a) Department of Modern Physics and State Key Laboratory of Particle Detection and Electronics, University of Science and Technology of China, Hefei, China; (b) Institute of Frontier and Interdisciplinary Science and Key Laboratory of Particle Physics and Particle Irradiation (MOE), Shandong University, Qingdao, China; (c) School of Physics and Astronomy, Shanghai Jiao Tong University, KLPPAC-MoE, SKLPPC, Shanghai, China; (d) Tsung-Dao Lee Institute, Shanghai, China 61 (a) Kirchhoff-Institut für Physik, Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany; (b) Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany 62 Faculty of Applied Information Science, Hiroshima Institute of Technology, Hiroshima, Japan 63 (a) Department of Physics, Chinese University of Hong Kong, Shatin, N.T., Hong Kong, China; (b) Department of Physics, University of Hong Kong, Hong Kong, China; (c) Department of Physics and Institute for Advanced Study, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China 64 Department of Physics, National Tsing Hua University, Hsinchu, Taiwan 65 Department of Physics, Indiana University, Bloomington, IN, USA 123 Eur. Phys. J. C (2019) 79:970 Page 33 of 36 970 66 (a) INFN Gruppo Collegato di Udine, Sezione di Trieste, Udine, Italy; (b) ICTP, Trieste, Italy; (c) Dipartimento Politecnico di Ingegneria e Architettura, Università di Udine, Udine, Italy 67 (a) INFN Sezione di Lecce, Zona Monte, Italy; (b) Dipartimento di Matematica e Fisica, Università del Salento, Lecce, Italy 68 (a) INFN Sezione di Milano, Milan, Italy; (b) Dipartimento di Fisica, Università di Milano, Milan, Italy 69 (a) INFN Sezione di Napoli, Naples, Italy; (b) Dipartimento di Fisica, Università di Napoli, Napoli, Italy 70 (a) INFN Sezione di Pavia, Pavia, Italy; (b) Dipartimento di Fisica, Università di Pavia, Pavia, Italy 71 (a) INFN Sezione di Pisa, Pisa, Italy; (b) Dipartimento di Fisica E. Fermi, Università di Pisa, Pisa, Italy 72 (a) INFN Sezione di Roma, Rome, Italy; (b) Dipartimento di Fisica, Sapienza Università di Roma, Rome, Italy 73 (a) INFN Sezione di Roma Tor Vergata, Rome, Italy; (b) Dipartimento di Fisica, Università di Roma Tor Vergata, Rome, Italy 74 (a) INFN Sezione di Roma Tre, Rome, Italy; (b) Dipartimento di Matematica e Fisica, Università Roma Tre, Rome, Italy 75 (a) INFN-TIFPA, Povo, Italy; (b) Università degli Studi di Trento, Trento, Italy 76 Institut für Astro- und Teilchenphysik, Leopold-Franzens-Universität, Innsbruck, Austria 77 University of Iowa, Iowa City, IA, USA 78 Department of Physics and Astronomy, Iowa State University, Ames, IA, USA 79 Joint Institute for Nuclear Research, Dubna, Russia 80 (a) Departamento de Engenharia Elétrica, Universidade Federal de Juiz de Fora (UFJF), Juiz de Fora, Brazil; (b) Universidade Federal do Rio De Janeiro COPPE/EE/IF, Rio de Janeiro, Brazil; (c) Universidade Federal de São João del Rei (UFSJ), São João del Rei, Brazil; (d) Instituto de Física, Universidade de São Paulo, São Paulo, Brazil 81 KEK, High Energy Accelerator Research Organization, Tsukuba, Japan 82 Graduate School of Science, Kobe University, Kobe, Japan 83 (a) AGH University of Science and Technology, Faculty of Physics and Applied Computer Science, Krakow, Poland; (b) Marian Smoluchowski Institute of Physics, Jagiellonian University, Krakow, Poland 84 Institute of Nuclear Physics Polish Academy of Sciences, Krakow, Poland 85 Faculty of Science, Kyoto University, Kyoto, Japan 86 Kyoto University of Education, Kyoto, Japan 87 Research Center for Advanced Particle Physics and Department of Physics, Kyushu University, Fukuoka , Japan 88 Instituto de Física La Plata, Universidad Nacional de La Plata and CONICET, La Plata, Argentina 89 Physics Department, Lancaster University, Lancaster, UK 90 Oliver Lodge Laboratory, University of Liverpool, Liverpool, UK 91 Department of Experimental Particle Physics, Jožef Stefan Institute and Department of Physics, University of Ljubljana, Ljubljana, Slovenia 92 School of Physics and Astronomy, Queen Mary University of London, London, UK 93 Department of Physics, Royal Holloway University of London, Egham, UK 94 Department of Physics and Astronomy, University College London, London, UK 95 Louisiana Tech University, Ruston, LA, USA 96 Fysiska institutionen, Lunds universitet, Lund, Sweden 97 Centre de Calcul de l’Institut National de Physique Nucléaire et de Physique des Particules (IN2P3), Villeurbanne, France 98 Departamento de Física Teorica C-15 and CIAFF, Universidad Autónoma de Madrid, Madrid, Spain 99 Institut für Physik, Universität Mainz, Mainz, Germany 100 School of Physics and Astronomy, University of Manchester, Manchester, UK 101 CPPM, Aix-Marseille Université, CNRS/IN2P3, Marseille, France 102 Department of Physics, University of Massachusetts, Amherst, MA, USA 103 Department of Physics, McGill University, Montreal, QC, Canada 104 School of Physics, University of Melbourne, Victoria, Australia 105 Department of Physics, University of Michigan, Ann Arbor, MI, USA 106 Department of Physics and Astronomy, Michigan State University, East Lansing, MI, USA 107 B.I. Stepanov Institute of Physics, National Academy of Sciences of Belarus, Minsk, Belarus 108 Research Institute for Nuclear Problems of Byelorussian State University, Minsk, Belarus 109 Group of Particle Physics, University of Montreal, Montreal, QC, Canada 110 P.N. Lebedev Physical Institute of the Russian Academy of Sciences, Moscow, Russia 123 970 Page 34 of 36 Eur. Phys. J. C (2019) 79:970 111 Institute for Theoretical and Experimental Physics of the National Research Centre Kurchatov Institute, Moscow, Russia 112 National Research Nuclear University MEPhI, Moscow, Russia 113 D.V. Skobeltsyn Institute of Nuclear Physics, M.V. Lomonosov Moscow State University, Moscow, Russia 114 Fakultät für Physik, Ludwig-Maximilians-Universität München, München, Germany 115 Max-Planck-Institut für Physik (Werner-Heisenberg-Institut), München, Germany 116 Nagasaki Institute of Applied Science, Nagasaki, Japan 117 Graduate School of Science and Kobayashi-Maskawa Institute, Nagoya University, Nagoya, Japan 118 Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USAUSA 119 Institute for Mathematics, Astrophysics and Particle Physics, Radboud University Nijmegen/Nikhef, Nijmegen, The Netherlands 120 Nikhef National Institute for Subatomic Physics and University of Amsterdam, Amsterdam, The Netherlands 121 Department of Physics, Northern Illinois University, DeKalb, IL, USA 122 (a) Budker Institute of Nuclear Physics and NSU, SB RAS, Novosibirsk, Russia; (b) Novosibirsk State University Novosibirsk, Novosibirsk, Russia 123 Institute for High Energy Physics of the National Research Centre Kurchatov Institute, Protvino, Russia 124 Department of Physics, New York University, New York, NY, USA 125 Ochanomizu University, Otsuka, Bunkyo-ku, Tokyo, Japan 126 Ohio State University, Columbus, OH, USA 127 Faculty of Science, Okayama University, Okayama, Japan 128 Homer L. Dodge Department of Physics and Astronomy, University of Oklahoma, Norman, OK, USA 129 Department of Physics, Oklahoma State University, Stillwater, OK, USA 130 Palacký University, RCPTM, Joint Laboratory of Optics, Olomouc, Czech Republic 131 Center for High Energy Physics, University of Oregon, Eugene, OR, USA 132 LAL, Université Paris-Sud, CNRS/IN2P3, Université Paris-Saclay, Orsay, France 133 Graduate School of Science, Osaka University, Osaka, Japan 134 Department of Physics, University of Oslo, Oslo, Norway 135 Department of Physics, Oxford University, Oxford, UK 136 LPNHE, Sorbonne Université, Université de Paris, CNRS/IN2P3, Paris, France 137 Department of Physics, University of Pennsylvania, Philadelphia, PA, USA 138 Konstantinov Nuclear Physics Institute of National Research Centre “Kurchatov Institute”, PNPI, St. Petersburg, Russia 139 Department of Physics and Astronomy, University of Pittsburgh, Pittsburgh, PA, USA 140 (a) Laboratório de Instrumentação e Física Experimental de Partículas-LIP, Lisbon, Portugal; (b) Departamento de Física, Faculdade de Ciências, Universidade de Lisboa, Lisbon, Portugal; (c) Departamento de Física, Universidade de Coimbra, Coimbra, Portugal; (d) Centro de Física Nuclear da Universidade de Lisboa, Lisbon, Portugal; (e) Departamento de Física, Universidade do Minho, Braga, Portugal; (f) Universidad de Granada, Granada, Spain; (g) Dep Física and CEFITEC of Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa, Caparica, Portugal; (h) Av. Rovisco Pais, 1 1049-001 Lisbon, Portugal, Portugal 141 Institute of Physics of the Czech Academy of Sciences, Prague, Czech Republic 142 Czech Technical University in Prague, Prague, Czech Republic 143 Charles University, Faculty of Mathematics and Physics, Prague, Czech Republic 144 Particle Physics Department, Rutherford Appleton Laboratory, Didcot, UK 145 IRFU, CEA, Université Paris-Saclay, Gif-sur-Yvette, France 146 Santa Cruz Institute for Particle Physics, University of California Santa Cruz, Santa Cruz, CA, USA 147 (a) Departamento de Física, Pontificia Universidad Católica de Chile, Santiago, Chile; (b) Departamento de Física, Universidad Técnica Federico Santa María, Valparaíso, Chile 148 Department of Physics, University of Washington, Seattle, WA, USA 149 Department of Physics and Astronomy, University of Sheffield, Sheffield, UK 150 Department of Physics, Shinshu University, Nagano, Japan 151 Department Physik, Universität Siegen, Siegen, Germany 152 Department of Physics, Simon Fraser University, Burnaby, BC, Canada 153 SLAC National Accelerator Laboratory, Stanford, CA, USA 154 Physics Department, Royal Institute of Technology, Stockholm, Sweden 155 Departments of Physics and Astronomy, Stony Brook University, Stony Brook, NY, USA 123 Eur. Phys. J. C (2019) 79:970 Page 35 of 36 970 156 Department of Physics and Astronomy, University of Sussex, Brighton, UK 157 School of Physics, University of Sydney, Sydney, Australia 158 Institute of Physics, Academia Sinica, Taipei, Taiwan 159 (a) E. Andronikashvili Institute of Physics, Iv. Javakhishvili Tbilisi State University, Tbilisi, Georgia; (b) High Energy Physics Institute, Tbilisi State University, Tbilisi, Georgia 160 Department of Physics, Technion, Israel Institute of Technology, Haifa, Israel 161 Raymond and Beverly Sackler School of Physics and Astronomy, Tel Aviv University, Tel Aviv, Israel 162 Department of Physics, Aristotle University of Thessaloniki, Thessaloniki, Greece 163 International Center for Elementary Particle Physics and Department of Physics, University of Tokyo, Tokyo, Japan 164 Graduate School of Science and Technology, Tokyo Metropolitan University, Tokyo, Japan 165 Department of Physics, Tokyo Institute of Technology, Tokyo, Japan 166 Tomsk State University, Tomsk, Russia 167 Department of Physics, University of Toronto, Toronto, ON, Canada 168 (a) TRIUMF, Vancouver, BC, Germany; (b) Department of Physics and Astronomy, York University, Toronto, ON, Canada 169 Division of Physics and Tomonaga Center for the History of the Universe, Faculty of Pure and Applied Sciences, University of Tsukuba, Tsukuba, Japan 170 Department of Physics and Astronomy, Tufts University, Medford, MA, USA 171 Department of Physics and Astronomy, University of California Irvine, Irvine, CA, USA 172 Department of Physics and Astronomy, University of Uppsala, Uppsala, Sweden 173 Department of Physics, University of Illinois, Urbana, IL, USA 174 Instituto de Física Corpuscular (IFIC), Centro Mixto Universidad de Valencia-CSIC, Valencia, Spain 175 Department of Physics, University of British Columbia, Vancouver, BC, Canada 176 Department of Physics and Astronomy, University of Victoria, Victoria, BC, Canada 177 Fakultät für Physik und Astronomie, Julius-Maximilians-Universität Würzburg, Würzburg, Germany 178 Department of Physics, University of Warwick, Coventry, UK 179 Waseda University, Tokyo, Japan 180 Department of Particle Physics, Weizmann Institute of Science, Rehovot, Israel 181 Department of Physics, University of Wisconsin, Madison, WI, USA 182 Fakultät für Mathematik und Naturwissenschaften, Fachgruppe Physik, Bergische Universität Wuppertal, Wuppertal, Germany 183 Department of Physics, Yale University, New Haven, CT, USA 184 Yerevan Physics Institute, Yerevan, Armenia a Also at Borough of Manhattan Community College, City University of New York, New York NY, USA b Also at CERN, Geneva, Switzerland c Also at CPPM, Aix-Marseille Université, CNRS/IN2P3, Marseille, France d Also at Département de Physique Nucléaire et Corpusculaire, Université de Genève, Genève, Switzerland e Also at Departament de Fisica de la Universitat Autonoma de Barcelona, Barcelona, Spain f Also at Departamento de Física, Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal g Also at Department of Applied Physics and Astronomy, University of Sharjah, Sharjah„ United Arab Emirates h Also at Department of Financial and Management Engineering, University of the Aegean, Chios, Greece i Also at Department of Physics and Astronomy, Michigan State University, East Lansing MI, USA j Also at Department of Physics and Astronomy, University of Louisville, Louisville, KY, USA k Also at Department of Physics, Ben Gurion University of the Negev, Beer Sheva, Israel l Also at Department of Physics, California State University, East Bay, USA m Also at Department of Physics, California State University, Fresno, USA n Also at Department of Physics, California State University, Sacramento, USA o Also at Department of Physics, King’s College London, London, UK p Also at Department of Physics, St. Petersburg State Polytechnical University, St. Petersburg, Russia q Also at Department of Physics, Stanford University, Stanford CA, USA r Also at Department of Physics, University of Adelaide, Adelaide, Australia s Also at Department of Physics, University of Fribourg, Fribourg, Switzerland t Also at Department of Physics, University of Michigan, Ann Arbor MI, USA 123 970 Page 36 of 36 Eur. Phys. J. C (2019) 79:970 u Also at Dipartimento di Matematica, Informatica e Fisica, Università di Udine, Udine, Italy v Also at Faculty of Physics, M.V. Lomonosov Moscow State University, Moscow, Russia w Also at Giresun University, Faculty of Engineering, Giresun, Turkey x Also at Graduate School of Science, Osaka University, Osaka, Japan y Also at Hellenic Open University, Patras, Greece z Also at Institucio Catalana de Recerca i Estudis Avancats, ICREA, Barcelona, Spain aa Also at Institut für Experimentalphysik, Universität Hamburg, Hamburg, Germany ab Also at Institute for Mathematics, Astrophysics and Particle Physics, Radboud University Nijmegen/Nikhef, Nijmegen, The Netherlands ac Also at Institute for Nuclear Research and Nuclear Energy (INRNE) of the Bulgarian Academy of Sciences, Sofia, Bulgaria ad Also at Institute for Particle and Nuclear Physics, Wigner Research Centre for Physics, Budapest, Hungary ae Also at Institute of Particle Physics (IPP), Vancouver, Canada af Also at Institute of Physics, Academia Sinica, Taipei, Taiwan ag Also at Institute of Physics, Azerbaijan Academy of Sciences, Baku, Azerbaijan ah Also at Institute of Theoretical Physics, Ilia State University, Tbilisi, Georgia ai Also at Instituto de Fisica Teorica, IFT-UAM/CSIC, Madrid, Spain aj Also at Istanbul University, Dept. of Physics, Istanbul, Turkey ak Also at Joint Institute for Nuclear Research, Dubna, Russia al Also at LAL, Université Paris-Sud, CNRS/IN2P3, Université Paris-Saclay, Orsay, France am Also at Louisiana Tech University, Ruston LA, USA an Also at LPNHE, Sorbonne Université, Université de Paris, CNRS/IN2P3, Paris, France ao Also at Manhattan College, New York NY, USA ap Also at Moscow Institute of Physics and Technology State University, Dolgoprudny, Russia aq Also at National Research Nuclear University MEPhI, Moscow, Russia ar Also at Physics Department, An-Najah National University, Nablus, Palestine as Also at Physics Dept, University of South Africa, Pretoria, South Africa at Also at Physikalisches Institut, Albert-Ludwigs-Universität Freiburg, Freiburg, Germany au Also at School of Physics, Sun Yat-sen University, Guangzhou, China av Also at The City College of New York, New York NY, USA aw Also at The Collaborative Innovation Center of Quantum Matter (CICQM), Beijing, China ax Also at Tomsk State University, Tomsk, and Moscow Institute of Physics and Technology State University, Dolgoprudny, Russia ay Also at TRIUMF, Vancouver BC, Canada az Also at Universita di Napoli Parthenope, Napoli, Italy ∗ Deceased 123