Upgrade of CMS Full Simulation for Run 2

We report on the status of the CMS full simulation software for Run 2 operations of the LHC. Initially, Geant4 10.0p02 was used and about 16 billion events were produced for analysis of 2015-2016 data. In 2017, the CMS detector was updated with a new tracking pixel detector, a modified hadronic calorimeter electronics, and extra muon detectors added. Corresponding modifications were introduced in the full simulation and Geant4 10.2p02 was adopted for 2017 simulation productions; that includes an improved Geant4 for multi-threaded mode, which became the default for 2017. For the 2018 Monte Carlo productions, the full simulation has been updated further. The new Geant4 version 10.4 is used, adopted for the production after detailed validations using test-beam and collision data. The results of validations will be described in details. Several aspects of the migration to Geant4 10.4 and modifications in CMSSW simulation software will be also discussed.


Introduction
The CMS full simulation is based on the Geant4 toolkit [1][2][3]. In our previous reports [4][5][6], the full simulation scheme for Run 2 was described. Several variants of Geant4 version 10 have been used for Run 2. Initially, Geant4 10.0p02 was adopted and new multi-threaded mode was operational [7]. CPU performance of the CMS simulation was substantially improved compared to the Run 1 simulation. The first CMS full simulation production campaign for Run 2 was performed and about 16 billion events were produced for analysing the data collected during 2015-2016. The multi-threaded mode was not yet used in the Monte Carlo production system.
The CMS detector has been updated for the 2017 operation: a new tracking pixel detector was installed, the hadronic calorimeter readout electronics was modified, and a few additional muon detectors were installed. Corresponding modifications have been introduced in the CMS full simulation and the Geant4 version 10.2p02 was adopted for 2017 simulation production. About 10 billion MC events are already produced in the multi-threaded mode.
For 2018 operations, the endcap hadron calorimeter readout was further modified, and the full simulation was updated accordingly for the 2018 Monte Carlo campaign. The new Geant4 version 10.4 has been adopted after detailed validations. Physics performance was validated using 2006 test-beam data for combined calorimeter setup and Run 2 collision data for isolated charged hadrons. In this work, results of these validations are described and several important aspects of migration to Geant4 10.4 and corresponding modifications of the CMS simulation software are discussed.

Migration to Geant4 10.4
The CMS decision to migrate to the newest 10.4 is based on following arguments: · the multithreaded mode of simulation production requires all technical fixes for Geant4 thread safety which are implemented in the version 10.4; · many other Geant4 problems are fixed since the version 10.2; · Geant4 10.4 is fully C++11 compliant, so newest compilers may be used to build the Geant4 library for CMSSW; · Geant4 10.4 may work with the VecGeom [8] geometry library, which provides faster computations than the native Geant4 geometry. In order to respect the CMS time schedule for 2018, a special strategy for deployment of Geant4 10.4 was proposed. CMS validation started 6 months ahead of Geant4 release date using Geant4 monthly reference versions. A special git branch was used coherently with the main development branch of CMSSW. This allowed validating simulation software via routine CMS software validation procedures. Checks on CMS calorimeter response were performed using test-beam data and Run 2 detector data. All necessary modifications in CMS simulation software were included both in the main development version of CMSSW, where Geant4 10.2p02 was used, and in this special branch with Geant4 development. Because minor Geant4 releases do not break any user interface, such a strategy allows adaptation of Geant4 just after the official release date in December 2017.

New features in CMS simulation
The following new features were implemented in CMS full simulation for the 2018 production: · the VecGeom external geometry library [8]; · the MixMax random number generator [9,10] for simulation of both hits and digi; · the new Geant4 stepper, G4DormandPrince745 [11], for computing of trajectories in magnetic field; · the "smart tracking" in magnetic field option allowing faster computations for lowenergy changed particles. The VecGeom geometry library includes an alternative set of classes to describe solid shapes and to compute geometrical parameters needed for Geant4 tracking. It was shown, that Geant4 becomes 5-10% faster without any loss of simulation quality.
The MixMax generator is the default for Geant4 10.4. This generator has much larger repetition period than that of the HepJamesRandom generator, which was traditionally used by CMS (logarithm of period is 294 and 43 respectively). In addition, MixMax is a true 61bit generator in contrary with HepJamesRandom, which implements a 32 bit algorithm. Being more powerful, MixMax does not reduce CPU performance of CMS hit and digi simulations.
The Geant4 stepper is responsible for the numerical integration of charged particle trajectories in non-uniform magnetic field. For a long time G4ClassicalRK4 was used in CMSSW; with Geant4 10.4 the new stepper, G4DormandPrince745 [11], becomes the default. CMS validation demonstrates statistical agreement between results obtained with G4DormandPrince745 and G4ClassicalRK4. At the same time, the new stepper is faster.
A "smart tracking" method was developed for the CMS simulation in order to speed up computations for tracking in magnetic field. This is achieved by dynamical switch of a set of parameters, which defines accuracy in the numerical integration of charged particle motion in the CMS non-uniform field. If the kinetic energy of a particle at a given step become below 2 MeV a set of loose parameters is applied. In the CMS case, the majority of such particles are electrons in calorimeters and for any low-energy particle there is no need to have an extremely accurate computation of a particle trajectory.
All these modifications in CMSSW pass validation procedures carried out by the CMS validation teams. It was confirmed, that physics performance of CMS simulation remains statistically equivalent with respect to the previous version of CMSSW used for 2017 analysis, but the CPU speed is increased approximately by 20%. The physics validations were performed on CMS hits and on reconstructed energy for isolated hadrons. Approximately 10% speedup is achieved due to migration to VecGeom, remaining 10% comes from faster integration in magnetic field. For QCD events more important is the usage of the new stepper, for hard scattered events the smart tracking method contributes more to the performance improvement.

Validation versus test-beam data
During 2006, CMS collected data with a prototype of the barrel hadron calorimeter (HCAL) and a super module of the barrel electromagnetic calorimeter (ECAL) in the H2 test beam area at CERN. The data correspond to single particle responses due to well-identified particles over a large momentum range (2 to 350 GeV) [12]. The test beam setup is described within CMS software and is used routinely to validate Geant4 models and CMSSW simulation software.
This test-beam data analysis allows selection of high purity samples for pions, kaons, protons and anti-protons. The same method of hit simulation is used in test-beam simulation as in the case of full CMS. The digitisation chain of CMSSW is not used, instead hit energy was smeared using the Gaussian distribution with fixed widths for ECAL (0.362 GeV) and HCAL (0.64 GeV) obtained from the test beam analysis. Hits are collected in the 7x7-crystal matrix of ECAL and in the 3x3 HCAL tower matrix. Hits in the outer hadron calorimeter layer are excluded in both simulation and data analysis. In addition, a time cut corresponding to test-beam data acquisition is applied. Visible energy response is computed using the following simple relation: Evis = EECAL•fECAL + EHCAL•fHCAL, where fECAL = 1.01 and fHCAL ~105.0, the latter value is obtained from simulation of special run with 50 GeV electron beam and HCAL alone. The same procedure is used in the analysis of the test beam data.
Since 2017 CMS simulation has been using the Geant4 version 10.2p02 and the FTFP_BERT_EMM Physics List. The hadronic interactions in this Physics List is provided by FTFP_BERT, which is the Geant4 default physics configuration [3]. Therefore, the hadronic part of FTFP_BERT_EMM is the Geant4 default as recommended by the Geant4 team. EMM provides a special configuration of Geant4 electromagnetic physics for CMS simulation, which includes: · applying production cuts for gamma induced processes; · simplifying step limitation due to multiple scattering of e ± ; · default step limitation due to multiple scattering of e ± inside CMS sampling calorimeters. The results of simulation with Geant4 10.2p02 and 10.4 are shown in Fig.1 (for pions) and Fig.2 (for protons and anti-protons). Both Geant4 versions provide compatible agreement for mean energy deposition for pions and protons within statistical accuracy of few %. For antiprotons and kaons the agreement is better than 10%. The energy resolution for pion and protons is also compared for π⁻ and protons (Fig.3). There is the statistical agreement between data and simulation at momentum above 8 GeV/c. At low momentum, simulation underestimates energy resolution, which could be due to simplified simulation of the detector response.

Validation versus Run-2 data
The analysis method [13] for comparing data and Monte Carlo simulation of calorimeters is applied to the new Geant4 10.4 simulation and to a well established CMSSW with Geant4 10.2p02. The ratio of calorimeter energy measurement to track momentum for isolated charged hadrons is studied. For that, good charged tracks reaching the ECAL surface are selected by imposing isolation criteria based on the NxN matrix analysis on the calorimeter surface. Final cuts are following: · No extra tracks in the isolation region; · Energy cut of 2 GeV for neutral isolation; · No additional good primary vertex in the event in order to avoid PU effect. Calorimeter energy is measured in a NxN matrix of calorimeter cells around the impact point of the track. Two versions of NxN matrix are defined: · 7x7 or 11x11 matrix for ECAL; · 3x3 or 5x5 matrix for HCAL. Two low luminosity data sets from the 2016B runs were used for this analysis. Results of comparisons are shown as a function of particle momentum (Figs.5-7) for three different eta regions (Fig.4) and two versions of Geant4 10.2p02 (used for 2017 simulation) and 10.4 (prepared for 2018).
The level of disagreement between data and MC is between 1.5% and 5.7%. For Geant4 10.2p02 these numbers are 1.5% and 4.9%. This level is computed as an average absolute deviation at all momentums excluding the 1 GeV/c point, which has a large uncertainty. In both cases, results are slightly different for different calorimeter regions. https://doi.org/10.1051/epjconf/201921402012 CHEP 2018

Summary
In this work, we demonstrate that the Geant4 version 10.4, when coupled to the new VecGeom geometry library, can be used for production CMS simulation. The difference between two Geant4 versions (10.2p02 and 10.4) is not distinguishable in describing the test beam as well as the collision data. In general, the hadronic response in the main CMS calorimeters is reproduced on level of few percent. The CPU performance of CMS simulation has been improved by about 20%.