Data Acquisition and Software for the ATLAS Tile Calorimeter Phase-II Upgrade Demonstrator

Abstract. After a series of upgrades, the High Luminosity LHC (HL-LHC) will have an instantaneous luminosity of 5-7 times larger than the LHC design value. The readout electronics of the ATLAS Tile Calorimeter (TileCal) will undergo a substantial upgrade during the Phase-II upgrade to accommodate the HL-LHC requirements. After the Phase-II upgrade, the TileCal detector signals will be digitized by on-detector electronics and transferred to the the TileCal PreProcessors (TilePPr), which is a part of the off-detector electronics. In the TilePPr, the digitized data will be stored in pipeline buffers and be packed and readout to the Front-End Link eXchange (FELIX) system upon receiving a trigger decision. At the same time, the energy information will be reconstructed from the detector data and transferred to the Level-1 Calorimeter Trigger (L1Calo) system in different granularity for every bunch crossing. The TileCal Demonstrator is designed to evaluate the performance of the TileCal with new readout electronics without compromising the present data taking. This contribution describes in detail the data processing and the hardware, firmware, software components of the TileCal Demonstrator system, together with the results of beam tests performed at CERN.


Introduction
The Tile Calorimeter (TileCal) [1] is the hadronic calorimeter covering the central region of the ATLAS detector [2] at the LHC. It is divided into four partitions : two long barrels (LBs) covering the central region (−1.0 < η < 1.0) and two extended barrels (EBs) that flank LBs on both sides (0.8 < |η| < 1.7). Each partition is segmented further into 64 wedges, each of which covers about 0.1 radians in φ, as shown in Figure 1. The TileCal is a sampling calorimeter using iron as the absorber and plastic scintillator as the active material. The scintillation light is collected by wavelength shifting fibers and then read out by photomultiplier tubes (PMTs) via light guides. A total of about 10,000 PMTs are used in the TileCal.
After a series of upgrades, the High Luminosity LHC (HL-LHC) [3] will have a much higher luminosity compared to the LHC nominal value. An instantaneous luminosity of up to 7.5 × 10 34 cm −2 s −1 is foreseen at the location of ATLAS/CMS, which corresponds to an average of 200 proton-proton interactions per bunch crossing. The higher luminosity will present great challenges to the detectors and the Trigger and Data AcQuisition (TDAQ) system because of the higher event pileup condition and increased trigger rate. To accommodate to the HL-LHC requirements, the TileCal will undergo a substantial upgrade. The TileCal readout electronics will be completely redesigned for the ATLAS Phase-II upgrade [4]. In order to evaluate the new readout architecture and the technology choices of the implementation, the TileCal Demonstrator is designed with new readout electronics while preserving compatibility with the current system and is planned to be inserted into the ATLAS detector for verification during the second LHC Long Shutdown in 2019 after extensive standalone tests. This contribution summarizes the data processing and the hardware, firmware, software components of the TileCal Demonstrator project, together with the result of beam tests performed at CERN.

TileCal Phase-II Upgrade Readout Electronics
The architecture of the present TileCal front-end electronics is shown in Figure 2. The PMT signals are shaped at first and then amplified in two gains with a gain ratio of 64:1. The low gain outputs are summed up to analog Trigger Tower (TT) signals, which are transferred further to the L1Calo [5] system to make Level 1 (L1) trigger decisions. In parallel, both of the amplifier outputs are digitized with 10-bit ADCs at the LHC bunch clock frequency. The digital samples are then stored in pipeline memories for up to 6.4 µs. When receiving a L1 trigger, seven samples from one selected gain are transferred to back-end electronics via optical fibers. Figure 2: The Run I&II architecture of TileCal front-end electronics. All the front-end electronics are located in dedicated drawers at the outer edge of the calorimeter. [9] The TileCal Phase-II readout electronics will be a completely new design. As shown in Figure 3, the shaping and amplification of the PMT signals are done in the new front-end boards. The signals amplified in two gains will be digitized with 12-bit ADCs at 40 MHz on the Mainboard. The digitized samples are then concentrated on the Daughterboard and sent to the TilePPr [6] located off-detector through redundant optical links for every bunch crossing. The TilePPr receives digital data from on-detector electronics and reconstructs energy deposited in each calorimeter cell with the Optimal Filtering algorithm. The cell energy data are then sent to TDAQ interface (TDAQi) module, where they are grouped into trigger primitives with different granularity and distributed to then Phase-II Level 0 trigger system [7] through low-and deterministic-delay data links for every bunch crossing. In parallel, the data samples will be stored in pipeline memories waiting for trigger decisions, and the selected data will be transferred to the FELIX [8] system for further processing. In order to transfer all the digitized samples from on-detector electronics to off-detector electronics system, a significant increase in the system data bandwidth is required in the new readout electronics design. A total of 2048 optical links at a rate of 9.6 Gbps each will be used in the new design, another 2048 links will be used as redundancy or backup; while in the current design, only 256 optical links (plus another 256 links for redundancy) with a link speed of 800 Mbps are used. The increase of the system data bandwidth makes it possible to move the data pipeline memories from on-detector electronics to off-detector FPGAs, where the digitized samples can be stored for a longer time (10 µs in the single Level-0 trigger architecture [9]). Moreover, the latency of the trigger accept signal will be considerably reduced, because the off-detector FPGAs are closer to the trigger system than the on-detector electronics. These will give Trigger Processors more time to make trigger decisions under a given latency envelope.
Another advantage of this implementation is that the new trigger system will have access to the digital information of each calorimeter cell, the lower electronics noise and more accurate energy calibration will be beneficial to improve the trigger selectivity.

TileCal Phase-II Upgrade Demonstrator
The TileCal Phase-II upgrade Demonstrator is a hybrid design based on the new readout electronics architecture while preserving the compatibility with the current system. It is designed to evaluate the performance of the TileCal with new readout electronics without compromising the present data taking.

On-detector and Off-detector Electronics
The architecture of the TileCal Demonstrator is shown in Figure 4. In order to preserve the backward compatibility with the current system, the on-detector electronics provides an optional analog summed Trigger Tower signals to the current L1Calo system; on the offdetector electronics side, the new TilePPr sends selected event data to the current Read Out Driver (ROD) and the new FELIX at the same time.
The on-detector electronics of the Demonstrator is modularly located in extractable Mini-Drawers as shown in Figure 5a. Each Mini-Drawer houses up to 12 PMTs with associated electronics, power supplies and cooling modules. A Super-Drawer composed of four identical Mini-Drawers will be used to read out one TileCal detector wedge. The on-detector electronics, power supply modules and mechanics designed for the demonstrator are very close to the final Phase-II electronics, except that the Analog Summing Board that provides the analog trigger signals will be removed in the final design. The off-detector electronics of the Demonstrator is shown in Figure 5b. The TilePPr demonstrator and the TDAQi demonstrator comply with Advanced Telecommunications Computing Architecture (ATCA) standard [10], which has been selected as the hardware platform for the Phase-II upgrade of ATLAS by the Electronics groups at CERN [11]. The TilePPr demonstrator is a double mid-size Advanced Mezzanine Card (AMC), each one is able to process the data from one Tile Super Drawer; the TDAQi demonstrator is designed as an ATCA Rear Transition Module (RTM), which sends the selected event data to ROD and FELIX. The off-detector modules used in the TileCal Demonstrator are light-weight designs. In the final Phase-II design, a complete TilePPr subsystem will consist of one ATCA motherboard blade, four Compact Processing Module (CPM) AMCs and one TDAQi RTM. Each TilePPr subsystem will be able to process the data from 8 Tile Super Drawers.

Data Flow and Software Components
The data flow of the TileCal Demonstrator is shown in Figure 6. The demonstrator system can be operated in two modes: legacy mode with the legacy TDAQi infrastructure and standalone mode with the Phase-II components only. It is also possible to operate the demonstrator in both modes in parallel. https://doi.org/10.1051/epjconf/201921401004 CHEP 2018 In legacy mode, the TilePPr receives the clock and trigger information from Time, Trigger and Control (TTC) module through dedicated optical links, that is used by the TilePPr for synchronization with the legacy system. Upon receiving a Level 1 accept signal, the selected data are packed in the legacy data format in the TilePPr and transmitted to the legacy ROD, which processes and transmits the data further to the ReadOut System (ROS) and finally to the Event Builder. The Event Builder will then save the data on a local file for off-line analysis.
In standalone mode, the selected data are packed in GigaBit Transceiver (GBT) [12] format and transmitted to FELIX, where the data are saved separately for off-line analysis. When the two modes work in parallel, clock and trigger information will also be distributed from TTC module to FELIX for synchronization.  Figure 6: The data flow of TileCal Phase-II upgrade demonstrator. [9] The data acquisition of the Demonstrator is controlled in the frame of the official ATLAS Trigger and DAQ (TDAQ) software. Additional software packages have been developed to provide support for Demonstrator system. In the TDAQ Graphical User Interface (GUI), a demonstrator super drawer behaves the same as a legacy super drawer.
For the off-line analysis, the ATLAS software framework (Athena) is used to reconstruct the energy and time information per cell from the raw data saved by Event Builder and FE-LIX, and to generate ntuples with the reconstructed information. During data-taking, Athena also collects some data from Event Builder and presents data quality information in the Data Quality Monitoring (DQM) for verification.

Beam Tests at CERN and Results
In order to test the detector performance of the TileCal Demonstrator, extensive tests with different particle beams (electrons, muons and hadrons) at different energies have been performed at the CERN SPS North Area.
The test beam setup is shown in Figure 7. Different kinds of beam detectors are installed for different purposes: two Cherenkov counters are used to identify proton, pions and electrons at energies below 50 GeV; a muon hodoscope placed behind the TileCal detector is used to detect muons; two wire chambers measures the exact position of the beam impact point; and the trigger is generated from coincidences of two scintillator detectors. The Tile-Cal demonstrator detector modules are mounted on a mobile table, as shown in Figure    The TileCal Demonstrator showed excellent performance during the beam test. Figure 9 presents the results obtained using muons, electrons and kaons beams from September 2017 test beam. Good agreement is achieved between the results from experimental data and the ones from the Monte Carlo simulation.  (c) The total energy distribution measured in data for hadron beam with identified kaons. Figure 9: Results using September 2017 test beam data [13]

Conclusion and Prospects
The TileCal Phase-II upgrade Demonstrator is developed to evaluate and qualify the detector performance with the Phase-II readout electronics while preserving the backward compatibility with the current system. The demonstrator design is well advanced, several system integrity tests with different beams at different energies have been performed at CERN. The beam test results show a good performance of the TileCal Demonstrator system.
For the next step, the TileCal Demonstrator is planned to be inserted into the ATLAS detector during the second LHC Long Shutdown in 2019 for further verification.