The Trigger and Data Acquisition System for the KM 3 NeT neutrino telescope

KM3NeT is a large research infrastructure in the Mediterranean Sea that includes a network of deep-sea neutrino telescopes. The telescopes consist of vertical detection units carrying optical modules, whose separation is optimised according to the different ranges of neutrino energy that shall be explored. Two building blocks, each one made of 115 detection units, will be deployed at the KM3NeT-IT site, about 80 km from Capo Passero, Italy, to search for high-energy neutrino sources (ARCA); another building block will be installed at the KM3NeT-Fr site, about 40 km from Toulon, France, to study the hierarchy of neutrino masses (ORCA). The modular design of the KM3NeT allows for a progressive implementation and data taking even with an incomplete detector. The same scalable design is used for the Trigger and Data Acquisition Systems (TriDAS). In order to reduce the complexity of the hardware inside the optical modules, the “all data to shore” concept is adopted. This implies that the throughput is dominated by the optical background due to the decay of 40K dissolved in the sea water and to the bursts of bioluminescence, about 3 orders of magnitude larger than the physics signal, ranging from 20 Gbps to several hundreds Gbps, according to the number of detection units. In addition, information from the acoustic positioning system of the detection units must be transmitted. As a consequence of the detector construction, the on-shore DAQ infrastructure must be expanded to handle an increasing data-rate and implement an efficient fast data filtering for both the optical and acoustic channels. In this contribution, the Trigger and Data Acquisition System designed for the Phase 1 of KM3NeT and its future expansion are presented. The network infrastructure, the shore computing resources and the developed applications for handling, filtering and monitoring the optical and acoustic data-streams are described.


Introduction
KM3NeT [1] deep-sea research infrastructure consisting of a network of neutrino telescopes will be installed in two sites in the Mediterranean Sea: off Toulon at a depth of about 2500 m, France, the ORCA (Oscillation Research with Cosmics in the Abyss) detector will study the neutrino mass hierarchy, using 115 string-like [2] Detection Units (DUs); off Portopalo di Capo Passero at a depth of 3500 m, Sicily, the ARCA (Astronomy Research with Cosmics in the Abyss) detector will study ultra high energy neutrinos expected by astrophysical and cosmologic cosmic rays sources.ARCA will be made by about 230 string-like and 8 tower-like [3,4] DUs arranged in a ∼ 100 meters-spaced grid.Each string carries 18 Digital Optical Modules (DOMs) [5,6], which are vertically separated by 6 and 36 meters in ORCA and ARCA, respectively.Each DOM is a 17 diameter glass sphere containing 31 3 photomultiplier tubes.The detection principle is based on the collection of the Cherenkov light emitted along the path of charged particles produced by neutrino interactions in the detector vicinity.
Due to the high optical background caused by decay of 40 K and bioluminescence phenomena in sea water, an on-line trigger is needed.In order to minimise the complexity of the underwater electronics, no hardware trigger is present and the all-data-to-shore approach is used: all detected optical signals are sent to shore via a computer network which comprehends both the shore-station and the detector.In the final configuration of the experiment, the expected data throughput from both sites is up to 500 Gbps.

Network topology
The DOMs, i.e. the off-shore detector, and the shore-station computing infrastructure are part of a computing network subdivided in 3 main subnetworks (see Fig. 1): • Raw data network: aims at connecting the off-shore detector elements (DOMs and DU bases) with both the computers dedicated to the aggregation and distribution of data (DataQueue, see the next section) and the ControlUnit of the experiment; • Managed data network: aims at connecting the DataQueues with the data filtering infrastructure; • Filtered data network: is devoted to the transportation of post trigger data.
Very Large Volume Neutrino Telescope (VLVnT-2015) The Raw data network represents the main networking peculiarity of the experiment.It makes use of a White Rabbit [8] asymmetric network, which also provides for sub-nanosecond time synchronisation.The asymmetry comes from the physical topology of the optical links between the underwater detector elements and the on-shore station.Every single DOM has a unidirectional 1 Gb/s uplink to reach the on-shore station delivering the acquired data and the slow control (SC) replies.On the other side, the on-shore station has a unique unidirectional 1 Gb/s downlink to steer all the DOMs.This is called the broadcast and is shared by every single DOM, so that when a DOM receives a packet, all other DOMs also receive it.This topology highly reduces the communication resources cost (i.e. the number of optical fibres), but it requires a specific customisation of the communication elements as the switches and the DOMs.
The Managed and Filtered data networks are standard Ethernet networks that provide a data transfer speed of 10 Gbps.
A 1 GbE Control network allows the connection of all the shore-station computing elements with the ControlUnit and provides access to the Internet through a Bastion Server, which protects the internal networks.

The KM3NeT TriDAS
The KM3NeT Trigger and Data Acquisition System [7] (TriDAS) is the software and computing infrastructure deputed to the read-out, aggregation and filtering of all the detector data.All TriDAS components are connected through the network described in Sect. 2.
Each DOM carries 31 3 PMTs and a piezo-electric acoustic sensor that produce the two main data streams from off-shore: the optical and acoustic data streams, respectively.The DOM digitises the signals from the PMTs as photon hits -defined by the integrated electric charge of the PMT signal over a certain threshold, the crossing time and the duration of the signal over the threshold -and collects them into optical data frames.Acoustic signals reaching the piezo-electric sensor are sampled at a constant frequency and with a proper resolution -both tunable from on-shore -and compose an acoustic data frame.Once a data frame is completed, the DOM sends it to shore using a UDP/IP protocol.
On-shore, the first data processing stage is represented by the DataQueue (DQ) farm which is in charge of the aggregation of unfiltered data and of their distribution to the computers dedicated to the on-line analysis and trigger, namely DataFilters (DFs).For each data stream a specific implementation of the DF exists, called AcousticDataFilter (aDF) and OpticalDataFilter (oDF) for acoustic and optical data streams, respectively.Each DQ is responsible for the collection of the data coming from a sector (i.e. a subset of DOMs) of the detector.Each oDF receives from all the DQs a bunch of data belonging to a precise time interval called Time Slice (TS).The duration of a TS is fixed, typically 100 ms.In this way, each oDF has a complete snapshot, limited to the TS duration, of the status of the whole detector.Triggered data coming out from the oDFs are sent to the DataWriter (DW) that collects them on permanent storage as ROOT [9] files.Each aDF, differently from the optical case, receives all data from a single DQ continuously, thus having the complete data time series from a subset of the acoustic sensors.The aim of the aDF is to provide on-line reconstruction of the Time Of Arrival (TOA) of the acoustic signals emitted by the acoustic beacons of the acoustic positioning system [10].The output of all the aDFs is collected by the DataBaseWriter (DBW, see below) and written to a database.
The ControlUnit (CU) [11], which represents the user interface to the detector, aims at coordinating the TriDAS and operating the DOMs through a dedicated SlowControl (SC) protocol.It is composed of the following sub-systems: • Master Control Program (MCP): is the main user interface and allows to schedule data taking of the experiment; • TriDAS Manager (TM): steers the on-shore computing facility, setting up the TriDAS processes;

Figure 1 .
Figure 1.Sketch of the network topology of the KM3NeT detector and shore-station infrastructure.For a detailed description see the text.