A MOdular System for Acquisition , Interface and Control ( MOSAIC ) of detectors and their related electronics for high energy physics experiment

In this work the MOSAIC (“MOdular System for Acquisition, Interface and Control”) board, designed for the readout and testing of the pixel modules for the silicon tracker upgrade of the ALICE (A Large Ion Collider Experiment) experiment at teh CERN LHC, is described. It is based on an Artix7 Field Programmable Gate Array device by Xilinx and is compliant with the six unit “Versa Modular Eurocard” standard (6U-VME) for easy housing in a standard VMEbus crate from which it takes only power supplies and cooling.


Introduction
This paper describes the main features and the preliminary tests of the electronics board, called MOSAIC, developed for the characterization of the detector building blocks of the new Inner Tracking System (ITS), which is a key element of the ALICE upgrade strategy for the Run 3 of LHC.ALICE [1] is the CERN LHC experiment designed to study the physics of strongly interacting matter at extreme conditions of energy density (∼ 15 GeV/fm 3 ) and temperature over a large volume (∼ 1000fm 3 ), and in particular the properties of the Quark Gluon Plasma (QGP), using nucleus-nucleus, proton-nucleus and proton-proton collisions.The ALICE long-term physics plans include a substantial upgrade of the central barrel detectors, which will be installed during the LHC Long Shutdown scheduled for the years 2019-20, in order to fully exploit the scientific potential of the LHC for fundamental studies of QCD [2].The planned upgrades, which will enhance the ALICE vertexing and tracking capabilities at low p T and allow data taking at substantially higher rates, include in particular a completely new ITS made of 7 cylindrical layers entirely based on an innovative active monolithic pixel sensor called ALice PIxel DEtector (ALPIDE) [3].The ALPIDE chip is the full custom pixel sensor that will equip all modules of the new ITS [4].

Hardware Description
Mainly conceived for the testing of detectors and their related electronics for high energy physics experiments, the MOSAIC board is able to read data through ten highspeed serial links at rates up to 6.6 Gbps or using up to 126 slower channels, such as general-purpose Low Voltage Differential Signals LVDS lines.
The board (Fig. 1) is equipped with four programmable LEMO Input/Output complying with Nuclear Instrument Module standard and two FMC-LPC mezzanine slots in order to provide additional connectivity and flexibility.Two Robinson Nugent P50E-068-P1-SR1-TG connectors are housed on the front panel, providing 33 LVDS signals for each connector plus the ground reference.Each LVDS couple is configurable as input or output and is protected against electrostatic discharges.The FPGA firmware can be designed to read data from these inputs at a maximum rate of 1.2 Gbps or to control ancillary electronic, like power control and monitoring interfaces.
To allow communication with high speed devices, a Samtec Eye Speed HDI6-035 connector is available on the front panel.Ten high speed transceivers, running up to 6.6 Gbps are connected to it.Moreover it provides two configurable LVDS couple of lines and two M-LVDS bidirectional differential lines, intended to control and monitor the front-end electronics.
All communication with the board for data readout, setup and status monitoring are performed through an Ethernet interface capable of running at 10, 100 and 1000 Mbps.The interface is able to configure itself automatically based on the connected device, so it can be connected directly to a PC without a network switch or a crossed cable.
The system can be expanded plugging up to two "FPGA Mezzanine Card" (FMC) Low pin count or one Double Width Card.Each FMC connector provides 34 differential pairs lines, configurable as 68 single ended lines, two clock differential lines and the connection to one high speed transceiver running up to 6.6 Gbps.The FMC slot is mechanically and electrically compliant to the ANSI/VITA 57.1 standard.

Firmware
The FPGA firmware has been designed to provide a common infrastructure and a set of modules specific for the external hardware that it will control or read.The hardware specific modules ("High Speed Receiver" and "Control line interface" in Fig. 2) are connected to the infrastructure in two ways: a) using a custom interface designed to move data as fast as possible to DDR3 main memory for data acquisition b) through a Wishbone bus for configuration and monitoring The infrastructure is made by the blocks described in the following sections.
The Data Collector is devoted to move data from the hardware specific block buffers to the DDR3 memory.When a High Speed Receiver needs to move data from the buffer, it sets a read request flag and, as soon as possible, the Data Collector moves the requested amount of data to the DDR3 memory at a sustained rate of 5 GB/s.A header is inserted before each data block; it contains a detailed description of the block like its size, source, errors and additional flags from the data source.The Gigabit Ethernet interface has been designed to allow high transfer rate with a very low CPU load.While the communication protocols are handled by the local CPU, all data are transferred by hardware using the DMA engine in the Data Controller module.Thanks to the hardware offloads implemented in the interface, like IP fragmentation and TCP checksum, the board is able to send data on the TCP connection at a transfer rate of 120 MB/s (the Gigabit Ethernet speed limit) with a CPU load of 30%.
The CPU implemented in the firmware is a 8-bit microprocessor running at 50 MHz with 12 kB of program memory and 2 kB of RAM, plus additional peripherals such as Interrupt logic, TIMER, I/O etc.The microprocessor has a Complex Instruction Set Computer (CISC) architecture, that has the advantage to allow the programmer to write complex programs even though its small size.The software running into the microprocessor needs less then 10 kB and implements the TCP/IP and UDP/IP protocols and accessory functions such as board startup diagnostic tests, firmware upgrade, IP configuration etc.
The IPBus Transactor translates the commands coming from the PC encapsulated in UDP/IP packets into Wishbone bus transactions.A single network packet can contain many bus read/write commands: the CPU forwards the request packet to the transactor that processes it and builds the answer packet containing transaction results.
The MOSAIC firmware further provides utility blocks such as the Pulse Logic, which implements a programmable periodic test-pulse/trigger generator to speedup the characterization procedures, and the Trigger Logic  to control and monitor the internal and external Trigger pulse production.
The whole structure in Fig. 2, as needed for the ALICE pixel module test, requires less than 10% of the FPGA resources, meaning that additional data processing can be potentially implemented in the hardware specific blocks and many external devices can be controlled with a single board.

Prototype test results
Three board prototypes have been tested to validate the design before final production.Two critical blocks have been identified: the DDR3 interface and the High Speed Transceivers.
The DDR3 interface requires special care in the PCB design due to the timing constraint, signal integrity and crosstalk for all signals connecting the FPGA to the SODIMM connector.The parameter suggested by Xilinx to measure the design quality is the time margin on sampling the signal on data and address busses.This measure has been performed on the MOSAIC board and on Xilinx AC701 evaluation board and the set of results are shown in Fig. 3, where the setup+hold margin is plotted for each byte lane of the bus for read and write cycles.The performance of the High Speed data link has been evaluated connecting in loopback one of the FPGA transceivers through the board connector and a 2 m long cable.The Xilinx tool [6] has been used to perform the statistical eye scan and evaluate the Bit Error Rate (BER) as a function of the time offset of the sampling time from the ideal time (on the horizontal axis) and voltage offset (on the vertical axis).Data are plotted in the top picture of Fig. 4 where time is in Unit Interval (UI), i.e. the symbol period, and voltage in DAC codes, equivalent to 2 mV.BER for voltage offset equal to zero is shown on the bottom plot of Fig. 4 to obtain the bathtub diagram.The vertical bars at +/-0.175 UI are the maximum time error allowed by the receiver.From the diagram we can extrapolate the BER curve to estimate the eye shrink for an eye opening of 0.60 UI at 10E-15.The width and the height margins can be evaluated as 70% and 180%, respectively.

Conclusions
A custom board for readout and testing of detectors and their related electronics has been designed.From the tests carried out on the prototype, it fulfills all the design speci-fications.The board is currently used to perform the functional test of the modules of the new pixel tracker for the upgrade of the ALICE experiment.The firmware architecture provides the possibility to easily add new custom interfaces to other front-end modules.

Figure 1 .
Figure 1.The MOSAIC board

Figure 2 .
Figure 2. Block diagram of the FPGA firmware for the test of the ALICE new tracker modules

Figure 3 .
Figure 3. DDR time margin for read operations (top) DDR time margin for write operations(bottom)

Figure 4 .
Figure 4. Eye scan for the transceiver at 6.4 Gbps (top) and bathtub diagram from the eye scan data (bottom).The eye scan shows the Bit Error Rate (BER) as a function of time (horizontal axis) and voltage offset (vertical axis) with respect to the ideal sampling point.Fixing the value of the voltage offset at 0, the recorded BER as a function of time expressed in UI give rise to the bathtub diagram in the bottom panel.