Front-End Board with Cyclone V as a Test High-Resolution Platform for the Auger_Beyond_2015 Front End Electronics

Transcription

1 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 62, NO. 3, JUNE Front-End Board with Cyclone V as a Test High-Resolution Platform for the Auger_Beyond_2015 Front End Electronics Zbigniew Szadkowski, Member, IEEE Abstract The surface detector (SD) array of the Pierre Auger Observatory needs an upgrade which allows space for more complex triggers with higher bandwidth and greater dynamic range. To this end this paper presents a front-end board (FEB) with the largest Cyclone V E FPGA 5CEFA9F31I7N. It supports eight channels sampled with max. 250 MSps@14-bit resolution. Considered sampling for the SD is 120 MSps; however, the FEB has been developed with external anti-aliasing filters to retain maximal flexibility. Six channels are targeted at the SD, two are reserved for other experiments like: Auger Engineering Radio Array and additional muon counters. The FEB is an intermediate design plugged into a unified board communicating with a micro-controller at 40 MHz; however, it provides 250 MSPs sampling with an 18-bit dynamic range, is equipped with a virtual NIOS processor and supports 256 MB of SDRAM as well as an implemented spectral trigger based on the discrete cosine transform for detection of very inclined old showers. The FEB can also support neural network development for detection of young showers, potentially generated by neutrinos. A single FEB was already tested in the Auger surface detector in Malargüe (Argentina) for 120 and 160 MSps. Preliminary tests showed perfect stability of data acquisition for sampling frequency three or four times greater. They allowed optimization of the design before deployment of seven or eight FEBs for several months of continuous tests in the engineering array. Index Terms DCT, FPGA, front-end, neural network, NIOS, Pierre Auger Observatory, trigger. I. INTRODUCTION T HE surface detector (SD) array of the Pierre Auger Observatory containing at present 1680 water Cherenkov detectors spread over an area of km has been in operation since The front-end boards currently in use are equipped with obsolete ACEX and Cyclone FPGAs (40 MSps/15-bit dynamic range). Huge progress in electronics and new challenges from physics require a significant upgrade of SD electronics either to improve the quality of measurements (much higher sampling rate and much wider dynamic range) or to pick up extremely Manuscript received June 12, 2014; revised December 19, 2014; accepted April 20, Date of publication May 21, 2015; date of current version June 12, This work was supported in part by the Polish National Center for Research and Development under NCBiR Grant ERA/NET/ASPERA/02/11 and in part by the National Science Centre (Poland) under NCN Grant 2013/08/M/ ST9/ The author is with the University of Łódź, Department of Physics and Applied Informatics, Faculty of High-Energy Astrophysics, Łódź, Poland ( zszadkow@kfd2.phys.uni.lodz.pl). Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /TNS Fig. 1. Comparison of dynamic ranges for the standard (10-bit) and new designs. We ignored two LSB in 14-bit ADCs using only the 12-bit resolution. Signals in high-gain channels are amplified by a factor of 16, those in low-gain channels suppressed by a factor of 4. The FEB with two 12-bit ADCs provides the 18-bit dynamic range with a 6-bit overlap. rare events from the background (new FPGA algorithms based on sophisticated approaches such as spectral triggers or neural networks). Much higher SD sensitivity is necessary to confirm or reject hypotheses critical for modern astrophysics. The data obtained with the Pierre Auger Observatory [1] have led to a number of major breakthroughs in the field of ultra-high energy cosmic rays: suppression of the cosmic ray flux at energies above ev; proof that top-down source processes such as the decay of super-heavy particles cannot be a significant part of the observed particle flux; observation of anisotropy of the arrival directions of particles with energies greater than ev. The primary objective of the upgrade of the Pierre Auger Observatory is to answer the question about the origin of flux suppression at the highest energies, i.e. the differentiation between the GZK-effect and the maximum energy of nearby astrophysical sources. To address all scientific targets, we propose an upgrade of the Pierre Auger Observatory to improve the physics potential of the data set. The aim of the Pierre Auger Observatory is to measure cosmic rays at the highest energies with unprecedented statistics and resolution. The first part of the Observatory is located in Argentina. It contains 1680 water Cherenkov detector stations distributed over an area of km for measuring charged particles associated with extensive air showers (EAS) and 24 telescopes with degrees of field of view and m mirror area each to observe the fluorescent light produced by charged particles in the EAS during operation on clear moonless nights. The simultaneous observation of EAS by ground array and the fluorescent light known as a hybrid event [2], [3] improves the resolution of the reconstruction considerably IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 986 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 62, NO. 3, JUNE 2015 Fig. 2. Fabricated Front-End board (12 layers) for the Auger-Beyond-2015 project with a temperature distribution on components of FEB in the laboratory environment. Upper temperatures are given without, lower with, radiators. Temperatures are measured by an infra-red sensor (IR-66) for 160 MSps. Cyclone V, ADCs and linear regulators even in an ambient room temperature indicate very hot (above ) packages. and, thanks to the calorimetric nature of the fluorescent light emitted, provides energy measurements virtually independent of hadronic interaction models. The Cherenkov light is detected by three 9-inch photomultiplier tubes (PMTs) of which the signals of the anode and last dynode are digitized by 10-bit ADCs. The front-end boards currently in use are equipped with ACEX [4] and Cyclone [5] FPGA sampled with 40 MHz. However, both FPGA families are already obsolete and ACEX chips are no longer produced. Data readout of the enhanced surface detector stations can be facilitated by replacing the current readout electronics with state-ofthe-art electronics offering sampling which is three times faster and a significantly enhanced dynamic range, and will provide enhanced trigger and monitoring capabilities. PMTs read out the Cherenkov light from the m of purified water contained in each tank. The signals from the anodes (low-gain (LG) channel) and dynodes (high-gain (HG) channel) are transported on shielded cables of equal-length to the front-end board (FEB), which is attached as a daughter board to a Unified Board (UB). The UB contains a micro-controller which manages all processes related to the data acquisition in the detector station. The splitting of the signals allows extension of the dynamic range of the measured energy range to 15 bits with five bits overlapping. The system digitizes the six analog signals of each detector station in the multistage differential pipeline architecture ADC. Signal filtering is performed by anti-aliasing five-pole Bessel filters. The outputs of the six 10-bit ADCs are processed by an Altera FPGA working as trigger/memory circuitry (TMC) (comparefig.3withfig.2in[4]andfig.1in[5]).thetmc evaluates the ADC outputs for interesting trigger patterns, stores Fig. 3. Structure of the Front-End Board and a data flow inside the FPGA. Quantized data from the ADC are received in LVDS receivers (there are no termination resistors on the board because of the OCT in the Cyclone V FPGA). The FPGA contains two buffers for the fast channel (an analysis of the profile of showers) and buffers (FIFO) for the slow channel (muon counting). FIFO replaced a previous external dual-port RAM. The FPGA also supports a spectral trigger based on the discrete cosine transform (DCT) and a trigger based on the artificial neural network (ANN). data in a buffer memory, and sends an interrupt to the UB if a trigger occurs [6]. II. REQUIREMENTS FOR AN UPGRADE The electronics currently used in surface detectors were designed at least years ago. Ten years in this context is an epoch. Several components are no longer produced so the replacement of failed components is a significant problem. On the other hand, increasingly better understanding of fundamental processes imposes new challenges requiring higher resolution, faster measurements with higher accuracy, and more sophisticated algo-

3 SZADKOWSKI: FRONT-END BOARD WITH CYCLONE V AS A TEST HIGH-RESOLUTION PLATFORM 987 rithms, etc. Fortunately, a significant number of such challenges can be overcome with new and powerful energy-efficient electronics with dedicated, embedded signal processing blocks allowing the implementation of much more complicated mathematical algorithms in real time. A. Faster Timing for the Surface Detector Stations The 40 MHz sampling in current use provides relatively smooth digitization. ADC samples at 25 ns intervals do not extract nano-second details from the analog signal, of course, but timing information is crucial for the separation of muon signals from electromagnetic components. Very inclined old showers (starting their development early in the atmosphere) with only muon components surviving generate very sharp rising ADC pulses with an exponential attenuation in the water Cherenkov detectors and can be a sign of old inclined showers. The precision of the rising time measurement is a factor. On the other hand higher sampling increases desynchronization of signals in three PMTs, decreasing the probability of threefold coincidences used in a standard threshold trigger. A desynchronized signal in a time domain can be recognized in the Fourier space by means, for example, of a discrete cosine transform algorithm [7] [9]. Neutrinos can generate showers starting their development deeply in the atmosphere, known as young. They contain a significant amount of an electromagnetic component, usually preceded by a muon bump. Simulations show [10] that it is often fully separated from the EM fraction. However, in this case very precise timing is essential. Generally, faster electronics provide more detailed ADC profiles, which are sometimes crucial for confirmation or rejection hypotheses. The Auger-Beyond-2015 project assumes 120 MHz sampling. The newly developed front-end board can operate even with 250 MHz sampling. In the laboratory we even detected 2 ns pulses (see Section VII). However, it should be pointed out that the use of 250 MHz sampling in the new FEB is only to verify (in what could be a wide range) several variants of data acquisition. The geometrical distance between PMTs is m. Depending on the azimuth angle differences between PMT excitations by direct light reach between 6 and 8 ns for horizontal showers. With 25 ns resolution (corresponding to the 40 MHz sampling currently in use) it is rather difficult to get valuable information on time distribution asymmetries between PMTs. Nevertheless, for 120 MHz or faster, sampling delays between PMTs can be registered as data shift to the next time-bin. An analysis of relative shifts for ADC profiles in the same surface detector can provide significant hints for trigger improvement for very inclined or horizontal showers. B. Increased Dynamic Range The 15-bit dynamic range with two 10-bit ADCs and 5-bit overlap is too narrow for an investigation of showers close to the core, where saturations even in the low-gain channel appear. The front-end board with Cyclone V developed in the University of Łódź, as a high-resolution test platform for the Auger-Beyond-2015 Front End Electronics, contains two 14-bit ADCs per PMT. The laboratory measurements showed that the noise is higher than we expected and planned 14-bit resolution is actually reduced to 12-bit only. We have to ignore two LSBs (as pure noise). Nevertheless, two 12-bit ADCs (with 6-bit overlap) give the 18-bit dynamic range. Two additional channels are designed for potential extra sensors. III. ANALOG SECTION The ACEX/Cyclone Front-Ends use six identical channels driven by PMT dynodes (high-gain channels) or PMT anodes (low-gain channels). The amplification in the HG is 32 times higher than that in the LG. In the new design we increase the sensitivity of measurements. Unfortunately, PMT dynodes provide relatively noisy signals. We therefore use anodes only and split analog signals directly on the FEB. ADCs in the UŁ design require a smaller amplification factor of 16. The total number of LVDS receivers only allows implementation of ADC. The proposed electronics upgrade provides a flexible interface allowing the other enhancements co-located with the surface detector stations to make use of the data processing and communication infrastructure of the stations. The final design extracts the best solutions as a compromise between resolution, speed and cost. In the analog section we used very fast components recommended by Texas Instruments: THS3201D (1.8 GHz, low-distortion current-feedback amplifier) two chips for two channels (instead of THS3202) to avoid cross-talk, THS4509 (wide-band, low-noise, low-distortion, fully differential amplifier with 1.9 GHz bandwidth). This very fast chain allows the detection of nanosecond peaks [Fig. 4(b)], albeit at the cost of relatively large power consumption. IV. FPGA SELECTION In the current design front-end boards are plugged into the UB on a 96-pin connector. The University of Łódź developed an intermediate design for the plugged-in FEB with all features required for the final design: fast sampling, a wider dynamic range, a possibility to merge other experiments. The following criteria were crucial for the FPGA selection: 1) number of LVDS receivers - to receive data from at least eight ADCs with at least 12-bit (preferably 14-bit) resolution, 2) an equivalent number of logic elements (LEs) - for implementation of general algorithms, 3) capacity for embedded memory - for implementation of large, fast buffers, typically as dual-port RAM, 4) a number of variable-precision digital signal processing (DSP) blocks which can implement embedded multipliers - important for sophisticated algorithms such as the trigger based on discrete cosine transform (DCT) [7] [9] for detection of very inclined old showers, the trigger based on an Artificial Neural Network (ANN) [10] for detection of young showers, potentially generated by neutrinos [11], [12], 5) a number of fractional clock synthesis phase-locked loops (PLLs) - to create individual PLL clocks corresponding to each ADC LVDS clock output,

4 988 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 62, NO. 3, JUNE 2015 firmware should either be written in VHDL or Verilog from scratch or should be translated from AHDL. Experiences with FPGA code development suggest we need to be very careful about simple code translation. Timing in VHDL, for instance, is not the same as in AHDL. AHDL code translated into VHDL implies varied timing optimization by the Quartus compiler. To keep the timing perfectly verified in real conditions for years and to save spending time on additional arduous simulations for timing verification we decided to adjust the existing AHDL code for new Cyclone V FPGAs. Only two chips obey the above criteria: 5CEFA7 (150 kle, 6.86 Mbit and 312 DSP 18x18 multipliers) and 5CEFA9 (301 kle, 12.2 Mbit and 684 DSP 18x18 multipliers). We decided to implement the biggest chip 5CEFA9F31I7 (industrial version with a temperature range -because of the large daily variation of temperature ) to retain maximal flexibility in the development of sophisticated, resource-consuming algorithms (e.g. ANN for recognition of young inclined showers). Fig. 4. Examples of ADC traces for 160 MSps DAQ. The upper graph shows an acquisition for a standard threefold coincidence in a single time-bin. A possible deflection from Tyvec (the 2nd peak in the PMT2 channel) is visible. The trigger appeared in 256th time-bin. The event shown in the lower graph would never have been registered for a standard trigger. It has been detected for extended subtriggers: four-time-bin pulses were generated for any signal above the threshold. A potential overlap of extended sub-triggers significantly increases the probability of the final trigger: threefold coincidences for four-time-bin pulses. 6) low power consumption. All previous generations of the FEB for surface detectors were equipped with Altera FPGAs [4][5]. However, we had to decide between two large FPGA producers, Altera and Xilinx, and opted for the latter s FPGA families integrating the standard fast logic with embedded micro-controllers (Hard Processing Systems-HPS)andSystemonChip(SoC).BothAlteraand Xilinx offer chips with HPS and SoC. Altera Cyclone V FPGAs offer the industry s lowest system cost and power, with high performance levels sufficient for highvolume applications. The SoC Altera Cyclone V SE family with ARM-based HPS offers the biggest 896-pin FPGA with only 72 LVDS lines [13]. Unfortunately, it is not enough even for six 12-bit channels (we also have to take into account the differential clock lines from ADCs). Other Altera equipments such as Arria V are too expensive. The University of Łódź decided to use the FPGA from the Altera Cyclone V E family without the SoC and HPS as they were not seen as absolutely necessary for a plugged-in FEB. The new design focuses on the reliability of the firmware (LVDS protocol with high sampling, standard and new trigger cooperation, potential lossless data compression, various modes of remote and on-side FPGA programming), optimization of the data acquisition for the final design, measurements of FEB parameters in real Argentinian pampas conditions as well as new trigger algorithms. Additionally, all old firmware written in AHDL for ACEX, Cyclone, Cyclone III and tested in pampas conditions for many years can easily be adjusted to the Cyclone V FPGAs. The Xilinx platform, however, does not support AHDL. The V. FPGA PROGRAMMING The surface detector is anticipated to be in operation for at least 15 to 20 years. It is therefore very likely that the algorithm of the detection controlling the FPGA will have to be updated. The system should be reconfigurable remotely, if possible. The FPGAs currently in operation in the surface detectors can be reprogrammed via the passive serial mode driven by the external host, the UB. The prototype FEB based on Cyclone V E is equipped with several programming modes so we can select the optimal one in the final version and keep a spare in case some modes fail. The prototype FEB is considered as an intermediate version, and is well equipped in the very sensitive analog section (14-bit ADCs with standard 120 MHz sampling: even 250 MHz sampling is possible) and a powerful digital trigger/memory circuitry, but plugged into the old UB (communication via DMA at 40 MHz with one wait-state). This intermediate FEB will allow verification of the new powerful system (both analog and digital sections) without a change in the existing infrastructure of the data transmission to the Central Data Acquisition Center (CDAS). The size of the programming files for ACEX CPLD (2nd generation - EP1K100Q208I7 [4]), Cyclone 3rd generation - EP1C12Q240I7 [5]) and Cyclone III (4th generation - EP4C40F324I7 [8]) are on the level of kb, 150 kb and 440 kb, respectively. However, the size of the rbf file for the largest chip from the Cyclone V family, 5CEFA9F31I7N (selected for the design), can reach 12 MB. Cyclone V E can be programmed from the UB level (as it receives the configuration file from the CDAS via radio). However, the UB contains only 2 MB oframforafulloperational system. The programming file has to be transmitted in pieces and stored in a nonvolatile memory. The transfer speed via radio is only 2400 bps. The 12 MB file will be transmitted for more than 14 hours. This is too long a time to be wasted on new file transmission. The transmission is considered against a background without a break in normal operation. The non-volatile memory used has a capacity of 128 MB, large enough to store

5 SZADKOWSKI: FRONT-END BOARD WITH CYCLONE V AS A TEST HIGH-RESOLUTION PLATFORM 989 eight different configuration files. The prototype is equipped with several programming modes in order to increase reliability of the entire system. All previous generations of the FEBs used only a single mode, passive serial, and the configuration file was uploaded to the FPGA from the external host (UB). The FPGA in the current design can be programmed in the following modes: 1) JTAG, considered for aboratory tests for temporary programming of the FPGA (J17 in Fig. 2), 2) passive serial via a 10-bit connector and the USB-Blaster programmer for laboratory tests - requires reprogramming after power down (J18 in Fig. 2), 3) combining JTAG programming of configuration device and FPGA with Active Serial configuration of FPGA using the EPCQ256 and the USB-Blaster (the MSEL[4..0] pins cannot be driven from the digital sources, as they have to be connected directly to the GND or VCC; a dynamical switch between Active and Passive Serial modes is not allowed) (J19 in Fig. 2), 4) passive serial (as in the previous generations of the FEBs) via the MAX V CPLD; the configuration file is uploaded sequentially from the UB to the serial NOR Flash memory N25Q00AA13GSF40F and next reloaded in a single shot from the memory to the FPGA. 5) passive serial via MAX V CPLD using the SDHC card (2 GB) located on the daughter board plugged into the 10-pin connector (J23 in Fig. 2). This mode is considered as the main one for the field operation. During the SD maintenance on the field, the local staff has min for a single SD, too short to upload several config files via a standard UB serial port even with speed of kbps. UARTs in the NIOS also support a maximum of kbps only. The SDHC card will be programmed in the laboratory and only plugged into the connector on the pampas. A single command from the CDAS is enough to reprogram the FPGA: the command selecting the address of the config file is in the SDHC. VI. FPGA CODE The FPGA code has been developed by the author for the APEX [15], ACEX [4], Cyclone [5] and Cyclone III [8] designs. Data from the FPGA are transferred via DMA protocol to the UB, analyzed by the T2 software trigger and then transferred via radio to the CDAS. Data are available for users in root files. Each trace for the events contains data for 256 time-bins before the trigger and 512 time-bins after the trigger. For 40 MHz sampling frequency 768 time-bins correspond to a signal profile for s. Analysis of a huge number of traces showed that the last eight words in each trace contain practically only noise. Monitoring files collect information on the working system; however, they cannot register specific information related to particular events. This information has been inserted on the end of the real traces in so-called diagnostic mode. For one year (2010) six surface detectors worked with the diagnostic mode on the Engineering Array. The current FEB provides bit data for 768 time-bins for each event. A matrix is standard format for SD data. Higher resolution, higher sampling and longer traces require a new format; however, this is a task for the updated unified board (UUB) which is currently in the development phase. This intermediate FEB has to use the standard data format in order not to change any cell in a data transmission chain. The trigger in the surface detector array is hierarchical. Two levels of trigger (T1 and T2) are used. T2 triggers are combined with those from other detectors and examined for spatial and temporal correlations, leading to an array trigger, T3, which initiates data acquisition by the CDAS. Data for each triggered event are stored on the local UB memory for 10 s, waiting for a possible T3. Two independent trigger modes are implemented as the T1, having been conceived to detect, in a complementary way, the electromagnetic and muonic components of EAS. The first T1 mode is a simple threshold trigger (Thr) which requires threefold coincidence above the This trigger is used to select large signals that are not spread over time. The Thr trigger is particularly effective for the detection of very inclined showers which form a flat pancake of muons and generate sharp narrow ADC traces. This trigger reduces the rate owed to atmospheric muons from 3 khz to 100 Hz. The Thr trigger rate is suppressed in the T2 trigger to Hz. The second T1 mode, ToT (Time over Threshold), is used for the detection of particles and photons at the detector dispersed in time. The ToT rate is on the level of 2 Hz and is not suppressed by the T2. Six channels for 14-bit ADC provide 84 bit data for each time-bin and 160 MHz sampling reduces the time window from sto s only for 768 time-bins. Only HG channels participate in trigger generation. Several configurations of data format (compatible with the standard one in order not to violate the interpretation of the eventbythet2,t3andthecdas)will be tested. All additional information will be inserted into the eight last words of the trace (the diagnostic mode). Ten fairly in significant bits from the 14-bit HG ADC will be put in place of the standard 10 bits of the HG channel. If the 14-bit value is greater than 1023 (10 LSBs may even be zeros) these 10 LSBs will be artificially set to #3FF to inform the T2 of the saturation in the HG channel. The LG channels can contain: four highly significant bits of the HG ADC on the four low significant bits of the LG channel and the six most significant (but non-zero) bits from the LG 14-bit ADC or 10 bits from the LG 14-bit ADC adjusted to neglect zeros for highly significant bits if the signal is small. Such a format provides non-overlapping information for strong signals as well as potentially dynamically overlapping information for relatively small signals. The s window can be too narrow, especially for events registered far from the shower core which are spread over time. The FPGA stores event data in two switching buffers (to reduce dead time) with four times longer length (1024 time-bins before a trigger and 2048 time-bins after it). The length of the buffer corresponds to s. The additional FPGA procedure checks a signal contribution for [0..768] and [ ] time-bins (it means earlier than 256 time-bins before the trigger and later than 512 time-bins after the trigger). If a significant signal contribution is found, data can be transmitted either: as the sum of three or four neighboring time-bins or with an additional one or three transmissions of extended events.

6 990 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 62, NO. 3, JUNE 2015 TABLE I LIST OF NEEDED ANALOG INPUTS AND DIGITAL I/O LINES The first variant provides loss compression; the second offers lossless data transfer, but it requires significant software modification in the UB and the CDAS. The second variant is treated as the possible backup. However, such a format modification has to be considered because data from two 12-bit ADCs (e.g. from the SPMT) should also be transmitted in the final design. The FPGA code receives data on 108 ( ) LVDS lines in total. Cyclone V families support the OCT (on-chip termination) option. No external 100 resistors for LVDS termination are needed. The ADS4249 ADCs provide the outgoing clock driving the PLL in the FPGA for optimal LVDS data reception. Four PLL circuits (each for double ADCs in ADS4249) with individual PLL clocks drive three synchronization register stages before the global FPGA clock. The Cyclone V FPGA also implements a NIOS processor supporting an external SDRAM (128 MB - U24 and U25 in Fig. 2), two RS232 UARTs (J11, J12 connectors on Fig. 2) and two 16-bit I/O buffers for additional experiments (J13, J14 in Fig. 2). DCT engines [9] contain a sigma-delta algorithm which can cope with huge daily temperature variations [14]. Each bank of the Cyclone V E can be supplied and drive I/O with various voltage standards. Table I lists I/O lines connected to the Cyclone V E banks. The previous version of the Quartus II v did not report any warnings for the FPGA code. The latest version, however, Quartus v. 13.1, reported for the previous code hundreds of critical warnings because of cross-talk between LVDS and single-ended lines. The reason was the relatively small distance between LVDS pin inputs and single-ended CMOS outputs. Movements of pin assignments with a gap of two or three pins removed all warnings. Nevertheless, it is a warning to us to select an FGPA chip with a large number of pins (e.g. the BGA896 package instead of cheaper version of F672, U484 or U324) to provide sufficient flexibility in terms of pin assignments. VII. DESIGN OPTIMIZATION IN LABORATORY TESTS The proposed sampling frequency for the new FEB is 120 MHz. However, for the prototype we developed both analog electronics with a digitizer and the FPGA code much faster. We used the Altera development kit with Cyclone V E FPGA 5CEFA7F31I7 [16] [17]. As the ADC we used Texas Instruments ADS4249EVM (Evaluation Module) [18] with a double channel 14-bit ADC with max. 250 MHz sampling. ADS4249EVM sends data to the 5CEFA7F31I7 via the Altera HSMC-ADC-BRIDGE [19] in the LVDS standard. Connection between the ADC and the FPGA was cm long and operated with very high reliability. The outgoing ADC clock supported the internal PLL in the FPGA. Each PLL for a single ADC chip (two channels each) optimized the reading of LVDS, interleaving data in the middle of a stable region. Even for a distance of 20 cm the LVDS transmission was perfect. In a developed FEB the distance between the ADCs and the FPGA is much shorter (Fig. 2). Input signals were even tested with 240 MHz sampling. The RMS measured for long traces is on the level of a 2.45 ADC unit (14-bit data processing). Nevertheless, note that the noise was measured for a non-standard configuration: the ADC was connected to the FPGA via an ADC-HSMC-BRIDGE with two HSMC connectors (total distance cm) with manually assembled resistors and capacitors for high-frequency response optimization. The test setup, developed originally for the optimization of the upgrade design, has also been used to test an FIR filter based on a linear predictor to suppress the RFI contaminations in the radio detectors of the AERA [20], [21], [22]. A. PLL Phase Optimization The main PLL circuit (PLL_0) provides several clocks: the main clock for data processing (from 40 to 240 MHz), the clock for the external clock splitter driving all SD ADCs (the same frequency as the main clock), the clock for the seventh and eighth channels (data processing with possible independent sampling frequency), the clock for data transfer to the UB - fixed 40 MHz). Each ADC chip provides its own clock to the FPGA, driving a separate PLL circuit (PLL_1 - PLL_4). Distances from ADC chips(u17-u20onfig.2)tothecyclonevfpga(u21)are not the same. In order to synchronize all ADC outputs with the global main clock the phases of PLL_1 to PLL_4 have to be tuned the LVDS signal in all channels to be converted into a single-ended signal without any of the distortions which can arise from incorrect phase compensation. Table III shows the phases for PLL_1 - PLL_4 necessary for a stable DAQ. Note that individual channel compensations are not needed for 120 MHz. B. Heat Removal from Components Temperatures of the Cyclone V, ADCs and linear regulators become really high even in the laboratory environment, especially for highest sampling frequencies. The first prototype FEB operated in the Auger test detector for 10 days with a radiator on the Cyclone V FPGA. The other chips were not at that time so equipped. According to the documentation the maximal operating temperature is, but long-term operation is definitely not recommended. Fig. 2 shows temperatures on components with and without radiators on chips. Daily temperature variations [14] may reach and temperature inside the electronic box even more than.all electronic components used in the surface detector are of industrial temperature with at least operational range of up to. Nevertheless, for a operationinpampasallfebsare equipped with radiators according to Fig. 2.

7 SZADKOWSKI: FRONT-END BOARD WITH CYCLONE V AS A TEST HIGH-RESOLUTION PLATFORM 991 TABLE II POWER CONSUMPTION FOR LABORATORY MEASUREMENTS (LABORATORY POWER SUPPLIES -UPPER TABLE) AND FOR DC-DC CONVERTERS FOR A DEPLOYMENT IN THE REAL FIELD ENVIRONMENT (LOWER TABLE) TABLE III PHASES FOR THE PLL CIRCUITS VS. AFREQUENCY SAMPLING VIII. MEASUREMENTS IN A TEST SURFACE DETECTOR A single FEB was already tested in the Auger surface detector in Malargüe (Argentina) for 120 and 160 MSps. Preliminary tests show perfect stability of data acquisition for sampling frequency three and four times greater. However, the FEB requires some optimizations before the deployment of seven or eight boards for several months of continuous tests in the engineering array. A. Optimization of a power consumption According to Table II the power consumption reaches 17 W for 120 MHz sampling frequency for only six SD channels. Theoretically, the power productivity of the solar panel is 10 W. However, this value corresponds to continuous work during several cloudy days when battery charging is poor. Practical tests on 10 sunny days show that 17 W of power consumption does not discharge batteries significantly and continuous work is possible. Fig. 6 shows the voltage on the battery for 18.5 hours (day and night). Depending on the sun, the battery can be temporarily charged to as much as 26 V. Nevertheless, voltage decrease during the night is slow and on normal fairly cloudless days we expect continuous operation. The DC-DC converters for the analog and digital power supplies are equipped in a remote control. In case of emergency both converters can be remotely switched off. This reduces power consumption on W (not 7.92 W as in Table II because of leakage between the digital and analog power supplies in chips with both supply sources). This value is still above the theoretical 10 W power productivity; nevertheless, this 10 W is estimated for very bad weather conditions, more likely in winter months. The system will be tested with a standard solar panel. If the uptime in real DAQ proves critical, we may consider mounting a second panel to provide sufficient power for continuous operation. Fig. 5. T1 and ToT trigger rates. The T1 rate for threefold coincidences for four-time-bin pulses as sub-triggers is on the level of 100 Hz, as in the standard design. The ToT rate was investigated for increasing occupancy from 30 to 52. The rate of course decreased. The optimal value for the occupancy seems to be 39. B. Remote Control In the case of dramatic battery discharge the analog power supply will be disabled. A command from the CDAS decoded in the MAX_V FPGA will disable the DC-DC converter (JCM1524D05 by XP Power) reducing the total power consumption to 11.5 W. This of course stops DAQ. The UB provides the 40 MHz clock to the FEB. If the FEB receives power before the UB, the FPGA cannot be programmed from the EPCQ256. The UB must be supplied first. DC-DC converters providing power to the UB and the FEB are supplied from the same V line from the batteries. During the turning on of the power some races may occur. There is no guarantee that the UB starts to work as the first (the clock necessary for the FPGA programming is taken from the UB). We add an emergency procedure for turning the digital voltage off and on (a similar control to that of the DC-DC converter supplying the digital section ( V). An additional command from the CDAS allows remote reprogramming of the Cyclone V. C. Optimization of the Firmware Increasing the sampling frequency also requires modification of the trigger circuitry. The probability of threefold coincidences (corresponding to the standard T1 trigger) for three or four times narrower time slots is extremely small. The trigger rate was almost on an unmeasured level. Fig. 4(b) shows an event registered in the test detector for a modified FPGA code. For a standard code this event would not have been detected because of desynchronization of signals. The modified code generates four-time-bin pulses in each channel for signals above a threshold.evenaveryshortpulse[e.g.6.25nsinpmt2from Fig. 4(b)] extended to 25 ns may overlap with pulses from neighboring channels sub-triggered in the following time-bins. The extension of the sub-trigger length allows for a 100 Hz trigger rate for the T1 as in the standard 40 MHz DAQ (Fig. 5). The ToT trigger requires at least 13 bins ( occupancy ) above a threshold of 0.2 VEM in twofold coincidences of any three PMTs in 120 time-bins of a sliding window ( s). For example, for 160 MSps of the DAQ simple scaling of the ToT parameters by a factor of four does not give satisfactory results. For the same width of the sliding window s (480 time-bins) the occupancy should be for a rate similar to the previous ToT rate Hz. Fig. 5 shows the dependence of the ToT rates for increasing the occupancy from 30 to 52.

8 992 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 62, NO. 3, JUNE 2015 REFERENCES Fig. 6. Voltage dependence on a battery. Discharging of the battery during the night is not relevant. Fig. 7. Location of the SDE Upgrade Engineering Array on the Pierre Auger Surface Detector Array. IX. CONCLUSION The Technical Board of the Pierre Auger Collaboration selected eight surface detectors (hexagon twin in the center for an investigation of possible GPS jitter) in a north-west region of the SD array (Fig. 7) for tests of the new FEBs on the Cyclone V platform with three or four times higher sampling and 12-bit resolution with cooperation of the SPMT and possibly other detectors. DCT triggers will be implemented simultaneously in parallel with the standard ones to verify the detection of very inclined showers based on an online analysis of the shape of signals in a frequency domain. We plan several months of tests to verify 14-bit and 12-bit DAQ (two LSBs neglected for future cheaper 12-bit ADCs), variants of data transmission to the CDAS in the current narrow radio channel (dynamical selection of a significant range of data for lossless DAQ - both in amplitude and time domain with dynamical summation of neighboring time-bins for large signals, when details cease to be significant), new trigger algorithms and NIOS support for temporary data storage. We believe that data obtained from the intermediate FEBs constitute a significant improvement on the final design for the Auger-Beyond-2015 and will enable new discoveries in astrophysical research. ACKNOWLEDGMENT The author would like to thank Y. Kolotaev from the Siegen University for the PCB design. [1] J. Abraham et al., Properties and performance of the prototype instrument for the pierre auger observatory, Nucl. Instrum. Meth. A, vol. 523, pp , May [2] P. Abreu et al., The exposure of the hybrid detector of the pierre auger observatory, Astroparticle Phys., vol. 34, no. 6, pp , Jan [3] M. Settimo et al., Measurement of the cosmic ray energy spectrum using hybrid events of the pierre auger observatory, Eur. Phys. J. Plus, vol. 127, no. 8, p. 15, Aug. 2012, 87. [4] Z. Szadkowski, The concept of an ACEX cost-effective first level surface detector trigger in the pierre auger observatory, Nucl. Instrum. Meth. A, vol. 551, pp , Oct [5] Z. Szadkowski, K.-H. Becker, and K.-H. Kampert, Development of a new first level trigger for surface array in the pierre auger observatory basedonthecyclonealterafpga, Nucl. Instrum. Meth. A, vol. 545, pp , Jun [6] J. Abraham et al., Trigger and aperture of the surface detector array of the pierre auger observatory, Nucl. Instrum. Meth. A, vol. 613, pp , Aug [7] Z. Szadkowski, A spectral 1st level FPGA trigger for detection of very inclined showers based on a 16-point discrete cosine transform for the pierre auger observatory, Nucl. Instrum. Meth. A, vol. 606, pp , Jul [8] Z. Szadkowski, Trigger board for the auger surface detector with 100 MHz sampling and discrete cosine transform, IEEE Trans. Nucl. Sci., vol. 58, no. 4, pp , Aug [9] Z. Szadkowski, Optimization of the detection of very inclined showers using a spectral DCT trigger in arrays of surface detectors, IEEE Trans. Nucl. Sci., vol. 60, no. 5, pp , Oct [10] Z. Szadkowski and K. Pytel, Artificial neural network as a FPGA trigger for a detection of very inclined Young showers, in Contribution to the IEEE Real Time Conference, Nara, Japan, May 26 30, [11] P. Abreu et al., Search for ultrahigh energy neutrinos in highly inclined events at the pierre auger observatory, Phys. Rev. D, vol. 84, no. 12, Dec. 2011, [12] P. Abreu et al., Search for point-like sources of ultra-high energy neutrinos at the pierre auger observatory and improved limit on the diffuse flux of tau neutrinos, Astrophys. J. Lett., vol.755,no.1,p.7,aug. 2012, L4. [13] [Online]. Available: [14] J. Abraham et al., Atmospheric effects on extensive air showers observed with the surface detector of the pierre auger observatory, Astroparticle Phys., vol. 32, no. 2, pp , Sep [15] Z. Szadkowski and D. Nitz, Implementation of the first level surface detector trigger for the pierre auger observatory engineering array, Nucl.Instrum.Meth.A, vol. 545, pp , Apr [16] [Online]. Available: [17] [Online]. Available: pl?language=english&categoryno=167&no=742 [18] [Online]. Available: [19] [Online]. Available: [20] Z. Szadkowski, E. D. Fraenkel, and A. M. van den Berg, FPGA/NIOS implementation of an adaptive FIR filter using linear prediction to reduce narrow-band RFI for radio detection of cosmic rays, IEEE Trans. Nucl. Sci., vol. 60, no. 5, pp , Oct [21] Z. Szadkowski, E. D. Fraenkel, D. Głas, and R. Legumina, An optimization of the FPGA/NIOS adaptive FIR filter using linear prediction to reduce narrow band RFI for the next generation ground-based ultra-high energy cosmic-ray experiment, Nucl. Instrum. Meth. A, vol. 732, pp , Jun [22] Z. Szadkowski, D. Głas, C. Timmermans, and T. Wijnen, First results from the FPGA/NIOS adaptive fir filter using linear prediction implemented in the AERA radio stations to reduce narrow bandl RFI for radio detection of cosmic rays, in Proc.IEEE Real Time Conf., Nara, Japan, May 26 30, [23] P. Abreu et al., Search for first harmonic modulation in the right ascension distribution of cosmic rays detected at the Pierre Auger Observatory, Astroparticle Phys., vol. 34, no. 8, pp , Mar