SBS VME DAQ. SBS collaboration Meeting Alexandre Camsonne July 7 th 2014

Transcription

1 SBS VME DAQ SBS collaboration Meeting July 7 th 2014

2 Outline SBS DAQ overview GEM readout Fastbus readout HCAL readout Plan Manpower Conclusion 2

3 SBS DAQ Overview Calorimeter ECAL : Fastbus HCAL SBS GEM APV25 INFN MPD BigBite Scintillator Shower preshower Coordinate detector 3

4 SuperBigbite Spectrometer Focal Plane Polarimeter setup 4

5 HCAL Calorimeter Rates (CDR section 5.1.7) Most demanding Hadron rate estimate using SLAC & DESY data, Wiser code: w/4.5 GeV threshold: 1.5 MHz ECAL From Hall A Real Compton Scattering experiment NB: Good resolution 16%/ E Background rate vs. cut on deposited energy (MC studies in progress) Electron rate estimate w/2.5 GeV threshold (73% of E elas ): 200 khz 9 khz coincidence rate w/ 30 ns window 5

6 DAQ concept Hybrid Fastbus and pipelined electronics Level ns latency by analog summing and discrimination Generated by electron arm ( 200 khz rate) Gate for Fastbus & non-pipelined VME for BigCal Level 2 : coincidence proton in HCAL and electron in ECAL Assume up to 1.8 μs latency ( L2 800 ns max + Fast Clear 1 μs) 9 KHz with 30 ns coincidence windows FPGA-based coincidence logic using geometrical constraints reduction by factor 5 2 khz physics DAQ rate Fast Clear FB & VME after L2 timeout 13% Electronics Dead Time 6

7 e -p Kinematic Correlation 11 x 22 HCAL blocks 20 x 76 ECAL blocks (CDR section F.3) Using geometric correlations from elastic kinematic one can reduce final rate by a factor of 5 and tracker data by at least a factor of 3 7

8 Data flow 100 Mb/s CODA GEM APV25 VME digitizer VME CPU GigE GigE Event builder Event recorder Summing L1 trigger TS SAS RAID BigCal Bigbite HCAL L1 trigger Fastbus Digital Sum VME CPU VME CPU Logic L2 trigger Compression + 10 GigE Data file Silo 8

9 GEM GEM APV25 APV25 BigCal Bigbite HCAL VME digitizer L1 trigger FEC SRS SRU VME CPU Summing Fastbus Digital Sum GigE 100 Mb/s GigE GigE L1 trigger VME CPU VME CPU 10 GigE PC Event builder 10 GigE Logic TS Event recorder L2 trigger SAS RAID Compression + 10 GigE CODA Data file Silo 9

10 GEM readout INFN MPD VME64X board New version with fast VME protocol Working with CAEN controller Adapting software package to Intel VME CPU to use with CODA SRS readout Using Mississippi State SRS Running with DATE, starting to look integration into CODA Standard UDP based protocol Request to use GEM for PREX and Tritium in addition to A1n Will have baseline performance number before November 10

11 2048 channel 1 MPD Front Tracker layout 2048 channel 1 MPD 11

12 1 MPD 2048 Front tracker Region of Interest from BigCal position Trackers layout 1 MPD 2048 Back tracker Region of Interest from HCal position Middle tracker Interpolated from both front and back information 12

13 Trackers layout Worst case configuration 13

14 Suppression schemes Several algorithm can be implemented in FPGA for further data reduction Thresholds Timing Fitting c2 slope 14

15 2048 FPP Tracker layout

16 Tracker event size with 3 samples readout Detector Rate Channels Occupancy Time window Hits Geometr ical factor Data size Effective data size Data rate 3KHz Mb/s Front Tracker % 75ns Kb 14.3Kb 43 Second Tracker % 50ns Kb 1.6Kb 5 Third Tracker % 50ns Kb 0.8Kb 2 Electron arm GEM % 50ns Kb 2.3Kb 6 Calorimeters Kb 0.5Kb 1.5 Total 67.8Kb 19.5Kb 58.5 Mb/s 16

17 Data rates Detector Rate Channels Occupancy Hits Data size Bytes Data rate MB/s 5 KHz Geometrical factor Data size no sup Data rate MB/s Front Tracker Second Tracker Third Tracker Total

18 Fastbus readout Time : 1877S Amplitude 1881M or MQT+1877S Fastbus max transfer speed : 40 MB/s can use either Intel or Old vxworks VME CPU Test Lab : 4 sets of 3 crates, will be able to test performance 18

19 HCAL readout 288 channels 2 VXS crates, 18 s 1.5 MHz singles 16 block clusters 250 MHz 12 bit = 2 bytes 10 samples : 320 bytes Need HCAL occupancy 19

20 Tape price Experiment Days Rate (MB/s) seconds Total TB Double DLO5 DLO6 GEp GEn GMn SIDIS Total 397 K$ 239K$ 20

21 Man power A. Camsonne : General, MPD readout S. Abraham, M. Jones : Fastbus Students Jessica Campbell (SMU) : Fastbus Jessie Twigger (FIT SULI),?: SRS CODA readout HCAL trigger readout :? 21

22 Plan summary Setup on 3 Fastbus crate setup and test performance APV25 performance with SRS and MPD and integration with CODA Getting started with for HCAL Start write-up about DAQ 22

23 Conclusion Need good ways to reduce data Fastbus setup almost ready for testing Need to start thinking about SIDIS Start writeup 23

24 Backup

25 VETROC JLAB Electronics group 25

26 CPU CTP SD TI CPU GTP SD TI CPU SD SD TS L1 Trigger Diagram VXS Crate CTP VXS Crate MHz, 16 ch Sums amplitude from all channels Transfer total energy or hit pattern to CTP VXS Serial Link MHz: 4 Gbps VXS Crate Fiber Optics MHz Crate Trigger Processor Sums energies from s Transfer total energy or hit pattern to 26

27 CPU CTP SD TI CPU GTP SD TI CPU SD SD TS L1 Trigger Diagram VXS Crate VXS Crate Sub-System Processor Consolidates multiple crate subsystems Report total energy or hit pattern to GTP VXS Serial Link MHz: 8 Gbps VXS Crate Copper Ribbon Cable MHz: 8 Gbps Global Trigger Processor Collect L1 data from s Calculate trigger equations Transfer 32 bit trigger pattern to TS 27

28 Level-1 Trigger Electronics Custom Designed Boards at JLAB Detector Signals (16) (1) (1) (1) Fiber Optic Links Clock/Trigger 62.5MHz (12) (1) (1) ( ) Number in parentheses refer to number of modules Fiber Optic Link (~100 m) 125 MHz) fadc250 CTP Crate Trigger Processor SD Signal Distribution (8) (2) (1) Copper Ribbon Cable (~1.5 m) 250 MHz) TI Trigger Interface VXS Backplane Pipelined detector readout electronics: fadc Trigger Latency ~ 3 μs 28

29 CTP GTP CTP Pipelined Hall D DAQ Calorimeter Light Gas Cerenkov Heavy Gas Cerenkov C T P C T P 29

30 Pipelined Hall D DAQ Electron shower accidental accidental 3 us latency Above threshold 30

31 Read Out Controllers Raid Disk L3 Farm blocked event fragments partially recombined event fragments full events All nodes connected with 1GB/s links ROC ROC ROC ROC ROC ROC ROC ROC ROC EB1 Event Builder stage 1 EB1 Event Builder stage 1 EB1 Event Builder stage 1 EB2 Event Builder stage 2 EB2 Event Builder stage 2 node node node L3 Farm node node node node Switches connected with 10GB/s fiber optics ER Event Recorder Front-End Crates ~60 crates ~50MB/s out per crate Staged Event Building N x M array of nodes (exact number to be determined by available hardware at time of purchase) Level-3 Trigger and monitoring Event Recording 300MB/s in 300MB/s out 31

32 Energy BCAL (GeV) Energy BCAL (GeV) Energy BCAL (GeV) Hall D L1 Trigger-DAQ Rate Low luminosity (10 7 g/s in 8.4 < E g < 9.0 GeV) 20 khz L1 High luminosity (10 8 g/s in 8.4 < E g < 9.0 GeV) 200 khz L1 Reduced to 20 khz L3 by online farm Event size: 15 kb; Rate to disk: 3 GB/s SC Detectors which can be used in the Level-1 trigger: Forward Calorimeter (FCAL) Barrel Calorimeter (BCAL) Start Counter (SC) Time of Flight (TOF) Photon Tagger Basic Trigger Requirement: Energy Energy Hits Hits Hits Electromagnetic background Hadronic E g < 8 GeV Hadronic E g > 8 GeV E BCAL + 4 E FCAL > 2 GeV and a hit in Start Counter Energy FCAL (GeV) Energy FCAL (GeV) Energy FCAL (GeV) 32

33 Custom Electronics for JLab VME Switched Serial (VXS) backplate 10 Gbps to switch module (J 0 ) 320 MB/s VME-2eSST (J 1 /J 2 ) All payload modules are fully pipelined 125 (12 bit, 72 ch) 250 (12 bit, 16 ch) F1-C (60 ps, 32 ch or 115 ps, 48 ch) Trigger Related Modules Crate Trigger Processor (CTP) Sub-System Processor () Global Trigger Processor (GTP) Trigger Supervisor (TS) Trigger Interface/Distribution(TI/D) Signal Distribution (SD) 125 F1-C 33

34 CPU CTP SD TI CPU GTP SD TI CPU SD SD TS L1 Trigger Diagram VXS Crate VXS Crate Trigger Distribution Distribute trigger, clock and synchronize signals to TI in each Crate Fiber Optics MHz: 1 Gbps VXS Crate VXS Serial Link MHz: 1 Gbps Trigger Supervisor Calculate 8 bit trigger types from 32 bit trigger pattern Prescale triggers Transfer trigger and sync signal to (16 bit total) 34

35 CPU CTP SD TI CPU GTP SD TI CPU SD SD TS L1 Trigger Diagram VXS Crate VME Readout Controller Gigabit ethernet Signal Distribution Distribute common signals to all modules: busy, sync and trigger 1/2 TID VXS Crate VXS Crate Trigger Interface Receive trigger, clock and sync signals from Make crate trigger decision Pass signals to SD VXS Serial Link MHz: 1 Gbps 35

36 The GlueX Detector 2.2 Tesla Solenoid 2.2T superconducting solenoidal magnet Fixed target (LH 2 ) 10 8 tagged g/s ( GeV) hermetic TOF time of flight SC start counter Charged particle tracking Central drift chamber (straw tube) Forward drift chamber (cathode strip) Calorimetry Barrel Calorimeter (lead, fiber sandwich) Forward Calorimeter (lead-glass blocks) PID Time of Flight wall (scintillators) Start counter Barrel Calorimeter 36

37 LHC JLab BNL CHEP2007 talk Sylvain Chapelin private comm. GlueX Data Rate Front End DAQ Rate Event Size L1 Trigger Rate Bandwidth to mass Storage GlueX 3 GB/s 15 kb 200 khz 300 MB/s CLAS GB/s 20 kb 10 khz 100 MB/s ALICE 500 GB/s 2,500 kb 200 khz 200 MB/s ATLAS 113 GB/s 1,500 kb 75 khz 300 MB/s CMS 200 GB/s 1,000 kb 100 khz 100 MB/s LHCb 40 GB/s 40 kb 1000 khz 100 MB/s STAR 50 GB/s 1,000 kb 0.6 khz 450 MB/s PHENIX 0.9 GB/s ~60 kb ~ 15 khz 450 MB/s * ** * Jeff Landgraf Private Comm. 2/11/2010 ** CHEP2006 talk MartinL. Purschke 37

38 CODA3 What s different CODA 2.5 CODA 3 Run Control (X, Motif, C++) (rcserver, runcontrol) Communication/Database (msql, cdev, dptcl, CMLOG) Event I/O C-based simple API (open/close read/write) Event Builder / ET System / Event Recorder (single build stream) Front-End vxworks ROC (Interrupt driven event by event readout) Triggering: 32 ROC limit, (12 trigger bits -> 16 types) TS required for buffered mode Experiment Control AFECS (pure JAVA) (rcplatform, rcgui) cmsg CODA Publish/Subscribe messaging EVIO JAVA/C++/C APIs Tools for creating data objects, serializing, etc EMU (Event Management Unit) Parallel/Staged event building Linux ROC, Multithreaded (polling event blocking) 128 ROC limit, (32 trigger bits -> 256 types) TI supports TS functionality. Timestamping (4ns) 38

39 Encoding Example 39

40 GTP Trigger Bit Example 40