Trigger and Data Acquisition at the Large Hadron Collider

Trigger and Data Acquisition at the Large Hadron Collider

Acknowledgments This overview talk would not exist without the help of many colleagues and all the material available online I wish to thank the colleagues from ATLAS, CMS, LHCb and ALICE, in particular R. Ferrari, P. Sphicas, C. Schwick, E. Pasqualucci, A. Nisati, F. Pastore, S. Marcellini, S. Cadeddu, M. Zanetti, A. Di Mattia and many others for their excellent reports and presentations 17-June-2006 A. Cardini / INFN Cagliari 2

Day 1 - Summary LHC Accelerator parameters The experiments Triggering General Concepts LHC Requirements Trigger architecture Implementation of First Level Trigger ATLAS and CMS LHCb ALICE The First Level Trigger Technology 17-June-2006 A. Cardini / INFN Cagliari 3

LHC

LHC Accelerator Complex Proton-proton CM energy = 14 TeV L = 10 34 cm -2 s -1 ATLAS, CMS L = 10 32 cm -2 s -1 LHCb Heavy Ions (ex.: Lead-lead) CM Energy = 1312 TeV (!) L = 10 29 cm -2 s -1 for ALICE 17-June-2006 A. Cardini / INFN Cagliari 5

Why LHC? We need a high luminosity and a high center-of-mass energy proton-proton collider to Search of Higgs boson(s) Search for SUSY particles Standard Model Physics CP violation studies in the B sector New Physics beyond SM Ultra High Energy Heavy Ions Collisions hope that we will see many other things we do not expect 17-June-2006 A. Cardini / INFN Cagliari 6

LHC vs. LEP vs. Tevatron One way to increase the luminosity is to increase the number of bunches circulating in the ring: LHC will have ~3600 bunches 27 km (LEP tunnel) ring 27000 m / 3600 = 7.5 m between bunches 7.5 m / 3x10 8 m/s = 25 ns 17-June-2006 A. Cardini / INFN Cagliari 7

Proton-proton cross section Interactions/second L = 10 34 cm -2 s -1 = 10 10 Hz/b σ inel (pp) ~ 70 mb 7x10 8 interactions/s σ inel (pp) ~ 70 mb Events/crossing @ 40 MHz ( t = 25 ns) 17.5 interactions/crossing Not all bunches are full Only about 4/5 (2835/3564) 22 interactions/ real crossings What are we looking for? 1 interesting physics event (Higgs for example) superimposed to ~20 minimum bias events!!! 17-June-2006 A. Cardini / INFN Cagliari 8

Here it is! 20 minimum bias events overlapping + H ZZ 4 muons (the cleanest golden signature) This mess (not the Higgs! - its production cross section in very small so we expect 0.01-0.1 Hz Higgs production) repeats every 25 ns! 17-June-2006 A. Cardini / INFN Cagliari 9

The Physics at LHC Cross sections for various physics processes vary of many orders of magnitude At the standard LHC luminosity we have: Inelastic (min. bias): 10 9 Hz W lν: 10 2 Hz ttbar: 10 Hz Light Higgs (100 GeV): 0.1 Hz Heavy Higgs (600 GeV): 0.01 Hz bbbar: huge (10 6 Hz) An efficient selection mechanism capable of selecting 1 event over 10 10-10 11 is needed: this is the TRIGGER 17-June-2006 A. Cardini / INFN Cagliari 10

How to build a LHC Experiment? Depends obviously on the physics New (very) heavy particles (Higgs, for example) are produced centrally with large transverse momentum symmetric detector ATLAS, CMS Lighter particles (B, for example), are produced mainly at small angles. One can take advantage of the boost forward detector for B physics LHCb When running in heavy ions mode ALICE will search for Quark-Gluon Plasma, needing both central and forward coverage 17-June-2006 A. Cardini / INFN Cagliari 11

How to build a LHC Experiment? (2) Each experiment is made of Inner trackers Calorimeters Muon detectors This will allow to Resolve the tracks Measure the energy depositions Identify the particles Measure the decay vertices Experiment size and granularity is determined by Required accuracy Particle multiplicity @ ATLAS/CMS O(1000) particles/b.c. This determines Number of detector elements Number of electronic channels Data size and throughput 17-June-2006 A. Cardini / INFN Cagliari 12

ATLAS 44 m length 22 m diameter 17-June-2006 A. Cardini / INFN Cagliari 13

CMS 17-June-2006 A. Cardini / INFN Cagliari 14

LHCb 17-June-2006 A. Cardini / INFN Cagliari 15

ALICE ACCORDE HMPID TOF TRD TPC PMD PHOS ITS Muon arm 17-June-2006 A. Cardini / INFN Cagliari 16

Triggering

General Trigger Requirements The role of the trigger is to make the online selection of particle collisions potentially containing interesting physics Need high efficiency for selecting processes of interest for physics analysis, for which: Efficiency should be precisely known Selection should not have biases that affect physics results Need large reduction of rate from unwanted high-rate processes (capabilities of DAQ and also offline computers): Instrumental background High-rate physics processes that are not relevant for analysis (min. bias) System must be affordable Limits complexity of algorithms that can be used Not easy to achieve all the above simultaneously! 17-June-2006 A. Cardini / INFN Cagliari 18

LHC Trigger Challenges N channels ~ O(10 7-10 8 ) and 20 interactions/25 ns Need huge number of connections Need information super-highway Information coming from different detector parts should correspond to the same interactions Need to synchronize detectors to (better than) 25 ns Note however that in some cases detector signals and/or time-offlight exceeds 25 ns Some detector will integrate information coming from more than 1 bunch crossing Can store data at 100 MB/s 100 Hz for ATLAS/CMS (1 MB/ev.), 1 khz for LHCb (100 kb/ev.) Need to reject most interactions What is discarded is lost forever! Need to careful monitor the selection 17-June-2006 A. Cardini / INFN Cagliari 19

Triggering Howto Look at (almost) all bunch crossings, select most interesting one, collect all detector information and store it for off-line analysis (for a reasonable amount of money) Since the detector data are not all promptly available and the selection function is rather complex, T() is evaluated by SUCCESSIVE APPROXIMATIONS called TRIGGER LEVELS (which should have possibly zero dead time) 17-June-2006 A. Cardini / INFN Cagliari 20

A multi-level trigger system provides: The multi-level trigger Rapid rejection of high-rate background without incurring in (much) dead-time: the fast first-level trigger (custom electronics) Needs high efficiency, but rejection power can be comparatively modest Short latency is essential since information from all (up to O(10 8 )) detector channels needs to be buffered (often on detector) pending trigger decision High overall rejection power to reduce output to mass storage to affordable rate: one or more High Trigger Levels: Progressive reduction in rate after each stage of selection allows use of more and more complex algorithms at affordable cost Final stages of selection, running on computer farms, can use comparatively very complex (and hence slow) algorithms to achieve the required overall rejection power Example: ATLAS 17-June-2006 A. Cardini / INFN Cagliari 21

First Level Selection First level (level 1) reduces event rate from 40 MHz to O(100) khz This step exist in all experiments Not enough, still to go down by factor 100-1000 in one or more extra step 17-June-2006 A. Cardini / INFN Cagliari 22

Successive Selection: 3 steps Additional processing in intermediate step reduces bandwidth requirements 17-June-2006 A. Cardini / INFN Cagliari 23

or 2 steps! This solutions reduces the number on building blocks and could rely on commercial components for what concerns calculations and network 17-June-2006 A. Cardini / INFN Cagliari 24

Triggering @ LEP vs. LHC e + e crossing rate 45 khz (4 bunches) p p crossing rate 40 MHz (L=10 33-10 34 cm -2 s -1 ) 45 khz 22µs Level 1 40 MHz 25 ns 6 µs Level 1 100 Hz Level 2 30 µs 10 Hz Readout 100 khz µs Level 2 8 ms 1 khz ms 8 Hz Level3 100 ms Level n LEP t L1 < inter bunch time no event overlapping LHC t L1» inter bunch time 22 overlapping events/bc 17-June-2006 A. Cardini / INFN Cagliari 25

Trigger/DAQ at LHC!!! 100 (~10 3 )! 17-June-2006 A. Cardini / INFN Cagliari 26

Trigger/DAQ: past, present and future 17-June-2006 A. Cardini / INFN Cagliari 27

Level-1 (L1) Trigger

Algorithms for Level-1 Trigger The Physics pp collision produce hadrons with p t ~ 1 GeV Interesting Physics has particles with large p t : W lν: M(W) ~ 80 GeV/c 2, p t (l) ~ 30 40 GeV H(120 GeV/c2) 2 photons, with p t (photon)~ 50 60 GeV Trigger Requirements Impose high thresholds on specific interaction products: easy for muons, electrons and jets, then need complex algorithms Typically: Single muon with p t > 20 GeV 10 khz dimuons with p t > 6 GeV 1 khz Single electron with p t > 30 GeV 10 20 khz dielectrons with p t > 20 GeV 5 khz Single jet with p t > 300 GeV 200 400 Hz ATLAS/CMS requirements 17-June-2006 A. Cardini / INFN Cagliari 29

P t cut in minimum bias events All tracks p t > 2 GeV Simulated H ZZ 4 µ event + 17 minimum-bias events 17-June-2006 A. Cardini / INFN Cagliari 30

Which Detectors at Level-1? 17-June-2006 A. Cardini / INFN Cagliari 31

Which Detectors at Level-1? (2) In Muon detector / calorimeter low occupancy and patter recognition is easy Simple reconstruction algorithms fast Small amount of data Can take regional decisions In inner detectors Complicated events! Complex analysis algorithms slow Huge amount of data Need to link to other detector for additional information 17-June-2006 A. Cardini / INFN Cagliari 32

Still is not easy It is not possible to generate a trigger in 25 ns detector 50 ns Example: CMS FE 100 ns Primitive Gen 100 ns ~3 µs pipeline delay O(100) deep 600 ns Local trigger Global trigger 600 ns 500 ns 300 ns 500 ns derandomizer Need a massive concurrent, pipelined processing to implement a dead-timeless L1 trigger 17-June-2006 A. Cardini / INFN Cagliari 33

Level-1 Processing 17-June-2006 A. Cardini / INFN Cagliari 34

Information Flow 17-June-2006 A. Cardini / INFN Cagliari 35

Information Flow (2) 17-June-2006 A. Cardini / INFN Cagliari 36

L1 trigger architecture in the LHC Experiments

Region of Interest (RoI) The L1 selection can based on local signatures called Region of Interest (RoI) Based on coarse granularity, no inner tracker info Local analysis allows an important further rejection The geographical location of interesting signatures are identified by L1 This allow access only to local data for each relevant detector 17-June-2006 A. Cardini / INFN Cagliari 38

Region of Interest (2) Region-of-Interest Not RoI RoI data ~1% of L1 output Complex mechanism for data access Many control messages Smaller readout network thanks to an intermediate trigger level which only processes local (in η,φ space) information Very high throughput Very large readout network Simpler system Flexible MORE COMPLEX SYSTEM MORE DEMANDING ON TECHNOLOGY 17-June-2006 A. Cardini / INFN Cagliari 39

The ATLAS Architecture RoI based L1 trigger 17-June-2006 A. Cardini / INFN Cagliari 40

ATLAS Muon O(10 6 ) RPC (Barrel)/TGC (Endcap) trigger channels Barrel and Endcap LOCAL Trigger Processor to estimate p t Muon Central Trigger Processor Calorimeter LAr (ECAL) and Tile/LAr (HCAL) Analog preprocessor (analog pipeline!) to estimate E t LOCAL Jet/Energy Sum and Cluster Processor stage The local triggers are sent to the Central Trigger Processor (CTP), which makes the FINAL decision The L1 trigger decision is sent to the Front End via the TTC (Timing- Trigger-Control) system For every accepted event the L1 trigger sends readout information to the Region-of-Interest (RoI) Builder which assembles the list of RoIs to be used by L2 Note: all digital design except input stage of calorimeter trigger pre-processor 17-June-2006 A. Cardini / INFN Cagliari 41

The ATLAS Muon Trigger The L1 Muon Trigger requires coincidence of RPC/TGC hits within a road, which is related to the p t cut applied A high and a low p t algorithms are applied Multiple cuts can be used at the same time thanks to programmable coincidences Fast and high redundancy system 40 khz expected at L=10 34 /cm 2 /s - Wide pt threshold range - Safe bunch crossing identification (fast detectors) - Strong rejection of fake muons (noise, physics background) 17-June-2006 A. Cardini / INFN Cagliari 42

The CMS Architecture Similar to ATLAS, but no RoI The Global Calorimeter trigger selects the best 4 e,γ, τ and jets and calculate E t and E t miss The Global Muon Trigger receives 4 muon candidates of maximum p t and select the best quality ones The Global Trigger applies the thresholds and performs the trigger algorithms. Up to 128 algorithms can run in parallel: arbitrary combinations of trigger objects passing thresholds and topological correlations 17-June-2006 A. Cardini / INFN Cagliari 43

The CMS Calorimeter Trigger Divide Calorimeter in towers Match towers between ECAL and HCAL Isolation and deposit shape criteria to identify electrons, photons, jets 17-June-2006 A. Cardini / INFN Cagliari 44

L1 Trigger Rates ATLAS CMS Selection Thr. (GeV) Rate (khz) Thr. (GeV) Rate (khz) Single e/γ 17 11 22 5.7 Double e/γ 12 1.4 12 2.7 τ 20 16.3 85 3.2 Jet 100 6.0 140 1.2 E t miss 51 1.0 275 0.01 Total E T 336 1.0 - - Single µ 6 (l) / 20 (h) 23.2/3.8 10 8.7 Double µ - - 3.3 1.6 L1 trigger rates at L = 10 33 when applying 90% efficiency thresholds 17-June-2006 A. Cardini / INFN Cagliari 45

CMS vs. ATLAS 17-June-2006 A. Cardini / INFN Cagliari 46

The LHCb Architecture 40 MHz crossing rate, but only 30 MHz real crossings Luminosity: 2 10 32 cm -2 s -1 (50 times lower than ATLAS and CMS) Minimum bias rate: 10 MHz bb rate is ~ 100 khz (15 khz in detector acceptance) cc rate is ~ 600 khz First level trigger (here called L0) selects high pt particles (muon, e,gamma, ) and events with only one interaction by means of the pile-up veto Calorimeters + Muon system 10 MHz L0: hight p T tracks+ not too busy events Fully synchr. (40 MHz), 4 µs latency On custom boards 1 MHz 17-June-2006 A. Cardini / INFN Cagliari 47

LHCb Muon Trigger The LHCb muon system: 5 stations Variable segmentation Projective geometry Trigger strategy: Straight line search in M2-M5 in every quadrant Look for compatible hits in M1 Momentum measurement ( p/p~20% for b-decays) µ >90% π/k decay Nominal threshold Sent to L0 decision unit: 2 highest p T candidates per quadrant Typical Performance: ~88% efficiency on B->J/ψ(µµ)X Algorithm latency ~1 µs 17-June-2006 A. Cardini / INFN Cagliari 48

LHCb Pile-up veto LHCb needs to identify secondary (decay) vertices This is performed @ a higher trigger level Works well if there is only and only one interactions per bunch crossing A veto against double interactions is implemented with 2 silicon detectors planes Hits are fitted by means of 4 large FPGAs and results are sent to L0 Decision Unit Typical Performance for identifying double interactions 60% efficiency 95% purity Latency ~ 1 µs 17-June-2006 A. Cardini / INFN Cagliari 49

The ALICE Architecture Heavy ions runs L=10 27 cm -2 s -1 Interaction rate < 10 khz Very high multiplicity & Huge event size (~ 50 MB) Modest requirements on lower level triggers pp (or pa) runs Interaction rate up to 200 khz, limited by TPC pileup Small event size (~ 2 MB) Strong requirements on lower level triggers To accommodate all the different running conditions the first level trigger is split in 3 distinct levels (L1, L1 and L2) 17-June-2006 A. Cardini / INFN Cagliari 50

ALICE First Level Trigger(s) Some of the Alice FEE is not pipelined but await a trigger before processing Some detector need very early strobe (e.g. TOF), so a first early decision is taken in 1.2 µs (L0) For detectors which require longer time L1 is used, which arrived 6.5 µs after interaction L2 comes after 88 µs, at the end of the drift time in TPC. Purpose of L2 is to wait for the end of the past-future pileup protection, in order to make sure that there is only one event in TPC 17-June-2006 A. Cardini / INFN Cagliari 51

ALICE: Optimizing Trigger Efficiency Requirements Some subdetectors need a long time to be read out after a L2 (silicon drift detector: 260 µs) However some interesting physics events need only a subset of detectors to be readout Concept of Trigger Clusters Group of sub-detectors Even if some sub-detectors are busy, triggers for not busy clusters can be accepted, increasing acquired statistics Rare Events When readout buffer almost full only accepts the so called rare event 17-June-2006 A. Cardini / INFN Cagliari 52

L1 Trigger Implementation

L1 implementation issues At L1 every operation must be extremely fast L1 logic is usually built using: ASIC (Application Specific Integrated Circuits) Can be produced radiation tolerant (to be installed on detectors) Can contain both analog and digital part Full-custom or Standard- Libraries (or mixed) design Long development cycle Extensive simulation necessary Production cycle expensive, but cost/asic can be extremely low FPGA (Field Programmable Gate Arrays), Extremely versatile nowadays Might contain memory, processors, high speed serial links Complex design possible: processors, PCI interfaces, WEB-servers Easy to implement the requested design Reprogrammable (even insitu!) (very) Expensive units 17-June-2006 A. Cardini / INFN Cagliari 54

LHCb Muon Off Detector Electronics SYNC ASIC CMOS IBM 0.25 um 8x4 bit TDCs 12 bit counter for Bxid generation L0 buffer and derandomizer Interface to L0 trigger logic Programmable via i2c Radiation resistant Triple-voted auto-correcting registers for radiation immunity many other things 17-June-2006 A. Cardini / INFN Cagliari 55

L1 implementation issues (2) Communication Technologies Very high-speed serial links (copper or fiber) LVDS, G-link, Vitesse, Backplanes Very large number of connection, data multiplexing 17-June-2006 A. Cardini / INFN Cagliari 56

CMS Regional Calorimeter Trigger Receiver card (there are O(20) cards/crate and O(20) crates in the system): ~ 400 Receives 64 trigger primitives, 32 from ECAL and 32 from HCAL Forms two 4x4 towers for Jet Trigger and 16 Et towers for electron isolation card Overall system input bandwidth: ~4000 Gb/s 17-June-2006 A. Cardini / INFN Cagliari 57

Today s Conclusions LHC: a very challenging environment Very demanding requests on trigger system, in particular on first level trigger Pipelines (analog or digital) everywhere for a (almost) dead timeless L1 trigger Different philosophies of the experiment: ATLAS vs. CMS want to study the same physics but adopt different approaches: RoI vs. not RoI Technology is progressing very rapidly and its performance appears adequate (but still systems are very complicated ) 17-June-2006 A. Cardini / INFN Cagliari 58