LHCb Trigger & DAQ Design technology and performance Mika Vesterinen ECFA High Luminosity LHC Experiments Workshop 8/10/2016
2 Introduction The LHCb upgrade will allow 5x higher luminosity and with greatly increased trigger efficiency. Underpinned by triggerless readout and the full software trigger. And by real time methods that are being pioneered in Run-II.
3 Heavy flavour signatures Beauty M 5 GeV ~10 mm Charm M 2 GeV ~4 mm
Heavy flavour signatures Beauty M 5 GeV ~10 mm Charm M 2 GeV ~4 mm Hardware trigger thresholds * Muon pt > 2 GeV Hadron ET > 4 GeV...,... *Very approximate Run-II values. They are still evolving since the LHC hasn t yet provided the complete number of colliding bunches that we can expect during Run-II. Mika Vesterinen 4
5 Heavy flavour signatures Beauty M 5 GeV ~10 mm Charm M 2 GeV ~4 mm At Run-II 45 khz 1 MHz luminosity in LHCb acceptance 4 x 10 32 cm -2 s -1 in LHCb acceptance Inclusive High Level Triggering ~250 exclusive selections
6 Run-I/II and lumi levelling Luminosity levelled at 4 x 10 32 cm -2 s -1 Lint [cm -2 s -1 ] L Run-I 4 x 10 32 3 fb -1 Run-II 4 x 10 32 5 fb -1 LHCb
7 Hardware trigger bottleneck CERN-LHCC-2011-001 Muonic Hadronic design Run-I/II
8 Hardware trigger bottleneck CERN-LHCC-2011-001 Muonic Triggerless readout in the upgrade allows 2x higher efficiency for hadronic b decays at 5x higher luminosity Hadronic Lint design [cm -2 s -1 ] Run-I/II L Run-I-II 4 x 10 32 8 fb -1 Run-III-IV 2 x 10 32 50 fb -1
9 Run-I Trigger 40 MHz bunch crossing rate L0 Hardware Trigger : 1 MHz readout, high ET/PT signatures 450 khz h ± 400 khz µ/µµ 150 khz e/γ Software High Level Trigger Introduce tracking/pid information, find displaced tracks/vertices Offline reconstruction tuned to trigger time constraints Mix of exclusive and inclusive selections 5 khz (0.3 GB/s) to storage 2 khz Inclusive beauty 2 khz Charm 1 khz Muons Alignment and calibration
10 Run-I Trigger 40 MHz bunch crossing rate L0 Hardware Trigger : 1 MHz readout, high ET/PT signatures 450 khz h ± 400 khz µ/µµ 150 khz e/γ Software High Level Trigger Introduce tracking/pid information, find displaced tracks/vertices Offline reconstruction tuned to trigger time constraints Mix of exclusive and inclusive selections 5 khz (0.3 GB/s) to storage 2 khz Inclusive beauty 2 khz Charm 1 khz Muons Alignment and calibration
11 Run-I to Run-II 40 MHz bunch crossing rate L0 Hardware Trigger : 1 MHz readout, high ET/PT signatures 450 khz h ± 400 khz µ/µµ 150 khz e/γ Software High Level Trigger Introduce tracking/pid information, find displaced tracks/vertices Offline reconstruction tuned to trigger time constraints Mix of exclusive and inclusive selections 5 khz (0.3 GB/s) to storage 2 khz Inclusive beauty 2 khz Charm 1 khz Muons Alignment and calibration
12 Run-I to Run-II 40 MHz bunch crossing rate L0 Hardware Trigger : 1 MHz readout, high ET/PT signatures Real time 450 khz h ± 400 khz µ/µµ 150 khz e/γ Sub detectors fully aligned and calibrated in real time Software High Level Trigger HLT performs offline quality Introduce tracking/pid information, find displaced tracks/vertices Offline reconstruction tuned to trigger time constraints Mix of exclusive and inclusive selections reconstruction, incl. RICH PID. 5 khz (0.3 GB/s) to storage 2 khz Inclusive beauty 2 khz Charm 1 khz Muons Alignment and calibration
13 Run-I to Run-II 40 MHz bunch crossing rate L0 Hardware Trigger : 1 MHz readout, high ET/PT signatures Real time 450 khz h ± 400 khz µ/µµ 150 khz e/γ Sub detectors fully aligned and calibrated in real time Software High Level Trigger HLT performs offline quality Introduce tracking/pid information, find displaced tracks/vertices Offline reconstruction tuned to trigger time constraints Mix of exclusive and inclusive selections reconstruction, incl. RICH PID. 5 khz (0.3 GB/s) to storage Real time physics analysis 2 khz Inclusive beauty 2 khz Charm 1 khz Muons with the Turbo stream Comp. Phys. Com. 208, 35-42, (2016) Alignment and calibration
14 Run-I to Run-II 40 MHz bunch crossing rate L0 Hardware Trigger : 1 MHz readout, high ET/PT signatures 450 khz h ± 400 khz µ/µµ 150 khz e/γ Software High Level Trigger Introduce tracking/pid information, find displaced tracks/vertices Offline reconstruction tuned to trigger time constraints Mix of exclusive and inclusive selections 5 khz (0.3 GB/s) to storage 2 khz Inclusive beauty 2 khz Charm 1 khz Muons The fully deferred split HLT allows us to Alignment and calibration employ the HLT farm throughout the year
15 Run-II to Upgrade
16 Run-II to Upgrade
Triggerless readout Data network throughput [ Tbit / sec ] Figure courtesy of Niko Neufeld, based on presentations at DAQ@LHC 2016 Mika Vesterinen 17
18 LHCb upgrade DAQ Cavern Surface data center
19 Full software trigger
Track reconstruction Scintillating Fibre Tracker VELO Upstream Tracker LHCb Tracker Upgrade TDR, LHCB-TDR-015 Mika Vesterinen 20
Track reconstruction Scintillating Fibre Tracker VELO Upstream Tracker LHCb Tracker Upgrade TDR, LHCB-TDR-015 Mika Vesterinen 21
Physics performance The 30 MHz tracking sequence only compromises a few % in efficiency compared to the ideal version. LHCb Trigger and Online Upgrade TDR LHCB-TDR-016 Mika Vesterinen 22
Speed Reconstruction time for nominal LHCb upgrade conditions measured on a node 1 from the current LHCb EFF Algorithm Time [ms] VELO tracking 2.0 VELO-UT tracking 1.3 Forward tracking 1.9 Primary vertexing 0.4 Total 5.4 LHCB-TDR-016 Within estimated budget 2 of 13 ms, allowing for subsequent selection and reconstruction 1. Dual socket x86 server with Intel x5650. 12 physical cores + 12 hyper threads, and 24 GB RAM. 2. Based on CPU growth projections Mika Vesterinen 23
Speed Reconstruction time for nominal LHCb upgrade conditions measured on a node 1 from the current LHCb EFF Algorithm Time [ms] VELO tracking 2.0 VELO-UT tracking 1.3 Forward tracking 1.9 Primary vertexing 0.4 Total 5.4 LHCB-TDR-016 Ongoing effort to (i) adapt LHCb software to better exploit modern CPU architectures & (ii) evaluate use of co-processor technologies. Mika Vesterinen 24
25 How much signal do we need to record?
26 Anatomy of upgrade events LHCb-PUB-2014-027 Beauty Charm τ > 0.2 ps Hyperons
Anatomy of upgrade events The goal of the trigger is no longer to distinguish signal from background, but rather to distinguish different signals. Mika Vesterinen 27
Selectivity example 60M Right sign decays Candidates / (0.1 MeV/c 2 ) 8 7 6 5 4 3 2 1 6 10 LHCb (a) D +! D 0 + 3 D +! D 0 + Data Fit Background 0 2.005 2.01 2.015 2.02 M (D 0 π s + ) [GeV/c 2 ] Candidates / (0.1 MeV/c 2 ) 10 60 LHCb 50 (b) 40 30 20 10 250k Wrong sign decays Data Fit Background 0 2.005 2.01 2.015 2.02 M (D 0 π s + ) [GeV/c 2 ] Figures correspond to full Run-II dataset PRL 111, 251801, 2013 Mika Vesterinen 28
29 HLT output rate Beauty Charm Muons Other 2011 2012 Run-II 0 2 4 6 8 10 12 14 16 Approximate HLT output rate in khz
30 HLT output rate Beauty Charm Muons Other 2011 2012 Run-II un-iii* Run-III 0 50 100 150 200 Approximate HLT output rate in khz
31 HLT output rate Beauty Charm Muons Other 2011 2012 Run-II un-iii* x5 luminosity Run-III 0 50 100 150 200 Approximate HLT output rate in khz
32 HLT output rate Beauty Charm Muons Other 2011 2012 Run-II un-iii* Run-III x5 luminosity No hardware trigger bottleneck 0 50 100 150 200 Approximate HLT output rate in khz
HLT output rate Beauty Charm Muons Other 2011 2012 Run-II un-iii* Run-III x5 luminosity No hardware trigger bottleneck 0 50 100 150 200 Approximate HLT output rate in khz E.g., Triggerless readout provides great potential for kaon physics 1,2,3 1. LHCb-PUB-2016-017 2. LHCb-PUB-2016-016 3. KAON 2016 indigo link Mika Vesterinen 33
34 HLT output rate Beauty Charm Muons Other 2011 2012 Run-II un-iii* Run-III 0 50 100 150 200 Approximate HLT output rate in khz Real time physics analysis methods needed to manage these data
35 HLT output rate Beauty Charm Muons Other 2011 2012 khz isn t the appropriate unit... Run-II un-iii* Run-III 0 50 100 150 200 Approximate HLT output rate in khz Real time physics analysis methods needed to manage these data
36 The Turbo Stream Comp. Phys. Com. 208, 35-42, (2016) Run-II HLT already performs full offline reconstruction Charm analysis is mostly based on the signal candidate We can do real time physics analysis which discards the majority of the event. Stream Full Turbo Event size 70 kb 5 kb
37 The Turbo Stream JHEP10 (2015) 172 Comp. Phys. Com. 208, 35-42, (2016) 2015: 13 TeV J/ψ μμ cross section measurement presented within 1 week of recording the data! 2016: More than 200 Turbo HLT2 lines in operation.
Evolution of Turbo Example: What if, unlike the D-D mixing example, you want all of the favoured charm decays, and not just the candidate? E.g. D*X spectroscopy Mika Vesterinen 38
39 Evolution of Turbo Turbo in 2015 Appropriate for analyses with exclusively triggered candidate
40 Evolution of Turbo Turbo in 2015 Turbo in 2016 Appropriate for analyses with exclusively triggered candidate Applicable to most physics analyses
Evolution of Turbo Turbo in 2015 Turbo in 2016 Appropriate for analyses with exclusively triggered candidate Applicable to most physics analyses We approach the flexibility to perform any type of physics analysis in real time with the Turbo stream! Mika Vesterinen 41
42 Beyond 2 x 10 33 cm -2 s -1? Proposed phase II LHCb upgrade in LS4 Record 300 fb -1 at 2 x 10 34 cm -2 s -1 See Chris Parkes talk in this workshop (link).
43 Trigger considerations We wouldn t go back to a triggered readout. - Increased occupancy and #channels should easily be absorbed by a larger #links and technology evolution. The main constraint is likely to be offline computing. The real time approach will be key.
Summary The LHCb upgrade requires a revolutionary advance in trigger and DAQ which is needed to better exploit the LHC potential for HF physics. Good progress towards the triggerless readout and the full software trigger Real time methods are being proven in Run-II. Bold but realistic plans are taking shape for future upgrades. Mika Vesterinen 44
45 Backup slides
46 Anatomy of upgrade events
Output bandwidth LHCB-TDR-016 Consider 2, 5 and 10 GB/sec scenarios assuming 120 kb raw event size Mika Vesterinen 47
LHCB-TDR-016 Mika Vesterinen 48
49 HLT output rate More than 1 billion charm signal D 0 D+ Ds Approximate HLT output rate in khz D * Λc