On the Off-chip Memory Latency of Real-Time Systems: Is DDR DRAM Really the Best Option? Mohamed Hassan

Transcription

1 On the Off-chip Memory Latency of eal-time Systems: Is DD DAM eally the Best Option? Mohamed Hassan

2 Motivation 2 PEDICTABILITY DAMs 3 LDAM 4 esults 5 Outline

3 Historically, SAMs have been the option for real-time safety-critical embedded systems With the increase in data demand the cost became unaffordable SAMs MOTIVATION

4 Predator [Akesson et al., IBM s CODES+ISSS] Acorn PET [eineke et al., CODES+ISSS] MCMC[Ecco et al., TCSA24] DCmc [Jalle et al., TSS24] [Kim et al., TAS24] OC[Krishnapillai et al., ECTS24] Smart Phones TMem[Li et al., ECTS24] eorder[ecco et al., ECTS26] [Guo and Pellizzoni, TAS27] /7 [Burchard et al, DATE] [Heithecker and Ernst, DAC] AMC [Paolieri et al., ESL] COP [Goossens et al.] MultiChannel [Gomony et al.] OP [Zheng et al.] PMC [Hassan et al., TAS25] [Kim et al., TAS25] MEDUSA[Valsan and Yun, CPSNA25] [Yun et al., ECTS25] eoorder[ecco and Ernst, TSS25] [Hassan and Pellizzoni, EMSOFT28] Work in Off-chip Memory MOTIVATION

5 Predator [Akesson et al., IBM s CODES+ISSS] Acorn PET [eineke et al., CODES+ISSS] MCMC[Ecco et al., TCSA24] DCmc [Jalle et al., TSS24] [Kim et al., TAS24] OC[Krishnapillai et al., ECTS24] Smart Phones TMem[Li et al., ECTS24] eorder[ecco et al., ECTS26] [Guo and Pellizzoni, TAS27] All in Double Data ate (DD) DAMs /7 28 [Burchard et al, DATE] [Heithecker and Ernst, DAC] AMC [Paolieri et al., ESL] COP [Goossens et al.] MultiChannel [Gomony et al.] OP [Zheng et al.] PMC [Hassan et al., TAS25] [Kim et al., TAS25] MEDUSA[Valsan and Yun, CPSNA25] [Yun et al., ECTS25] eoorder[ecco and Ernst, TSS25] [Hassan and Pellizzoni, EMSOFT28] Work in Off-chip Memory MOTIVATION

6 DD DAM is the commodity off-chip memory, Why? Low cost Large capacity High BW What is the most important requirement for real-time/safety-critical systems? 2 Yes, Predictability How is DDx for predictability? DDx andom Access Memories are not andom at all!! Access latency varies notably based on many factors access patterns transaction type (read or write) DAM state from previous accesses A Context about DDs MOTIVATION

7 Multiplexed address mode: The address bits are split into two segments provided to the device in two stages:. ow address row decoder 2. Column address column decoder 3 Low cost (less pin count) High latency Huge variability Background DAM

8 A request in general can consist of one, two, or three commands: ACTIVATE (A) command: Bring data row from cells into sense amplifiers 3 Background DAM

9 DAM Consists of multiple banks The memory controller (MC) manages accesses to DAM A request in general consists of: ACTIVATE (A) command: Bring data row from cells into sense amplifiers ead/write (/W) commands: To read/write from specific columns in the sense amplifiers 3 Background DAM

10 DAM Consists of multiple banks The memory controller (MC) manages accesses to DAM A request in general consists of: ACTIVATE (A) command: Bring data row from cells into sense amplifiers ead/write (/W) commands: To read/write from specific columns in the sense amplifiers PECHAGE (P) command: to write back a previous row in the sense amplifiers before bringing the new one 3 Background DAM

11 DAM Consists of multiple banks The memory controller (MC) manages accesses to DAM A request in general consists of: ACTIVATE (A) command: Bring data row from cells into sense amplifiers ead/write (/W) commands: To read/write from specific columns in the sense amplifiers PECHAGE (P) command: to write back a previous row in the sense amplifiers before bringing the new one ow Conflict: P+ A + /W ow Idle/Close: A + /W ow Hit: /W 3 Background DAM

12 DAM Consists of multiple banks The memory controller (MC) manages accesses to DAM A request in general consists of: ACTIVATE command /W commands PECHAGE command All commands have associated timing constraints that have to be satisfied by the controller (2+ timing constraints) A A tcd W twl twt tl Background DAM

13 4 Predator [Akesson et al., IBM s CODES+ISSS] Acorn PET [eineke et al., CODES+ISSS] MCMC[Ecco et al., TCSA24] DCmc [Jalle et al., TSS24] [Kim et al., TAS24] OC[Krishnapillai et al., ECTS24] Smart Phones TMem[Li et al., ECTS24] eorder[ecco et al., ECTS26] [Guo and Pellizzoni, TAS27]. It can not address the variability in the access latency of the DDx chips Still suffers 27 from high 2 WCLs due 24 to the complex 26/7 interactions between DDx commands [Burchard et al, DATE] [Heithecker and Ernst, DAC] AMC [Paolieri et al., ESL] COP [Goossens et al.] MultiChannel [Gomony et al.] OP [Zheng et al.] PMC [Hassan et al., TAS25] [Kim et al., TAS25] MEDUSA[Valsan and Yun, CPSNA25] [Yun et al., ECTS25] eoorder[ecco and Ernst, TSS25] [Hassan and Pellizzoni, EMSOFT28] Work in Off-chip Memory MOTIVATION

14 DD DAM is the commodity off-chip memory, Why? Low cost high capacity High BW What is the most important requirement for real-time/safety-critical systems? 5 Yes, Predictability How is DDx for predictability? DDx andom Access Memories are not andom at all!! access latency varies notably based on many factors access patterns ሽ transaction type (read or write) DAM state from previous accesses Comprehensively study these factors Assessing DDs for Predictability MOTIVATION

15 How is DDx for predictability? 5 Predictability has different definitions in the real-time literature One important measure is the relative difference between best- and worst-case execution times (or latencies in case of memories) [Wilhelm et al, TECS8] We define Variability Window (VW) to quantitatively measure the DAM predictability VW = WCL BCL WCL Assessing DDs for Predictability PEDICTABILITY

16 5 2 Latency [ns] Targets an open row (only command) (a) is best-case (e) arrives after a write to same rank (a) 4 tl 4 2 a b c d e f g h i j k l m n o (e) W twl twt tl Assessing DDs for Predictability PEDICTABILITY

17 5 2 Latency [ns] Targets an open row (only command) (a) is best-case (e) arrives after a write to same rank (a) 4 tl 4 2 a b c d e f g h i j k l m n o (e) W twl twt tl Assessing DDs for Predictability PEDICTABILITY

18 5 2 Latency [ns] Targets a close row (A + command) (f) is best-case (j) arrives after a closed write to same rank (f) A tl a b c d e f g h i j k l m n o (j) A A tcd W twl twt tl Assessing DDs for Predictability PEDICTABILITY

19 5 2 Latency [ns] Targets a close row (A + command) (f) is best-case (j) arrives after a closed write to same rank (f) A tl a b c d e f g h i j k l m n o (j) A A tcd W twl twt tl Assessing DDs for Predictability PEDICTABILITY

20 5 2 Latency [ns] Targets a conflict row (P + A + command) (k) is best-case (o) arrives after a conflict write to same rank (k) P A tl 2 a b c d e f g h i j k l m n o (o) P A tcd W twl tl P A tl Assessing DDs for Predictability PEDICTABILITY

21 5 2 Latency [ns] Targets a conflict row (P + A + command) (k) is best-case (o) arrives after a conflict write to same rank (k) P A tl 2 a b c d e f g h i j k l m n o (o) P A tcd W twl tl P A tl Assessing DDs for Predictability PEDICTABILITY

22 5 2 Latency [ns] cases for a read request Another 5 for a write request VW= 62% WCL=8ns a b c d e f g h i j k l m n o Assessing DDs for Predictability PEDICTABILITY

23 VW% 7 We calculate the VW for 8 of the state-of-the-art DDx DAM Controllers Assessing DD Controllers for Predictability PEDICTABILITY

24 VW% 7 We calculate the VW for 8 of the state-of-the-art DDx DAM Controllers 6 out of the 8 exceed 8% Assessing DD Controllers for Predictability PEDICTABILITY

25 VW% 7 We calculate the VW for 8 of the state-of-the-art DDx DAM Controllers 6 out of the 8 exceed 8% Achieve less variability at the expense of. complexity: Bank partitioning ank switching 2. Conservatism: e.g. using close-page (MCMC) Assessing DD Controllers for predictability PEDICTABILITY

26 VW% We calculate the VW for 8 of the state-of-the-art DDx DAM Controllers Even with the pessimism and complexity, 26% is 6 out of the 8 exceed 8% Achieve less variability at the expense of. complexity: Bank partitioning ank switching 2. Conservatism: e.g. using close-page (MCMC) still a significant variability for safety-critical systems 7 Assessing DD Controllers for predictability PEDICTABILITY

27 VW% We calculate the VW for 8 of the state-of-the-art DDx DAM Controllers Even with the pessimism and complexity, 26% is 6 out of the 8 exceed 8% 2 Achieve less variability at the expense of. complexity: Bank partitioning still a significant variability for safety-critical systems ank switching 2. Conservatism: e.g. using close-page (MCMC) Exploring other types of memories that address 8 6 these limitations is unavoidable towards providing 4 2 more predictable memory performance with less 7 variability and tighter bounds Assessing DD Controllers for predictability PEDICTABILITY

28 + 8 LDAM2 was introduced by Infinion and Micron LDAM Introduced by Infinion LDAM3 was introduced by Micron LDAM: An Alternative LDAM

29 Multiplexed Address Mode Non-Multiplexed Address Mode 9 2 CAS Col Address AS ow Address DD DAM All Address Bits LDAM ow Conflict: P+ A + /W ow Idle/Close: A + /W ow Hit: /W /W Why LDAM? LDAM

30 3 7 tl W tl W twl W twl tl C tc tl W twl C tc W twl Assessing LDAM for Predictability LDAM

31 Latency [ns] tl 4 8 W tl A total of 8 cases Variability window is 46.2% (3.4x reduction) WCL=28.5ns (3.79x reduction) W W twl twl tl C tc tl 46.2% W twl C tc W twl 5 a b c d e f g h Assessing LDAM for Predictability LDAM

32 Processor Decoder Arbiter LDAM type address Command Generation Address Mapping perpe Buffers Timing Checker cmd ba addr LDC to predictably manage accesses to LDAM ound obin Support both bank sharing and bank partitioning Simple timing checker good for analyzability, V&V, Certification LDC: A Predictable Controller for LDAM LDAM

33 Bank Sharing Scheme: 2 tc WCL share = N tc +tcl W W tl : Processor : Bank N is number of processing elements Bounding Memory Latency LDAM

34 Bank Partitioning Scheme: 2 W-to- Delay WCL part = N 2 + N 2 + tcl twl tl + BL 2 tl twl + BL 2 W W tl 26 : Processor : Bank N is number of processing elements -to-w Delay Bounding Memory Latency LDAM

35 3 PEs DAM LDAM DDx 4 Processors in-order pipeline a private 6KB L a shared MB L2 cache Either LDAM or DDx LDAM3-6 LDC manages accesses to LDAM DD3-6 AMC, PMC, TMem, DCmc, OP, MCMC, OC, or eorder manages access to DD3 Bank We experiment with both bank partitioning and bank sharing among PEs for LDC Management Benchmarks EEMBC Automotive Evaluation Setup ESULTS

36 WCL (ns) experimental analytical. DDx MC has 2.5x to 6.46x 4 worse analytical WCL than LDC Very similar numbers for exp. 2. elatively low WCL of MCMC, OC, eorder is due to 4 ranks! 3. For LDC: bank partitioning provide tighter WCL than sharing at the expense of flexibility 4. Gap between exp. vs analytical WCL is much higher for DD again due to inherent variability Worst-Case Latency ESULTS

37 VW experimental analytical. Already discussed analytical VW 2. Exp. VW for DDx MCS: >4% for 4 MCs, 3%-4% for 3 MCs and ~2% for MC. 3. for LDC: 76.9% for partitioned banks 84.6% for shared banks Variability Window ESULTS

38 6. The WCL latency gap between LDC and majority of DDx MCs increases drastically Scalability: # Processors ESULTS

39 cat cat2 cat3. The WCL latency gap between LDC and majority of DDx MCs increases drastically 2. DDx MCs can be categorized into three categories 6 Bank sharing Bank part Bank part + multi-rank Scalability: # Processors ESULTS

40 cat cat2 cat3 Bank sharing Bank part Bank part + multi-rank. The WCL latency gap between LDC and majority of DDx MCs increases drastically 2. DDx MCs can be categorized into three categories 3. LDC s WCL is less than all categories for all #PEs for both part and sharing without complex arbitration/ reorderings (better analyzability and composability) 6 Scalability: # Processors ESULTS

41 7. How mature is LDAM? Has been there since 999 Long-term Supported by Micron The main off-chip memory in networking and other low-latency needs Discussion ESULTS

42 7 2. Can I buy LDAM off-the-shelf? Definitely They are sold as discrete components (which is the norm for embedded systems anyway) There are also specialized DD-compatible sockets for LDAM Discussion ESULTS

43 7 3. Are there any platforms/boards/test-beds? Zynq UltraScale/+ MPSoC ATCA-935-NSP Processing blade Arria SoC Discussion ESULTS

44 7 4. If it is that good, why it did not take off then? Take off = replaces DD? It is not meant to be! It is a specialized type of memory for low-latency guarantees Commodity general-purpose market is looking for high BW, capacity, cost We should ask ourselves, what are we looking for? An analogy: FPGAs have been there for long-time, why are they taking off now? they satisfy a new need in the AI market Will they replace CPUs then? It is not an either-or decision Discussion ESULTS

45 7 5. Does that mean we no longer need DAMs in real-time systems? No Use-case dependent A heterogenous memory system? Mixed Criticality with different requirements? Discussion ESULTS

46 7 6. As a scheduling researcher, why should I care? Taking the shared resources interference into account is unavoidable to provide more accurate numbers DD DAM is very complex to account for its details (rd vs wr, conf vs hit,..etc) explodes the analysis LDAM alleviates this complexity..which can make the analysis more feasible Discussion ESULTS

47 7 7. Limitations/trade-off DDx LDAM SAM COST DDx LDAM SAM Latency Discussion ESULTS

48 VW% Latency [ns] Latency [ns] Processor Decoder Arbiter LDAM VW= 62% a c e g i k m o less timing variability 6.4 less WCL type address Command Generation Address Mapping 46.2% perpe Buffers Timing Checker cmd addr ba Main Lessons:. DD DAM is not designed for predictability, LDAM is. 2. Looking for solns that address our needs instead of starting from the mainstream soln? 3. Not either-or: A heterogenous memory to address conflicting needs of MCS 5 a b c d e f g h

49 VW% Latency [ns] Latency [ns] Processor Decoder Arbiter LDAM VW= 62% a c e g i k m o less timing variability 6.4 less WCL type address Command Generation Address Mapping 46.2% perpe Buffers Timing Checker cmd addr ba Main Lessons:. DD DAM is not designed for predictability, LDAM is. 2. Looking for solns that address our needs instead of starting from the mainstream soln? 3. Not either-or: A heterogenous memory to address conflicting needs of MCS 5 a b c d e f g h