Exploring the Software Stack for Underdesigned Computing Machines Rajesh Gupta UC San Diego.

Transcription

1 Exploring the Software Stack for Underdesigned Computing Machines Rajesh Gupta UC San Diego. 1

2 Exploring the Software Stack for Underdesigned Computing Machines 1

3 Exploring the Software Stack for Underdesigned Computing Machines

4 The Hardware-Software Boundary Application Application Operating System Hardware Abstraction Layer (HAL) 3

5 The Hardware-Software Boundary Application Application Operating System Hardware Abstraction Layer (HAL) 4

6 The Hardware-Software Boundary Application Application Operating System Hardware Abstraction Layer (HAL) 5

7 Performance Manufacturing Variability Meets Moore s Law: From Chiseled Transistors to Molecular Assemblies nominal scaling 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation 6

8 Performance Manufacturing Variability Meets Moore s Law: From Chiseled Transistors to Molecular Assemblies nominal scaling 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation 6

9 Manufacturing Variability Meets Moore s Law: From Chiseled Transistors to Molecular Assemblies nominal scaling Performance 249,403,263 Si atoms: 68,743 donors & 13,042 acceptors 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation 6

10 Manufacturing Variability Meets Moore s Law: From Chiseled Transistors to Molecular Assemblies nominal scaling Performance 249,403,263 Si atoms: 68,743 donors & 13,042 acceptors 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation 6

11 Manufacturing Variability Meets Moore s Law: From Chiseled Transistors to Molecular Assemblies nominal scaling Performance 249,403,263 Si atoms: 68,743 donors & 13,042 acceptors design for worst case: overdesigned scaling 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation 6

12 Manufacturing Variability Meets Moore s Law: From Chiseled Transistors to Molecular Assemblies Performance 249,403,263 Si atoms: 68,743 donors & 13,042 acceptors nominal scaling design for nominal, let software handle variation design for worst case: overdesigned scaling 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation 6

13 Active Power Variability Across Instances Cortex M3 Active Room Temperature Current (ma) P1 UCLA P2 P3 P4 P5 Processor Instance P6 P7 P8 P9 P10 Atmel SAM3U4E Cortex M3 Active Mode, 4MHz Internal Oscillator Room Temperature 7

14 Active Power Variability Across Instances Cortex M3 Active Room Temperature ~ 5% Variation Current (ma) UCLA P3 Processor Instance P10 Atmel SAM3U4E Cortex M3 Active Mode, 4MHz Internal Oscillator Room Temperature 7

15 Active Power Variability Across Temperature Current (ma) P1 P2 P3 P4 P5 P6 P7 P8 P9 P Temperature 8

16 Active Power Variability Across Temperature Current (ma) Temperature P5 P8 P10 9

17 Active Power Variability Across Temperature % Variation % Variation 28% Variation Current (ma) Temperature P5 P8 P10 9

18 Sleep Power Variability Across Instances Cortex M3 Sleep Current (Room Temperature) Current ( A) P1 P2 P3 P4 P5 Processor Instance P6 P7 P8 P9 P10 Atmel SAM3U4E Cortex M3 Sleep Mode, 32KHz Slow Oscillator Room Temperature 10

19 Sleep Power Variability Across Instances Cortex M3 Sleep Current (Room Temperature) x Variation Current ( A) 10 0 P3 Processor Instance P10 Atmel SAM3U4E Cortex M3 Sleep Mode, 32KHz Slow Oscillator Room Temperature 10

20 Sleep Power Variability Across Temperature Current ( A) P1 P2 P3 P4 P5 P6 P7 P8 P9 P Temperature 11

21 Sleep Power Variability Across Temperature Current ( A) Temperature P7 P9 P10 12

22 Sleep Power Variability Across Temperature x Variation 4x Variation Current ( A) % Variation Temperature P7 P9 P10 12

23 Source #1: Manufacturing Variability Example Frequency variation in an 80-core processor within a single die in Intel's 65nm technology [Dighe10] Frequency (GHz) GHz 5.7 GHz 1.2V 0.8V 25% 50% Observables Maximum speed, energy efficiency Mitigation mechanism Computation fidelity Core ID Permanence Spatial Granularity Temporal Rapidity Magnitude Permanent Within & across part Fixed Large 13

24 Source #2: Vendor Variability Example Power variation across five 512 MB DDR2-533 DRAM parts [Hanson07] Observables Relative cost of memory and compute operations Mitigation mechanism Algorithm selection Permanence Spatial Granularity Temporal Rapidity Magnitude Permanent Part-to-part Fixed Large 14

25 Source #2: Vendor Variability Example Power variation across five 512 MB DDR2-533 DRAM parts [Hanson07] NormalizedCurrent/Power Observables Relative cost of memory and compute operations Mitigation mechanism Algorithm selection Max Active (idd7) Max Idle (Idd3N) 0 Vendor 1 Vendor 2 Vendor 3 Vendor 4 Vendor 5 Permanence Spatial Granularity Temporal Rapidity Magnitude Permanent Part-to-part Fixed Large 14

26 Source #3: Ambient Variability Example Variation in Psleep with temperature across five instances of an ARM Cortex M3 processor Sleep Power (mw) Observables Sleep mode power Mitigation mechanism Adapt duty cycle ratio Temperature ( O C) Permanence Spatial Granularity Temporal Rapidity Magnitude Transient Part-to-part Medium Large 15

27 Source #4: Aging Example F/F (a.u.) Normalized frequency degradation in 65 nm due to NBTI [Zheng09] Model RO measurement Observables Speed degradation, increased error Mitigation mechanism Computation elasticity x10 3 4x10 3 6x10 3 8x Time (s) Permanence Spatial Granularity Temporal Rapidity Magnitude Permanent Within & across part Slow Medium 16

28 Sources of Variability Frequency variation in an 80-core processor within a single die in Intel's 65nm technology 7 1.2V 25% Frequency (GHz) GHz 5.7 GHz 0.8V 3 50% Core ID 17

29 Sources of Variability Frequency e (GHz) Frequency variation in an 80-core processor within a single die in Intel's 65nm technology Core ID 1.2V 0.8V 25% GHz Semiconductor 5.7 GHz Manufacturing 5 50% Power variation across five 512 MB DDR2-533 DRAM parts [Hanson07] 2.5 Vendor Differences Max Active (idd7) 0 Max Idle (Idd3N) Vendor 1 Vendor 2 Vendor 3 Vendor 4 Vendor 5 2 NormalizedCurrent/Power rren Variation in Psleep with temperature across five instances of an ARM Cortex M3 processor Normalized frequency degradation in 65 nm due to NBTI [Zheng09] Sleep e Power (mw) Ambient Conditions F/F (a.u.) Model RO measurement Aging Temperature ( O C) x10 3 4x10 3 6x10 3 8x Time (s) 17

30 Let us take another look at the HW/SW stack Application Application Operating System Hardware Abstraction Layer (HAL) 20x in sleep power 50% in performance }overdesigned hardware Time or part 18

31 Let us take another look at the HW/SW stack Application Application Operating System Hardware Abstraction Layer (HAL) 20x in sleep power 50% in performance }overdesigned hardware 40% larger chip 35% more active power 60% more sleep power Time or part 18

32 Let us take another look at the HW/SW stack Application Application Operating System Hardware Abstraction Layer (HAL) }overdesigned hardware 40% larger chip 35% more active power 60% more sleep power Time or part 18

33 Imagine a new hardware-software interface... Application Application Operating System Hardware Abstraction Layer (HAL) Time or part 19

34 Imagine a new hardware-software interface... Application Application Operating System Hardware Abstraction Layer (HAL) Underdesigned Hardware Time or part 19

35 Imagine a new hardware-software interface... Traditional Fault-tolerance Application Application Operating System Opportunistic Software Hardware Abstraction Layer (HAL) minimal variability handling in hardware Underdesigned Hardware Time or part 19

36 Hardware: Self-monitoring as opposed to selfhealing Measure hardware signatures, use fluid constraints in HW design, error possibility in operation using simple device monitors Static and Dynamic Reliability Management Worst case PVTS assumed Realistic PVTS conditions Number of Chips Reliability Slack Target Lifetime(T LT ) Lifetime 20

37 Hardware: Self-monitoring as opposed to selfhealing Measure hardware signatures, use fluid constraints in HW design, error possibility in operation using simple device monitors Static and Dynamic Reliability Management 20

38 Hardware: Self-monitoring as opposed to selfhealing Measure hardware signatures, use fluid constraints in HW design, error possibility in operation using simple device monitors Static and Dynamic Reliability Management Worst case PVTS assumed Realistic PVTS conditions Number of Chips Reliability Slack Traded with performance Realistic PVTS conditions with DRM Target Lifetime(T LT ) Lifetime 20

39 Hardware: Self-monitoring as opposed to selfhealing Measure hardware signatures, use fluid constraints in HW design, error possibility in operation using simple device monitors Static and Dynamic Reliability Management Worst case PVTS assumed Realistic PVTS conditions Number of Chips Reliability Slack Traded with performance Target Lifetime(T LT ) Realistic PVTS conditions with DRM Process Lifetime Circuit Functional System 20

40 21

41 21

42 An Underdesigned Multiplier Idea: change functional description of arithmetic units instead of voltage overscaling Basic building block: 2x2 multiplier Computes 11 x 11 = 111 (not 1001) Scalable to arbitrary bit widths by adding partial products ~40% power reduction but ~8% power overhead in correct mode Average error ~3.3%, max error ~22.2% Comparison with voltage overscaling (image filtering) a) Inaccurate multiplier, 41.5% power reduction, SNR : 20.3dB b) Voltage over-scaling, 30% power reduction, SNR : 9.16dB c) Voltage over-scaling 50% power reduction, SNR : 2.64dB 22 Puneet Gupta, UCLA

43 Variability-aware Duty-cycling Duty Cycle = f(psleep) 0.3 Sleep Power (mw) Atmel s ARM Cortex M3-based SAM3U Embedded Processor Mani Srivastava, UCLA Temperature ( O C) 23 Puneet Gupta, UCLA

44 Adaptable Duty Cycled Tasks in TinyOS Task (pmin, pmax) Task (imin, imax) Task 24

45 Adaptable Duty Cycled Tasks in TinyOS Task Adaptable (pmin, pmax) Task Adaptable Task (imin, imax) Task Traditional Task Task allowable DC Duty Cycle Kernel Scheduler: DC = f (PA, PS,...) PA, PS,... Hardware Signature 24

46 Hardware Variability Signatures Analytic modeling of sleep power Measured vs. modeled A and B are technology-dependent constants I gl is the temperature-independent gate leakage current T is the core temperature. Parameters of calibrated models are the hardware variability signatures passed to the software stack 25

47 Comparison of DC Scheduling Methods 2 Datasheet Worst-Case Variability-Aware Value Duty Cycle (%) Time (days) Lifetime: 1 year Processor Instance #7 Battery Capacity: 850 mah Temperature Profile: Stovepipe Wells, CA,

48 Improvement over Worst-Case Duty Cycle Average: 22x improvement Improvement (x) P1 P2 P3 P4 P5 P6 P7 P8 P9 0 Instance P10 27

49 Energy Untapped by Worst-Case Duty Cycle Average: 63% energy left untapped Remaining energy (%) P1 P2 P3 P4 P5 P6 P7 0 Instance P8 P9 P10 28

50 Duty-cycling Results from a Proof-of-concept UnO Embedded Sensing Stack 10 Quality of Sensing x Current UnO Benefit (Measured) Technology (in nm) UnO Sensor with Signature Measurement and Variability Prediction Conventional Non-Adaptive Sensor 29

51 Another Example: Underdesigned Radios Source App Tx Processing RF Amp channel Rx Processing Destination App Problem: error, deadline misses, & variability error, loss & variability error, deadline misses, & variability 30

52 Another Example: Underdesigned Radios Source App Tx Processing RF Amp channel Rx Processing Destination App Problem: error, deadline misses, & variability error, loss & variability error, deadline misses, & variability Current Practice: tolerate via protocol and app level recovery 30

53 Another Example: Underdesigned Radios Source App Tx Processing RF Amp channel Rx Processing Destination App Problem: error, deadline misses, & variability error, loss & variability error, deadline misses, & variability Current Practice: over-design for no error and minimum speed tolerate via protocol and app level recovery over-design for no error and minimum speed 30

54 Another Example: Underdesigned Radios Source App Tx Processing RF Amp channel Rx Processing Destination App Problem: error, deadline misses, & variability error, loss & variability error, deadline misses, & variability Current Practice: over-design for no error and minimum speed tolerate via protocol and app level recovery over-design for no error and minimum speed tolerate computational errors, deadline misses, and performance variation tolerate computational errors, deadline misses, and performance variation 30

55 Underdesigned & Opportunistic Computing (UNO) Machines: From Crash-and-Recover to Sense-and-Adapt Do Nothing (Elastic User, Robust App) Change Hardware Operating Point (Disabling parts of the cache, Changing V-f) Change Algorithm Parameters (Codec Setting, Duty Cycle Ratio) Change Algorithm Implementation (Alternate code path, Dynamic recompilation) Change to Algorithm with Different Characteristics (Dynamic linking to new library module) sensors & models Variability signatures: -cache bit map -cpu speed-power map -memory access time -ALU error rates Underdesign Mechanisms -stochastic processor -fluid hw constraints -application intent Variability manifestations -faulty cache bits -delay variation -power variation 31

56 Underdesigned & Opportunistic Computing (UNO) Machines: From Crash-and-Recover to Sense-and-Adapt Do Nothing (Elastic User, Robust App) Change Hardware Operating Point (Disabling parts of the cache, Changing V-f) Change Algorithm Parameters (Codec Setting, Duty Cycle Ratio) Change Algorithm Implementation (Alternate code path, Dynamic recompilation) Change to Algorithm with Different Characteristics (Dynamic linking to new library ry module) sensors & models Variability signatures: -cache bit map -cpu speed-power map -memory access time -ALU error rates - Underdesign esign Mechanisms -stochastic processor -fluid hw constraints -application intent Variability manifestations -faulty cache bits -delay variation -power variation 31

57 Rakesh Kumar, UIUC E_2 HW-based Error Resilience E_1 One Step Further: Active Fault Tolerance Rx: Treating bugs as allergies (SOSP 05) In case of errors, actively changing execution environment to avoid the error-triggering allergen Different layouts Memory padding Zero-filling Different scheduling Packet sizing, etc YY Zhou, UCSD

58 Realizing the Expeditions Project Vision 33

59 Realizing the Expeditions Project Vision 33

60 Realizing the Expeditions Project Vision 34

61 Realizing the Expeditions Project Vision Testbed 1: General Purpose Computing Job queue Scheduler node 7 1.2V Workload migration Frequency (GHz) V Core ID Instrumented Flash Servers in GreenLight Datacenter [UCSD] Off-line variability characterization and Run-time hardware signature sensing[ucsd, UCLA, UIUC] Software Mechanisms for DB Querying and Map-Reduce Apps [UCSD, UCI] 34

62 Realizing the Expeditions Project Vision 34

63 Realizing the Expeditions Project Vision 35

64 Realizing the Expeditions Project Vision Testbed 2: Embedded Processing for Body Sensor Networks Sleep Power (mw) ISF based Alcohol Sensor (on Arm) Skin Temperature Skin Conductance 3- Axis Accelerometer 3 Lead ECG Respiration Band Photoplethysm ography (PPG) Temperature ( O C) Sensor Node with ARM Cortex M3 CPU with in situ Variability Sensor [UM, UCLA] Off-line variability characterization and Run-time hardware signature sensing [Stanford, UCLA, UM] OS, PL, and App Mechanisms for Distributed Sensing [UCLA, UCI, UCSD] 35

65 Realizing the Expeditions Project Vision 35

66 Realizing the Expeditions Project Vision 36

67 Realizing the Expeditions Project Vision Testbed 3: Software Radio ARM Cortex M3 CPU & Underdesigned DSP Accelerators [UM, UCLA] Off-line variability characterization and Run-time hardware signature sensing[ucla, Stanford, UM] Variability-aware GNU Radio + N/W Protocol Stack under Linux [UCLA, UCSD] 36

68 Realizing the Expeditions Project Vision 36

69 Realizing the Expeditions Project Vision 37

70 Realizing the Expeditions Project Vision Testbed 4: Mobile Computing for Multimedia Instrumented Android Smartphone [UCLA] Off-line & S/W-inference based run-time power & error variability characterization [UCLA] Variability Adaptation Mechanisms for VP8 Codec [UCI, UCLA] 37

71 Realizing the Expeditions Project Vision 37

72 Realizing the Expeditions Project Vision 38

73 Realizing the Expeditions Project Vision Outreach Physically-minded Computing COSMOS LACC... 38

74 Variability Expedition: A Paradigm Shift to Fluid HW-SW Interfaces Opportunistic SW Radical departure from hard failures to soft variability - Work through hardware variability - rather than over-designed hardware and fault-handling software - Software becomes a significant part of the solution to variability Software adapts to part as manufactured rather than as designed - opportunistically exploit application elasticity - adaptation simplifies the structure of software layers 39

75 Variability Expedition: A Paradigm Shift to Fluid HW-SW Interfaces Opportunistic SW 39

76 Variability Expedition: A Paradigm Shift to Fluid HW-SW Interfaces Opportunistic SW No rigid constraints s on hardware implementation Sense-and-adapt, rather than crash-and-recover Leverage application intent, adaptability, and error resilience 39

77 Variability Expedition: A Paradigm Shift to Fluid HW-SW Interfaces Opportunistic SW Underdesign HW 39

78 Variability Expedition: A Paradigm Shift to Fluid HW-SW Interfaces Opportunistic SW Underdesign HW Software-driven Equivalent Scaling Eliminate the last MHz Problems Overcome the Power- Performance Wall Towards Physicallyminded Computing 39

79 Variability-Aware Software for Efficient Computing with Nano-scale Devices Performance Problem: Increasing variability in nanoscale devices leading cause of overdesigned hardware. nominal scaling Goal: Re-architect the hardware-software stack Application Application Operating System Hardware Abstraction Layer (HAL) Traditional Fault-tolerance Opportunistic Software overdesigned scaling 130nm 90nm 65nm 45nm 32nm 22nm post-silicon Technology Generation minimal variability handling in hardware Time or part Underdesigned Hardware Sleep Power (mw) Frequency variation in an 80-core processor within a single die in Intel's 65nm technology Semiconductor Manufacturing Frequency (GHz) GHz 5.7 GHz V 0.8V Core ID Variation in Psleep with temperature across five instances of an ARM Cortex M3 processor Ambient Conditions Temperature ( O C) 25% 50% F/F (a.u.) Power variation across five 512 MB DDR2-533 DRAM parts [Hanson07] 2.5 Vendor Differences Max Active (idd7) Normalized frequency degradation in 65 nm due to NBTI [Zheng09] x10 3 Model RO measurement Max Idle (Idd3N) 0.5 4x10 3 6x10 3 8x Time (s) NormalizedCurrent/Power Vendor 1 Vendor 2 Vendor 3 Vendor 4 Vendor 5 Aging