Mirror Mirror on the Ceiling: Flexible Wireless Links for Data Centers

Transcription

1 Mirror Mirror on the Ceiling: Flexible Wireless Links for Data Centers Xia Zhou, Zengbin Zhang, Yibo Zhu, Yubo Li*, Saipriya Kumar, Amin Vahdat, Ben Y. Zhao and Haitao Zheng Department of Computer Science, UC Santa Barbara *Xi an Jiaotong University, China Google and UC San Diego

2 Data Centers are Everywhere No longer a luxury for tech companies Retailers Governments Universities, hospitals 1

3 Today s Data Centers Wiring is complex and costly Planning, deploying, testing 10K+ fibers Takes several weeks or even months Difficult to change wiring High labor cost Significant interruptions to operations Overprovisioning is difficult Traffic demands unpredictable Limited by hardware costs 2

4 Destination rack To Top of Rack Switch Dealing with Traffic Hotspots Name # Servers Description Cosmos O(1K) Map-Reduce Measurements show sporadic congestion losses IndexSrv O(10K) Index lookup caused Neon by traffic O(100) hotspots Car Simulation: HPC 3Cars O(100) Car Simulation: HPC Traffic hotspots are unpredictable, can appear anywhere Table 2: Datasets Can double failure rate for some jobs Demand 0.4 From Top of Rack Switch Source rack Figure 14: Traffic Demands(normalized) between ToR Switches Figure source: Halperin, D., et al. Augmenting data center networks with multi-gigabit wireless links. In Proc. of SIGCOMM (2011) 3

5 Dealing with Traffic Hotspots Measurements show sporadic congestion losses caused by traffic hotspots Traffic hotspots are unpredictable, can appear anywhere Can double failure rate for some jobs Hard to add bandwidth using wires Do not know where to add capacity Rewiring is complex, high labor cost Interrupt current operation Need alternative solutions! 4

6 Augmenting via Wireless Links Key benefit: on-demand links Create links on-the-fly at congestion hotspots Adapt to traffic dynamics New wireless technology: 60 GHz beamforming Multi-Gbps data rate Small interference footprint B A C D 5

7 Existing Work: Connecting Neighboring Racks 60GHz flyways [1] address local traffic hotspots by connecting neighboring racks wirelessly [1] Halperin, D., et al. Augmenting data center networks with multi-gigabit wireless links. In Proc. of SIGCOMM (2011) 6

8 Our Goal: Any-to-any Communication Traffic hotspots can appear between any rack pair Connect any rack pair wirelessly Hard to do using existing 60GHz beamforming! 7

9 Challenge #1: Link Blockage 60GHz transmissions are blocked by small obstacles (anything larger than 2.5mm!) A B C Confirmed by our testbed measurements Signal strength dropped by 10-30dB Up to 15-90% throughput loss Must use multi-hop forwarding Antenna rotation delay Reduce throughput by at least half 8

10 Challenge #2: Radio Interference Beam interferes with racks in its direction Exacerbated by dense rack deployment Signal leakage makes it worse RX Verified via testbed measurements A single link causes 15-20dB drop in signal quality for 15 nearby links TX Links interfere with each other Very few links can run concurrently Put a hard limit on aggregate bandwidth 9

11 Outline Motivation Our solution: 3D beamforming Implications on data centers Deployment challenge 10

12 3D Beamforming Connect racks by reflecting signal off the ceiling! A B C 11

13 3D Beamforming Connect racks by reflecting signal off the ceiling! Key Benefits No more link blockage Much smaller interference RX 2D A B C 3D 12

14 Simple Setup Reuse existing hardware, low maintenance cost! Reflector Absorber A B C 13

15 4 feet 3D Beamforming Testbed Off-the-shelf 60GHz radio and horn antenna HXI radio with 0dBm transmission power 10 o horn antenna from Flann Microwaves 8 feet Reflector Ceiling Height Plumbbob 14

16 Received signal strength (dbm) Benchmark #1: Link Connectivity Propagation path Q1: Does reflection cause any energy loss? Even cheap metal plate provides perfect reflection! -50 Direct path Reflected path Propagation path length (m) 15

17 Data rate (Gbps) Benchmark #1: Link Connectivity 2D 3D Q2: How does longer propagation path impact data rate? Link distance Negligible data rate loss 8 6 2D w/o blockage 3D (h=2m) 3D (h=3m) Link distance (m) 16

18 Benchmark #2: Interference Footprint A transmitter (0,0) communicates with a receiver (2,0) Measure the received energy at multiple locations Energy Map 2D 3D 17

19 Signal degradation (db) Benchmark #3: Robustness to Alignment Errors How does alignment accuracy impact signal strength? Fine grain experiment Measure received signal when antennas perfectly tuned Measure signal strength while introducing artificial alignment errors at 1 o increments Today s rotators: o o accuracy Reflector Link distance = 3m Link distance = 10m o Alignment error (degree) 18

20 Benefits of 3D Beamforming Reflection overcomes link blockage Connect any rack pair w/ indirect LOS Bouncing the beam minimizes interference footprint Many links can run concurrently 19

21 Outline Motivation Our solution: 3D beamforming Implications on data centers Deployment challenge 20

22 Link Concurrency in Data Centers Example data center scenario Medium-sized data center: 250 racks in a 42m x 15m room One 60GHz radio per rack 125 randomly chosen bidirectional links w/ 5+Gbps data rate Results Connect any two racks via a single hop; 70% of links run concurrently w/ 5+Gbps rate! Create a highly flexible network with data rates close to wired networks 21

23 # of concurrent links Multiple Radios per Rack Each rack can talk to multiple racks concurrently Number of concurrent links increases linearly w/ the number of radios per rack! racks 5+Gbps links # of radios per rack 22

24 # of concurrent links Impact of Ceiling Height How does ceiling height impact performance? Higher ceiling increases signal arrival angle smaller interference region Also has longer propagation path signal degradation Sweet spot: 3-4m θ Distance from antenna to ceiling (m) 23

25 Addressing Traffic Hotspots Large-scale data center simulations 250 racks (5K servers), 8 radios/rack Synthetic hotspot traffic based on popular workloads Create 60GHz links for hotspots Result: Adding 3D beamforming links cuts completion time by half Highly effective to address traffic hotspots 24

26 Deploying 3D Beamforming Need clearance between ceiling and top of rack Raised floor to hide wires under racks Cover wires by aluminum-plated ducts Reuse wall or existing metal surface 25

27 Deploying 3D Beamforming Cost of 60GHz radios Affordable thanks to the low-cost silicon implementation A pair costs ~ $130 (25m+ LOS range) Antenna arrays becoming the cheaper option Transmitter Receiver 26

28 Mirror Mirror on the Ceiling: Flexible Wireless Links for Data Centers (on the job market) 27