Silicon-Photonic Clos Networks for Global On-Chip Communication

Transcription

1 Silicon-Photonic Clos Networks for Global On-Chip Communication Ajay Joshi, Christopher Batten, Yong-Jin Kwon, Scott Beamer, Imran Shamim, Krste Asanović, Vladimir Stojanović NOCS 2009 Massachusetts Institute of Technology University of California, Berkeley

2 Our target manycore system 64-tile system 1 or more cores per tile 5 GHz clock ~20 W power budget for network 2 Ajay Joshi

3 On-chip network topology spectrum Mesh CMesh Clos Crossbar Increasing diameter Increasing radix 3 Ajay Joshi

4 Landscape of on-chip photonic networks Mesh CMesh Clos Crossbar [Shacham 07] [Petracca 08] [This work] [Vantrease 08] [Pan 09] [Psota 07] [Kirman 06] 4 Ajay Joshi

5 Outline Mesh CMesh Clos Crossbar Photonic interconnect technology Photonic networks Electrical vs Photonic networks 5 Ajay Joshi

6 Photonic technology Silicon photonic link 6 Ajay Joshi

7 Silicon photonic link Coupler Coupler loss = 1 db 7 Ajay Joshi

8 Silicon photonic link Ring modulator Energy spent in E-O conversion = fj/bt (independent of link length) Modulator insertion loss = 0 1 db 8 Ajay Joshi

9 Silicon photonic link Waveguide Waveguide loss = 0 5 db/cm 9 Ajay Joshi

10 Silicon photonic link Ring filter, photodetector Energy spent in O-E conversion = fj/bt (independent of link length) Photodetector loss = 0.1 db Receiver sensitivity = -20 dbm Filter drop loss = 1.5 db 10 Ajay Joshi

11 Silicon photonic link WDM Through ring loss = 1e-4 1e-2 db/ring Dense WDM (128 λ/wg, 10 Gbps/λ) improves bandwidth density 11 Ajay Joshi

12 Silicon photonic link Energy cost E-O-E conversion cost fj/bt (independent of length) Thermal tuning energy (increases with ring count) External laser power (dependent on losses in photonic devices) 12 Ajay Joshi

13 Electrical technology Repeaters Repeaters FF FF FF Design constraints Repeater inserted pipelined wires 22 nm technology 500 nm pitch 5 GHz clock Design parameters Wire width Repeater size Repeater spacing 10.0 mm 7.5 mm 5.0 mm 2.5 mm 1.0 mm 13 Ajay Joshi

14 Electrical technology Repeaters Repeaters FF FF FF Design constraints Repeater inserted pipelined wires 22 nm technology 500 nm pitch 5 GHz clock Design parameters Wire width Repeater size Repeater spacing 10.0 mm 7.5 mm 5.0 mm 2.5 mm 1.0 mm 14 Ajay Joshi

15 Electrical vs Optical links Energy cost Elec: Electrical Opt-A: Optical-Aggressive Opt-C: Optical-Conservative Optical laser power not shown (dependent on the physical layout) Thermal tuning energy Transmitter -Receiver energy 15 Ajay Joshi

16 Electrical vs Optical links Energy cost 6x 2x Optical laser power not shown (dependent on the physical layout) Thermal tuning energy Transmitter -Receiver energy 16 Ajay Joshi

17 Electrical vs Optical links Bandwidth density Repeaters Repeaters FF FF FF Repeater inserted pipelined wires 10 Gbps/μ Wavelength-division multiplexed photonic link 320 Gbps/μ 30x bandwidth density advantage using optical links 17 Ajay Joshi

18 Outline Mesh CMesh Clos Crossbar Photonic interconnect technology Photonic networks Electrical vs Photonic networks 18 Ajay Joshi

19 Outline Mesh CMesh Clos Crossbar Photonic Interconnect technology Photonic networks Electrical vs Photonic networks 19 Ajay Joshi

20 Distributed Multiplexer Crossbar Electrical design Photonic design 20 Ajay Joshi

21 Distributed Multiplexer Crossbar Electrical design Photonic design 21 Ajay Joshi

22 Centralized Multiplexer Crossbar Electrical design Photonic design 22 Ajay Joshi

23 Centralized Multiplexer Crossbar Electrical design Photonic design 23 Ajay Joshi

24 Photonic crossbar for a 64-tile system 24 Ajay Joshi

25 Photonic crossbar for a 64-tile system 25 Ajay Joshi

26 Photonic crossbar for a 64-tile system 26 Ajay Joshi

27 Photonic crossbar for a 64-tile system 27 Ajay Joshi

28 Photonic crossbar for a 64-tile system 64 tiles 64 waveguides (for tile throughput = 128 b/cyc) 128 modulators per tile 63 x 64 = 4032 ring filters per tile Total rings > 500K 10W (thermal tuning) 28 Ajay Joshi

29 Photonic device requirements in a crossbar Optical laser power (W) contour Percent area of photonic devices contour 29 Ajay Joshi

30 Photonic device requirements in a crossbar Optical laser power (W) contour Percent area of photonic devices contour Waveguide loss and Through loss limits for 2 W optical laser power (30% laser efficiency) constraint 30 Ajay Joshi

31 Outline Mesh CMesh Clos Crossbar Interconnect technologies Photonic networks Electrical vs Photonic networks 31 Ajay Joshi

32 Clos network using point-to-point channels Electrical design Photonic design 32 Ajay Joshi

33 Clos network using point-to-point channels Electrical design Photonic design 33 Ajay Joshi

34 Photonic Clos for a 64-tile system 34 Ajay Joshi

35 Photonic Clos for a 64-tile system 35 Ajay Joshi

36 Photonic Clos for a 64-tile system 36 Ajay Joshi

37 Photonic Clos for a 64-tile system 37 Ajay Joshi

38 Photonic Clos for a 64-tile system 64 tiles 56 waveguides (for tile throughput = 128 b/cyc) 128 modulators per cluster 128 ring filters per cluster Total rings 28K 0.56W (Thermal tuning) 38 Ajay Joshi

39 Photonic device requirements in a Clos Optical laser power (W) contour Percent area of photonic devices contour Waveguide loss and Through loss limits for 2 W optical laser power (30% laser efficiency) constraint 39 Ajay Joshi

40 Photonic device requirements in a Clos Optical laser power (W) contour Percent area of photonic devices contour Optical loss tolerance for Crossbar Optical loss tolerance for Clos 40 Ajay Joshi

41 Photonic Crossbar vs Photonic Clos Crossbar Clos 10 W power for thermal tuning circuits (1 μw/ring/k) 0.56 W power for thermal tuning circuits (1 μw/ring/k) For 2 W optical laser power For 2 W optical laser power Waveguide loss < 1 db/cm Waveguide loss < 2dB/cm Through loss < db/ring Through loss < 0.05 db/ring 41 Ajay Joshi

42 Outline Mesh CMesh Clos Crossbar Photonic interconnect technology Photonic networks Electrical vs Photonic networks 42 Ajay Joshi

43 Simulation setup Cycle-accurate microarchitectural simulator Traffic patterns based on partition application model Global traffic UR, P2D, P8D Local traffic P8C 64-tile system, 512-bit messages Events captured during simulations to calculate power CMesh Clos 43 Ajay Joshi

44 Power-Bandwidth tradeoff EClos PClos CMeshX2 Channel width = 128b Clos Channel width = 64b 44 Ajay Joshi

45 Power-Bandwidth tradeoff 2-3x on-chip power savings for global traffic (off-chip laser) EClos PClos CMeshX2 Channel width = 128b Clos Channel width = 64b 45 Ajay Joshi

46 Power-Bandwidth tradeoff EClos PClos EClos Comparable on-chip power for local traffic (off-chip laser) PClos CMeshX2 Channel width = 128b Clos Channel width = 64b Clos Channel width = 128b 46 Ajay Joshi

47 Conclusion Mesh CMesh Clos Crossbar Accurate baseline electrical design required Need to carefully account for the energy components in optical interconnects E-O-E conversion, Thermal tuning power, Optical laser power Clos network provides comparable throughput at lower energy for global traffic patterns More work required on the photonic device design 47 Ajay Joshi

48 Acknowledgement MIT photonic device team Franz Kärtner, Rajeev Ram, Judy Hoyt, Henry Smith Jason Orcutt, Anatoly Khilo, Benjamin Moss, Charles Holzwarth, Jonathan Leu, Michael Georgas, Jie Sun, Miloš Popović, Hanqing Li Funding sources DARPA Intel Corp. 48 Ajay Joshi

49 Backup 49 Ajay Joshi

50 Clos network using intermediate crossbar Electrical design Photonic design 50 Ajay Joshi

51 Clos network using intermediate crossbar Electrical design Photonic design 51 Ajay Joshi

52 Clos network using intermediate crossbar Electrical design Photonic design 52 Ajay Joshi