Overview of System Interfaces

Transcription

1 Overview of System Interfaces Stefanos Sidiropoulos Aeluros Inc (1) System Bandwidth Growth Corollary of: IO Bandwidth tracks gate count Rent s rule: E.F. Rent ca IC Integration (and hopefully performance) doubles every two years Moore s law: G. Moore ca (2)

2 Bandwidth Requirements: Computers CPU DVI, HDMI FSB, HT AGP, PCI-E DDR, RDRAM FBDIMM Display >1GB/s Graphics Controller >4GB/s System Controller >4GB/s Memory PCI-X PCI-E Storage Network I/O >0.1GB/s I/O Controller PCI*, *ATA, USB.. (3) Bandwidth Requirements: Network Systems Switch Cards CPU Cards >100Gb/ s Line Cards Control Memory LU, Classification Packet Buffer Memory >2GB/s 5GB/s DDR, RDRAM Proprietary OIF-CEI >10Gb/s B-Plane PHYs *MII 2.5GB/s NPU SxI*, *MII 2.5GB/s MAC/ PHY Eth, FC, SONET >10Gb/s (4)

3 Outline Design Space Topology, Parallelism, Signaling, Timing Interface Evolution Signaling, Timing, Performance Specifications Examples Multi-drop: Rambus SPI4.2 Point-point: XFI SATA (5) Simple Interface Model S Tx (t) Tx Channel PCB, Cable, Fiber Rx S Rx (t) Timing: Drive and capture signals at the correct time Signaling: Send and receive symbols robustly < 400-mV < 1-ns (6)

4 Design Space Main goal: Maximize system performance with minimum cost Parameters: Data Rate/Symbol Rate Latency Desired Bit Error Rate: measurable vs 0 Parallelism: single vs multi-link Topology: multi-drop vs point-to-point Clocking: synchronous vs asynchronous Power/Area/Cost Operating modes Different applications have different priorities Many different interfaces (7) Interface Taxonomy: (1) Parallelism Parallel: Data split over multitude of channels Transfer rate 1x-2x on chip clock Tx D 0 D 1 D N Rx Serial: Data serialized over a single channel Transfer rate multiple of chip clock Tx Rx D 0 D 1 D 3 Bidirectional data flow Time division multiplexed Simultaneous Unidirectional data flow (8)

5 Interface Taxonomy: (2) Topology Shared Bus: Lower cost Lower aggregate bandwidth Channel Impairments Shared medium scheduling Lower end to end latency e.g.: ISA/PCI, Memory, 10BaseT #1 #2 #N Shared Channel Point to point: Higher Cost Higher aggregate bandwidth #1 #2 #N High end to end latency e.g.: MP networks, 100BT, PCI-Express #3 (9) Interface Taxonomy: (3) Timing Synchronous: Same frequency and phase Conventional busses t t F 0 Mesochronous Source Synchronous: Same average frequency, unknown phase Fast memories MAC/Packet interfaces t A t A t B F 0 t B Plesiochronous: Almost the same frequency Mostly everything else today F 1 F 2 F 1 F 2 (10)

6 Interface Taxonomy: (4) Signaling Driver Impedance: Low - Voltage Mode V TX R 0 Z 0 R 0 High - Current Mode I TX R 0 Z 0 R 0 Wire Utilization: Single Ended shared signal return path Differential explicit signal return path - + Pseudo Differential ref - + (11) Performance Limitations Tx V Tx (t) V Rx (t) Rx S Tx (t) Channel - F C S R (t) V Rx (t + t) = F C ( V Tx (t) + V N ) t, F C,V N are neither zero nor constant: Varying t Timing uncertainty: Tx, Rx, Channel AC jitter, DC skew Nonideal F C Nonzero V N Channel imperfections: Bandwidth, Discontinuities Interference, random noise Affect both timing and voltage system margins Distinction is more elusive at higher speeds (12) Chip Design Goal: Minimize contribution to t, V N, compensate/filter F C

7 Outline Design Space Topology, Parallelism, Timing, Signaling Interface Evolution Signaling: From *TL to equalized CML Timing: From static delays to CDRs Methodology: From V/T budget to Power Penalty Examples Multi-drop: Rambus SPI4.2 Point-point: XFI SATA (13) Voltage Mode Signaling Example: HSTL (High Speed Transceiver Logic) up to 1Gbps/line V DDQ V SSQ - Z 0 = 50 + V REF shared + - Need to control: Output Impedance: Match Z0 to minimize reflections Slew Rate: Slow down fast edges to limit signal return noise (SSO) Need to achieve: Low input offset High gain Reference Noise Rejection (14)

8 Voltage Mode Driver Control Segmented Driver for Impedance Control: ep[n:1] en[n:1] D p n p n p n Q ep R P R R P << R Actively servoed with FSM using replica pullup, pulldown en Further Segmentation for Slew Rate Control: td td D Q td may be servoed either by timing loop or by PVT detector (15) Current Mode Bus Signaling Shared channel with passive pull-up/termination up to 1.2 Gbps/pin Active drivers with high output impedance avoid wired-or glitch i.e. Tx2 driving immediately after Tx1 R 0 Z 0 Z 0 Z 0 Z 0 R 0 Tx2 Tx1 Rx D Ictl[N:0] Must ensure driver MOSFETs stay saturated at 3/2*Vsw Control output current instead of impedance with Ictl[N:0] Slew rate control similar to voltage mode drivers (16)

9 Current Mode Differential Signaling Most common interface in higher speed systems - up to 40Gbps/pair R CTL [N:1] C C Z 0 Tx, Rx termination control similar to other systems Slew rate control may be necessary to control EMI AC coupling: Required to allow hot-plug Allows independent Tx, Rx supplies Enables Tx, Rx common mode optimization Requires data encoding to ensure run-length << R CTL *C C 8b/10b, 64b/66b, scrambling (17) Bandlimited Channel and ISI Example: coaxial 2.5Gbps 2m 3m 20m (18) Pre-emphasis/De-emphasis: Attenuate low-frequency components to match high frequency Discrete time transmitter FIR Usually open loop Equalizing receiver: Continuous time FIR Nonlinear (DFE) Can be made adaptive

10 Current Mode Signaling w/ De-emphasis z -1 R K 1 K 2 + D Q D Q I 1 I2 K 1 + K 2 =1 (I 1 +I 2 )R=V SWMAX I 2 R I 1 R Emphasis off Emphasis on (19) Outline Design Space Topology, Parallelism, Timing, Signaling Interface Evolution Signaling: From *TL to equalized CML Timing: From static delays to CDRs Methodology: From V/T budget to Power Penalty Examples Multi-drop: Rambus SPI4.2 Point-point: XFI SATA (20)

11 Timing: Simple Synchronous Systems d1 CK X CK X D I CK C1 CK C2 D I d2 on-chip logic CK C1 CK C2 Long bit times compared to on chip delays: Rely on buffer delays to achieve adequate timing margin Higher speed designs delay CK x off-chip relative to D I and match d1, d2 (21) Timing: Faster Synchronous Systems CK X PLL/DLL CK C CK X D I on-chip logic D I CK C On-chip might be a multiple of system clock: Synthesize on-chip clock frequency On-chip buffer delays do not match Cancel clock buffer delay (22)

12 Timing: Source Synchronous Systems CK SRC PLL/DLL CK RCV data rcvr logic ref CK SRC data D 0 D 1 D 2 D 3 CK RCV Position on-chip sampling clock at the optimal point i.e. maximize timing margin (23) Phase and Delay Locked Loops VCO VCDL clk K V /s K DL clk V C ref clk N K PD PD Filter F(s) ref clk K PD PD Filter 1/s Second/third order loop VCO integrates V C to generate phase Filter needs integrator for zero phase error Need zero in filter to stabilize N provides for frequency synthesis Filters reference clock jitter Accumulates phase error (1 s / z1) F( s) s K PD KV (1 s / z1) H ( s) 2 s K K (1 s / z ) PD V 1 First order loop Only controls delay through VCDL Limited phase range (conventional) Clock synthesis problematic All pass filter on reference clock jitter Phase error does not accumulate KPDKDL H( s) sk K PD DL (24)

13 Timing: Plesiochronous Systems rcvr logic D IN D 0 D 1 CK R D IN CDR CK R Recover incoming data fundamental frequency Position sampling clock at the optimal point (25) Conventional CDRs CK R D IN Vc PD VCO ref Acquisition Aid Analog Phase Detection Binary Phase Detection up dn D 0 D 1 early late Subtracts data-to-clock edge time distance from half-cycle time Phase error is difference of up/dn widths Systematic phase offset due to DFF delay Detect transition and assert up/dn according to previous/current data values Phase error information is binary Non-linear loop dynamics (26)

14 Digital CDRs Interpolating CDR: refclk Clock Generator Phase Interpolator D IN D 0 D 1 D IN CK R Digital Controller CK R Oversampling CDR: D R refclk Clock Generator D IN D 0 D 1 CK L CK L D IN Digital Controller D R (27) Both require precise phase generation Variable Phase Interpolation w = 0..N ( N w) w N If selectively span 2: Can generate any can be generated by DLL/PLL (28)

15 Phase Interpolator Implementation ctl I W V + V - I MAX - I W ctlb V + V - V + V - Interpolating CDR Oversampling CDR (29) ctl (variable) ctl = 0.5 (fixed) Outline Design Space Topology, Parallelism, Timing, Signaling Interface Evolution Signaling: From *TL to equalized CML Timing: From static delays to CDRs Methodology: From V/T budget to Power Penalty Examples Multi-drop: Rambus SPI4.2 Point-point: XFI SATA (30)

16 Conventional Performance Specification CK X D IN V IH V IL t S t H Treat amplitude and timing parameters independently: t S setup time t H hold time V IH input high margin V IL input low margin Easy to verify and implement production testing using ATE Applies mostly to synchronous systems 2.5 V DD (Volts) t S 780-ps t H t (ns) 1.0 (31) Eye Mask Performance Specification (i) Specify Limits on both timing and amplitude for both Tx and Rx: (ii) Specify minimum sinusoidal phase variation that the CDR must track: Difference between Tx and Rx mask allocated to the channel (iii) Specify integrated transmitter output phase noise over limited band (32)

17 Power Penalty Specification Used to specify receive equalizer effectiveness when input eye is closed 1) Measure BER vs Tx power with an ideal channel: V TX1 Ideal Channel P TX2 BER IDEAL 2) Repeat measurement with ISI impaired channel BER vs Signal Level V TX1 P TX2 Real Channel BER REAL Amplitude/Power Penalty is the difference between the two measurements for a given bit error rate BER 1E-06 1E-07 1E-08 1E-09 1E-10 1E-11 1E-12 (33) 1E Amplitude (x 100 mv) Signal Integrity Specifications Used to treat package and silicon as lumped elements: L I C I R I Lumped element model not adequate at high frequencies: Higher speed (>1 Gbps) interfaces specify scattering parameters: e.g. reflections characterized by S11: S11 for 3.2 Gbps Interface reflected energy specification IC loss (db) measurement variable frequency source (34) Frequency (GHz)

18 Outline Design Space Topology, Parallelism, Timing, Signaling Interface Evolution Signaling: from *TL to equalized CML Timing: from PLLs/DLLs to CDRs Methodology: from V/T budget to Power Penalty Examples Multi-drop: Rambus SPI4.2 Point-point: XFI SATA (35) Example #1: RDRAM Memory Channel M 1 M 2 M 16 M 1 M 2 M Controller CTM Clk Gen 533MHz CFM 2.1-GB/s (1066 Mbps/pin): Current mode signaling: 0.8-V swing, referenced at 1.4-V 28- channel impedance CTM/CFM Source synchronous clocking D0 D1 D2 (36)

19 RDRAM Signaling: Output Driver Subsystem Driver Bias Voltage Generator V GREF - + V GATE CC[6:0] EN V G [6:0] 7 DQ 0 7 DQ 1 7 DQ 8 Q 0 Q 1 Q Requirements: Constant output current: CC set by calibration FSM Maximize saturation margin: Bias with constant gate overdrive (37) RDRAM Clocking: DLL PD/CP/Bias Input Clock Amp Amp Mux+Interpolator Decoder 8 Counter up/dn FB Clock PD CFM Main Clock to I/O Uses dual loop DLL with digital control: Enables low power mode with no loss of lock Enables in-situ margin optimization (38)

20 Example #2: System Packet Interface SPI4.2 tclk *CLK NPU tdata (15+1) rclk MAC PHY *data D 0 D 1 C 0 D 2 rdata (15+1) ctl Mbps/pair (10Gbps + overhead per direction) LVDS signaling: mv swing 100- channel impedance Source Synchronous Clocking Intra-bit skew not well controlled by channel Up to +/- 1-UI skew relative to clock Receiver needs to de-skew individual bits (39) SPI4.2 Receiver Clocking clk PLL (x1) Phase Interpolator 0 o 90 o FSM repeated for every pair D IN System guarantees periodic calibration pattern Guarantees minimum transition density Enables intra bit alignment Alternative implementations delay D IN relative to on-chip clock (40)

21 Example #3: Serial ATA Commodity Storage Standard Replaces PC ribbon cables Two speeds: 1.5Gbps, 3.0Gbps Backward compatible Controller Hot Plug Connectors Cable/Backplane Storage Device Plesiochronous clocking with up to 600ppm base offset Requires spread clock compliance =5000ppm, f m =30-33 KHz, > 7db f no m (1- )f nom 0.5 / f m t 1 / f m Requires oversampling or second order digital CDR (41) Serial ATA Out of Band Signaling Enables speed negotiation and power mode transitions Two basic sequences used: ComReset/Cominit ComReset/ComInit T1 T2 Comwake ComWake T1 T1 T1 = 160-UI, T2 = 480-UI Need Tx to force zero output differential voltage and Rx to detect it Idle T+ T- 10mA RX + RX transition detector peak detector to FSM (42)

22 Example #4: XFP - 10Gbps Small Form Factor Pluggable Gbps AC coupled DC balancing left to higher level protocol 10GE: 64/66b SONET: scrambling MAC/PHY ASIC CDR Optics Channel has substantial loss 12 FR4 Transfer Function Tx Output Rx Input 0.38-V 55-mV 0.33-UI 0.15-UI (43) XFP Adaptive Receiver Feedback loop equalizes high and low frequency signal components: 0 o 180 o 5-GHz LPF 10 Gb/s VGA + LA 90 o 270 o de-mux HPF HPF LPF + - (44)

23 XFP System Measurements Transfer Function Tx Output Rx Input Jitter Tolerance Tolerance (UI) (45) Frequency (MHz) Summary/Foreword Different system needs dictate different interface specifications: Parallel, Serial Multi-drop, point-to-point Synchronous, Asynchronous Different design techniques/methodology apply to each class: Cost sensitive memory applications: synchronous, multi-drop, busses High-speed chip-to-chip communications: plesiochronous, point-to-point Challenges: Imperfect channels require extensive signal processing High data rates dictate different testing/characterization methods Increased I/O bandwidth challenges signal integrity and packaging Aggregate system bandwidth follows Moore s law >5Tb/s chip bandwidth and >100 Tb/s system bandwidth by 2010 (?) (46)