Fundamentals of Precision Time Protocol. Rudy Klecka Cisco Systems. October 14, 2015

Size: px

Start display at page:

Download "Fundamentals of Precision Time Protocol. Rudy Klecka Cisco Systems. October 14, 2015"

Audra Manning
5 years ago
Views:

1 Fundamentals of Precision Time Protocol Rudy Klecka Cisco Systems October 14, 2015

2 Abstract This session will provide a general background on IEEE 1588 Precision Time Protocol (PTP), how it works, some basic terminology, and its main uses in the market. There will be a discussion on PTP implementations (with a primary emphasis on Industrial products). The session will also touch on other related timing protocols and future enhancements to PTP. 2

3 Objectives Converse About PTP Basics Understand the different Time Protocols Understand the various PTP Profiles and Standards Understand how PTP relates to Controls Engineering 3

4 Timing is Everything Motion Control Scan-based to time-based control operation Load sharing of robots

5 Timing is Everything Line PLC HMI Ring Slow Loops DLR Ring Fast Loops DLR Ring 5

6 Various Time Distribution Protocols One Way Time Transfer Two Way Time Transfer GPS GLONASS GNSS BeiDou Galileo NTP IRIG PTP/1588 6

7 GNSS Systems Around the World 7

8 Why (and Why Not) GNSS Examples: GPS, GLONASS, COMPASS/BeiDou-2, Galileo, Nearly globally available Traceable to UTC Reliability (weak satellite signal) Not available indoor (or urban canyons) Government Requirements for Backup Cost of Equipment 8

9 One Way Time Transfer Basics 9

10 One-Way Time Transfer (OWTT) Basics Since my phone only has a cheap oscillator and I just powered it on initially I have no concept of true time 10

11 S1 t1 t2 R Delay 1. Frequency Lock to Satellite 1 To Frequency Lock only t1 & t2 timestamps are needed FFO = ((t 2 - t 2 )-(t 1 - t 1 ))/(t 1 - t 1 ) Steps to GPS Time t1 Continually adjusting your frequency (yellow) until it matches GPS (blue) This gets your cheap Cell Phone oscillator locked to a stable GPS frequency t2 This will allow you to more accurately measure time differences (up next) Also note: the Delay here is constant speed of light through atmosphere 11

12 Steps to GPS Time S1 R S2 t1 t2 t2 t2 t1 t1 t1 S3 S4 2. Find Time Differences of Arrival (TDOAs) TDOA S1 S2 = t2 t2 TDOA S1 S3 = t2 t2 TDOA S1 S4 = t2 t2 t2 These differences are Hyperboloids Note: angles on the arrows are the same as the delays through the atmosphere are the same 12

point Source: http://ieeexplore.ieee.org/stamp/stamp.jsp?

13 Multilateration Overlapping Hyperboloids It takes the intersection of 3 of the Hyperboloids to narrow it to a single point Source: & 13

14 Two Way Time Transfer Basics 14

15 PTP v2 messages and transmission A set of event messages providing significant instances and consisting of: - Sync - Delay_Req - Pdelay_Req - Pdelay_Resp A set of general messages consisting of: - Follow_Up - Delay_Resp - Pdelay_Resp_Follow_Up - Announce - Signaling - Management Mappings: L2 Ethernet, IPv4, IPv6 (others possible) Transmission modes: either unicast or multicast (can be mixed) Variable rate and timeout values Various TLVs and flexible TLV extensions

16 Basic Message Flow PTP (TWTT) Event Messages: Sync Del_Req Pdel_Req Pdel_Resp General Messages: Follow_Up Announce Del_Resp Pdel_Resp_FU Management Signaling Master Slave t1 t4 Sync Follow-Up (t1) Del_Req Del_Resp (t4) t2 t3

17 How do we get there?

18 1. Frequency Lock to GMC To Frequency Lock only t1 & t2 timestamps are needed FFO = ((t 2 - t 2 )-(t 1 - t 1 ))/(t 1 - t 1 ) Think of a counter on each system. From t1 to t1 the Master uses its clock to Count the time First you Synchronize (or Syntonize ) Master t1 t1' Sync Follow-Up (t1) Slave t2 From t2 to t2 the Slave uses it clock to Count the time Sync Follow-Up (t1') The difference in the number of counts is how different the two clocks are. t2' (Assuming the Delays are the same) Fractional Frequency Offset (FFO) = diff in frequency from Master to Slave

19 2. Time Align Next you Time (Phase) Align Master t1 Slave To Time Align you need t3 and t4 timestamps Sync Follow-Up (t1) Delay 1 Once you are Frequency Locked, you can Calculate the Delay and Offset from the Master Delay 2 Del_Req t2 t3 t4 Del_Resp (t4)

20 Delay = ((t 2 t 1 )+(t 4 t 3 ))/2 Delay = (Delay1+Delay2)/2 Master t1 Calculating Delay Slave Delay is really an Average Sync Follow-Up (t1) Delay 1 An assumption is made: Delay 1 = Delay 2 Assumption is usually wrong (to some degree) Delay 2 t4 Del_Req t2 t3 t1, t2 Protocol Mechanisms can correct for known Asymmetry Del_Resp (t4) Asymmetry cannot be detected t1, t2, t3, t4

21 Delay = ((t 2 t 1 )+(t 4 t 3 ))/2 Delay = (Delay1+Delay2)/2 Master t1 Calculating Time Offset Slave Offset = ((t 2 t 1 )-(t 4 t 3 ))/2 Offset = (Delay1 Delay2)/2 or Offset = Master Time Slave Time Delay Delay 2 Sync Follow-Up (t1) Del_Req t2 t3 t1, t2 Delay 1 In other words: t4 Del_Resp (t4) If you are Frequency Locked and you assume Delay1 = Delay2, then any difference is due to error in Time. t1, t2, t3, t4

22 Time Error Visualizing Time Error Fast Slow Averages to Zero Time GMC OC

23 Time Error Visualizing Time Error Fast Offset Slow Averages High Time GMC OC

24 Assumptions we made: 1. Symmetrical Master/Slave & Slave/Master 2. Same delay each time Master t1 Real World Problems Sync Follow-Up (t1) Slave t2 Delay 1 In the Real World both are not true to varying degrees Delay 2 Del_Req t3 The Two Biggest Problems: t4 1. Asymmetry 2. Packet Delay Variation (PDV) Del_Resp (t4)

25 Asymmetry Forward and backward delays are not identical. Every Node and Link in the network can introduce asymmetry. Master Slave 25

26 Asymmetry: Within a Node Uplink If the internal components of the System are symmetrical, then the delays will be symmetrical. However PHY Switch ASIC Switch PHY Downlink

27 TimeStamp Reference If different PHYs are used on different ports, Even that slight difference can cause asymmetry When trying to get to nanoseconds of Precision, Every little bit counts And where you time stamp matters. Asymmetry: Within a Node PHY Switch ASIC Uplink TimeStamp Reference Switch PHY Downlink

28 Typically, transmission delays are fairly symmetrical: Bidirectional over same pairs, small differences in pair lengths However, some things can make it more likely: EtherChannel, Fiber (especially rings; East v. West) Nanoseconds matter => Meters matter Asymmetry: Transmission Master Slave

29 PDV is primarily due to Varying Queue Delays Packet Delay Variation (PDV) Master Packet Network Hop 1 Hop 2 Hop 3 Hop 4 Hop n Slave PTP Session PTP Session Even High Priority packets get behind a 1518 from time to time. The variance comes from the fact that sometimes you do and sometimes you don t. It s all statistics

30 Time Slave Master Servo Boundary Clocks and Transparent Clocks Servo Monitor Freq Only 30

31 Boundary Clock v. Transparent Clock Error Sources Boundary Clock Transparent Clock TimeStamp Reference Switch TimeStamp Reference Switch PHY PHY Time & Message Transfer Time Slave Master Servo Oscillator Time & Message Transfer Monitor Servo Freq Only Oscillator TimeStamp Reference TimeStamp Reference PHY PHY

32 Pros and Cons of Boundary Clocks Pros: Breaks up the PTP message domain Breaks up the PTP timing domain Spans across VLANs Shields Slaves from Transients due to hierarchy changes (BMCA) Filters PDV Cons: Adds low frequency (wander) time error (hard to filter) Message Transfer Time Transfer 32

33 Pros and Cons of Transparent Clocks Pros: Maintains tight timing throughout a domain Peer-to-Peer TCs can converge faster after network topology changes Cons: End-to-End TCs can have scalability issues Time Transfer Message Transfer 33

34 GM GM Two Types of TCs: End-to-End and Peer-to-Peer Non-PTP BC BC E2E TC S M S M S M M P E2E TC BC BC P2P TC BC S M S M S M S M M P Delay and pdelay Mechanisms Delay Brownfield SYNC Delay_Resp Delay_Req SYNC pdelay_resp pdelay_req P2P TC 34

35 Peer to Peer Transparent Clock t 1 MASTER Master time = T M SYNC P2P Transparent Clock SLAVE Slave time = T S = T M + offset Offset = ((t 2 t 1 ) (t 4 t 3 ))/2 Residency Time (rt) pt 2 pdelay_request pt 1 SYNC pdelay_request t 2 pt 1 Offset = (t 2 t 1 ) mpd2 correctionfield pt 3 pdelay_response pt 2 correctionfield = mpd1 + rt mpd1= ((pt 2 pt 1 )+(pt 4 pt 3 ))/2 pdelay_response mpd2= ((pt 2 pt 1 )+(pt 4 pt 3 ))/2 pt 4 pt 3 pt 4 mpd : meanpathdelay 35

36 Thermal Changes Affect Base Oscillators Environmental Aspects

37 PTP Profiles (And the Proliferation of said ) 37

38 Main Current Industrial Profiles IEEE Default (CIP Sync) IEEE 802.1AS IEEE C ITU-T G Segments Industrial Solutions AVB (residential) TSN Transport IP, L2 Ethernet, industrial specifics Transmission Multicast (default) Multicast non-forwardable Delay mechanism Delay (Annex J.3) Pdelay (Annex J.4) Power Industry (SmartGrid substation) Telecom Mobile Backhaul Substation Backup L2 Ethernet L2 Ethernet L2 Ethernet Multicast non-forwardable pdelay pdelay delay Clock mode One- & two-step Two-step Two-step Any BMCA Default Alternate Default Alternate TLV Extensions Optional Yes Yes No Clocks OC, BC, TC time-aware bridge and end station OC, TC (BC in future revision) Multicast (both address types) T-GM, T-BC, T-TSC Deployment model Not defined Full support Full support Full Support + PHY layer freq.

39 Performance Specifications IEEE Default (CIP Sync) IEEE 802.1AS-2011 IEEE C Network limits No 7-hop network: time accuracy jitter and wander Clocks No LocalClock: frequency accuracy time granularity noise generation 16-hop (TC) network TC timeinaccuracy limit +-50 nsecs Grandmaster Frequency accuracy TimeAware systems: residence time pdelay turnaround time Error in rate ratio (or frequency offset) measurement Grandmaster timeinaccuracy limit

40 White Rabbit Sub-nanosecond synchronization! 40

41 Real World Impact PTP Errors Translated to Machine Error 41

42 Using Multiple Unmanaged Switches in a Large System 42

43 Simulated Network Load ITU Telecom Profiles ITU-T G.8261 Timing & Sync Aspects in PSNs Appendix VI.5 Test for Two Way Protocols Baseline Test (no Network Master/Slave back to back) Performance Tests (Network & Load) Test Case Description Network Load 12 Static Packet Load 13 Sudden large and persistent Load Changes 14 Slow Load Change over extremely long Time 15 Temporary Network Outage 80% 20% 80% 10% 20% 3h 80% 55% 12h 50% 1h 20% 6h 24h 16 Temporary Congestion 17 Routing Changes caused by failures 43 43

44 Count Simulated Network Load ITU Telecom Profiles GMC Background Traffic ITU G TC 13 TM 1 Background Traffic PDV Mean = 64 usecs Minimum = 57 usecs Maximum = 105 usecs Std. Dev. σ = 6 usecs Sample Count = Recovered Clock To To To To To To To To To msec/bin 44

45 Count Simulated Network Load ITU Telecom Profiles GMC Background Traffic ITU G TC 13 TM 1 Background Traffic Mean = -1.6 μsecs Minimum = -35 usecs Maximum = 11 usecs Std. Dev. σ = 7.4 μsecs Sample Count = 900 QoS Only Recovered Clock To To To To To -4 0 To 2 6 To 8 1 μsec/bin 45

46 Count Simulated Network Load ITU Telecom Profiles GMC Background Traffic ITU G TC 13 TM 1 Background Traffic QoS + PTP Recovered Clock Mean = 43 nsecs Minimum = -54 nsecs Maximum =154 nsecs Std. Dev. σ = 35 nsecs Sample Count = To To To 0 10 To To To To To To To and over 10 nsec/bin 46

47 Count Simulated Network Load ITU Telecom Profiles GMC Background Traffic ITU G TC 13 TM 1 Background Traffic Mean = -1.6 μsecs Std. Dev. σ = 7.4 μsecs Sample Count = 900 Mean = 43 nsecs Std. Dev. σ = 35 nsecs Sample Count = 900 With and Without PTP Same Bin Scale 200X Difference w/ PTP! Recovered Clock To To To To To To -6-4 To -2 0 To 2 4 To 6 8 To 10 1 μsec/bin 47

by this value to determine position error due to network jitter Average System Clock Jitter (Max) ~ 35 nanoseconds 0.

48 So what do these numbers really mean? 16 Axis Star, Linear K6500, Stratix 8000 Switch 16 Axis Star, K350, Stratix 2000 Switch Multiply your application speed by this value to determine position error due to network jitter Average System Clock Jitter (Max) ~ 35 nanoseconds s x 6000 RPM/ 60s/min = Revs Note: Sample from Axis 2 Average System Clock Jitter (Max) ~ 7.4 microseconds s x 6000 RPM/ 60s/min = Revs Note: Sample from Axis 1, off switch 1 48

49 THANK YOU

Measuring Time Error. Tommy Cook, CEO.

Measuring Time Error. Tommy Cook, CEO. Measuring Time Error Tommy Cook, CEO www.calnexsol.com Presentation overview What is Time Error? Network devices. PRTC & Grand Master Clock Evaluation. Transparent Clock Evaluation. Boundary Clock Evaluation.