Identifying and Addressing Issues Arising From Advancements & Trends in Industrial Wireless Data Communications

Transcription

1 Identifying and Addressing Issues Arising From Advancements & Trends in Industrial Wireless Data Communications Presenting today: EVOLVE CONSOLIDATE Mike Larson, BS MS Sr. Systems Engineer ACCELERATE

2 Larson Data Communications, Inc. Total solutions provider of GE MDS industrial wireless data communications systems & services. * Established in 2002 * Specializing in all things relating to Wireless Data Communications MDS Triple Certified Full Service Partner for: Iowa, Minnesota, North Dakota, South Dakota, Nebraska, Montana, & Wyoming

3 A well designed, well installed, well maintained wireless system is a beautiful thing

4 Advancements & Trends A growing number of utilities are experiencing Industrial Wireless Data Communications network issues.

5 Advancements & Trends Often, these issues are the result of either unaddressed or inadequately addressed rapidly occurring changes in the operational environments within which these networks operate and are maintained.

6 Advancements & Trends #1 Once nice to have telemetry systems have evolved to become critical operational tools. #2 Control & monitoring systems as critical operational infrastructure greatly increases the need for their reliability. #3 Highly beneficial new technology has created a need for faster over-the-air data rates. #4 Modulation methodologies have thusly become extremely sophisticated and complex. #5 Higher order modulation techniques require stronger, clearer, cleaner receive signals. #6 Higher data rate requirements have pushed system operating frequencies ever higher. #7 Higher frequencies experience more loss, yet are usually limited to lower power levels. = Higher frequencies & complex mod schemes make system reliability harder to achieve.

7 Advancements & Trends The rapid adoption of highly sophisticated, high frequency, high data throughput wireless systems & complex Ethernet control system networks has pushed radio operating frequencies from more forgiving lower frequency, higher power level, frequency bands up into much less forgiving, higher frequency, lower power level bands making communications network reliability harder to achieve & control while also resulting in expertise gaps between the level of expertise required & that readily available in either the available employee pool or hirable vendor pool. Good help is hard to find

8 Advancements & Trends Issues Resulting From Expertise and/or Staff Capacity Gaps Include: * Poor Initial System Design * Poor product selection and/or installation * Spectrum issues interference, licensing issues, & bandwidth limitations * Lack of periodic system design review/update procedures * System aging & end of life issues availability of seamless upgrades paths * Personnel turnover & organizational restructuring (In-house & Vendors) * Inadequate staff awareness of current wireless operating environment * Inadequate wireless test equipment and/or the knowledge to use it * Limited knowledge of available system configuration techniques/solutions

9 So What to do? What to do?

10 A Brief History 150 Years Of Advancements & Trends In Wireless Systems Technology

11 Well, in the beginning James C. Maxwell predicts the existence of electromagnetic waves: A Dynamical Theory of The Electromagnetic Field (1864) Spark Gap Transmitters (1900s 1910s) Electron Tubes (1910s 1950s) Transistors (1950s s)

12 Analog Communications But, this was the age of Analog Data. And, analog waveforms are highly susceptible to noise & distortion. So, what starts out at the transmitter looking like this: ends up at the receiver looking like this:

13 Digital Communications Then came digital

14 Digital Communications and the Digital Signal Processor! Which allowed radios to perform multiple tasks simultaneously: * Noise Filtering * Forward Error Correction * Over-the-air Diagnostics

15 Legacy Point-To-Multipoint RS-232 Serial Radio System Point-To-Multipoint RS-232 Serial Radio System ELECTRIC UTILITY GAS/OIL WELL WATER UTILITY MDS REMOTE PIPELINE DIAGNOSTICS COMPUTER LOTTERY / FINANCIAL MDS MASTER STATION HOST COMPUTER (1980s-2000s)

16

17 Ethernet Communications and then came Ethernet and what started out in the office was quickly adapted to industrial communications systems.

18 Ethernet Communications and then came Ethernet Networking and the ability to send any packet, to any port, on any device, anywhere - on a given network.

19 The Evolving Wireless Landscape Growing Demand for Additional Network Connectivity More smart machines producing more data, passing over more connections. GE MDS Wireless Industrial Communications

20 The Evolving Wireless Landscape Growing Demand for Additional Network Connectivity Increased wireless network demands require optimal wireless links, sound network plans, and network equipment configurations that minimize OTA traffic.

21 The Evolving Network Landscape Convergence of IT and OT: Information Technology: Servers, PCs, Printers, Switches, & Routers all configured to support routine business & administrative network requirements usually consisting of non-time-sensitive communications. Typically: * over only short distances within or between office/plant buildings. * connected via copper or fiber (near unlimited network capacity). Operation Technology: Servers, PCs, Printers, Switches, & Routers AND PLCs, RTUs, Sensing/Controlling/Metering/Pumping/(ad infinitum) devices all configured to support critical industrial automation & control network requirements usually consisting of time-sensitive (real/near real time) communications. Typically: * over long distances between system sites and/or control points. * connected via wireless circuits (limited network capacity).

22 The Evolving Wireless Network Landscape Increasingly Complex Wireless Networks that work! Multiple network segments on a single, integrated, manageable, secure network.

23 Ethernet Networking Issues What could go wrong?

24 Ethernet Networking Issues Knowledge of, and experience working with, legacy RS-232 serial connectivity radios and field devices is of little use when dealing with state-ofthe-art Ethernet based wireless networks.

25 Ethernet Networking Issues The rapid adoption of highly sophisticated wireless systems & complex industrial control Ethernet networks has resulted in expertise gaps between what is required and what is available.

26 Ethernet Communications Issues For example, a few dozen bits of RS-232 Serial data becomes several hundred bits of Ethernet data (Layer 3) (Layer 2)

27 Ethernet Networking Issues Some advanced Ethernet networking knowledge is a must. Other requirements: * Realistic expectations. * A sound network IP address scheme employing network segmentation where appropriate/needed. * Network equipment (managed switches and/or routers) configured to shield low bandwidth wireless networks from unnecessary Ethernet traffic.

28 Licensed vs Unlicensed Options

29 Common SCADA System Frequency Bands MHz Licensed MHz Licensed MHz Licensed MHz Licensed MHz Licensed MHz Unlicensed GHz Unlicensed GHz Registered GHz Registered GHz Unlicensed GHz Unlicensed GHz Licensed

30 & 3/29/18

31 Upper 700 MHz A Block 2x1 MHz paired (2 MHz total) Block Licenses for Sale 52 Major Economic Areas Population coverage of 319 million (POPs) MHz MHz Favorable FCC Rules Unencumbered clear licensed spectrum Superior propagation characteristics SCADA, Voice, Telemetry, Backhaul, Unmanned Aerial Systems Base station transmission: up to 1000 and 30 watts ERP Antenna height: up to 1000 feet (305 meters) HAAT Operate in TDD or FDD

32 January 2018 Grey = Available NorthWest ern Energy NorthWestern Energy Great River Energy First Energy California High Speed Rail Authority DPPD Navopache Electric GRE NWE Salt River Project ONCOR Enterprise Products

33 Critical Infrastructure Industry Applications Critical Infrastructure & Industry SCADA Distribution Automation AMI/AMR Fixed Data and Site Sensing Demand side Management Distributed generation Renewables Secure Backhaul Mobile Data Narrow Band WiMax: GRIDMAN s Land Mobile Radio Digital Mobile Radio Tele protection Well Head monitoring Unmanned Vehicle Systems /Drones: o o o Pipeline Inspection Drones Other Asset Remote Site Inspections Asset Mapping & Survey Mapping

34 One Issue That Has Been Resolved: Equipment! Previous Licensed SCADA Radio Data Throughput Rates: 1200 bps >> 19,200 bps Current Licensed SCADA Radio Data Throughput Rates: 1200 bps >> 240,000 bps Previous Licensed SCADA Radio Ethernet processing, switching, routing, 2-way dynamic adaptive modulation, compression, data packet prioritization, forward error correction features: ~None Current Licensed SCADA Radio Ethernet processing, switching, routing, 2-way dynamic adaptive modulation, compression, data packet prioritization, forward error correction features: Excellent - in all of these areas!

35 Licensed vs. Unlicensed SCADA Radio Systems NEW: A major leap in over-the-air data throughput technology.

36 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors in choosing between a Licensed vs. an unlicensed radio system?

37 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors? 1) First, and most importantly, every situation is different 44 55' Tower G Sioux WTP Booster D 48' Tower E Tower F Tower B Kones Corner Resv/Bstr Krause Resv/Bstr Castlewood WTP 41' Booster C Tower A 44 34' 97 30' 30' 20' 10' 96 50'

38 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors? 1) First, and most importantly, every situation is different 44 6' 44 40' County Y 4' Fountain Ave Snell Prairie court Northside Tank 4 Shorewood 20' Iroquois City Valve DeSmet WTP Lake Preston City Tower Lake Preston City Valve Lake Preston Reservoir Badger Reservoir Arlington City Tower Arlington Office Bruce WTP Oakwood BPS Arlington Reservoir 2' Algoma Punhoqua Bowen Murdock Melvin Hickory Lakeview Carthage City Tower Sinai BPS Sawyer Creek Witzel W.W.T.P. Howard Tower Ramona Tower Tank 2 Broad St. 44 0' Junius Reservoir South Main 44 0' Orland Reservoir Chester WTP 28th ST. Montrose Tower Washburn Waukau 43 58' 88 36' 33' 88 30' Montrose City Tank 43 40' 98 40' 20' '

39 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors? 1) First, and most importantly, every situation is different 46 53' 24" Old Red Trail Highland Road 51' 36" Roughrider Porsborg FD Tower Tera Valley Big Sky Riverbend 49' 48" Viaduc Office Kinsella Master Lift Weedas Twin City Dr Whey 2 Go Riverwood Speedway SIS Football Midway New Station HWY 1806 Kist Pirates Loop 19th St 48' 40th Ave McKenzie Marina Bay Lakewood 46 46' 12" ' 48" 58' 48" 57' 55' 12" 53' 24" 51' 36" '

40 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors? 1) First, and most importantly, every situation is different 49 0' Whitetail Reservoir Raymond Reservoir Opheim BPS & Res. Dooley BPS & Reservoir Plentywood City Vault Flaxville BPS Plentywood BPS & Res Flaxville Reservoir Dagmar Reservoir 40' St. Marie North BPS & R Pleasant View Reservoir Pleasant View BPS Medicine Lake BPS Ry Road BPS 2 & Res. 2 Medicine Lake Water Twr St. Marie Water Twr Lustre BPS & Res. Ry Road BPS 1 & Res. 1 20' St. Marie South BPS & R McCabe BPS Volk BPS & Res. Glasgow BPS & Res. Culbertson Reservoir Brockton Reservoir N Bainville Reservoir Milk River BPS Nashua BPS Porcupine Creek Res. Wolf Point BPS & Res. WTP Poplar BPS Brockton BPS Culbertson BPS DPRW Office Bainville BPS Frazer Meter Vault Wolf Point Meter Vault Poplar Water Tower (West) Frazer BPS 48 0' Ft Peck Rptr ' 20' ' 20' ' 20' 104

41 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors? 1) First, and most importantly, every situation is different 47 50' CDP Site Sweetgrass Woman 22-15HC Nathan Hale 4-25H Maggie Old Dog 19-18H KYW 27-34H Mandree 30-31H 45' Gerald Hale 33-28H Station 1 Morsette 35-26H Morsette 26-35HB Walterpackswolf 31X-12 Helen Ruth Grant 33-34H Buffalo 1-36H Goeseverywhere 31X-11 MHA H Questar Twins Wells 32-29H TAT 15-1H Smith 11X-10 MHA H TAT 14-2H MHA H High Hawk 4-9H Plenty Sweetgrass Stevenson 15-8H Dakota Cross 2-13H Fox 14-8H Alicia Fox 16-9H TSB H Huntsmedicine 24X-8 Nathan Hale 3-18H Patricia Charging Station 6 40' Blue Buttes 3-21H Wolf 27-34H Lucy Lone Flight 16-22H Spotted Horn 26-35H Bearstail 32-29H TAT 2-1H Brugh 31-30H Station 5 FB D-4-1H Bird 31X-19 Clara 14-17H Skunk Creek H Birdsbill 14-16H TSB H Skunk Creek 23.14HC1 Guyblackhawk 24X-27 Lawerence 24X-26 Beaks 36-35H Baker / Walker TSB H Packineau 15-32H Black Hawk 15-34H Wicker 34-27H Station 4 Youngbear 31X-9 Skunk Creek H3 Ironwoman 21X-10 Skunk Creek 1-12H Skunk Creek H Skunk Creek H3 Ethan Hall 2-14H FB A-31-1H (Q1)#1 FB D-8-1H FB A-27-1H FB A-36-1H Nellieoldmouse 13X-13 Darcie 34X-14 FB D-26-1H MHA H FB A-12-1H 47 35' ' 50' 45' 40' 35' 30' 25' ' FB A-10-1H MHA H FB A-11-1H MHA H

42 Licensed vs. Unlicensed SCADA Radio Systems What are the key decision factors? 1) First, and most importantly, every situation is different 2) A SCADA system is only as good as its comm system ) Existing and/or expected electromagnetic environment 4) Terrain, vegetation, and man-made structures & obstacles 5) Utility specific operational requirements Number of sites requiring communications Number of devices at each site Number of bits of data required to/from each device Number of times each day/hour/minute data is needed Location of individual sites requiring communications Degree of reliability required Budget

43 Licensed vs. Unlicensed SCADA Radio Systems Key factors: Making a Licensed vs. Unlicensed decision

44 Whichever system you choose, in order to move the maximum amount of data in the fewest possible transmissions, that system has to be performing OPTIMALLY!

45 So, a few physics basics & a little math Antenna Theory Electromagnetics & Wave Propagation

46 Antenna System Issues Knowledge of, and experience working with, legacy low frequency radio antenna systems is of little use when dealing with higher throughput, higher frequency wireless networks.

47 Antenna Fields Fixed vs Electromagnetic Fields Magnetic Field Electromagnetic Field

48 Antenna Fields Frequency Wavelengths Wavelength (λ) = Propagation Speed/ Frequency Electromagnetic Wave Propagation Speed = Speed of light ( c ) in a vacuum c = 299,792,458 m/sec; = 186,000 miles/sec = 11,802,852,677 in/sec Frequency = 915 MHz = 915,000,000 cycles/sec c / 150 MHz = λ 78.7 => ~6 c / 450 MHz = λ 26.2 => ~2 c / 915 MHz = λ 12.9 => ~13 c / 5.80 GHz = λ 2.00 => =2

49 Fresnel Zones Dimensions per changes in frequency f1 f2 f4 f1 = frequency at 1.4 GHz f2 = frequency at 900 MHz f4 = frequency at 450 MHz 49 / GE Title or job number / 9/27/2018

50 Risk Management How do we know a wireless system will actually work? What Wireless System Tools do we have in our industrial data connectivity toolbox?

51 Wireless system design tools & engineering services have been developed and are available Electromagnetic Propagation Modeling Path Analysis The first step toward eliminating the costs of: * scope of work disagreements * extended on-site start-up times * poor wireless network performance * post-startup warranty call-backs

52 Electromagnetic Propagation Modeling

53 Free-Space Path Loss Calculation The formula for calculating free-space path loss is: α fs = (log f) + 20 (log d) Where α fs = free-space path loss in db d = path length in km f = frequency in GHz

54 Additional Losses Resulting from Poor First Fresnel-Zone Clearance

55 System Planning, Analysis & Design Path Profile Modeling Q: Radio Line of Sight Is This a Good or Bad Link? Elevation (ft) Path length (5.51 mi) Co./City Rptr Latitude N Longitude W Azimuth Elevation 1580 ft ASL Antenna CL ft AGL Frequency (MHz) = K = 1.33 %F1 = Apr Latitude N Longitude W Azimuth Elevation 1519 ft ASL Antenna CL 40.0 ft AGL

56 System Planning, Analysis & Design Link Summary A: Actually, VERY good! Co./City Rptr 107 Elevation (ft) Latitude N N Longitude W W True azimuth ( ) Vertical angle ( ) Antenna model A A14 Antenna height (ft) Antenna gain (dbi) (dbd) TX line type LDF5-50A LDF4-50A TX line length (ft) TX line unit loss (db /100 ft) TX line loss (db) Miscellaneous loss (db) Fade Margin Measure of Receive Signal Strength exceeding a particular radio system s Receive Sensitivity specification. Critical for maintaining link wireless connectivity despite variations in atmospheric & electromagnetic conditions. Frequency (MHz) Polarization Horizontal Path length (mi) 5.51 Free space loss (db) Diffraction loss (db) Net path loss (db) Radio model inet 900 inet 900 TX power (watts) (dbm) Effective Radiated Power (Watts) (dbm) RX Sensitivity Criteria 1x100^-6 BER 1x100^-6 BER RX Sensitivity (µv) (dbm) RX Signal (µv) (dbm) RX Field Strength (µv/m) Fade Margin (db) Rayleigh Fade Probability (%) Log Normal Fade Probability (%) 4.11E E-04 Accounts for impingement into radio line-of-site Predicted Receive Signal Strength Indication (RSSI) =/>20 db design minimum f/critical 900 MHz systems Thu, Apr CntyCty Rptr-107.pl4 Location - Woodland (sigma = 6 db)

57 System Planning, Analysis, & Design Multipath Analysis You now can know in advance where, or where NOT, to mount your antenna!

58 System Planning, Analysis, & Design Multipath Analysis 1530 TX Antenna Ht Elevation Above Sea Level (ft) Antenna Ht: 35 ft Antenna Ht: 25 ft Antenna Ht: 15 ft Antenna Ht: 10 ft } } } Antenna Heights at which undesirable out-of-phase signal energy (reflected from either H or V surfaces) can be expected to destructively interfere with desired receive signal energy. Optimal RX Antenna Height Path Distance Between Tx & Rx Antennas (miles) 2.18

59 System Planning & Design Obstruction Analysis **Variable Antenna Height vs. Obstruction Loss Analysis** Should your antenna be mounted at 30 feet and working instead of 50 feet and not working?! Path Obstruction Losses (db) 26 db (eg. RSSI = -92) 12 db (eg. RSSI = -78) Antenna Height (ft)

60 Antenna System Components Not the place to skimp *Transmission Lines * Grounding *Lightning Protection

61 Transmission Lines, Grounding, & Lightning Protection

62 Grounding and Lightning Protection who needs it?

63 Grounding, & Lightning Protection - 1

64 Grounding, & Lightning Protection - 2

65 Grounding, & Lightning Protection - 3

66 Wireless System Components Not the place to skimp Antenna Selection & Installation... can make, or break, your system!!

67 Common Higher Frequency System Issues * Failure to Optimize Antenna Selection For Maximize Performance * Failure To Optimally Place Obstructed Radio Link Antenna Systems * Inadequate Transmission Line Selection - Excessive Line Losses * Inadequate Grounding System Storm Related Equipment Losses Use of higher frequency, higher data throughput radio systems requires more than just a casual knowledge of electromagnetic, antenna, & transmission line theory and system grounding.

68 Antenna Options Differences? Omni Antennas Directional Antennas

69 Selecting Antennas (2) Select the antenna with the narrowest beamwidth that provides acceptable system operation; this helps to prevent interference and provides more signal strength A 3-element Yagi has 6 dbd of gain and a 72 beamwidth A 7-element Yagi has 10 dbd of gain and a 45 beamwidth A Miniflector has a 14 dbd gain and an 16 beamwidth A Paraflector has an 16 dbd gain and a 12 beamwidth Panel antennas are available with various gains and beamwidths from 60 to 180

70 Antenna Mounting Positions & Electromagnetic Radiation Patterns

71 Antenna Fields Remember science class? Unobstructed 3D Magnetic Field

72 Antenna Fields Obstructions Impeded Magnetic Field

73 BCR H1 Pattern

74 BCD H2 Pattern

75 Antenna Selection Criteria & Interference Mitigation Planning

76 Here in the upper Midwest & northern plains states, we almost never have bad weather

77 or have to worry about the effects of icing

78 or this

79 Self Interference (2) [GS-16 ]

80 Self Interference (4)

81 Self Interference (8)

82 Self Interference (10)

83 System Design Tools Antenna Space & System Isolation Interference Mitigation & Co-located System Isolation Transmitter frequency separation Transmitter timing synchronization and/or coordination Antenna physical separation Antenna directivity separation Polarization diversity Terrain segmentation ( Dirt is your friend )

84 Wireless System Planning Oshkosh, WI Oshkosh - Initial Topology Oshkosh Final (LDC) Topology 44 6' County Y Prairie court Snell Fountain Ave 4' Northside Tank 4 Shorewood Bowen Hickory Punhoqua Murdock Lakeview 2' Algoma Melvin Sawyer Creek Witzel W.W.T.P. Court Tower Broad St. Tank 2 South Main 44 0' 28th ST. Washburn Waukau 43 58' 88 36' 33' 88 30' * =/>40 Second Poll Cycle * 2 single points of failure * All data half speed * No frequency separation * No directivity separation * No polarization separation * Severe Self-Interference * 3 APs, 23 Remotes (26) * +/<4 Second Poll Cycle * No single points of failure * All data full speed * Frequency separation * Directivity separation * Polarization separation * 6 APs, 22 Remotes (28) 44 6' County Y Prairie court Fountain Ave Snell 4' Northside Tank 4 Shorewood Bowen Hickory Punhoqua Murdock Lakeview 2' Algoma Melvin Sawyer Creek W.W.T.P. Witzel Broad St. Tank 2 South Main 44 0' 28th ST. Washburn Waukau 43 58' 88 36' 33' 88 30'

85 To Do List: Do commit to planning for (& riding ) waves of change. Do take advantage of highly beneficial new technology. Understand the limits of limited wireless system expertise. Ensure that both the design & installation of sensitive systems are performed by those with proven expertise. Do plan for funding & facilitating additional staff training. Do seek out and work only with support & product vendors known to have invested in advanced wireless system and Ethernet network training for their staffs.

86 The path to an optimally designed, optimally functioning, optimally maintained wireless system can be difficult But with the right technology partners the ride can be a smooth one!

87 Thoughts? Questions?

88 Larson Data Communications, Inc. Toll Free: (866) EVOLVE CONSOLIDATE ACCELERATE