Lessons Learned During the Development of GNSS Integrity Monitoring and Verification Techniques for Aviation Users Sam Pullen Stanford University spullen@stanford.edu ITSNT Symposium 16 November 2016 Toulouse, France
2 Outline Introduction Four Key Lessons summarized: GPS performance and reliability have been outstanding. GNSS anomalies are difficult to characterize. Safety requirements for civil aviation are complex and challenging to meet. Significant design conservatism is needed. Key integrity methodologies and issues Probability distributions of GNSS errors Development and use of threat models based upon very limited data Summary
Introduction 3 I have been working on GNSS integrity and safety assurance (for civil aviation and other users) since completing my Ph.D. at Stanford in 1996. Ground-based Augmentation Systems (GBAS; LAAS) Space-based Augmentation Systems (SBAS; WAAS) Receiver Autonomous Integrity Monitoring (RAIM) In this research, similar issues have presented themselves multiple times lessons learned. These lessons help focus on the key issues underlying GNSS integrity assurance and verification.
Lesson 1: GNSS Performance 4 In the early years of GPS, satellite anomalies occurred occasionally but relatively rarely. These anomalies have become even rarer over time, particularly since 2007 2008. At the same time, nominal GPS Signal-in-Space User Range Error (URE) has steadily decreased. GPS users have benefited tremendously from the resulting accuracy, integrity, and continuity.
Nominal GPS SPS Error Performance 5 Source: Col. Steve Whitney, GPS Program Update, CGSIC, Tampa, FL, Sept. 2015
User Range Error Probability per GPS Satellite Block (2008 2014) 6 Source: T. Walter & J. Blanch, Characterization of GPS Clock and Ephemeris Errors to Support ARAIM, ION Pacific PNT 2015. Combined IGS station data: 15 minute samples from 2008 2014 Meets GPS SPS requirement Would not bound below ~ 10-6
GPS Satellite Outages and Service Failures (2008 early 2016) 7 Source: T. Walter History of GPS Performance, May 2016 (unpublished). SV healthy with valid comparison SV unhealthy Service Failure (error > 4.42 URA) Five observed service failures over eight+ years Service failures only observed on one satellite at a time Mean Time to Alert (MTTA) ~ 21 minutes
Implications for Other GNSS Constellations 8 Today s GPS performance was demonstrated over many years of experience and system improvements. Other GNSS constellations will take time to demonstrate similar levels of fault robustness. GLONASS is an important cautionary example: Over the period from 2009 to mid-2016, almost 300 examples of individual satellite failures (unalerted range errors > 70 meters) were observed with mean duration 1 hour. In addition, simultaneous failures of multiple satellites ( constellation faults ) occurred in 2009, 2010, and 2014. Details to be presented by Stanford at upcoming ION ITM 2017 conference.
Lesson 2: GNSS Anomalies 9 Because nominal GPS performance has been so good, finding and studying anomalous GNSS behavior has been difficult. Limited observances of GNSS anomalies makes deriving threat models of anomalous behavior challenging. How can we be confident that threat models developed from observed behavior cover all possible events? More on this to come
Lesson 3: Aviation Requirements 10 Integrity is the primary aviation safety requirement. With a given probability (e.g., 1 10-7 per operation), bound errors to specified levels or alert users of unsafe conditions within a defined time-to-alert. This probability is interpreted under very strict conditions ( specific risk ). The continuity requirement protects safety as well as aircraft operations flow. With a given probability (e.g., 1 10-5 per 15 sec), prevent operations from being aborted once they have begun. Aborts due to detected failures (to protect integrity) count against this requirement. Integrity and continuity must both be met simultaneously, creating challenging trade-offs.
11 Specific vs. Average Risk Average Risk: the probability of unsafe conditions based upon the convolved ( averaged ) estimated probabilities of all unknown events. Probabilistic Risk Analysis (PRA) is based on this procedure Specific Risk: the probability of unsafe conditions subject to the assumption that all (negative but credible) unknown events that could be known occur with a probability of one. Evolved from pre-existing FAA and ICAO safety standards Risk aversion is buried inside specific risk analysis Difficult to apply in practice because engineering judgement about which events could be predicted varies widely
Lesson 4: Need for Conservatism 12 In order to meet continuity requirement, measurement removals must be infrequent. As a result, the error distributions to be bounded include various degrees of off-nominal as well as nominal behavior. Underlying distributions usually have non-gaussian tails. Mixing of different error conditions creates fatter-than Gaussian tails in the combined distribution. Gaussian distributions chosen to bound non- Gaussian data are often very conservative. Faulted conditions must be assessed in a worst-case manner, also leading to conservatism.
User Protection Levels for Civil Aviation 13 Under nominal conditions (H 0 ): VPL H 0 Extrapolation to H0 integrity risk probability (for Gaussian dist.) Under specific faulted condition (H f ): Error bias caused by faulted condition (converted to vertical position error) K ffmd The maximum protection level across all nominal and faulted conditions is applied by the user. Multiple different fault-condition protection levels exist. Fault conditions without computed protection levels must be bounded by the maximum protection level. N i 1 s 2 i, vert VPL f B f, vert Kmd, f vert, f 2 i Bounding range error variance Geometric conversion: range to vertical position Vertical position error std. dev. under faulted condition Extrapolation to faulted integrity risk, incorporating prior probability (for Gaussian dist.)
User Range Error Probability per GPS Satellite Block (2008 2014) 14 Source: T. Walter & J. Blanch, Characterization of GPS Clock and Ephemeris Errors to Support ARAIM, ION Pacific PNT 2015. Combined IGS station data: 15 minute samples from 2008 2014 Consistent nominal behavior conservatism Varied offnominal behavior (not necessarily faulted )
16 November 2016 Error Bounding in Practice (Simulated Data Set) GNSS Integrity Verification for Aviation 15 Inflated Gaussian distribution (to bound visible tails) m = 0 = 4.0 4.50 = 18.0 Gaussian Dist. m = 0 = 4.50 Normalized Observations (non-gaussian, non-symmetric, fat tails due to anomalous behavior) m = 2.0 = 4.50
Threat Model Development and Usage 16 Specific Threat or Anomaly Description Theory / Physics Collected Data Bounded, multidimensional parameter space System / User Impact Model (incl. monitoring) Deterministic simulation Worst-case user impact (and relevant points within threat model)
LAAS Ephemeris Threat Model 17 MI due to Erroneous Satellite Ephemeris Type A Threat: Satellite maneuver (orbit change) Type B Threat: no satellite maneuver Type A1: error after satellite maneuver Type A2: error during satellite maneuver Error in generating or updating ephemeris parameters Erroneous (or unchanged) ephemeris after maneuver completed Type A2a: intentional OCS maneuver, but satellite flagged healthy Type A2b: unintentional maneuver due to unplanned thruster firing or propellant leakage Mitigation not required for CAT I ops.
Type A Threat: Observed GPS SPS 3-D Position Errors on April 10, 2007 18 Source: FAATC GPS SPS PAN Report #58, 31 July 2007 (at Honolulu)
Type B Ephemeris Failure on PRN 19, 17 June 2012 19 Source: T. Walter & J. Blanch, Characterization of GPS Clock and Ephemeris Errors to Support ARAIM, ION Pacific PNT 2015. Very large orbit errors appear after ephemeris message changeover.
Ephemeris Threat Model Implementation 20 Type A Type B Model scheduled orbit maneuvers Bound maneuver directions / DVs GPS SV operations GPS ICD parameter ranges Apply LAAS monitor MDEs Model permissible navigation data blunders Select ranges of blunders most difficult to detect Simulate maneuver impacts on GPS satellite orbits Random maneuver parameters Apply LAAS monitor MDEs Random blunder parameters (from selected ranges) Simulate blunder impacts on GPS satellite orbits Missed detections Look for Hazardous Errors Modify as needed Modify as needed Missed detections Look for Hazardous Errors
Severe Ionospheric Storm in CONUS on 20 November 2003 16 November 2016 Lessons Learned on GNSS Integrity Verification 21 20:15 UT 21:00 UT
CONUS Ionospheric Gradient Threat Model Gradient [mm/km] Source: Jiyun Lee, et al, Long-Term Iono. Anomaly Monitoring, ION ITM 2011 450 400 Flat 375 mm/km Linear bound (mm/km): y = 375 + 50(el - 15)/50 Flat 425 mm/km 350 300 250 200 150 100 50 16 November 2016 0 0 10 20 30 40 50 60 70 80 90 Elevation [deg] Front speed wrt. ground: 750 m/s Front width: 25 200 km Total differential delay 50 m Lessons Learned on GNSS Integrity Verification 22
Ionospheric Threat Model Development (using LTIAM Software Tool) 23 Pre-select periods with anomalous behavior Search for large gradients in ionospheric truth data Confirm validity of observed gradients
Ionospheric Threat Model Implementation (GBAS Simulation and Geometry Screening) 24 SV almanac and current time (simulate satellite geometries) Subset Geometry Determination (N-2 constraint) LGF acts to make potentially unsafe user geometries unavailable. (simulate iono. gradient impact) Ionosphere Anomaly Threat Model Worst-Case Ionosphere Error Determination Inflated pr_gnd, vig, and/or P- values Iterative Sigma/P- Value Parameter Inflation Yes Airport Approach Layout and Ops. Limits Approach Hazard Assessment Compare MIEV to Ops. Limits for Available Subset Geometries Do Any Unsafe Subsets Exist? No Approved Sigmas/P-Values for Broadcast by VDB Inflate broadcast parameters as needed to eliminate (make unavailable) all subset geometries with max. vertical errors exceeding OCS-based safety limit. This makes many safe (max. error < limit) geometries unavailable as well and thus reduces system availability (conservatism).
16 November 2016 Conservatism in WAAS (SBAS) Vertical Protection Levels Lessons Learned on GNSS Integrity Verification 25 Source: WAAS PAN Report #34, Oct. 2010. http://www.nstb.tc.faa.gov/ DisplayArchive.htm Max. VPE 7 m (at Barrow, AK)
VPL (m) CAT I GBAS Vertical Protection Levels at Houston (20 June 2015) Uninflated VPLs at Houston are typically < 3 5 meters Values approaching 10-meter VAL are due to geometry screening 10 9 8 Source: http://laas.tc.faa.gov/iah_graph.html Rwy 27 Rwy 08R Rwy 08L Rwy 26L 7 6 5 4 Rwy 26R 3 2 Rwy 09 Time (UTC) 26
27 Summary Similar challenges influence design for user safety in different GNSS systems. While GPS performance to date has been exemplary, anomalies and faults exist that require monitor exclusion or inflated error bounding. Because of the rarity of anomalous GNSS behavior, it is difficult to assure that threat models cover all possible fault scenarios. Assessment seeks out worst combination of fault parameters within threat model bounds. Significant conservatism results provides margin against faults not considered ( unknown unknowns ).
28 Ongoing Work Collect data to better understand characteristics of new and evolving GNSS constellations Needed to set per-satellite and constellation fault parameters for ARAIM Consider applications of specific and average risk to different hazard classes For example, loss of continuity presents a lower hazard and can usually be evaluated based upon average risk Adapt civil-aviation risk requirements to other modes of transport that apply GNSS Train control and signaling Autonomous vehicles and driver assistance
29 Backup Slides follow
SV Age and Duration (years) Individual GPS Satellite Outages: Age, Number, and Duration (1999 2011) 20 18 Source: S. Pullen, P. Enge, Using Outage History to Exclude High-Risk Satellites from GBAS Corrections, Navigation, Spring 2013. Blk II Blk IIA Blk IIR (and RM) 16 14 12 10 SV age at end of outage Outage duration SV age at start of outage 8 6 4 Record begins in 1999 2 0 15 20 25 30 35 40 45 50 55 60 SVN Number 30
User Range Error Probability per GPS Satellite Block (2008 2014) 31 Source: T. Walter & J. Blanch, Characterization of GPS Clock and Ephemeris Errors to Support ARAIM, ION Pacific PNT 2015. conservatism Combined IGS station data: 15 minute samples from 2008 2014 Meets GPS SPS requirement Would not bound below ~ 10-6
User Protection Levels for Civil Aviation 32 GNSS civil aviation users verify integrity in real time using position-domain protection levels: H0 (nominal) protection level: VPL H 0 Nominal UCL multiplier (for Gaussian dist.) K ffmd N i 1 s 2 i, vert Faulted Protection levels (one per fault hypothesis j): 2 i Bounding range error variance Geometric conversion: range to vertical position VPL j Bias error generated by fault B K j, vert md vert _ fault Faulted UCL multiplier (computed for Gaussian dist.) Vertical error std. deviation under fault condition
Example: GPS SV Orbit Error (from FAA GPS PAN Report #94, July 2016) 33 Orbit Error Histograms for GPS PRN-08 (SVN-72), April June 2016)
Slant Iono Delay Slant Iono Delay (m) (m) Moving Ionosphere Delay Bubble in Ohio/Michigan Region on 20 Nov. 2003 35 30 Data from 7 CORS stations in N. Ohio and S. Michigan 25 20 Initial upward growth; slant gradients 60 120 mm/km Sharp falling edge; slant gradients 250 330 mm/km 15 10 5 Valleys with smaller (but anomalous) gradients 0 0 50 100 150 200 250 300 350 WAAS Time (minutes from 5:00 PM to 11:59 PM UT) 34
Ionospheric Anomaly Front Model: Potential Impact on a GBAS User 35 Simplified Ionosphere Wave Front Model: a ramp defined by constant slope and width Front Speed 200 m/s Front Slope 425 mm/km LGF IPP Speed 200 m/s Airplane Speed ~ 70 m/s (synthetic baseline due to smoothing ~ 14 km) Front Width 25 km Max. ~ 6 km at DH GBAS Ground Station Stationary Ionosphere Front Scenario: Ionosphere front and IPP of ground station IPP move with same velocity. Maximum Range Error at DH: 425 mm/km 20 km = 8.5 meters
Aviation Integrity Strategy 36 Detection and exclusion of individual GNSS measurements that appear faulted or anomalous ( integrity monitoring ). Bounding of remaining errors (using conservative statistical models) Removal of anomalous measurements is needed to prevent error bounds from getting too large. But every measurement removal puts continuity at risk
Critical Parameters in Protection Levels 37 Satellite geometry known to user aircraft K-factors to extrapolate integrity bounds based on integrity and continuity requirements allocations Bounding error model i2 is the variance of a zero-mean Gaussian distribution that bounds nominal range errors on satellite i out to allocated H0 integrity probability K ffmd B i,j is the bias error on satellite i due to fault mode j Finding these parameters is the primary challenge for integrity verification for civil aviation.
Impacts on Schedule and Cost 38 Unpleasant surprises have been a major factor in delaying completion and approval of GNSS augmentations, leading to cost increases. Severity of anomalous ionospheric spatial decorrelation Behavior of satellite C/A-code signal deformation ICAO/RTCA/EUROCAE standards are typically completed before extensive operational experience is gained and are difficult (and expensive) to change. Retaining standards flexibility is key to limiting cost and schedule impacts, but this is very difficult to achieve in practice.