Failure Analysis Engineering Services & Technology Licensing

Transcription

1 Presents Upgrading Space Vehicle Equipment Design, Manufacture, Test, Integration, Launch and On-Orbit Spacecraft Operations Using Prognostic Technology Developed for the U.S Air Force Global Positioning System Program September 22, 2008

2 Company Information Founded in October, 2006 to develop and sell advanced technology Located in Salinas California, 70 miles south of NASA Ames Research Center, at the Creek Bridge Commercial Office Complex, 50 miles from Silicon Valley, CA Private, small Business Woman owned 8 Employees 1500 ft 2 office Creek Bridge Office Salinas, CA Customers include U.S commercial, NASA and military space system owners, builders, computer and telecommunication companies 1/10/2013 2

3 Company Services & Products Space vehicle and space systems development Engineering Services Space vehicle customer consulting Systems Engineering Payload design requirements Satellite & LV RFI/RFP generation Contract law Proposal Generation & Review Contract waste, fraud & abuse law Technology Licensing Prognostics Reliability Technology Prognostic-based technology programs for NASA & DOD Satellite & Launch Vehicle Independent Failure Analysis (IFA )Service Oracol In-Orbit Telemetry Behavior Predictions Service Telemetry Science Seminars GPS Block I & II Operational Constellation 3

4 History of Innovation First to prove need for nuclear blast test ban treaty with France, 1978 First UHF/TDMA satellite crosslink system for space (1979) First NUDET Identification and location system using GPS in space (1983) First use of qualified atomic clocks First to develop prognostic algorithms to predict equipment failures (1980) First prognostic technology data processing system (1981) First Satellite & Ground Station Ku-band TC&R system (1984) First micro-processor-based digital satellite central control system (1986) First to eliminate infant mortality failures in factory acceptance testing (1994) First to design a TT&C RF ground station in a Pentium/Windows/PC tower (1997) First to integrate Pentium/Windows NT into space vehicle factory test (1997) First to publish on prognostic technology for space vehicle applications (1995) First to integrate prognostic technology into telecommunications servers (2001) First to explain infant mortality failures using highly reliable parts (2006) First to adopt signal theory to telemetry behavior from space (2006) L3 Astra RF Ground Station in a PC First to prove piece-part and mechanical system failures do not have Markov property (2006) 4

5 Company Experience 20+ years GPS space, ground and user segment design and development 25+ years in advanced satellite, launch vehicle, missile system design & test 25+ years systems engineering process 15+ years spacecraft payload design & test 30+ years Aerospace & commercial RF & digital design & test 29+ years in prognostic technology research, development and use 25+ years space vehicle failure analysis 25+ years satellite constellation design & development 20 years space vehicle/equipment marketing & sales 20 years satellite TT&C/TC&R ground station design 10 years Advanced technology for RF Factory I&T test consoles 8 years as consultant to Air Force, NASA HQ and NASA GSFC 5 years telecommunications industry, prognostic-based high-reliability computer servers 5 years waste, fraud & abuse law 5

6 Companies & Organizations Experienced with FA Prognostic Technology 1/10/2013 6

7 Why is an upgrade necessary? Space Vehicle Yearly Failure Rate 1/10/2013 7

8 Satellite & Launch Vehicle Industry is the only industry using exhaustive equipment and vehicle acceptance testing and still has one of the highest infant mortality rates Industry Product Return/Failure Rate Aircraft 25% Satellite & Launch Vehicle 24% Consumer Electronics 20% CD s 20% Software 20% Semi-Conductor Chips 15% Computer 15% Video Recorders 10% Office Equipment 6% Cameras 4% 1/10/2013 Infosys 8 Food 0.5%

9 Why is an upgrade possible By proving that electrical piece-parts and mechanical assembly failures do not have the Markov property Markov property means an absence of memory for a random process - future behavior depends only on the current information and not on the information in the past. System Noise Has Markov Property 1/10/2013 9

10 Equipment failures that were a supposed to be random and instantaneous Due to believing that failures had the Markov property, where failure analysis has been focused Where failure analysis is now focused from prognostics Recently Proven Actual Behavior of an Electronic and Mechanical System Failure 1/10/

11 Examples of States without the Markov Property Disease Home Construction Becoming a Beauty Contestant 1/10/

12 Failure Analysis proved the failure process exists on Air Force, NASA and commercial space programs The U.S. Air Force Global Positioning System Program Office funded full failure analysis continuously for 25 years across 6 satellites and launch vehicles resulting in the successful development and use of prognostic algorithms for satellites and launch vehicles Personnel from Lockheed Martin, General Dynamics Convair and many GPS satellite equipment subcontractors provided the data to create prognostic algorithms Documented in 10 years contract requirement reports Prognostic algorithms Developed On 6 GPS Block 1 satellites 40 GPS Block 2 & 2A Satellites were Designed Based on Results from Block 1 1/10/

13 What is the upgrade? The expansion of diagnostics to include prognostics into the design and test process What is prognostic? The analysis of diagnostic data The algorithms to illustrate the information that identifies that electronic piece-parts and mechanical assemblies are going to fail in the near future Why are prognostic algorithms important? The information illustrated is very difficult to find Requires proprietary algorithms Data looks like system noise Requires training in disciplines not normally attained Data Generation & Collection Diagnostics Prognostics Prednostics 1/10/2013

14 Analogy With the Medical Field Technical field (non Markov) Identify a problem exists Identify symptoms of failed equipment Make tests to collect information the problem (acceptance testing) Use the information obtained during testing to diagnose if a problem exists (diagnosis) Use the algorithms to predict if and when a failure is going to occur (prognosis) Use the prognosis to make a conclusion when a failure will occur (prednosis) Medical field (disease is non Markov) Identify a problem exists Identify symptom for a disease/illness Make tests to collect information the symptom (tests) Use the information obtained from tests to diagnose the disease/illness (diagnosis) Use the test data to predict if and when a disease/illness exists (prognosis) Use the prognosis to make a conclusion (prednosis) 1/10/

15 Lockheed Martin s History with FA s Prognostic Technology Between 1978 and 1988, LMTO personnel provided the motivation for the creation and use of prognostic technology on the Air Force GPS program LMTO personnel, on contract to the U.S. Air Force, collected and stored the GPS satellite telemetry using AFSCN resources used to develop prognostic technology for Boeing engineering team to evaluate Between 1995 and 1997 published papers announcing FA s prognostic technology (LM TI) LMSS Advanced Technology Center, Sunnyvale, CA funded the successful independent validation of prognostics technology for use for use on the Trident Fleet Ballistic Missile Program (1997) Published interim and final reports 1/10/

16 Failure Analysis Decades Old Commercial Satellite Bus Product Lines Continue to Suffer High Infant Mortality Failures Today Satellite Bus Boeing 376 LM A2100 SSL 1300 Boeing 601 Spacebus 3000 Eurostar E2000 Boeing years 14 years 24 years 23 years 14 years 20 years 21 years

17 Commercial Satellite Bus Reliability Performance is Available Commercial Satellite Bus Reliability History Number Satellites Ordered Number of Insurance Claims Number of Years Produced Boeing 376 LM A2100 SSL 1300 Boeing 601 Spacebus 3000 Eurostar E2000 Boeing 702 1/10/

18 Launch Vehicle/Missile Demonstrated Reliability Never Achieves 100% 18 Aerospace Corporation

19 Game Theory Steep Learning Curve Shallow Learning Curve Learning Curve Having Previous Experience 1/10/ Learning Curve Reflects Learning Slowly and Building on Experience Learning Curve Using Intuitive Reasoning

20 Game Theory Desired Learning Curve No Learning Curve with Prognostics 1/10/

21 Relevant Space Vehicle Failures Studied in Space Industry U.S. Air Force Global Positioning System Program In 1983, after completing 25 years of round-the-clock failure analysis, Boeing/GPS Engineering Management ordered a stop to predicting failures Result: $80M Boeing/GPS/NAVSTAR 5 satellite failed catastrophically within weeks, suffering from a dual, reaction wheel failure that was predicted but not identified NASA Space Shuttle Program In 1986, Morton-Thiokol SRM Engineering staff predicted the O ring would fail during Challenger launch, Morton-Thiokol management over-road engineering s direction not to launch Result: Challenger failure/7 astronauts died In 2003, NASA failed to predict the destruction of the Columbia during reentry from damage to the wing s outer edge, data was available but was not evaluated Result: Columbia destruction/7 astronauts died 1/10/

22 Why did these space vehicle failures occur? Current diagnostics practices includes generating and storing data for post-failure, failure analysis Little need to evaluate data real-time, if a failure occursanalyze it later Current diagnostic practices discourages continued data analysis when no problems occur that can be used to justify the costs Not necessary, save money 1/10/

23 Challenger Video at Lift Off Showed O-Ring Failure NASA Space Shuttle Challenger SRM at Lift-Off 1/10/

24 Challenger Telemetry Showed Failure Normal Thrust Behavior Actual Thrust behavior 1/10/

25 How to Eliminate Infant Mortality Failures Identify the Problem Vehicle/equipment builders do not generate the information necessary to identify equipment that is going to fail Identify the Solution Generate the information necessary to predict what equipment is going to fail during test and within 1 year of use 25

26 Prognostic Analysis Recognizes that electronic and electro-mechanical assemblies do not fail instantaneously but go through a long-term process that is identifiable, predictable and can be managed to a positive conclusion Piece-parts that fail that are not the most unreliable but the most susceptible to circuit transients caused when piece-parts age either normally or abnormally in operating circuits Explains why factory testing produces equipment that will fail immediately after use and what is needed to eliminate the failures regardless of the stated reliability performance 1/10/

27 How to arrive at prognostic analysis from diagnostic analysis α[ ] = Analyze operator: Start with Time Series Data: α α α [Data] = Diagnostic Data Identify failures that have occurred [Diagnostic Data] = Prognostic Data [Prognostic Data] = Prednostic data Apply the data behavior to identifying characteristics about the system producing the data Used to: Identifies the information used to predict if a future failure will occur Identifies when the failure will occur in the future 1/10/

28 Prognostic Approaches Data-driven Algorithms that use real-time and stored data to determine remaining usable life for equipment Generic, equipment independent Well suited for low volume aerospace equipment In-expensive Model-based Uses experts to define future normal behavior well in advance Suited for high volume products such as consumer electronics Well suited for pattern recognition software Well suited for steady-state equipment behavior Expensive Hybrid 28

29 Data Driven Prognostic Algorithms ALGORITHM Purpose of Algorithm BASELINE ANALYSIS Identifies normal data behavior CHANGE ANALYSIS Determines change from normal behavior. COMPARISON ANALYSIS Determines when a change in normal behavior is occurring DATA INTEGRATION Collects and display all the data available to begin the process of illustrating failure DATA MINING Search large data sets for common behavior during the same time DIGITAL PROCESSING Replace values that are outliers improving image resolution DISCRIMINATION ANALYSIS Identify behavior that has changed from normal behavior FAILURE PATTERN RECOGNITION Identify data that predicts behavior MATHEMATICAL MODELING Generates mathematical equation that approximates normal telemetry behavior MULTI-VARIANT LIMIT ANALYSIS Compare behavior from several telemetry measurements simultaneously RATE CHANGE ANALYSIS Identify data to be investigated at a deeper level REMAINING USABLE LIFE Determine when equipment will fail STATISTICAL SAMPLING Reduce large data set before analyzing STATE CHANGE ANALYSIS Identify data to be analyzed further for failure signature SUPER IMPOSITIONING Creates continuous data behavior when little data is available SUPER PRECISION Improve image resolution AUTHENTICATION Eliminates corrupted data VIRTUAL DATA Creates normal data behavior DATABASE CREATOR Creates minimal data set when access to the full data set is not available 1/10/

30 Failure Analysis Prognostic Data-Driven Algorithms Algorithm Flight Equipment Supplier Factory Satellite Factory LV Factory Launch Pad Mission Control Data Integration X X X X X Baseline Analysis X X X X X Change Analysis X X X X Comparison Analysis X X X X X Data Mining X X X X Day of Failure X X X X X Digital Processing X Discrimination Analysis X X X X X Mathematical Modeling X X X X X Multi-Variant Limit Analysis X X X X X Rate Change Analysis X X X X Remaining Usable Life X X X X X Statistical Sampling X X X X State Change Analysis X X X X Super Impositioning X X X X Super Precision X Telemetry Authentication X Virtual Telemetry X X X X X Data Integration X X X X X Data Base Creation X 30

31 Robust General Purpose Data-Driven Prognostic-Based Engine Diagnostics Prognostics Database Generator Data Minor Data Summer/ Condenser Data Authenticator Data Filter Authenticator Data Filter Smoother Authenticator Filter Smoother Digital Processing Smoother Digital Processing Digital Processing Baseline Generator Baseline Virtual Generator data generator Baseline Virtual Generator Super data Impositioner generator Baseline Virtual Generator Baseline Super Super Generator data Impositioner generator Virtual Precisioner Super data Super Impositioner generator Virtual Statistical data generator Precisioner Super Sampler Super Impositioner Super Statistical Impositioner Precisioner Data Modeler Sampler Super Statistical Precisioner Super Data Precisioner 1.Wavelets Modeler Sampler Statistical Data 1.Wavelets Modeler Sampler Statistical 2.Fourier Sampler Data Transforms 1.Wavelets Modeler Data Modeler 2.Fourier 3.Regression Transforms 1.Wavelets Analysis Wavelets 2.Fourier 3.Regression Transforms 4.Curve Fitting Analysis 2.Fourier 3.Regression Transforms Fourier 4.Curve 5.FFT Transforms Fitting Analysis 3.Regression 4.Curve 5.FFT Fitting Analysis Regression Analysis 4.Curve 5.FFT Fitting Curve Fitting 5.FFT FFT Kalman Filter Neural Net Predictor Data Comparator Data Change Comparator Analyzer Data Change Comparator Comparator Rate Change Analyzer Analyzer Change Rate Discriminator Change Analyzer Change Analyzer Rate Discriminator Change Analyzer Rate Multivariate Change Analyzer Discriminator Discriminator Multivariate Analyzer Multivariate Analyzer Multivariate R e c o g n i z e r RUL Prednostics Architecture of a Robust Data-Driven Behavior Predictor in a Noise Environment 1/10/

32 Robust Data-Driven Algorithm Features Highly reliable results Used to predict outcome with >99% certainty Doesn t use any probability modeling to determine results Uses noise effectively Eliminates/corrects corrupted/data errors Insensitive to amount of data available Insensitive to the source of information 1/10/

33 Predicting Day of Failure (Prednosis) Prednostic algorithm is based on Failure Analysis database of flight hardware failures past actual remaining usable life predictions Failure Analysis updates database of failure durations on a continuous basis Failure database are proprietary information and not disseminated outside the company 1/10/

34 Probability of Reaching a Duration of Remaining Life (%) Failure Analysis Prednostic Algorithm Performance 120 Probability of Reaching A Duration of Remaining Life (%) VS Months of Remaining Usable Life Months 1/10/

35 mamps mamps mamps Failure Analysis Example of Data-Driven Prognostic Failure Behavior Predictor NASA EUVE Satellite Rate Gyro Current Analog Measurement Seconds After Summing Algorithm Unit Failure Years After algorithms used on processed data 1/10/ Minor Frames

36 Accuracy of Remaining Usable Life False Negatives False Positives Failure Analysis Reliability of Data-Driven Predictor from Over 30 Years of Use Time Interval Between Data Samples (Hours) 100% 50% 25% Where technology was developed 3 hours 6 hours 1/10/ hours 0 6 Hours 36 Time Interval Between Samples Time Interval Between Data Samples

37 Elimination of False Positives 1/10/2013 Prognostics technology developed for applications in which there are enormous penalties associated with false alarms False positives reduces the reliability of technology People simply ignore alerts Space Shuttle $5M/day to delay a launch to postpone a launch on the pad Commercial Nuclear Industry False alarms result in plant shutdowns, which cost $1M per day. EPRI studies have shown that 25% of nuclear plant trips are false alarms from degrading sensors (many sensors have much shorter MTBFs than the assets they are monitoring). Result: After a rigorous 2 year evaluation, the US NRC formally approved the use of prognostics for continuous calibration validation of all safety-critical and life critical sensors in all US nuclear plants. 37

38 Financial Impact To Space Vehicle Programs Using Data- Driven Prognostic Technology Increase in Cost Due to Price of Prognostic Technology Decrease in Cost of Space Vehicle Equipment Total Savings to Vehicle Customer 40% Decrease in Cost of Space Vehicle Production Decrease in Equipment or vehicle Sales Potential Increase in Sales of Vehicle, Equipment or Services 41% 50% 1% 0% 1% 20% 1% 20% 4% Program Insurance 4% Underwriters -3% -5%-10% -5% -5% Satellite -14% Launch -9% Equipment -30% Supplier Vehicle Supplier Supplier Satellite Customer 1/10/ % 38

39 Prognostic Technology Experience Launch Pad Integration Equipment Manufacturing Satellite I&T Launch Vehicle I&T On Orbit Satellite Operations Telecommunications Highly Reliable Servers Missile 1/10/

40 Engineering Services Areas of Prognostics that can be Improved Vehicle/equipment instrumentation Add more vehicle measurements Circuit/unit voltage and/or current and/or temperature Not real-time, but can be If used real-time, the number of false positives increase LSB resolution increase to improve accuracy Requires human interface 1/10/

41 Benefits of FA s Prognostic Technology Many financial rewards Lowers program cost Through the reduction in vehicle testing Fewer equipment spares needed Elimination for the need of space vehicle insurance or a reduction in premium cost Fewer vehicle replacement contracts Scheduling improves by having less equipment to test less down time doing diagnostics/removing/replacing unreliable equipment during test Increases equipment/vehicle reliability for the 1 st year of use Provides generic, independent failure analysis for any vehicle failure Allows the long-term (20+ years) prediction of normal spacecraft behavior to help eliminate surprise catastrophic vehicle failures Allows for generic, failure analysis across anyone s equipment 1/10/

42 Commercial Applications Every industry that produces electrical and electro-mechanical products 1/10/

43 Why Use FA Data-Driven Prognostic Algorithms Developed by U.S. Air Force on a $10M/10 year effort Used on GPS Block I satellites and launch vehicle (Atlas) use Results were used to design GPS Block II and IIA satellites Proven on military, NASA and commercial satellite and launch vehicle programs Lockheed Martin has a long history with and participated in the development and use of prognostic algorithms Drove the need for their use on the Air Force GPS program Published the first paper on using prognostics on satellites and launch vehicles (1995) Independently validated by Lockheed Martin Advanced Technology Center, Sunnyvale, CA for the Trident Missile Program (1997) 1/10/

44 FAILURE ANALYSIS Creek Bridge Business Complex 12 Glen Falls Circle Salinas, CA