Why is Systems Integration understood so poorly? Reflections on 3 decades of unforeseen failures

Transcription

1 Reflections on 3 decades of unforeseen failures by Gerrit Muller University of South-Eastern Norway-NISE gaudisite@gmail.com Abstract Nearly all systems developments run into problems in the late project phases, where unforeseen surprises disrupt careful planning. We will discuss a framework for systems development and integration and use a number of examples to explore what happens during systems integration. We assert that the entire project plan should be designed in reverse order, taking systems integration as driving concern. integration examples 1 viewing 1980 performance 2 acquisition performance theory fundamentals of integration fundamentals of systems engineering 7 great presentations Distribution This article or presentation is written as part of the Gaudí project. The Gaudí project philosophy is to improve by obtaining frequent feedback. Frequent feedback is pursued by an open creation process. This document is published as intermediate or nearly mature version to get feedback. Further distribution is allowed as long as the document remains complete and unchanged. status: planned solution information physical skip; too little time products/projects skip; too little time 4 gas turbines 5 subsea oil&gas role of software and users reflections on systems integration wrap-up

2 Figure of Contents TM integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 2 Gerrit Muller SIRKlogo

3 Example 1: Integration of Treatment Planning System user 1980, first job: display firmware host display integration drama: image retrieval 20s (spec: less than 1s) disk next image 19s monitor 1s image on monitor cause: too much overhead too many layers too much process communication root cause: lack of system design 20 seconds 3 Gerrit Muller SIRKfirmwareExample

4 Why is Systems Integration so Poorly Understood Why do we always get delays and cost overruns during integration? Why seems everything OK until integration? Why do so few people understand what happens during integration? component 1 scheduled delivery date realized delivery date component 2 component 3 integration and test component 4 Do you have any design issues for the design meeting? The default answer is: No. delay and cost overruns During integration numerous problems become visible 4 Gerrit Muller SIRKintegrationQuestions

5 How do you rank your project or program? poor sufficient good very good excellent perfect Outside world Customers Lifecycle support Specifications Design Technology People Process 5 Gerrit Muller SIRKsurvey

6 Practical Limitations Outside world Customers Lifecycle support Specifications Design Technology People Process poor sufficient good very good excellent perfect Perfect processes, people, technologies, designs, or specifications do not exist Imperfections sometime, somewhere, will show up; always at an inconvenient moment x x x x x x x x x expected answers from Kongsberg industry 6 Gerrit Muller SIRKlimitations

7 Fundamentals of Integration integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 7 Gerrit Muller SIRKlogoFundamentalsIntegration

8 V-Model needs validation specification verification system design system test subsystem design subsystem test component design component test component realization 8 Gerrit Muller TPSEPvModel

9 Conventional Integration View needs specification and design specification system design subsystem design component design validation verification system test subsystem test component test integration and test component realization 9 Gerrit Muller SIRKvModel

10 Limitations in Front-End Cause Failures failures found during integration can be traced back to unknowns, unforeseens, and wrong assumptions needs specification and design specification system design subsystem design component design ε ε subsystem test component test validation verification system test integration and test component realization 10 Gerrit Muller SIRKvModelFailures

11 Typical Concurrent Product Creation Process policy requirements design integrate test policy requirements and specification design integrate test -1 strategy 0. feasibility 1. definition 2. system design 3. engineering 4. integration & test 5. field monitoring 6. product operational life cycle 11 Gerrit Muller SINTproductCreationProcess

12 Integration Takes Place in a Bottom-up Fashion parts view functional dynamic qualities integrate alpha test component subsystem system function system context 12 Gerrit Muller SIRKlevels

13 Fill in this form during KSEE 2013! KSEE 2013 work form Niels Braspenning System Integration at ASML: Linking Technical Content, Test Configurations, Timing... And People! Current Status What type of failures pop-up during your Integration? Potential Improvements How could these failures be found earlier? What means or strategies can you employ to find them earlier? Alejandro Salado Validation risks of using development methodologies in a hierarchical fashion - When contracts meet architecture ownership Andreas Thorvaldsen Changing A System From Within And Get Hit By The Unexpected Surprises Benoît Le Bihan Laggan Tormore Project System Test: when new Subsea Solutions For Harsh Environment Meet Reality Jim Armstrong Systems Integration: What Are We Waiting For? Terje Jensvik A software centric approach to Electronic Systems Engineering. Eldar Tranøy Early phase need analysis Can we ease systems integration? Gerrit Muller Why is Systems Integration understood so poorly? Reflections on 3 decades of unforeseen failures 13 Gerrit Muller SIRKformKSEE

14 Example 2: Performance Again integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 14 Gerrit Muller SIRKlogoAcquisition

15 Example 2: Integration of MRI Acquisition Subsystem host new hardware new software acquisition reconstruction display gradient amplifiers RF amplifiers RF receiver 15 Gerrit Muller SIRKscanner

16 Repetition Time MRI imaging = repeating similar pattern many times G y =0 G y =127 RF transmit TE TR typical TE: 5..50ms receive problem: TR > 1 s. spec less than 10 ms more than factor 100 off! Gz Gx causes: floating point arithmetic too many layers Gy root cause: functionality focus 16 Gerrit Muller SIRKimaging

17 Fundamentals of Systems Engineering integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 17 Gerrit Muller SIRKlogoFundamentalsSE

18 SE Rule 1: Partition and Define Interfaces system subsystem 1 subsub system A subsub system B subsub system N atomic part atomic part atomic part subsub system A subsystem 2 subsub system B subsub system N subsub system A subsystem n subsub system B subsub system N 18 Gerrit Muller SPFrecursion

19 99% of Organization has a Parts Focus engineering knowledge parts data base procurement system specification engineering production procedures production system design qualification procedures installation source data system documentation quality assurance lifecycle support knowledge DB doc DB CAD SCM ERP PDM past experience project documents mechanical electrical design database source code management resource planning, e.g. SAP product data management 19 Gerrit Muller SPFengineering

20 10%? Understands Dynamic Behavior or Functionality image from database interpolate Look up table invert contrast / brightness graphics merge colour LUT monitor spatial enhancement bi-linear bi-cubic output brightness contrast legend input SW HW 20 Gerrit Muller MICVpresentationPipeline

21 Few Understand Key Performance Parameters Systems Engineering: responsible for customer key drivers and key performance parameters of system process quality throughput reliability mechanical engineering mechatronics electrical engineering optics measurements embedded control software engineering 21 Gerrit Muller SCSEsystemsEngineering

22 Typical Order of Integration Problems 1. The (sub)system does not build. 2. The (sub)system does not function. 3. Interface errors. 4. The (sub)system is too slow. 5. Problems with the main performance parameter, such as image quality. 6. The (sub)system is not reliable. 22 Gerrit Muller SINTproblems

23 Solutions: Integration of Multiple Products integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 23 Gerrit Muller SIRKlogoSolutions

24 Example 3: Integrated ClinicalSolutions Integrated Clinical Solutions: integrate stand-alone products to offer clinical integrated functionality MR "MPR" URF CT CT Vascular Recepti on Reading Room Office Note the similarity with Kongsberg Maritime s achievements with K- master and operator stations Radiologist at home Radiologist at other hospital Radiologist somewhere in the hospital Printer RAD RAD RAD teaching Radiology department Referring Physician Referring Physician Operating theatre trauma room Conference room IT infrastructure in basement 24 Gerrit Muller SIRKintegratedSolutions

25 The Information Model Swamp takes more than 5 years Every application, release, product, product family, and vendor has its particular interpretation of information, despite standardization. Convertors, wrappers, and adapters are nearly everywhere. The cynical name of our product was Shit Concentrator since the integrating product has to resolve any inconsistency standardization stack high innovation rate global standardization cardio analyse CT MRI Siemens ACR/NEMA high interoperability bolus chase cardio vascular GE vascular analyse URF DICOM medical imaging Philips RF legend applications product family vendor world standard 25 Gerrit Muller SIRKinformationModels

26 Risks of Near Identical Data Models URF monitor output: fixed size letters at fixed grid Workstation workstation tumor> tumor> X-ray system MRI scanner legacy systems other rendering causing a dangerous mismatch between text and image multiple near-identical data models with near-identical interpretations 26 Gerrit Muller SIRKfontRendering

27 Role of Software integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 27 Gerrit Muller SIRKlogoRoleSoftware

28 Software Characteristics and Role quantified properties productivity: images per hour speed: 100 frames/second max latency: 50ms max down time: 4 hrs/year SW determines and limits properties dynamic behavior control function function function SW defines functionality and dynamic behavior captures applications conducts all technologies parts subsub system A atomic part system subsystem 1 subsub system B atomic part subsub system N atomic part SW has its own partitioning in e.g. components, units SW has zero delivery time production is costless is ideal to solve last minute problems subsub system A subsystem n subsub system B subsub system N SW is abstract and intangible is alien to physical engineers 28 Gerrit Muller SIRKpartsFunctionsQualities

29 Hardware and Software Typically Meet at the End SW unit 1 SW unit 2 SW unit 3 SW unit 4 SW component A SW component B SW (sub)system typically, SW uses old hardware or stubbed hardware system HW el. 1 HW el. 2 HW el. 3 HW el. 4 HW module A HW module B HW (sub)system typically, HW uses old, vendor-specific, or special software Segregation of hardware and software is a typical organizational problem. Such segregation ignores close coupling of hardware and software. Erroneous assumptions about hardware are discovered late. Key performance parameters are visible late. 29 Gerrit Muller SIRKintegrationHWSW

30 User Behavior is a.o. Determined by environmental factors social status education personal factors relation family group influence fashion culture taboo cultural mental status trauma emotional status physical status allergy handicap religion location taboo time preferences taste 30 Gerrit Muller ETexperienceFactors

31 Role of Users Users: are autonomous behave under influence of internal and external drivers are creative solve problems have limited knowledge of the system have limited insight in their impact on the system Users do the unexpected 31 Gerrit Muller SIRKusers

32 Today in Kongsberg integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 32 Gerrit Muller SIRKlogoToday

33 Errors Found after Functional Analysis and Quantification PID before PID after changed due to functional analysis and quantification from Knowledge Capture, Cross Boundary Communication and Early Validation with Dynamic A3 Architectures by Vickram Singh 33 Gerrit Muller SIRKdesignErrorsDR

34 Analysis of Subsea System Test design base functional spec System requirements verifying and validating the system 8,12,17,64,99, 113 System Test 8,17,99 subsystem spec modes of operation XX = event no. detail design spec 17,64 Subsystem requirements 8,17,64,99, 113 XX XX 8,12,17,64 Product requirements 12,17,99, 113 Where it was specified Where it was tested verifying and validating the subsystem verifying and validating the product XX XX Factory Acceptance Test 8,12,17, Extended Factory Acceptance Test 8,17,64,99, ,64 Where it should have been specified Where it should have been tested from master project by Åke Törnlycke and Rune Henden, FMC, Gerrit Muller SIRKsubseaVmodelAnalysis

35 Reflections on Systems Integration integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 35 Gerrit Muller SIRKlogoReflections

36 Imperfect Processes Outside world Customers Lifecycle support Specifications Design Technology People Process result and delivery oriented artifact oriented (documents!) check mark syndrome 36 Gerrit Muller SIRKreflectionsProcess

37 Imperfect People Outside world Customers Lifecycle support Specifications Design Technology People Process see only a small part of the big picture are unaware of their blind spots are adaptable and intelligent 37 Gerrit Muller SIRKreflectionsPeople

38 Imperfect Technology Outside world Customers Lifecycle support Specifications Design Technology People builds on math, physics, etc. even experts do not understand all vendors may supply it Process 38 Gerrit Muller SIRKreflectionsTechnology

39 Imperfect Design Outside world Customers Lifecycle support Specifications Design Technology multi-disciplinary many faceted (parts, functions, qualities) People Process 39 Gerrit Muller SIRKreflectionsDesign

40 Imperfect Specifications Outside world Customers Lifecycle support Specifications Design Technology are never complete are often polluted with solutions are often internally inconsistent tend to lack sharpness People Process 40 Gerrit Muller SIRKreflectionsSpecifications

41 Imperfect Lidecycle Support Outside world Customers Lifecycle support Specifications many lifecycles many stakeholders many rhythms Design Technology People Process 41 Gerrit Muller SIRKreflectionsLifecycle

42 Imperfect Customers Outside world Customers Lifecycle support Specifications complicated environment politics do not know what they need do the unexpected Design Technology People Process 42 Gerrit Muller SIRKreflectionsCustomers

43 Imperfect Outside World Outside world Customers Lifecycle support Specifications social complexity (humans) natural complexity interaction between natural and artificial world Design Technology People Process 43 Gerrit Muller SIRKreflectionsOutsideWorld

44 Without Measures it only gets Worse... trends required time to market increased integration features performance expectations amount of software globalization use globalization in development and logistics increased black box re-use increased black box complexity integration surprises 44 Gerrit Muller SIRKtrends

45 Wrap-up integration examples theory viewing performance fundamentals of integration 7 great presentations 2 acquisition performance fundamentals of systems engineering solution information physical role of software and users skip; too little time 2000 products/projects skip; too little time gas turbines 5 subsea oil&gas reflections on systems integration wrap-up 45 Gerrit Muller SIRKlogoWrapUp

46 Conclusion on Reflections Outside world Customers Lifecycle support Specifications Design plenty of imperfections! Technology People Process 46 Gerrit Muller SIRKreflectionsConclusion

47 How to Counter all of this? Outside world Customers Lifecycle support Specifications Design Technology People Process plenty of imperfections! Fail Early: proof key performance ASAP use partial integrations trends increased integration features performance expectations amount of software required time to market globalization use globalization in development and logistics increased black box re-use increased black box complexity integration surprises Improve System Development: modeling, analysis, tools process, people Focus on Systems Engineering 47 Gerrit Muller SIRKwrapUp