Maritime Graphics Visual Computing for Offshore Applications 07.11.2017 Uwe Freiherr von Lukas Fraunhofer-Institut für Graphische Datenverarbeitung - IGD Joachim-Jungius-Str. 11 18059 Rostock - GERMANY Tel.: +49 381 4024 100 Fax: +49 381 4024 199 Email: uwe.von.lukas@igd-r.fraunhofer.de https://www.igd.fraunhofer.de/en 1
Fraunhofer-Gesellschaft, the largest organization for applied research in Europe 67 institutes and independent research units More than 23,000 staff 2 billion annual research budget totalling Roughly two thirds of this sum is generated through contract research on behalf of industry and publicly funded research projects Roughly one third is contributed by the German federal and Länder governments in the form of base funding International cooperation Fraunhofer lines 2
The German Research Landscape Basic research Applied research Industrial Research (Companies) The Fraunhofer- Gesellschaft undertakes applied research of direct utility to private and public enterprise and of wide benefit to society. Max Planck Society Universities Public funding Private funding Source: BMBF Germany (simplified) Joint Research Contracted R&D Licensing Our Offer 3
Subsea@Fraunhofer Multidisciplinary community Itzehoe Rostock 13 involved Fraunhofer Institutes Sensors ICT Material Energy Broad range of underwater technologies test facilities Initiated and coordinated by Prof. von Lukas Gelsenkirchen Oberhausen Euskirchen Aachen Sulzbach Duisburg Köln Wachtberg Oldenburg Münster Dortmund Schmallenberg Bonn Sankt Augustin Remagen Karlsruhe Ettlingen Freiburg Kandern Efringen-Kirchen Frankfurt Pfinztal Bremerhaven Bremen Lemgo Paderborn Kassel Alzenau Aschaffenburg Würzburg Darmstadt Wertheim/ Mannheim St. Ingbert Bronnbach Kaiserslautern Saarbrücken Gießen Stuttgart Esslingen Hamburg Hannover Braunschweig Goslar Göttingen Lübeck Wolfsburg Erfurt Augsburg Weßling Potsdam-Golm Magdeburg Halle Bayreuth Sulzbach-Rosenberg Erlangen Nürnberg, Fürth Freising München, Garching Holzkirchen Regensburg Deggendorf Straubing Rosenheim Prien Teltow Wildau Cottbus Schkopau Leipzig Leuna Dresden Moritzburg Jena Freiberg Zittau Hermsdorf Chemnitz Ilmenau Coburg Berlin 4
Subsea navigator Structuring multidisciplinary expertise One stop shop for subsea technology Our offer Consulting Technologies Systems (e.g. ROV) Complete solutions 5
Our focus: Digital Transformation of the Offshore Industry Rely on digital data in all processes over the whole lifecycle Advantages of digital data Instant access Worldwide availability Flexible usage Benefits for the end users Increased efficiency Higher quality Chance for new business models / revenue streams 6
Cyber-physical Equivalence Foundation for Digital Transformation of the Offshore Industry Production/ operation Sensors Analysis/ Modelling Reale Welt Validation Interact. Visualization Digital Twin Actuators Monitoring Inspection Repair/Overhaul 3D- Printing Simulation/ Prediction/ Planning 7
Applying Expertise in Visual Computing Underwater Image Enhancement 3D Reconstruction Machine Learning Augmented Reality Large Model Visualization 8
Fast generation of 3D interior ship models for AR training Feasibility Study Tango device for mobile 3D capturing and AR application 3D Reconstruction Reconstructed 3D world 3D object library Augmented Reality 9
Underwater computer vision Our goal: Automatic extraction of information out of images or image sequences Challenges Poor quality of available images Real-time requirements Necessary robustness Applications Monitoring and observation Technical structures (piles, quay walls, ) Biology (fish, mussels, algae, Visual assistance Underwater operations (diver, ROV, AUV) 10
Super Resolution for underwater image enhancement Low-resolution Blur estimation High-resolution Low-resolution High-resolution 11
Underwater Video Processing Tool Customer: German Navy Flexible combination of different algorithms for underwater image enhancement Usable as stand-alone tool or as a library for integration Video up to 4k PC (Windows, Linux) or embedded system Hardware acceleration 13
Convolutional Neuronal Networks Fish Detection 5 classes (species) are trained and can be detected in real-time Robust detection even with partial occlusion 14
Offshore Surveillance Using AIS data Pan/Tilt/Zoom- Camera Coupled with AIS- Tracks 3D world as a basis for visualization Alignment of virtual camera and real camera Source: Signalis/Fraunhofer IGD 15
Subsea environmental monitoring Near-realtime monitoring dashboard Post-mortem data analysis Visual forecast of gas leakage situations Fully web-based (HTML5), distributed displays Multi-touch interaction and navigation 16
Exploration of heterogeneous Sensor Data Software solution for a Subsea Lander 17
Interactive Visualization of Massive 3D Product Models Complete ship model with 55 Mio triangles 19
Our strategy: VisualComputing as a Service Powerful platform Initially developed for 3D data in product development (CAD, CFD, ) Now extended for other application scenarios (GIS, city models, sensor data, ) Open interfaces for easy integrat. Supporting all relevant operating systems High scalability 20
Support Data Analysis with Virtual Reality Displays Select appropriate hardware Use our VR software Data preparation Integrate interaction concepts Optional coupling with (physical) simulation 21
22.11.2017 Visual Analytics Overview of massive amounts of data Visual Analytics allows the exploration of extremely large and complex data repositories through interactive user interface The passive observer of condensed analysis results becomes an active user of visual analytics tools to guide the analysis process Combination of the strengths of human and machine 22
Sensor analysis over time Analysis of large amounts of sensor data over time Detection of similar sensor behaviour Combination of visualization and automatic clustering algorithms
Concept for an Underwater Lab in the Baltic Sea Ocean Technology Center Rostock 24
Mariscope Technology Network Mini-ROV Joining forces for developing modular inspection ROV Focal areas Modular platform Sensor integration Propulsion and Energy Manneuovering Data services 10 companies, 4 research institutes Managed by Fraunhofer IGD www.mini-rov.de 26
Technology Network Munitect Developing new technologies for efficient detection of underwater unexploded ordnance device Focal Areas Improving specific sensors Sensor fusion and integration Datenmanagement und -analyse Aufbau reales Munitions-Testfeld 10 comapnies and 5 resarch institutes Managed by Fraunhofer IGD UXO (Estimation) Baltic Sea: 300.000 t conventional 5.000 t chemical North Sea: 1,3 Mio t conventional 90 t chemical