SELF-EXPLAINING AUTONOMOUS SYSTEMS Werner Damm

Size: px

Start display at page:

Download "SELF-EXPLAINING AUTONOMOUS SYSTEMS Werner Damm"

Scarlett Garrett
5 years ago
Views:

1 SELF-EXPLAINING AUTONOMOUS SYSTEMS Werner Damm Interdisciplinary Research Center on Human Cyber Physical Systems Managing University Other Participating Organizations Institutions as Cooperation Partners Stadt Oldenburg, Klinikum Oldenburg, Hanse Wissenschaftskolleg

2 THE APPLICATION DOMAINS

3 The operating Room of the Future

4 The future Energy System 4

5 The Future Mobility System 5

6 A sample Scenario Paul always loved the fun of driving. Yet, being 76 as he was, he had acquired multiple chronic diseases, which caused him to buy one of these new cars which can actually take over driving whenever you want it. This particular car he picked was marketed as "The Car that Cares"- it could actually identify a number of critical health states with its inbuilt sensor systems and driver monitoring systems and could be personalized to his individual health state by allowing the car to take over when such critical health states were detected, which would drastically impair his driving capabilities. He just loved the dynamics and drag of the car to unfolds under his guidance, but also several time tested its fully automatic modes.

7 Because the car kept him in the loop by telling him about why it handled this critical situation or that road conditions so well, he was able to build trust in his car and thus decided to now capitalize on the car that cares by personalizing it to his particular health needs, such as his COPD. His medical records were transferred to the car, and he authorized the car to take over control in asthmatic attacks, and to forward his medical records electronically to authentified doctors. A sample scenario (cont.) Almost two years later he encountered a pulmonary attack while driving on this beautiful highway, which the car picked up, proposing this diagnosis of his health state, and asking him to confirm. He was so relieved to recall that all it took to be safe now was to confirm - the car would now take care of him, issuing an emergency call broadcast

8 and negotiating with surrounding cars and the truck platoon on the right a safe trajectory for taking the car of the highway. At the same time, the emergency broadcast was picked up by an ambulance close by on the highway, transmitting the call to Dr. Brenner monitoring a patient on this hospital transfer. Dr. Brenner immediately connected to Pauls car with his credentials, allowing the car to pass on all health information about the current asthmatic attack, as well as health A sample scenario (cont.) and also provide a video image of Paul allowing Dr. Brenner to confirm the diagnosis. He ordered the ambulance to perform a maneuver putting the ambulance right behing Pauls car. Rushing out with oxygen ventilators, he came just in time to prevent an otherwise deadly outcome of the asthmatic shock state Paul had reached.

9 THE TECHNOLOGY BEHIND THIS

10 Increasing level of autonomy = delegation of decision making from humans to technical systems National Roadmap Embedded Systems ZVEI, VDI, SafeTRANS Chairman of Editorial Board W. Damm 10

11 National Roadmap Embedded Systems ZVEI, VDI, SafeTRANS Chairman of Editorial Board W. Damm 11

12 AUTONOMOUS CYBER-PHYSICAL SYSTEMS large scale orchestration of hundreds. thousands.millions. billions of physical systems made intelligent through embedded systems and hundreds. thousands. of humans cooperating with these using web-services An artificial living organism the SWARM global awareness highly intelligent acting globally AND WE ARE BUT ONE SMALL PART OF IT

13 Market Pull McKinsey estimates that the on-going digitalization will add an additional 1.5 Trillion US $ to the US GDP by 2025, and 1 Trillion EUR to the GDP in Europe Soure: James Manyika, Michael Chui, Peter Bisson, Jonathan Woetzel, Richard Dobbs, Jacques Bughin, Dan Aharon: The Internet of Things: Mapping the value Beyond the Hype, McKinsey Global Institute, June 2015

14 THE SOCIETAL RISKS OF ACPS

15 Critical Issues I given that the very enabler for this boost is their increasing "smartness : Do we know what decisions are taken when and for what reasons?

16 Critical Issues II Can I assess at all, whether the rational underlying such decision-making is compliant to our societal values and consistent with my own value system?

17 Critical Issues III Will, ultimately, global system companies determine value systems globally by tightly integrating the company s proprietary ethical code into their products?

18 Critical Issues IV As the levels of smartness are pushed by integrating self-learning systems, enabling them to carry out tasks never programmed explicitly by humans, how can we at all ensure, that their emergent behavior is compatible with the norms and principles of our society?

19 THE SEE APPROACH: SELF-EXPLAINING AUTONOMOUS SYSTEMS

20 The SEE Vision To provide transparency in decision making in large systems of autonomous cyber-physical systems, thus fostering their individual and societal acceptance across individuals of wildly varying skill levels and cognitive abilities as well as diverse societal groups, enabling their societal, democratic, and legal control, and safeguarding and optimizing autonomous decisions through high-level interaction with humans

21 The SEE Mission SEE researches into how to make decision-making of autonomous systems transparent and enable debating perceptions and strategies between humans and ACPS. We solve the challenge of providing capabilities for human-understandable justifications and debate of decisions at any relevant layer of decision-making in ACPS. We create design paradigms for building socially accepted self-explaining ACPS with demonstrated societal and industrial relevance. In addressing this challenge in an interdisciplinary approach and in the presence of increasing levels of learning and self-reflection, we provide the scientific, sociological, and ethical basis for further increases in levels of automation of ACPS while paving their way for societal acceptance.

22 Justifications Those facts about the real world in which the system is acting and those predictions about its future evolution under the proposed and alternate moves of the ACPS which the ACPS used to make its own choice for the suggested move

23 Feb 16 resolution of European Parliament Highlights the principle of transparency, namely that it should always be possible to supply the rationale behind any decision taken with the aid of AI that can have a substantive impact on one or more persons lives; considers that it must always be possible to reduce the AI system s computations to a form comprehensible by humans; considers that advanced robots should be equipped with a black box which records data on every transaction carried out by the machine, including the logic that contributed to its decisions; European Parliament resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics (2015/2103(INL))

SEE Challenges I observations of the real worlds by an ACPS are inherently incomplete hence ACPS does not know facts about real world ACPS has only beliefs about real worlds, based on sensors

24 SEE Challenges I observations of the real worlds by an ACPS are inherently incomplete hence ACPS does not know facts about real world ACPS has only beliefs about real worlds, based on sensors communication with other ACPS connection to cloud / data base human input All these can be subject to failures cyber attacks poor quality (too old, too imprecise, ) Scope of Research Area 1: Design Principles and Methods for Self- Explaining ACPS

25 SEE Challenges II methods used for predicting future evolutions are typically based on statistical models and what if in current situation the actual evolution of the system differs: statisical model not validated in current system state unidentified common modes rare events when decisions are based on learning algorithms, there are currently no methods available to generate such justifications Scope of Research Area 1: Design Principles and Methods for Self- Explaining ACPS

SEE Challenges III the sheer amount of data involved in such justifications can not possibly be processed by humans Need to identify the most relevant aspects of beliefs Relevance depends on

26 SEE Challenges III the sheer amount of data involved in such justifications can not possibly be processed by humans Need to identify the most relevant aspects of beliefs Relevance depends on criticality of state of physical system e.g. how close to collapse / accident / loss of grid state of human(s) interacting with ACPS e.g. stress level, skill level,... How can we communicate such relevant aspects across the HMI in a way focussing the attention of the human on the most critical aspects? How can humans debate such justifications? Scope of Research Area 2: Communicating and Debating Justifications

27 Challenges IV Scope of Research Area3: Ethical, Societal, and Legal Principles of Self-Explaining ACPS how to assure by design that justifications are compliant with societal norms, ethical principles and regulations from post mortem analysis to first-class principles observed throughout all phases of design process assessing trust in ACPS do justifications build or destroy trust? can algorithms for conflict resolution take decisions with ethical dimensions e.g. utilitarian measures: minimize overall damage how to create justifications respecting privacy and IP

SEE Challenges V how to create industrial impact paving the way for industrial take up we demonstrate that SEE concepts can be turned into actual innovations how to create societal impact can we

28 SEE Challenges V how to create industrial impact paving the way for industrial take up we demonstrate that SEE concepts can be turned into actual innovations how to create societal impact can we demonstrate societal acceptance of SEE concepts? impact on norms, standards, regulations, laws Scope of Research Area 3: Ethical, Societal, and Legal Principles of Self-Explaining ACPS Scope of Research Area 4 Generating SEE based Innovations building on Living Labs Virtual Organization

29 Research Areas RA 1: Design Principles and Methods for Self-Explaining ACPS RA 2: Communicating and Debating Justifications RA 3: Ethical, Societal, and Legal Principles of Self-Explaining ACPS RA 4: Generating SEE based Innovations

30 30

Basic research Applied research Cross organizational

applications Cutting edge simulator and brain imaging

360 deg driving Whole head fnirs simulator in driving

development and integration of Driving scenarios that allow

31 Basic research Applied research Cross organizational experimental basis for Human in the Loop automotive applications Cutting edge simulator and brain imaging infrastructure MRI-scanner with driving simulator hardware 360 deg driving Whole head fnirs simulator in driving simulators Coupled multi-subject driving simulators Enables development and integration of Driving scenarios that allow for social and cultural sensitive decision making. Realistic driving simulation in MR-scanner.

32 Living Labs I The operating room of the Future In this Living Lab, patient simulators (full scale minikins) can be used to simulate various medical situations in anesthesia, intensive care and emergency medicine. The entire technology - team interaction can be monitored and analyzed. Concepts will be developed on how the relevant information can be prepared in such a way that it is perceived in a timely manner and interpreted correctly by the medical staff It is planned to set up the Living Lab as a simulation center in previous operating rooms of the Klinikum Oldenburg, in a close cooperation with the University Clinic for Anesthesiology, Intensive Care Medicine, Emergency Medicine and Pain Management. The Living Lab brings together the competences Medical Informatics, Medical Device Engineering, Patient Safety / Human Factors (including technical, social and psychological factors), medical science and clinical practice.

33 Living Labs II: The Smart Energy Living Lab The Smart Energy Living Lab comprises the Smart Energy Simulation and Automation Laboratory (SESA-Lab in full operation since the beginning 2013) research infrastructure of the University of Oldenburg and OFFIS, which integrates hardware-in-the-loop cosimulation, energy data analytics together with SCADA/DMS control system interaction for investigating large-scale energy system scenarios, governed by communication-reliant (i.a. human decision) processes. It offers a unique laboratory setup for investigating the dynamics and system effects of communication-reliant sociotechnical energy systems the SESA-Lab is about to be significantly expanded to simulate and research cyber-security aspects and threat scenarios targeting end-user systems as well as key infrastructure components (e.g. station/substation automation systems, control rooms)

34 Living Labs III: Highly Automated Driving in Urban Environments This Living lab integrates the Application Platform for intelligent Mobility (AIM in full operation since the beginning 2014 in Braunschweig) and the Testbed Lower Saxony (TLS the TLS extends AIM to the metropolitan area tensed by the cities Wolfsburg, Braunschweig, Hannover, and Hildesheim and will be available in 2018 for practical use). AIM and TLS provide an infrastructure for a wide range of research and development activities in the area of intelligent mobility services.. For this purpose, AIM and TLS provide virtual environments and very different simulation-based tools, driving simulators, test beds, dedicated test tracks and test tracks implemented in the city of Braunschweig (e.g. providing tools for very detailed traffic observation as well as for communication in the sense of Car2X), a fleet of research vehicles (in particular, for automated and connected driving), an ICT-infrastructure etc. All of these components can be combined to generate multiple options for their integrated application

35 Living Labs IV: Distributed Collision Avoidance in the German Bight The Maritime Living Lab provides a seamless experimental laboratory and testbed ranging from an open co-simulation environment to an in situ physical testbed at the German Bight. It covers maritime traffic, environment and sensor system simulators as well a physical test bed with ship and shore equipment. The physical experiment and test bed offers a instrumented reference waterway in the river Elbe mouth the research boat Zuse, an mobile integrated navigation system / full bridge and a vessel traffic services (VTS) - System for traffic surveillance and control It is part of the emaritime Integrated Reference Platform emir, a national industrial initiative for maritime safety.

36 Living Labs V: Smart City Lab Oldenburg The Fliegerhorst urban development area in Oldenburg offers demonstration capabilities form smart energy and smart health applications and autonomous vehicles based services. The so-called Smart City Living Lab Fliegerhorst builds upon a consensus-driven strategy process between the general public, urban administration and industry partners as well as potential research&development parties. The successful concept of merging above mentioned critical application domains for the sake of a high-quality and sustainable living-environment led to the City of Oldenburg acquiring the 3,9 hectares area a former military airfield in close proximity to the city center and defining a significant part of it as a Living Lab for experimenting on novel integrated automation concepts together with community, industry and research partners.

37 A Cross-Discipline Cross Domain Approach SEE Project Types Concept Creation Projects Concept Assessment Projects Study Projects Foundation Projects Transfer Project Building on Living Labs

IT and Systems Science Transformational Impact on Technology, Society, Work, Life, Education, Training

IT and Systems Science Transformational Impact on Technology, Society, Work, Life, Education, Training John S. Baras Institute for Systems Research and Dept. of Electrical and Computer Engin. University