Alan Turing Institute: May 30, 2017

Transcription

1 Alan Turing Institute: May 30, 2017 Algorithmic Accountability: Design for Safety Ben Founding Director ( ), Human-Computer Interaction Lab Professor, Department of Computer Science Member, Institute for Advanced Computer Studies Member, National Academy of Engineering Photo: BK Adams University of Maryland

2 Interdisciplinary research community - Computer Science & Info Studies - Psych, Socio, Educ, Jour & MITH hcil.umd.edu vimeo.com/

3 The New ABCs of Research Guiding Principles Applied & Basic Combined Science, Engineering & Design

4 Designing the User Interface Design Theories Direct manipulation Menus, speech, search Social Media Information Visualization Sixth Edition: 2016

5 Web links Tiny touchscreen keyboards Photo tagging

6 Spotfire Treemaps FinViz NodeXL EventFlow

7 Wikimedia

8 Obama Unveils BIG DATA Initiative (3/2012) BIG DATA challenges: Developing scalable algorithms for processing imperfect data in distributed data stores Creating effective humancomputer interaction tools for facilitating rapidly customizable visual reasoning for diverse missions. `

9 White House Big Data Strategies (5/2016) - Create next-generation capabilities: foundations & technologies - Support R&D on trustworthiness of data & resulting knowledge better decisions, breakthrough discoveries & confident action - Build cyberinfrastructure for agency missions - Increase value of data promote sharing & management of data - Understand collection, sharing & use privacy, security & ethics - Improve the landscape for education & training - Enhance connections in the innovation ecosystem

10 White House Big Data Strategies (5/2016) - Create next-generation capabilities: foundations & technologies - Support R&D on trustworthiness of data & resulting knowledge better decisions, breakthrough discoveries & confident action - Build cyberinfrastructure for agency missions - Increase value of data promote sharing & management of data - Understand collection, sharing & use privacy, security & ethics - Improve the landscape for education & training - Enhance connections in the innovation ecosystem

11 White House Big Data Strategies (5/2016) - Support R&D on trustworthiness of data & resulting knowledge better decisions, breakthrough discoveries & confident action promote transparency including tools that provide detailed audits to show the steps that led to a specific action.

12 Weapons of Math Destruction Cathy O Neil Opacity, scale & damage These algorithms, slam doors in the face of millions of people, often for the flimsiest of reasons, and offer no appeal. They re unfair. Permission: Cathy O Neil

13 Weapons of Math Destruction Cathy O Neil Opacity, scale & damage These algorithms, slam doors in the face of millions of people, often for the flimsiest of reasons, and offer no appeal. They re unfair. Algorithms can be biased, harmful & deadly! Permission: Cathy O Neil

14 Designing the User Interface Balancing automation & human control First Edition: 1986

15 Designing the User Interface Ensuring human control while increasing automation Sixth Edition: 2016

16 Apple Design Guidelines User Control people - not apps - are in control. Flexibility users complete, fine-grained control over their work.

17 PNAS Opinion: To mitigate the dangers of faulty, biased, or malicious algorithms requires independent oversight (November )

18

19 Algorithmic Transparency & Accountability 1. Awareness: Owners, designers, builders, users & other stakeholders should be aware of possible biases and harm

20 Algorithmic Transparency & Accountability 1. Awareness: Owners, designers, builders, users & other stakeholders should be aware of possible biases and harm 2. Access and redress: Regulators should adopt mechanisms that enable questioning & redress

21 Algorithmic Transparency & Accountability 1. Awareness: Owners, designers, builders, users & other stakeholders should be aware of possible biases and harm 2. Access and redress: Regulators should adopt mechanisms that enable questioning & redress 3. Accountability: Institutions should be held responsible even if it is not feasible to explain how the algorithms produce their results

22 Algorithmic Transparency & Accountability 1. Awareness: Owners, designers, builders, users & other stakeholders should be aware of possible biases and harm 2. Access and redress: Regulators should adopt mechanisms that enable questioning & redress 3. Accountability: Institutions should be held responsible even if it is not feasible to explain how the algorithms produce their results 4. Explanation: Systems & institutions that use algorithmic decisionmaking are encouraged to produce explanations of the procedures & decisions

23 Algorithmic Transparency & Accountability 5. Data Provenance: Algorithm builders should maintain a description of how training data was collected

24 Algorithmic Transparency & Accountability 5. Data Provenance: Algorithm builders should maintain a description of how training data was collected 6. Auditability: Models, algorithms, data & decisions should be recorded so that they can be audited

25 Algorithmic Transparency & Accountability 5. Data Provenance: Algorithm builders should maintain a description of how training data was collected 6. Auditability: Models, algorithms, data & decisions should be recorded so that they can be audited 7. Validation & Testing: Institutions should use rigorous methods to validate their models

26 Designing the User Interface Ensuring human control while increasing automation Sixth Edition: 2016

27 PNAS Opinion: (November ) To mitigate the dangers of faulty, biased, or malicious algorithms requires independent oversight

28 Corporate internal audit committees & advisory boards & External audits required by SEC University Accreditation, NSF & EPSRC Advisory Boards Zoning Boards, Planning Commissions, Environmental Impact Statements NASA, FAA, FDA, DHS, Federal Reserve, etc. ALGORITHMS

29 Planning oversight

30 Continuous monitoring Federal Reserve

31 Retrospective analysis NTSB Tweet: 6:25 PM - 7 Jul 2013

32 - Degree of Independence, subpoena power - Powers to enforce recommendations PNAS Opinion: (November ) To mitigate the dangers of faulty, biased, or malicious algorithms requires independent oversight

33 Designing the User Interface Ensuring human control while increasing automation ALGORITHMS Sixth Edition: 2016

34 Clarifying responsibility accelerates quality Independent oversight: Transparency: Accountability: Liability: Open adversarial reviews Open the black box Open failure reporting No hold harmless contracts