Governance in the Age of Algorithms, Cognitive Computing, & Systems of Systems 1

Transcription

1 Governance in the Age of Algorithms, Cognitive Computing, & Systems of Systems Prepared by Kevin C. Desouza 2 Foundation Professor, School of Public Affairs, Arizona State University Nonresident Senior Fellow, Brookings Institution kev.desouza@gmail.com For BIG Ideas October 206 Dublin, OH BIG Ideas Sponsors: The opinions expressed here are my own and do not represent official views of Arizona State University or the Brookings Institution. These views also do not represent official views of any organization that funds or participates in any of my research projects. Kevin C. Desouza is a Foundation Professor in the School of Public Affairs at Arizona State University. For four years, he served as associate dean for research for the College of Public Service and Community Solutions. Dr. Desouza is also a Nonresident Senior Fellow at the Brookings Institution. He holds a visiting professorship at the Faculty of Economics, University of Ljubljana. Previously, Desouza held faculty and research positions at the University of Washington, Virginia Tech, London School of Economics and Political Science, University of the Witwatersrand, and Accenture, among others.

2 This document is prepared to stimulate your thinking for the opening panel of BIG Ideas 206. It is our hope that you read this document, think about how some of these issues might impact your community and/or organization, and bring your questions and ideas to Dublin, OH. As we think about the future of governance and systems of governance, one thing is clear technological advancements will render our current methods and approaches obsolete. Algorithms will have much to say when it comes to deciding how public services are delivered. Today, all social platforms (e.g., Facebook) have evolved from being purely social to being arbitrators of how citizens consume news. Cognitive computing systems that learn are being deployed in almost all sectors of our society from healthcare to law enforcement and education. These systems are on track to outperform humans on many tasks, resulting in a whole slew of both intended and unintended consequences that will reshape how we think of organizations, service delivery, and communities. We are no longer living in an age of isolated systems. Today, systems are connected at multiple levels and across multiple environments resulting in an ecosystem of systems systems of systems (SoSs). The challenge of governing SoSs is intensified by the fact that right now, these systems exist entirely in the private domain. These systems are rarely subject to a discussion of how public values are accounted for and who is to be held accountable in the case of failures. Our level of dependency on SoSs to perform is rarely matched with an adequate level of curiosity into how these artifacts are designed, managed, and cared for. We need a thoughtful conversation about algorithms, the nextgeneration systems and SoSs they will create and propagate if we have any chance of ensuring that these technologies are deployed responsibly and in a manner that will benefit our communities. Having a thoughtful conversation will require us to unpack technology as a concept and examine its finer components (e.g., algorithms) along with the underlying mechanisms and outcomes they generate. In this brief commentary, I want to provoke your thinking on a handful of salient issues that need to be debated as we think about the future of technology and the precarious nature of governing in a fully technology-mediated and technology-driven society. Algorithms that are the backbone of software systems are being designed to be adaptive, intelligent, and social. As one example, algorithms on social network platforms are trained to increase user engagement on the platform (after all that is how these platforms generate revenue). As such, the manner in which information is shared and displayed in these environments raises interesting dilemmas from a public policy and civic engagement perspective. Research has already shown that the algorithms used to display news content are more likely to share news that already supports a predisposed position of the individual (i.e. you get data that already confirms what you thought of the world), which results in increased polarization. In addition, these algorithms learn from interactions (i.e. every click you make and how much time you spend on a particular information element, who you share information with, what combination of search words you use, etc.). Recent research has demonstrated that adaptive and social nature of algorithms result in some disturbing outcomes. For example, Google s online advertisement selectively showed high-paying jobs to men compared to women. Research has also shown us that arrest records are more likely to show up on searches involving distinctively African-American names. Currently, our methods for detecting these biases upfront are limited. The algorithms that go into these systems are rarely inspected 2

3 from an ethical or policy perspective. To do so, developers of these systems will need to open them up for inspections, which poses its own challenges because it is quite contrary to how these systems are currently being developed and deployed. The nature of our computing systems continues to radically evolve from simple computation machines to machines that can learn and adapt. IBM s well-known Watson is just one of the many cognitive computing environments that are already in deployment. But have you ever wondered how these systems learn and process data? These systems live off of the ability to process large quantities of data and learn from experts. Simply put, they learn through various machine intelligence techniques that either employ supervised or unsupervised learning. However, there is a limited ability to easily trace the logic that goes into these systems because of the complex nature of the underlying mechanisms by which they process data to arrive at their recommended solutions for humans to consider or to implement autonomously. Put differently, if you have every built formula in Excel, you can use the trace function to look at what components went into an equation and make corrections as needed. This ability to audit increases our confidence in the systems and their outputs. Now, think of next-generation systems that are processing much more data than Excel can handle and are working on more complex tasks. What happens when we lose the ability to trace and audit these systems? How do we ensure that these systems perform to function? Complicating matters is the fact that these systems will require new approaches to how we acquire technology in the public sector. The costs of housing these systems within every local government community will be prohibitively high. In addition, these systems are never finished, so new models of financing are needed for new conditions under which they are delivered, maintained, and rented. The renting of solutions and cognitive computing systems will raise thorny data governance issues. The nature of our next-generation computing systems will impact how citizens interact with public services and public agencies. Consider this example: in September of 205, a UK programmer launched a cognitive computing inspired bot in London, which allows residents to contest their parking tickets. By February, more than $3 million traffic ticket appeals were initiated through the bot. So far, the bot has successfully appealed 64 percent (60,000 of 250,000) of parking ticket cases both in London and New York. While the benefits of cognitive computing systems can be seen through increased efficiencies, real challenges exist in optimizing these systems and integrating them into the workforce. Human experts must invest time in training cognitive computing systems. More generally, the future from a workforce development perspective needs to be considered. First, who we hire into our public workforce will require some rethinking. Are we hiring people today with the mind that they will be augmented with technology in the future? What tasks are we hiring people for? How are we assigning our current experts so that their knowledge can be captured within these systems and documented for future input? What are the knowledge transfer procedures in place? These are just some of the questions that will need to be debated. Ultimately, think of a world where, just like in our current financial markets, most of the activities (the equivalent of trades) are done between machines without a large amount of human interventions. Public agencies need to start engaging various stakeholders (e.g. businesses, educational institutions) to understand the skills and capacities needed to train the future workforce generation to thrive in the age of cognitive computing. 3

4 Today, our systems are tightly coupled and linked giving rise to SoSs. The upside is that we get a linked (or connected) experience as we move across systems and environments. We do not need to re-enter information within each system, we can just link our different accounts. Of course this is not without risk; SoSs are highly vulnerable to attacks that result in cascading failures across the ecosystem. Consider the case of autonomous vehicles. Many car companies are developing software to automate features to improve customer experiences and reduce accidents. Tesla, for instance, is experimenting with the Summon feature that allows users to use their phone for parking the car in the garage and shutting it down. The vehicle s computational system will be linked to your home sensors and those of your security systems. There is only one problem: in the last year there have been several cases where thieves have been able to hack the computational systems of cars and cause havoc. Taken further, autonomous vehicles rely on vehicle-to-vehicle communications, and vehicle-to-infrastructure communications. Recently, the Kaspersky Lab demonstrated that it does not take much to hack into systems that control traffic flows. The bottom line: SoSs are only as good as their weakest link. Unfortunately, recent cases have shown us that the public sector has a long way to go when it comes to securing systems. Other issues that impact our treatment of SoSs stems from the fact that today the private sector is the major investor in next-generation platforms on which SoSs reside. It was the U.S. government who was the principal funder of the Internet, the platform many of us take for granted today. Without U.S. government investment into the infrastructure, much of what we have in place today in the digital world would not exist. At present, public agencies lack the appetite to make fundamental investment into next-generation infrastructure, putting the public sector behind and in a vulnerable position in terms of playing an active role shaping the future of our societies. Think drones; think autonomous vehicles; or even X-Prize, the initiative that led to the creation of the private spaceflight market the public sector has had a reactive, and even confrontational, stance towards these innovations even though they made significant investments into the fundamental science behind these technologies (e.g. the U.S. Department of Defense was one of the first to invest in autonomous vehicles and drones). I have only scratched the surface when it comes to issues that we should consider as we think about the next-generation computational systems in the public sphere. I can go on and point to other issues, such as the rise of digital currencies and the changing role of the public sector in this space. Bitcoins are crypto-currency tokens that are digitally created and traded. They are a peer-to-peer created network of payments that are not regulated by government. Many restaurants and merchants accept Bitcoins as a mode of payment for goods and services. For instance, Bitcoin enthusiasts have created a map that locates the stores and restaurants that accept Bitcoins in Kansas City. Recently, the Polish government has initiated a Blockchain and cryptocurrencies stream project to develop strategies for promoting safer and faster use of Bitcoins. The city of Zug, Switzerland recently started a pilot to accept Bitcoins for payments for public services. Are you ready for Bitcoins? What are the implications for your agency and your community? Let me be clear, whether Bitcoin survives or not is not the issue, the innovation is in how people think about exchanges akin to the impact Napster had on music that led to the current digital music market. If digital currencies are not your cup of tea, have you thought about the dark web? What about how technologies are being implemented to make cities smarter (and 4

5 intelligent) which might actually disrupt communities as they do not fully account for civic and public values and emphasize efficiency rather than effectiveness? The bottom line: I do not want you to be scared about technological innovation. Quite the contrary, I want you to get more engaged with technological innovation. Too often the public sector is underrepresented when it comes to shaping the trajectories of emerging technologies in the early days. We need to increase the level of intellectual discourse on how technologies can be responsibly deployed to advance our communities. We need you to ask hard questions, which I hope you will at the opening panel of the 206 BIG Ideas Conference. See you there! 5