Analogy Engine November 2017 Jay Ulfelder Quantitative Geo-Analyst 202.656.6474 jay@koto.ai Mark Pipes Chief of Product Integration 202.750.4750 pipes@koto.ai PROPRIETARY
INTRODUCTION Koto s Analogy Engine can help analysts understand and anticipate developments in geopolitical situations of interest by objectively and efficiently identifying analogous cases from recent history. This research tool applies an algorithm to a rich global dataset to find cases similar to a reference case on political, economic, and social features that are selected and weighted by the analyst. More than 100,000 comparisons are performed almost instantly in every run, and the resulting list of historical analogies offers solid leads to more reliable insight and foresight on cases of current concern. By searching globally and rigorously, the Analogy Engine lets analysts expand their search for signposts, drivers, and plausible futures beyond familiar cases in a fast, reliable, and unbiased way. KOTO BACKGROUND Koto is the national security division of Kensho, a machine intelligence and data-analytics company founded out of Harvard University in 2013 and headquartered in Cambridge, MA and New York City. Kensho is the leading provider of machine learning-based analytics to Wall Street s premier global banks and financial institutions, including Goldman Sachs, J.P. Morgan, Bank of America Merrill Lynch, Morgan Stanley, Citi, and Soros Fund Management. Its lead investors include Google, Goldman Sachs, and In-Q-Tel. To enable its Wall Street-centric products, Kensho distills a vast amount of structured data from unstructured events (e.g. geopolitical developments, natural disasters, commercial product announcements, etc.) and deploys a cloud-based machine-learning engine tailored to analyze relationships between these often-sparse events and financial market data. The result is a capability to forecast likely market movements based on world events. For example, Kensho s models were used to forecast which London-traded equities were most likely to decrease in value after the Brexit vote. In early 2014, In-Q-Tel approached Kensho to explore how the company s expertise could be applied to intelligence problems. In-Q-Tel invested in Kensho in 2015, and that same year the company built and demonstrated an initial capability to use market data to generate anticipatory intelligence on geopolitical events. The success of this research and development effort combined with the unique constraints of developing technologies for the US intelligence and defense communities led Kensho to create Koto, which provides anticipatory intelligence to the US government. PROPRIETARY 2 / 6
METHODOLOGY Analysts and policymakers often use historical analogies as a source of new perspectives that can provide insight and foresight into situations of geopolitical concern. By identifying and exploring similar cases, we can learn a great deal about how those situations typically unfold, what kinds of variations in process and outcome we might expect to see in future iterations, and what factors drive those variations. Analogical reasoning works best when the supposedly similar cases really do resemble the reference case on as many salient dimensions as possible, and when the resulting set of analogies is as large as possible. Unfortunately, most attempts at analogical reasoning fail to meet these conditions for at least one of two reasons. 1. Our background knowledge sharply limits the scope of possible analogizing, geographically and historically. 2. Common cognitive biases often lead us to focus our search for analogies on cases that are either most familiar or most likely to confirm a particular conclusion. Analysts can try to overcome these challenges by searching for analogies in a more rigorous way. If they do, however, they will quickly discover that it is difficult and time-consuming to build and analyze the dataset necessary to perform large numbers of comparisons, and the appropriate metrics are not obvious. Koto s Analogy Engine is designed to search as widely and objectively as possible for similar historical cases, including ones that may fall outside the scope of the analyst s prior knowledge and biases. Analysts continue to direct the process by deciding which of the available structural features and events are most salient to the situation they want to explore, but the tool quickly leads them to a more reliable set of plausible analogies. The lists of historical cases and similarity scores returned by the Analogy Engine are not intended to provide a direct answer to an analyst s research question, but they should spark new thinking and provide strong leads for further research. We envision at least a few ways in which analysts might use this new tool. Tracing how politics unfolded across similar historical cases can help to identify important signposts and inflection points on the path to various outcomes. Where outcomes differ across structurally similar cases, analysts can confidently focus their search for the drivers of that variation on factors they did not include in their initial analogizing. Comparisons of outcomes across similar cases can improve foresight by identifying potential alternative futures and revealing distributions of outcomes. PROPRIETARY 3 / 6
In situations where analysts start their research process with strong assumptions about relevant analogies, this tool offers an efficient way to challenge those assumptions. ALGORITHM The workhorse of the Analogy Engine is an algorithm that measures the similarity of countries at particular points in time across a user-selected set of political, economic, and demographic features. This algorithm treats each country s values on this set of features as a vector in multidimensional space and computes the distance between those vectors. The result is a set of similarity scores that the Analogy Engine uses to generate a ranked list of cases closest to the analyst s country and month of interest. To fine-tune their comparisons, users can also choose to upweight or downweight individual features in those computations. Analysts may also narrow their search to moments that included a specific event, such as a national election, a terrorist attack, or a failed coup attempt. When an event is selected as a criterion, the Analogy Engine filters on the occurrence of that event, only comparing the similarity of historical cases in which it occurred. DATA To enable proper comparisons with past and current cases around the world, each dataset used in the Analogy Engine must satisfy several criteria. Global or near-global geographic scope Historical depth spanning at least a few decades Validity, i.e., the data is a reasonable representation of the thing it purports to measure Reliability, i.e., measurement is consistent over time Regular updates, to enable comparisons with current and recent cases Most of the data used in the Analogy Engine comes from the following sources. World Bank s World Development Indicators International Monetary Fund s World Economic Outlook Database One Earth Future Foundation s Rulers, Elections, and Irregular Governance (REIGN) Varieties of Democracy (V-Dem) PROPRIETARY 4 / 6
Center for Systemic Peace s Major Episodes of Political Violence The unit of observation in the AE database is the country-month, but not all of our sources report monthly values. In situations where only annual values are available (e.g., population size, GDP per capita, corruption index), we assign the annual value to all months within the relevant year. Also, some AE data sources update faster and more frequently than others. For measures that are not always current (e.g., oil wealth), we forward-fill the last valid observation from as far back as three years, as long as the measure represents a structural feature that is unlikely to change abruptly. PLANNED WORK The Analogy Engine is just one part of a larger platform Koto is building to enhance and accelerate analysts workflow with machine-assisted geopolitical analysis. Here are a few ways we intend to improve and expand on this initial piece. More dynamic measures. The Analogy Engine works best when it can run on data that updates monthly in near-real time, but most of the global political or economic datasets available now are only updated once a year, often with a significant delay. To help overcome this problem, we plan to develop dynamic indices of important geopolitical concepts using algorithms that can combine the information in those annual values with features extracted from news stories and other sources of alternative data. These indices will enable more dynamic comparisons of historical cases, and they will allow users to compare the trajectories of similar cases across a larger set of outcomes. Automated reports on analogous cases. As we build out our natural language processing and search capabilities, we aim to give analysts fast access to a rich set of information about events and entities in the analogous cases, accelerating the analyst s research process even further. More events. One of Koto s primary objectives is the efficient production of reliable data on a wide range of geopolitical events. As the array of Koto-made event sets continues to grow, so will the list of event types on which users can analogize. More data-exploration tools. The Analogy Engine draws on a rich global dataset that merges many time series from multiple sources. Analysts might want to explore that dataset in many other ways, from generating and exporting simple charts or summary statistics to performing more complex tasks like anomaly detection and predictive modeling. User feedback. One of the best ways to improve a tool like the Analogy Engine is to hear from users about their experiences applying it. We plan to convene a focus group to elicit this feedback in a structured way, and we welcome suggestions from all users. PROPRIETARY 5 / 6
LEGAL NOTICE 2017 Kensho Technologies, Inc., all rights reserved. Koto is a division of Kensho Technologies Inc. ( Kensho ) is a global analytics company that uses proprietary mathematical algorithms, machine learning, and various computational models to forecast geopolitical events (collectively Kensho Technology ). Kensho Technology is licensed as a research and analysis tool, and is not publicly available. The marks Kensho and Knowledge Graph, as well as this document and its contents, are the intellectual property of Kensho or the respective third parties ( IP ) and are protected by copyright, trademark, service mark, and intellectual property laws. This document has been distributed to specific recipients directly by Kensho, and redistribution or sharing of this document with any other party is strictly prohibited. Any use of the IP not otherwise expressly permitted herein is prohibited without express written permission from Kensho. All rights reserved. Contact Legal@Kensho.com if you have any questions about these restrictions, or contact info@koto.ai for permission to use this document in any manner not otherwise permitted. PROPRIETARY 6 / 6