Charlotte Informatics Forum 2012

Transcription

1 Charlotte Informatics Forum 2012 Opportunities and Challenges for Big Data Analytics May 15, 2012 More Information: JOSEPHKIELMAN

2 Digital World

3 Analytics in the Analog World

4 The Problem: Big Data In 2002, recorded media and electronic information flows generated about 22 exabytes (10 18 bytes) of information In 2006, the amount of digital information created, captured, and replicated was 161 EB In 2010, the amount of information added annually to the digital universe was estimated at 1,600 EB (1.6 ZB). In 2011, the Library of Congress collected 230 TB of data each month. Annual global consumer internet traffic was 140 EB in 2010; estimated to rise to about 600 EB in Each month, 30 billion pieces of digital content are shared on Facebook. 95% of the Digital Universe s information is unstructured 25% of the digital information produced in 2010 was images By 2014 file sharing and video will represent about 25% and 35%, respectively, of internet traffic Ref: IDC, UC-Berkeley, Cisco, and McKinsey Global Institute

5 New (Real) Homeland Security Concerns Environmental Stability Slow Catastrophes Resource Constraints Threat (All-Hazards) Awareness Population Dynamics and Ethnic Conflict Financial Stability Terrorism Natural and Manmade Disasters Disease Development Economic Stability Climate Change

6 Data Volumes in 60 Minutes of Emergency Response

7 The Formation and Growth of Visualization and Data Analytics PIE Transition VACCINE & CCICADA ACI RVACs UACs FODAVA NVAC The Idea 2004: NVAC Formed Dec, 2004: USA-UK MOA on S&T signed Nov, 2007: USA-Canada CAA signed Nov, 2009: VACCIN E Kickoff Dec, 2009: PIE Kick-off Feb, 2011: VASA Kick-off and beyond! 2007: IN-SPIRE to IC and PANYNJ 2007: NSF- S&T FODAVA Initiative begins March, 2009: USA- Germany MOA signed Dec, 2009: CCICADA Kick-off Dec, 2010: STARE 2011: IN-SPIRE to ICE BCSC June 2011: ERE CAUSE 2007: COMPSTAT II to PANYNJ 2011: IN-SPIRE to ICE OCIO SRS to PNW

8 Products and Tools

9 Predictive, Prospective Policy Prediction Cognition Organization Integration Aggregation Cognitive/Behavioral Analytics Visual Analytics Graph Matching Evidence Extraction Pattern Analysis Link Discovery Extraction Data Information Knowledge Wisdom Context, Culture, Genes Synthesis Analysis Content Management Connect the Dots Discovery

10 Analytics for Complex Information (ACI) Being Prospective rather than Retrospective means enabling users to harness Massive data, which comes in Multiple modes and Multiple types, through Multiple devices, in Diverse user environments, In order to make decisions in real-time. Dynamic Information Made Actionable In Real-Time

11 Information after Analytics What part is relevant? Am I missing something? What does it mean? When will I need it? How do I use it? Can I trust it? Is it private or personal? How do I convey it to others? When can I take actions with it? And which ones? How will others see it? How should I interact with it?

12 Other Issues with Digital Data Data Archiving How and where to store it? What do we do about clouds, warehouses, and websites? Data Permanence How long will it last? Data Rendering What will we use to run it? Regulatory or Legal Issues How to deal with Digital Rights Management and piracy? Who enforces the copyright?

13 Science and Technology Principles What Is the Problem? Information -There are real-world [resiliency] issues for which the availability and usefulness of information is problematic How Do We Structure the Solution? Utility -The capabilities being developed must address at least four information-related concerns: increasing amounts and diversity of data compounded by multiplicity of users; indeterminate nature of relevance coupled with multiplicity and complexity of [threats] or problems What Is the Value of the Work? Outcome -The improvements being provided must have real-world outcomes or implications for the [homeland security] enterprise NOTE: Please replace words in [ ] with terms appropriate to your industry or affiliation.

14 What We Want/Need/Expect from Analytics The Goal A coherent story using complete, up-to-date information and with which we are comfortable in making decisions But How coherent? How complete? How current? How comfortable? So Risk measures Confidence limits

15 The Future for Analytics Means Incorporating New Features Utility for long-term, large-scale issues Confidence limits or measures of risk Ability to address interacting or cascading effects Real-time view of dynamic, streaming data Reach-back into data streams for incipient relevance System (or holistic ) view Iterative customer/user engagement Acknowledgment or appreciation for human + machine interplay Addressing questions/possibilities rather than rules/prescriptions Accurate timings for PROSPECTIVE view