How to Lie With Statistics

Transcription

1

2 Welcome

3 # T C 1 8 How to Lie With Statistics David Sigerson Solutions Consultant Tableau

4

5 Key Takeaway Understand how statistics affect our decision making Engage with the numbers all around us

6 Who Am I?

7 Why Am I Here?

8 Who Are You? and Why Are You Here?

9 This Talk Inspired By

10 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

11

12 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

13

14 Truth isn t truth Rudy Giuliani

15 Voted #1 presentation!

16 80% would watch again!

17 4 out of 5 women would recommend to a friend

18 Chair of the UK Statistics Authority, Big Red bus Sir David Norgrove This confuses gross and net contributions. It is a clear misuse of official statistics

19 Often, We Don t Notice Being Lied To

20 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

21 Pre-Attentive Visual Attributes

22 The 3D Pie

23 Verna 56,486

24

25

26

27

28

29 The Picture Always Wins

30 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

31

32 What s More Likely? Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? Linda is a bank teller. Linda is a bank teller and is active in the feminist movement.

33 Bias: Anchoring

34 Anchoring Is All Around Us

35 Bias: Framing

36 Framing Positive Frame Negative Frame The product has been proven effective in 80% cases The product has failed to work on 2 out of every 10 cases 75% lean meat 25% fat meat 9/10 customers fully satisfied 10% of our customer not fully satisfied 10 discount on a new phone Avoid 10 surcharge on new phone Gym membership 1.37 per day Gym membership 500 per year

37 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

38 Mean, Median and Mode Mean Sum([Sales]) / Number of Records Median Mid value of your ranked data

39 Mean, Median and Mode Median great for hiding inequalities Mean great for making things seem better or worse if you have inequality in the data

40 Salaries What s the average increase of salary over two periods of +100% and -50%? 50% 25% 0%

41 Percentages & The Credit Card You ve got on a credit card with 20% APR, how much before it doubles? (assuming you don t pay anything back?) How long before it quadruples?

42 Pig In A Poke

43 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

44 Data Collection: Surveys

45 What should my next play be?

46 Questionnaires Measurement Error Is Nick Foles too short to play quarterback? (leading, anchoring) Did Nick Foles miss too many chances to be main quarterback (Leading) Where does Nick Foles enjoy drinking beer? (loaded question) Do you believe Nick Foles always give 110%? (jargon / lacking clarity)

47 Data Collection: Confidence in the Data?

48 Always Think About Data You Don t Have Extrapolation / Interpolation / Over-fitting

49 Data Collection: Correlations Happen All The Time

50

51 Data Collection: Machine Learning & AI

52 Data Bias And Selection Issues Impacts AI/ML as much as human analysts, surveys etc.

53 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

54 Statistics in the Media (and even in most supposedly data driven orgs) Narrative Assemble your data

55 Statistics in Data Science And Big Data World Assemble your data Narrative

56 Chocolate & The Misleading Media Journalist John Bohannon (Ph.D in molecular biology of bacteria) Gunter Frank GP (added authority ) 5 men, 11 women aged19-67 Recruited people via Facebook for 150 Euros Looked for correlations in the dataset The International Archives of Medicine pub BioMedCentral Institute of Diet and Health The networks pick it up

57 Where Statistics Really Add Value Hypothesis / Ideas / Beliefs Assemble your data Test the hell out of it / Peer review Narrative The Tableau impact

58 Sharing And Collaboration

59 Eyeballing The Data

60 Auto Recommendations Wisdom of the crowds

61 Trusting Experts, References

62 Empowering People To Question Things

63 Community Feedback

64 Agenda What s the problem? Is there a problem? How can we lie with stats? Visualisations Psychology Calculations Data collection Where do we go from here? Summary

65 We Are The Problem, Not The Math Take responsibility Acknowledge caveman but move beyond Always add context to your numbers Democratise the process for fact checking let others help you!

66 If You Want To Use Statistics To Lie Show things too-good-to-be true, people want to believe! Don t back up your claims with references Bad visuals Confuse with percentages Extrapolate! Use small inappropriate samples Slanted questions/bad data collection Correlate! Correlate! Correlate! Dress up in a doctors coat

67 Understand how statistics affect our decision making Engage with the numbers all around us

68 Coda: Stay Alert For Manipulation

69 100,000 s of Customers Love Tableau

70 Please complete the session survey from the Session Details screen in your TC18 app Most people score the session 5 But if they could, they d score it Did you consciously read this slide?

71 #TC18 Thank you! David Sigerson, Tableau UK

72