The Past, Present and Future of Statistical Graphics (An Ideo-Graphic and Idiosyncratic View)

Transcription

1 tablevsgraph (An Ideo-Graphic and Idiosyncratic View) Tables and graphs: Tasks, goals, audience Admit?: Yes Sex: Male Sex: Female Michael Friendly York University Admit?: No VIEWS, London, Nov, 4 Like good writing, effective graphical displays require an understanding of purpose what is to be communicated, and to whom Friendly (99) Tasks and Goals for information display Lookup read off exact numbers Comparisons which is more? Detecting patterns, trends, anomalies Different tables or graphs for different purposes: analysis, persuasion Visual presentation as communication: what do you want to say? what the the audience? Tables vs. Graphs Tables are best suited for look-up read off exact numbers Graphs are better for showing patterns, trends, anomalies, making comparisons VIEWS, London, 4 6 c Michael Friendly vpart challenger Part : Tables and Graphs: Some principles of Graphical Display Estimated Failure probability If I can t picture it, I can t understand it Graphical failures and successes Graphical comparisons and graphical perception Corrgrams: rendering and variable order Effect ordering for data display NASA Space Shuttle O-Ring Failures Temperature (deg F) Baseball data: PC/ order Dimension (7.4%) Albert Einstein Dimension (46.3%) Graphical failure: Challenger disaster What we have here is a failure to communicate Cool Hand Luke Few events in history provide as compelling an illustration of importance of appropriate ordering and display of information. Tables and charts presented to NASA by Thiokol engineers showed data from prior launches ordered by time (launch number), rather than by temperature the crucial factor. VIEWS, London, 4 6 c Michael Friendly VIEWS, London, 4 63 c Michael Friendly

2 challenger challenger Graphical failure: Challenger disaster Graphical failure: Challenger disaster The engineers charts were also remarkable for information muddeling extraneous information (wind), cryptically abbreviated labels, no clear assessment of damage ( blow-by (soot) vs. erosion depth (O-ring damage)). A better display shows all the data, some prediction, and an an indication of uncertainty. It is hard to imagine a launch at 3 F given this graph. NASA Space Shuttle O-Ring Failures..9 Estimated Failure probability Engineers did make the proper recommendatation: O-ring temperature must be 53 F at launch. NASA launch control over-rode the recommentation Temperature (deg F) VIEWS, London, 4 64 c Michael Friendly VIEWS, London, 4 66 c Michael Friendly challenger langren Tufte (997) notes: Graphical failure: Challenger disaster the fatal flaw is in the ordering of the data, the graphics... suggest there are right ways and wrong ways to display data; there are displays that reveal the truth and displays that do not. Thiokol engineers did prepare a graph but it was seriously misleading. (What are the flaws?) 3 5C Graphical success: van Langren s graph of longitude van Langren could have presented these data as a table sorted by date (priority), name (provenance), or value (range) Only his hand-drawn graph shows simultaneously: individual estimates and spacings along the scale associated names, offset to avoid overlap estimated, central value ( ROMA ) and wide variability Number of Incidents 4B 6C 4C 4D 6A Calculated Joint Temperature (F) VIEWS, London, 4 65 c Michael Friendly VIEWS, London, 4 67 c Michael Friendly

3 playfair Graphical success: Playfair s first barchart Imports and exports of Scotland (Playfair, 786) Horizontal, to show the country labels Grouped by country, so imports/exports could be directly d. Sorted by numerical value rather alphabetically by country (as would be done by most statistical graphing software) Graphical comparisons: Baselines Baselines data to model against a line, preferably horizontal Comparing observed and fitted discrete distributions: histogram and hanging histogram Frequency 75 Sqrt(frequency) Number of Occurrences Number of Occurrences See: for hanging histograms and hanging rootograms. VIEWS, London, 4 68 c Michael Friendly VIEWS, London, 4 7 c Michael Friendly Graphical comparisons: Make them easy Visual grouping connect with lines, make key comparisons contiguous Graphical comparisons: Tolerances Left: easier to across Level Right: easier to across Type Tolerances show an acceptable region around a comparison standard Normal QQ plot: Standard vs. Detrended Response Response log 986 salary Deviation From Normal Level A Type B Normal Quantile Normal Quantile See: Type A B Level VIEWS, London, 4 69 c Michael Friendly VIEWS, London, 4 7 c Michael Friendly

4 Graphical comparisons: Small multiples Graphical comparisons: Small multiples e.g., mosaic matrix for quantitative data: all pairwise mosaic plots Multiple, contiguous panels allow differences to be sensitively d e.g., Coplots of log(infant Mortality) vs. log(income) Life Expectancy Admit Admit Reject Admit Reject Life Life Life Life Male Female A B C D E F logimr Male Female Gender Male Female loginc loginc loginc loginc F E Admit Reject F A B C D E F See: D C A B A B C D E Dept Admit Reject Male Female VIEWS, London, 4 7 c Michael Friendly VIEWS, London, 4 74 c Michael Friendly e.g., scatterplot matrix for quantitative data: all pairwise scatterplots Visual codes for Quantative vs. Frequency data 87. Quantitative data: magnitude position along an axis Prestige Frequency data (Friendly, 995): count area Model: (DeptGender)(Admit) Educ Income 5879 Women 97.5 Admit?: Yes Sex: Male Admit?: No A B C D E F Sex: Female Fourfold display for table Admitted Rejected Male Female Mosaic plot for 3-way table VIEWS, London, 4 73 c Michael Friendly VIEWS, London, 4 75 c Michael Friendly

5 aspect smooth Graphical comparisons: Aspect ratios Shape of a plot (height/width) aspect ratio often determines what you can see. Typically chosen by software to fill the graphics device (landscape, portrait) y Smoothing often helps Our eyes can usually see patterns not easily captured in numbers. Sometimes relationships may be too weak to see the trend in a scatterplot. Drawing a smoothed curve helps show the trend. 4 USA Draft Lottery Data 3 E.g., plot with a square frame (aspect ratio=) Is there any evident pattern here? - Draft Priority value Can you see the trend? x 3 4 Brithday (day of year) VIEWS, London, 4 76 c Michael Friendly VIEWS, London, 4 78 c Michael Friendly aspect smooth Smoothing often helps Graphical comparisons: Aspect ratios The same data, replotted with an aspect ratio =.5 y General rule: Choose the aspect ratio so the slopes of connecting lines 45. x Our eyes can usually see patterns not easily captured in numbers. Sometimes relationships may be too weak to see the trend in a scatterplot. Drawing a smoothed curve helps show the trend. Draft Priority value 4 3 USA Draft Lottery Data Draft Priority value 4 3 USA Draft Lottery Data 3 4 Brithday (day of year) 3 4 Brithday (day of year) VIEWS, London, 4 77 c Michael Friendly VIEWS, London, 4 79 c Michael Friendly

6 Corrgrams Correlation matrix displays Corrgrams Variable ordering How to show a correlation matrix for different purposes? (Friendly, ) Render a correlation to depict sign and magnitude (tasks: lookup, comparison, detection) Correlation value (x ) Number Task-specific renderings: Task Lookup Comparison Detection Rendering Number Circle Shading Circle Ellipse Bars Shaded Reorder variables to show similarities: PC or angles (PC/PC) Dimension (7.4%) Dimension (46.3%) Generalizations to partial (R(Y X)), conditional correlations (r ij rest R ) VIEWS, London, 4 8 c Michael Friendly VIEWS, London, 4 8 c Michael Friendly Corrgrams Rendering Corrgrams Baseball data Baseball data: (lower) Patterns vs. (upper) comparison Baseball data: PC/ order Baseball data: (a) alpha vs. (b) correlation ordering (a) Alpha order (b) PC/ order See: VIEWS, London, 4 8 c Michael Friendly VIEWS, London, 4 83 c Michael Friendly

7 effect Corrgrams Auto data Displa Auto data: Alpha order Auto data: PC/ order Effect ordering for data displays Hroom Gratio MPG Length Rep77 Price MPG Gratio Rep77 Rep78 Rseat Trunk Hroom Price Information presentation is always ordered in time, or sequence (a talk, a written paper), in space (a table, or graph) Constraints of time and space are dominant can conceal or reveal the important message. Rseat Rep78 Weight Length Effect ordering for data display (Friendly and Kwan, 3) Turn Trunk Weight Gratio Displa Length Hroom Rep78 Rep77 Price MPG Trunk Rseat Weight Turn Turn Displa Sort the data by the effects to be seen Correlation ordering shows a coherent pattern Size variables positively correlated Gratio, MPG, repair record positively correlated Negative correlations between the two sets Turn Displa Weight Length Rseat Trunk Hroom Price Rep77 Rep78 MPG Gratio Applies to: unordered factors for quantitative data categories of variables in frequency tables arrangement of observations and variables in multivariate displays VIEWS, London, 4 84 c Michael Friendly VIEWS, London, 4 86 c Michael Friendly effect Corrgrams Other renderings Baseball data: schematic scatterplot matrix: 68% data ellipse + loess smooth Multiway quantitative data Effect ordering for data displays Main effects ordering sort unordered factors by means/medians Multiway frequency data Association ordering sort by CA Dim (SVD of residuals from independence) Multivariate displays Correlation ordering for variables Clustering/sorting for observations Different renderings for look-up, comparison, detection of patterns, anomalies! VIEWS, London, 4 85 c Michael Friendly VIEWS, London, 4 87 c Michael Friendly

8 effect Main effect ordering for tables and charts Main effects ordering: Tabular displays Playfair s 786 barchart of imports and exports of Scotland Average yield (over years) by Variety and Site, ordered alphabetically: Good for lookup Bad for seeing patterns, trends, anomalies Table : Average Barley Yields (rounded), Means by Site and Variety Variety Crookston Duluth Site Grand Morris Rapids University Farm Waseca Mean Glabron Manchuria No No No Peatland Svansota Trebi Velvet Wisconsin No Mean VIEWS, London, 4 88 c Michael Friendly VIEWS, London, 4 9 c Michael Friendly Quantitative data: Main effects ordering Enhanced tabular displays Quantitative response data, cross-classified by one or more factors Cleveland (993) Barley yields: varieties 6 sites years 3-way dot plot, varieties and sites sorted by main effects. All sites except one: higher yields in 93 than 93. Anomalous site (Morris) might have had years mislabeled. Trebi Wisconsin No. 38 No. 457 Glabron Peatland Velvet No. 475 Manchuria No. 46 Svansota Trebi Wisconsin No. 38 No. 457 Glabron Peatland Velvet No. 475 Manchuria No. 46 Svansota Trebi Wisconsin No. 38 No. 457 Glabron Peatland Velvet No. 475 Manchuria No. 46 Svansota Crookston University Farm Grand Rapids Barley Yield (bushels/acre) Waseca Morris Duluth Average yield (over years) by Variety and Site, ordered by main effect means: values shaded by (interaction) residual from additive model Yield = Variety + Site Color á la mosaic display: blue for e ij >, red for e ij <. Intensity: e ij > {, } MS E. Table : Average Barley Yields, sorted by Mean, shaded by residual from the model Yield = Variety + Site Variety Grand Rapids Duluth Site University Morris Crookston Waseca Mean Farm Svansota Manchuria No Velvet Glabron Peatland No No Wisconsin No Trebi Mean VIEWS, London, 4 89 c Michael Friendly VIEWS, London, 4 9 c Michael Friendly

9 tukeyway Enhanced tabular displays Yield difference ( y ij = 93 93) by Variety and Site, ordered by year effect difference shaded by value ( y ij > {, 3} ˆσ yij ) Table 3: Yield Differences, 93-93, sorted by mean difference, and shaded by value Variety Morris Duluth University Farm Site Grand Rapids Waseca Crookston Mean No Wisconsin No Velvet Peatland Manchuria Trebi Svansota No Glabron No Mean Barley yield differences: Two-way display Morris dominates the display Residuals, e ij > MS E shown by directed arrows Residual for Velvet at Grand Rapids stands out Yield Difference, No. 457 Glabron No. 46 Svansota Duluth Trebi Manchuria Peatland University Farm Velvet Wisconsin No. 38 No. 475 Crookston Waseca Grand Rapids Negative values for Morris immediately stand out Other differences have lower-triangular pattern - Morris VIEWS, London, 4 9 c Michael Friendly VIEWS, London, 4 94 c Michael Friendly tukeyway Effect ordering for frequency tables Automating main effect ordering: Two-way display Tukey (977) two-way display Show predicted values and residuals in a two-way table Additive model, Y ij = µ + α i + β j + ɛ ij Fitted values, Ŷij shown as rectangular grid at coordinates (x, y), Table 4: Hair color - Eye color data: Alpha ordered Hair color Eye color Blond Black Brown Red Blue Brown Green Hazel x i = µ + α i = row fit i Table 5: Hair color - Eye color data: Effect ordered y j = β j = col effect j Two-way display (45 rotation) plots: (x i + y j )=Ŷij = Fit vs. (x i y j ) scaled to keep rectangular e ij = Y ij Ŷij = Residual shown as vectors Hair color Eye color Black Brown Red Blond Brown Hazel Green Blue Model: Independence: [Hair][Eye] χ (9)= 38.9 Color coding: <-4 <- <- > > >4 n in each cell: n<expected n>expected VIEWS, London, 4 93 c Michael Friendly VIEWS, London, 4 95 c Michael Friendly