Computer Science 474 Spring 2010 Data Visualization

Transcription

1 DATA VISUALIZATION The modeling and rendering processes described thus far have been focused on representing and displaying objects and scenes in the world, whether real or imaginary. However, many of the principles and techniques for displaying images can be applied to data other than modeled physical objects. Data collected from satellites can be used for displaying weather maps; fluid dynamic computations can be used to show airflow over a surface; daily stock market activity can drive visual representations of market behavior; etc. Many of the principles of visual perception apply to how to effectively display data for human understanding. Some people distinguish between scientific visualization and information visualization. These will be briefly described, but the focus of the reading is on the more general topic of effective data visualization. Scientific visualization is the representation of scientific data, typically physically based, for purposes of better understanding structure, processes, and scientific principles. Data used in representations and animations may be measured directly using scientific measurement tools such as sensors, probes, or scanning devices. Medical imagery from MRI s and CAT scans is an example of measured physical data used in displays. Alternatively, data may be computer generated based on simulations or computations. For example, animating a bridge collapsing based on finite element analysis or showing how a tornado is formed based on simulating the interaction of air particles. Figure 1 shows an example. Figure 1. Visualizing magnetic field interactions from calculations. Since the early days of computer graphics, researchers saw the potential for using visualization as part of the analysis portion of their scientific investigations to better analyze, understand, and communicate results. As computers have increased in power and capacities, both the amount of data being generated, and the ability to visually display it increased tremendously. In 1987, NSF sponsored a special report on Visualization in Scientific Computing that recognized the potential impact of scientific visualization in a number of disciplines and, as a result, earmarked large sums of money to support and promote the development of visualization techniques at the national laboratories. Today, visualization 1

2 permeates all scientific disciplines and research publications often contain computer generated imagery as part of their scientific process. In contrast to scientific visualization, information visualization typically deals with non physical abstract data. Information such as stock market prices, census data, network traffic, voice messages, textual content, web pages, and multi media data (the so called digital universe ) is expanding at unfathomable rates. By one estimate, the amount of data generated and stored in 2010 is predicted to be 1 ZB (a Zettabyte equals bytes, or a billion terabytes). The challenge is not only how to store and access that much data, but, more importantly, how to make use of it. In other words, how do we change vast amounts of data into useful information? Techniques such as data mining, or automatically extracting patterns from data, are becoming important tools for dealing with the data explosion. Visualization tries to take advantage of the high bandwidth of the human visual system to discover patterns in raw data. As a simple example, assume you are interested in purchasing a new automobile and are concerned with costs. You do some research on cars that interest you, and come up with the following table: While it is easy to pick out certain piece of information, such as the highest city MPG, by scanning the appropriate column, it is virtually impossible to do much correlation between cost and efficiency. However, by doing a simple scatter plot, that graphs the two variables together, much more information can be gleaned as shown in Figure 2. For example, you discover the Honda Insight stands out from the group as significantly better fuel economy at a cheap price. More subtly, you may notice there is a correlation that more expensive cars tend to have worse gas mileage. Every car over 25 MPG is under $30,000, while conversely, the four most expensive cars are all under 20 MPG. From this, you may conclude that people who are willing to pay more for a car are less concerned about gas mileage. 2

3 Additional plotting features such as color coding by manufacture could add insights about efficiency groupings. Data Visualization Figure 2. Scatterplot of MPG versus Cost for selected car models. For purposes of this reading, we will not distinguish between scientific and information visualization, but rather the more general topics of principles and techniques in data visualization. Data visualization can be defined as the application of computer technology to create and interact with a visual representation of data for the purpose of increasing understanding about the data. Some people limit visualization to simply generating pictures from data. However, the most stunning image based on data, if it does not enhance understanding, is simply a picture. As Ben Shneiderman, well known visualization researcher and educator, states The purpose of visualization is insight, not pictures. People use data visualization for a variety of purposes. Searching for a specific value or pattern, such as a local minima or maxima may benefit from a geometric representation of data as a surface. A more general application may be to explore data looking for interesting patterns, without a preconceived idea about what those patterns are. For example, Figure 3 is a simple chart called a sparkline that shows the 2009 season wins and losses for the Colorado Rockies. There is one dot per game. A win raises the Y value for the dot, a loss lowers. When the line crosses the horizontal axis, the team has an equal number of wins as losses. Several patterns can be seen in this simple visualization such as the lengths of winning streaks versus losing streaks, the fact that winning streaks are most often followed by multiple losses, and the general pattern for each month. 3

4 Types of Data and Graphs Figure 3. Colorado Rockies 2009 season sparkline. There are different ways of categorizing types of data. At a basic level, we can classify data as nominal, ordinal, or quantitative. Nominal data are categories of data in which there is no order to the categories. Sex, race, and color are examples. Ordinal data implies there is an ordering to the data such as letter grades, education level, and t shirt sizes. Quantitative data are values that you can perform mathematical operations on such as age, weight, and grade points. In addition to these categories, data is often characterized by dimensionality. That is, the number of values associated with each data point. For example, univariate data, or single value, might simply be a list of ages. Bivariate data might be age and weight. Trivariate and hypervariate (three or more) variables could include factors such as ethnicity, height, hometown, and name. Multivariate data is often put into tabular form for access and manipulation. The most common method for visualizing multivariate data is the graph. Several types of graphs have been developed for viewing different features of data. In general graphs consist of some well defined structure such as axes, scale, and shape; the data as points, lines, areas, or symbols; and some labeling. The earlier examples of sparklines and scatterplots are types of graphs. Bar graphs, line graphs, and area graphs are all common examples people are used to seeing from spreadsheet applications. Graphing one and two dimensional data using such techniques is straightforward. Working with higher order dimensions becomes more challenging. The following presents a couple of possible approaches. As a sample data set for illustration purposes, the batting statistics for the 2009 Rockies starting lineup are used: Comparison bar charts can be used to show multiple values per data point, but can quickly become overwhelming. An alternative is to show stacked bar charts with each variable having it own chart. Figure 4 is an example from the sample data set showing base hits. The axis legend only has to be shown once for the stacked charts, and because the individual vertical scales are different, it is easier to perform relative comparison across the player positions. At the same time, viewing the relative performance of each player can be determined by looking along a single column of all charts. 4

5 Figure 4. Comparison bar chart versus stacked bar charts. Figure 5 shows an alternative representation of multivariate data in which data from each player is shown in a separate star, or radar, chart. Separate variables are shown on different axes arranged in a circular fashion. Different types of patterns can be quickly identified (depending on the data) that can provide general comparisons between players. Figure 5. Star charts of selected statistics by player position. Following the theme of multiple charts, multiple scatterplots allow all pairs of selected variables to be checked for correlations. The scatterplot matrix shown in Figure 6 quickly shows from the pattern of dots that there is a much stronger correlation in the data between At Bats and Hits versus At Bats and Strikeouts. 5

6 Figure 6. Scatterplot matrix for selected data. Another visualization that can be effective for multivariate data are parallel coordinates as shown in Figure 6. Each variable is normalized based on the maximum value for that variable, and datasets are plotted on separate axes parallel to each other. Each line represents a single player. By highlighting a single player, or small number of players, their statistics can be easily compared to the other values in the data. Figure 6. Parallel axes plot of sample data with single player highlighted. Many other graph types have been developed for displaying data. Some of the techniques are specific to certain sub categories of data such as time series data, 3D spatial data, and geo based information. 6

7 Pie Charts Just Say No One of the advantages of popular spreadsheet software, such as Microsoft s Excel, is that it has several pre defined chart types that allow quick generation of graphs from data. One of the drawbacks, however, is that it promulgates the myth that pie charts are a viable visualization method. Quite simply, they are not. This bias is not without basis. Perceptually, humans are not very adept at comparing relative areas. In Figure 7, for example, without the labels, it is virtually impossible (at least for the author) to tell which area is larger, B or D. Even with the labels, relative comparisons between areas are difficult. I, personally, cannot tell that A is half of B. It appears smaller than half. The exact same data is plotted as a bar chart in the middle graph of Figure 7. Rather than comparing areas, the comparison is to relative heights of the bars, which is much easier to perceive. Now, looking at B and D, even without the labels, the viewer can tell that D is slightly larger. Similarly, B appears twice as high as A. An even more austere representation of the data is the dot plot shown in the rightmost image of Figure 7. Here, a minimum amount of ink is used to present the data, yet the base information can be gleaned along with additional insight into the distribution of values. Many examples of the poor nature of pie charts to effectively display data abound in literature. However, according to famed data visualization expert, Edward Tufte, the only thing worse than pie charts are multiple pie charts on the same page. Now, you are increasing the level of perceptual difficulty by forcing the user to compare relative areas separated on the page in different charts. Figure 8 is a humorous look at one reasonable use of a pie chart. Figure 7. Random data in pie chart, vertical bar chart, and dot plot. Figure 8. Humorous take on pie charts. 7

8 Some General Principles Many guidelines have been proposed for how to design and create effective visualizations. Some of these come from the visual perception world, others from graphic design, still others from various scientific communities, and some from a growing community of information visualization researchers. Many of the techniques developed to date are a result of trial and error or variations on historical approaches. However, more and more research is being reported on attempts to identify, quantify, and verify general principles for visualization design. A few examples are given. Edward Tufte, previously mentioned, lists several guidelines in his classic book, The Visual Display of Quantitative Information. A summary of some of his principles are: Avoid chartjunk, the decorative elements of a chart that do not provide data; especially apparent in media graphics such as USA Today graphs Maximize data ink ratio, the amount of ink used to represent data versus total ink on the page; some ways to accomplish: avoid heavy grid lines, use small data markers, and only show as much scale information as necessary Maximize data density, the number of data elements per chart area Maintain graphical integrity; do not exaggerate scale or only show part of the data Utilize multi functioning graphical elements, elements that both convey data information and serve a design function Do not use pie charts Another well known visualization expert, Ben Shneiderman, is credited with the mantra overview first, zoom and filter, then details on demand. The idea is to provide a high level view of the entire dataset to allow the user to select the area of interest, quickly zoom in to the relevant data, and be presented with the necessary details when desired. Interactivity If one of the primary purposes of visualization is exploration, then the ability to interact with the data is critical. Static graphs and displays may be adequate for communicating information, but the ability to manipulate the viewing parameters, whether it is the part of the dataset being viewed, or the means by which it is being displayed, is necessary to recognize and understand patterns in data. Fortunately, with advances in processing power, storage capacities, and display capabilities, real time interaction is feasible even with very large datasets. A good example of a highly interactive, and fun, online dataset is the Baby Name Wizard by Laura Wattenberg ( A screen shot is shown in Figure 9. A database of thousands of historical baby names drives the interactive display. The user can select specific names, periods of times, or enter name filters to interactively search and explore the data. 8

9 Figure 9. Screen shot of Baby Name Wizard visualization tool. Several interactive techniques have been created to explore data through data selection, filtering, zooming, viewing in context, and displaying details. As in any user interaction, key features are rapid response, intuitive interface, judicious use of color, and meaningful interactions. Toolkits Many open source and commercial systems are available to create and interact with visualizations of data. Some of them are targeted at specific types of data or visualization approaches. One trend in the open source area is to create toolkits that allow users to build their own visualization applications from a set of pre defined software components and supporting architecture. Current toolkits (from wiki.net) include: BirdEye is a community project to advance the design and development of a comprehensive open source information visualization and visual analytics library Macrofocus InfoScope is an interactive visualization environment to explore large or complex datasets Prefuse is a user interface toolkit for building highly interactive visualizations of structured and unstructured data The flare visualization toolkit is a set of libraries for interactive visualizations on the web (based on prefuse) 9

10 The InfoVis Cyber Infrastructure web site provides access to a comprehensive set of software packages easing the exploration, modification, comparison, and extension of data mining and information visualization algorithms The InfoVis Toolkit is an interactive graphics toolkit written in Java to ease the development of Information Visualization applications and components VisAD is a Java component library for interactive and collaborative visualization and analysis of numerical data. The name VisAD is an acronym for "Visualization for Algorithm Development" Improvise is an information visualization system in which users build and browse multiview visualizations interactively using a simple shared object coordination mechanism coupled with a flexible, expression based visual abstraction language VisIt is a free interactive parallel visualization and graphical analysis tool for viewing scientific data on Unix and PC platforms (Lawrence Livermore National Laboratory, University of California) As an example of the types of capabilities provided, InfoVis is a Java based system that currently supports nine different views of data, many with available variations. Some features of the tool include: Common database format for all types of data Small memory footprint to support large datasets Unified set of interactive components that allow dynamic queries and data interaction Ability to take advantage of hardware accelerated graphics Extendable architectures that allows new visualization techniques to be easily added Figure 10 shows some of the visualizations generated using the InfoVis Toolkit. Figure 10. Visualizations from the InfoVis Toolkit. 10

11 LAGNIAPPE Historical Visualization One of the best known early examples of using visualization was by Dr. John Snow, a British physician in the 1800 s. Dr. Snow is considered a father of epidemiology because of his work in tracing the source of a cholera outbreak in Soho, England, in In August 1854, a major outbreak of cholera struck Soho which was later termed "the most terrible outbreak of cholera which ever occurred in the United Kingdom." Over the course of three days, 127 people died; within a month, 500 people had died with the mortality rate as high as 12.8 percent in some parts of the city. By the end of the outbreak, 616 people had died. The prevalent theory at the time for the disease was miasma, or bad air. Although knowledge about germs and how they spread disease were not well known at the time, Snow was skeptic of the miasma theory and set about to determine the root cause. By working with the townspeople and studying the patterns of deaths, he was able to conclude that the public water pump on Broad Street was the most likely source of the disease. Snow used a spot map to illustrate how cases of cholera were centered around the pump, and used statistics to make the connection between the quality of the water source and the cholera cases. It was discovered later that this public well had been dug only three feet from an old cesspool pit that had begun to leak fecal bacteria. Although Snow's chemical and microscope examination of a sample of the Broad Street pump water was not able to conclusively prove its danger, his studies of the pattern of the disease were convincing enough to persuade the local council to disable the well pump by removing its handle. John Snow died of a stroke at the age of