Exploring Data Patterns. Run Charts, Frequency Tables, Histograms, Box Plots

Transcription

1 Exploring Data Patterns Run Charts, Frequency Tables, Histograms, Box Plots 1

2 Topics I. Exploring Data Patterns - Tools A. Run Chart B. Dot Plot C. Frequency Table and Histogram D. Box Plot II. III. IV. Understanding Data Patterns Distribution Shapes Outliers Distribution Decomposition Multiple Box Plot Graphical Summary Analysis - QETools 2

3 I. Exploring Data Patterns Effective data analysis requires understanding data patterns or distribution for a data set. Necessary to select best analysis tool Necessary to help focus on type of problem that needs to be resolved (e.g., chronic vs. sporadic, mean off target, high dispersion, special cause) Necessary to make appropriate conclusions 3

4 Analysis Tools Assessing Stability (pattern over time) A. Run Chart SPC Chart (Other Module) Assessing Distribution (irregardless of time) B. Dot Plot (unique values) C. Histogram (grouped into bins or equal width ranges) D. Box Plot 4

5 Transactional Case Study Office Visit Excel Data File: Wait-time.xls Suppose you study how long patients are waiting during office visits. You collect 69 samples of wait times. What is the distribution or data pattern? 5

6 A. Run Chart Plot observed data in time order (may be used to show dispersion / trends) Run Chart: Total Wait (69 Samples) Total Wait Time (min

7 B. Dot Plot Graphical representation of unique observed values. Y-axis # of occurrences for each unique value (may use with discrete or continuous) Example: Test Scores: 77, 77, 73, 84, 84, 81, 93, Frequency Scores 7

8 C. Frequency Analysis / Histogram Frequency Analysis is used to summarize occurrences and analyze patterns by grouping data. It involves determining frequencies (# of occurrences) by classes Classes or (Bins) - values or ranges of values (for discrete or continuous variables). E.g., Test Scores: 77, 77, 73, 84, 84, 81, 93, 98 Bin 70-79: 3 values Bin 80-89: 3 values Bin 90-99: 2 values frequency table histogram 8

9 Frequency Table First, we create a frequency table with relative and cumulative frequency. Note: bins are in continuous order of values NOT ranked by frequency counts (as done in Pareto). Key Inputs needed: # of Bins (Classes) (usually calculated by software) 1 st Bin Bin Width (equal length) Total: Relative Frequency Cumulative Frequency Bin Frequency % 14.5% % 23.2% % 31.9% % 43.5% % 50.7% % 59.4% % 65.2% % 68.1% % 75.4% % 84.1% % 88.4% % 92.8% % 97.1% More 2 2.9% 100.0% Relative Frequency ~ Frequency / Total Cumulative Freq % ~ Running total of % 9

10 Creating # Bins and Bin Widths Most statistical software automatically creates # bins and the width of the bin given a data set. Rules of thumb, if creating own bin ranges: # of bins ~ #observations Usually between 5 20 bins is appropriate May experiment to find effective bin width 10

11 Observations Occurring on Frequency Range Limits In Excel, a range of implies all values greater than 20 and less than or equal to 30. In other software (e.g., Minitab), a range of could mean all values greater than or equal to 20 and < 30. Neither is wrong, it is merely a case of style. To avoid confusion, try to use more discriminatory scales: Example: ;

12 Histogram A histogram is a graphical representation of a frequency table. It is used to: show distribution shape for one variable (conveys the location, dispersion, and symmetry). Distribution Examples: Normal, Skewed Right, etc. identify outliers 12

13 Histogram Example Office Visit Wait Time Typical Y-Axis: frequency or relative frequency Frequency Histogram Bin 100 Wait Time, min Average 65.6 St Dev More 13

14 Creating Histogram in QETools QETools >> Graphical Tools >> Histogram Example: wait-time.xls timeinwaitingroom OR Enter Min and Width 1st Bin: 5 WIDTH: 5 Total: Relative Frequency Cumulative Frequency Bin Frequency % 4.35% % 21.74% % 34.78% % 40.58% % 50.72% % 62.32% % 68.12% % 71.01% % 78.26% % 78.26% % 84.06% % 86.96% % 92.75% % 95.65% % 97.10% % 98.55% % % Frequency Timeinwatingroom Histogram Bin 14

15 D. Box Plots Graphical technique to summarize distribution of a continuous (or discrete) variable. Box plot may be used to communicate: Location line for median Note: some software will also include a dot for mean. Dispersion box shows the 25 th 75 th percentile value range (Q1 and Q3). Departures from symmetry one box or whisker can be larger than the other side suggesting a lack of symmetry. Identification of mild and extreme outliers. 15

16 Box Plot Calculations Extreme Outlier(s) Mild Outlier(s) Upper Whisker: Highest value within upper limit Third quartile (Q3) Median First quartile (Q1) Lower Whisker: Lowest value within lower limit * * * Q +/- 1.5 f s Q3 75 th Percentile Median - 50 th Percentile Q1 25 th Percentile f s = Q3 Q1 Upper Limit: Q f s Lower Limit: Q1 1.5 f s < mild < Q +/- 3.0 f s outlier < extreme outlier 16

17 Single Box Plot Example: Wait Time What does this box plot show? Total Wait Wait Time, min Average 65.6 Median Total Wait Box Plot: QETools >> Graphical Tools >> Box Plot >> Variable = Wait Time 17

18 Box Plots Vs. Histogram Note: wider box and/or longer tail above median in box plot suggests more spread to right (skewed right). Similar pattern is shown in the histogram. Total Wait 12.0 TotalWait Histogram Frequency Total Wait Bin 18

19 Frequency Analysis and Sample Size General rules for creating histograms and box plots for assessing distribution shapes. minimum 30 samples ~ prefer 100 or more Avoid using relative frequency unless at least 30 samples are available. For smaller samples use dot plot. 19

20 II. Assessing Data Patterns Distribution Fitting Identify Distribution that best fits Data Outliers Assess if data set includes unusual observations. 20

21 Identifying Distributions Shapes Histograms may be used to help identify distribution shapes that best fit data. Exponential Bi-Modal Normal Skewed Right Skewed Left 21

22 Histogram Shape - Example 1 Which shape is shown in this histogram? 14.0 Timeinwatingroom Histogram Frequency Bin 22

23 Histogram Shape Example 2 Which distribution is shown in this histogram? Histogram Mean = 20 StdDev = < > 23 Frequency (N=100) Bin Range

24 Normal Vs. Skewed or Bi-Modal For some six sigma projects, the goal is to eliminate the causes for Non-normality. Skewed Bi-Modal 24

25 Outliers (Extreme Values) One issue is assessing a distribution is outliers. Outliers (extreme values) generally result from: measurement or data entry error (e.g., record using wrong units) An observation being obtained under a different set of circumstances (e.g., special cause). Examples: wait time recorded during an office emergency versus typical day. Wait time recorded during a day when number of patients is 50% lower than usual. Extreme values may be better or worse. 25

26 Outlier Example Suppose one patient had wait time > 150 minutes for Clinician 2. Some unusual reason likely explains this value as opposed to just inherent random variation. Waitforcinician Value Waitforcinician2 26

27 Outliers: Good Or Bad? Data Analysis Trap is to automatically exclude outliers. Outliers may suggest a better set of operating conditions are available. Still, they may skew our results. Note: outliers adversely impact calculations of statistics such as mean and standard deviation. Unfortunately, deciding whether to include or exclude outliers is an experience-developed skill. To build this skill seek to understand the source of outliers before discarding. 27

28 III. Decomposing Distribution Note: One reason for skewed data distributions (e.g., skewed right) is lack of consistency among some process setting or input variable. Example: compare distribution of all teams versus Team B only. Team B may be normal if enough data is taken. Frequency TotalWaitB Histogram Minutes 12.0 Frequency TotalWait Histogram Minutes N = 28 All Teams N = 69 Team B 28

29 Multiple Box Plots One powerful technique to help see distributions inside overall distributions are multiple box plots. Can you identify what input is causing the skew right? Sample Sizes A = 15 B = 28 C = TotalWait-A TotalWait-B TotalWait-C 29

30 Distribution Drill Down: Multiple Box Plots As with other tools, stratification variables may be used to drill down (from overall components) Suppose you decompose Team C total wait by its 3 components. Where is most of the variation occurring? WaitRoomTime WaitClinician1 WaitClinician2 Total Wait 30

31 Multiple Box Plot - QETools QETools >> Graphical Tools >> Box Plot Enter variable (timeinwaitingroom) and grouping variable (Team), OR enter multiple data columns. 31

32 IV. Graphical Summary Understanding data patterns, process stability and defect rates (DPM, DPMO) are common in most problem solving situations to identify (chronic vs. sporadic, mean off target) Thus, QETools provides a Graphical Summary tool to show key charts and summary statistics. 32

33 Graphical Summary Example Key Information: Summary Stats, PPM Defects Box Plot, Histogram Control Charts QETools >> Process Capability Summary - Normal 33

34 Summary Effective data analysis for continuous/discrete responses (Y) requires an understanding of data pattern. E.g., bi-modal distribution (likely some special cause creating two distributions within the overall) Some common tools to visualize data patterns are by: run chart (by time), dot plots (low sample size), histograms, and box plots. From here, we can drill down into variation sources. One effective technique for decomposing distributions is a multiple box plot. 34