Knowledge discovery & data mining Classification & fraud detection

Transcription

1 Knowledge discovery & data mining Classification & fraud detection Knowledge discovery & data mining Classification & fraud detection 5/24/00 Click here to start Table of Contents Author: Dino Pedreschi Knowledge discovery & data mining Classification & fraud detection Module outline The classification task Classification systems and inductive learning Train & test Train & test Training step Test step Prediction Machine learning terminology Comparing classifiers Classical example: play tennis? Module outline Bayesian classification Estimating a-posteriori probabilities Naïve Bayesian Classification Play-tennis example: estimating P(xi C) file:///c /My Documents/2DM_class/index.htm (1 of 3) [5/24/ :49:42 PM]

2 Knowledge discovery & data mining Classification & fraud detection Play-tennis example: classifying X The independence hypothesis Module outline Decision trees Classical example: play tennis? Decision tree obtained with ID3 (Quinlan 86) From decision trees to classification rules Decision tree induction Generate_DT(samples, attribute_list) = Criteria for finding the best split Information gain (ID3 C4.5) Information gain (ID3 C4.5) Information gain - play tennis example Gini index (CART) Gini index - play tennis example Entropy vs. Gini (on continuous attributes) Other criteria in decision tree construction The overfitting problem Stopping vs. pruning If dataset is large If data set is not so large Categorical vs. continuous attributes Summarizing Scalability to large databases Module outline Backpropagation file:///c /My Documents/2DM_class/index.htm (2 of 3) [5/24/ :49:42 PM]

3 Knowledge discovery & data mining Classification & fraud detection Backpropagation Prediction and (statistical) regression Other methods (not covered) Classification with decision trees What have we achieved? References - classification References - classification file:///c /My Documents/2DM_class/index.htm (3 of 3) [5/24/ :49:42 PM]

4 Knowledge discovery & data mining Slide 1 of 50 file:///c /My Documents/2DM_class/sld001.htm [5/24/ :50:12 PM]

5 Module outline Slide 2 of 50 file:///c /My Documents/2DM_class/sld002.htm [5/24/ :50:13 PM]

6 The classification task Slide 3 of 50 file:///c /My Documents/2DM_class/sld003.htm [5/24/ :50:14 PM]

7 Classification systems and inductive learning Slide 4 of 50 file:///c /My Documents/2DM_class/sld004.htm [5/24/ :50:15 PM]

8 Train & test Slide 5 of 50 file:///c /My Documents/2DM_class/sld005.htm [5/24/ :50:16 PM]

9 Train & test Slide 6 of 50 file:///c /My Documents/2DM_class/sld006.htm [5/24/ :50:17 PM]

10 Training step Slide 7 of 50 file:///c /My Documents/2DM_class/sld007.htm [5/24/ :50:18 PM]

11 Test step Slide 8 of 50 file:///c /My Documents/2DM_class/sld008.htm [5/24/ :50:19 PM]

12 Prediction Slide 9 of 50 file:///c /My Documents/2DM_class/sld009.htm [5/24/ :50:20 PM]

13 Machine learning terminology Slide 10 of 50 file:///c /My Documents/2DM_class/sld010.htm [5/24/ :50:20 PM]

14 Comparing classifiers Slide 11 of 50 file:///c /My Documents/2DM_class/sld011.htm [5/24/ :50:21 PM]

15 Classical example: play tennis? Slide 12 of 50 file:///c /My Documents/2DM_class/sld012.htm [5/24/ :50:23 PM]

16 Module outline Slide 13 of 50 file:///c /My Documents/2DM_class/sld013.htm [5/24/ :50:24 PM]

17 Bayesian classification Slide 14 of 50 file:///c /My Documents/2DM_class/sld014.htm [5/24/ :50:24 PM]

18 Estimating a-posteriori probabilities Slide 15 of 50 file:///c /My Documents/2DM_class/sld015.htm [5/24/ :50:25 PM]

19 Naïve Bayesian Classification Slide 16 of 50 file:///c /My Documents/2DM_class/sld016.htm [5/24/ :50:26 PM]

20 Play-tennis example: estimating P(xi C) Slide 17 of 50 file:///c /My Documents/2DM_class/sld017.htm [5/24/ :50:28 PM]

21 Play-tennis example: classifying X Slide 18 of 50 file:///c /My Documents/2DM_class/sld018.htm [5/24/ :50:29 PM]

22 The independence hypothesis Slide 19 of 50 file:///c /My Documents/2DM_class/sld019.htm [5/24/ :50:30 PM]

23 Module outline Slide 20 of 50 file:///c /My Documents/2DM_class/sld020.htm [5/24/ :50:31 PM]

24 Decision trees Slide 21 of 50 file:///c /My Documents/2DM_class/sld021.htm [5/24/ :50:32 PM]

25 Classical example: play tennis? Slide 22 of 50 file:///c /My Documents/2DM_class/sld022.htm [5/24/ :50:33 PM]

26 Decision tree obtained with ID3 (Quinlan 86) Slide 23 of 50 file:///c /My Documents/2DM_class/sld023.htm [5/24/ :50:34 PM]

27 From decision trees to classification rules Slide 24 of 50 file:///c /My Documents/2DM_class/sld024.htm [5/24/ :50:35 PM]

28 Decision tree induction Slide 25 of 50 file:///c /My Documents/2DM_class/sld025.htm [5/24/ :50:36 PM]

29 Generate_DT(samples, attribute_list) = Slide 26 of 50 file:///c /My Documents/2DM_class/sld026.htm [5/24/ :50:37 PM]

30 Criteria for finding the best split Slide 27 of 50 file:///c /My Documents/2DM_class/sld027.htm [5/24/ :50:38 PM]

31 Information gain (ID3 C4.5) Slide 28 of 50 file:///c /My Documents/2DM_class/sld028.htm [5/24/ :50:39 PM]

32 Information gain (ID3 C4.5) Slide 29 of 50 file:///c /My Documents/2DM_class/sld029.htm [5/24/ :50:40 PM]

33 Information gain - play tennis example Slide 30 of 50 file:///c /My Documents/2DM_class/sld030.htm [5/24/ :50:42 PM]

34 Gini index (CART) Slide 31 of 50 file:///c /My Documents/2DM_class/sld031.htm [5/24/ :50:43 PM]

35 Gini index - play tennis example Slide 32 of 50 file:///c /My Documents/2DM_class/sld032.htm [5/24/ :50:44 PM]

36 Entropy vs. Gini (on continuous attributes) Slide 33 of 50 file:///c /My Documents/2DM_class/sld033.htm [5/24/ :50:45 PM]

37 Other criteria in decision tree construction Slide 34 of 50 file:///c /My Documents/2DM_class/sld034.htm [5/24/ :50:46 PM]

38 The overfitting problem Slide 35 of 50 file:///c /My Documents/2DM_class/sld035.htm [5/24/ :50:47 PM]

39 Stopping vs. pruning Slide 36 of 50 file:///c /My Documents/2DM_class/sld036.htm [5/24/ :50:48 PM]

40 If dataset is large Slide 37 of 50 file:///c /My Documents/2DM_class/sld037.htm [5/24/ :50:49 PM]

41 If data set is not so large Slide 38 of 50 file:///c /My Documents/2DM_class/sld038.htm [5/24/ :50:50 PM]

42 Categorical vs. continuous attributes Slide 39 of 50 file:///c /My Documents/2DM_class/sld039.htm [5/24/ :50:51 PM]

43 Summarizing Slide 40 of 50 file:///c /My Documents/2DM_class/sld040.htm [5/24/ :50:52 PM]

44 Scalability to large databases Slide 41 of 50 file:///c /My Documents/2DM_class/sld041.htm [5/24/ :50:53 PM]

45 Module outline Slide 42 of 50 file:///c /My Documents/2DM_class/sld042.htm [5/24/ :50:53 PM]

46 Backpropagation Slide 43 of 50 file:///c /My Documents/2DM_class/sld043.htm [5/24/ :50:54 PM]

47 Backpropagation Slide 44 of 50 file:///c /My Documents/2DM_class/sld044.htm [5/24/ :50:55 PM]

48 Prediction and (statistical) regression Slide 45 of 50 file:///c /My Documents/2DM_class/sld045.htm [5/24/ :50:56 PM]

49 Other methods (not covered) Slide 46 of 50 file:///c /My Documents/2DM_class/sld046.htm [5/24/ :50:57 PM]

50 Classification with decision trees Slide 47 of 50 file:///c /My Documents/2DM_class/sld047.htm [5/24/ :50:58 PM]

51 What have we achieved? Slide 48 of 50 file:///c /My Documents/2DM_class/sld048.htm [5/24/ :50:59 PM]

52 References - classification Slide 49 of 50 file:///c /My Documents/2DM_class/sld049.htm [5/24/ :51:01 PM]

53 References - classification Slide 50 of 50 file:///c /My Documents/2DM_class/sld050.htm [5/24/ :51:02 PM]