Lecture IV Visual Data Descrip.on cont.

Transcription

1 Boğaziçi University EE Department ee58j spring Data Mining for Visual Media Lecture IV Visual Data Descrip.on cont. Ceyhun Burak Akgül, PhD in EE research.com

2 In This Lecture The Bag of Words Model Textons Subspace Methods: PCA and ICA Shape Contexts Projects 3/25/10 ee58j spring 2

3 Bag of Words Model *Picture credits: Li Fei Fei, Princeton University 3/25/10 ee58j spring 3

4 Bag of Words Model *Picture credits: Li Fei Fei, Princeton University 3/25/10 ee58j spring 4

5 Bag of Words Model *Slide credits: Li Fei Fei, Princeton University 3/25/10 ee58j spring 5

6 Bag of Words Model LimitaKon All the images below have the same representa.on Loca.on maoers *Picture credits: Li Fei Fei, Princeton University 3/25/10 ee58j spring 6

7 Textons Texture Examples I 3/25/10 ee58j spring 7

8 Textons Texture Examples II 3/25/10 ee58j spring 8

9 Textons Texture Examples III 3/25/10 ee58j spring 9

10 Textons Example ApplicaKon: Texture ClassificaKon *Slide credits: Visual Geometry Group, Oxford University 3/25/10 ee58j spring 10

11 Textons What are textons? (Julesz, 1981)*: local conspicuous features of texture (Malik et. Al, 1999)**: Opera.onal defini.on Prototype filter responses on a texture pixel * Nature 290, ** ICCV /25/10 ee58j spring 11

12 Textons Filter Banks: Leung Malik 48 Edge, bar, spot filters 2 Gaussian deriva.ve filters at 6 orienta.ons, 3 scales = 36 filters 8 Laplacian of Gaussian filters 4 Gaussian filters Matlab code available at hop:// 3/25/10 ee58j spring 12

13 Textons Filter Banks: Schmid 13 Gabor like filters Rota.onally invariant Matlab code available at hop:// 3/25/10 ee58j spring 13

14 Textons Filter Banks: Max. Response 38 Edge, bar, spot filters* 2 Gaussian deriva.ve filters at 6 orienta.ons, 3 scales = 36 filters 1 Laplacian of Gaussian filter 1 Gaussian filter Matlab code available at hop:// 3/25/10 ee58j spring 14

15 Textons Learning the Texton DicKonary *Slide credits: Visual Geometry Group, Oxford University 3/25/10 ee58j spring 15

16 Textons Leung Malik Texton DicKonary 3/25/10 ee58j spring 16

17 Textons Schmid texton dickonary 3/25/10 ee58j spring 17

18 Textons Learning the model for a texture image *Slide credits: Visual Geometry Group, Oxford University 3/25/10 ee58j spring 18

19 Textons Nearest neighbor based classificakon *Slide credits: Visual Geometry Group, Oxford University 3/25/10 ee58j spring 19

20 Textons Further reading Malik et al. (1999) Textons, Contours, and Regions: Cue IntegraCon in ICCV Varma & Zisserman (2005) A StaCsCcal Approach to Texture ClassificaCon from Single Images IJCV 62 (1/2): Zhu et al. (2005) What are textons? IJCV 62 (1/2): /25/10 ee58j spring 20

21 Subspace Methods Subspace methods can be used to Learn the filter banks from data Learn the prototypes directly General idea z = Wx such that SomeCriterion(z) is sa.sfied. PCA: What is the criterion? ICA: What is the criterion? NNMF: What is the criterion? 3/25/10 ee58j spring 21

22 Subspace Methods PCA ICA 3/25/10 ee58j spring 22

23 Subspace Methods LOCAL GLOBAL 3/25/10 ee58j spring 23

24 Subspace Methods Further reading Cootes et al. (2001) AcCve appearance models IEEE TPAMI 23 (6): /25/10 ee58j spring 24

25 Shape Contexts Matching two visual objects Find correspondences (between points) Es.mate the transforma.on Measure similarity 3/25/10 ee58j spring 25

26 Shape Contexts Describing a point on the shape *Slide credits: Serge Belongie, UCSD 3/25/10 ee58j spring 26

27 Shape Contexts Describing a point on the shape *Slide credits: Serge Belongie, UCSD 3/25/10 ee58j spring 27

28 Shape Contexts Comparing shape contexts *Slide credits: Serge Belongie, UCSD 3/25/10 ee58j spring 28

29 Shape Contexts TransformaKons revisited 3/25/10 ee58j spring 29

30 Shape Contexts EsKmaKng the transformakon For instance the affine model: Least Squares solu.on *Shape contexts use the thin plate spline model 3/25/10 ee58j spring 30

31 Shape Contexts Thin plate spline model 3/25/10 ee58j spring 31

32 Shape Contexts Thin plate spline model TPS interpolant minimizes the bending energy Square integrability of the 2 nd deriva.ves 3/25/10 ee58j spring 32

33 Shape Contexts Thin plate spline model Solve a linear system to find the parameters of the interpola.ng func.on? Regularized version: interpola=on approxima=on 3/25/10 ee58j spring 33

34 Shape Contexts Measuring similarity (+) Shape context distance (+) Local appearance distance Gradient orienta.ons Local descriptors (+) Bending energy 3/25/10 ee58j spring 34

35 Shape Contexts ApplicaKons 3D object recognikon COIL database Handwrieen character recognikon MNIST database Trademark retrieval 3/25/10 ee58j spring 35

36 Shape Contexts Further Reading Belongie et al. (2002) Shape Matching and Object RecogniCon with Shape Contexts IEEE TPAMI 24 (4): /25/10 ee58j spring 36

37 Project SuggesKons Fish categoriza.on from underwater images Texture/color/appearance based Contour based Image based medical diagnosis Sports video analysis Personal photo album organiza.on* Industrial Defect Detec.on Defect detec.on on iron surfaces Defect detec.on from cylindrical tank contour lines 3/25/10 ee58j spring 37

38 Assignments No assignment this week! Prepare your project proposals due April 1st 3/25/10 ee58j spring 38