Hash Function Learning via Codewords

Transcription

1 Hash Function Learning via Codewords 2015 ECML/PKDD, Porto, Portugal, September 7 11, Yinjie Huang 1 Michael Georgiopoulos 1 Georgios C. Anagnostopoulos 2 1 Machine Learning Laboratory, University of Central Florida, US 2 ICE Laboratory, Florida Institute of Technology, US September 09 th, 2015

2 Table of Contents 1 Introduction 2 Formulation 3 Algorithm 4 Experiments 5 Concentration Guarantees 6 Summary 7 References 8 Back Up Slides Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

3 Introduction Section 1 Introduction Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

4 Introduction What is Content-Based Image Retrieval? Figure: Content-Based Image Retrieval (CBIR) [Datta et al., 2008] Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

5 Introduction Challenges in CBIR There are two main challenges in CBIR: Search complexity considerations Nearest neighbor search. In practical settings (large amount of data), exhaustively comparing the query with each sample in the database is impractical. Storage space considerations Image features usually have hundreds or thousands of features. Storing all raw images in the database also poses a problem. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

6 Hash Function Learning Introduction Figure: Hashing based CBIR [Wang et al., 2012] Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

7 Hash Function Learning Introduction Design hash functions transforming the original data into compact binary codes. Hash functions to be learned aim to map similar data to similar hash codes. Benefits: Approximate nearest neighbors (ANN) search [Datta et al., 2008] using binary codes was shown to achieve sub-linear search time. Storage requirement advantages. For example, a 10 dimension real value vector needs 320 bits (for single precision), while the hash code (represent this vector) may need only 10 bits. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

8 Introduction Data-dependent hashing Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

9 Introduction Contributions One new hashing framework - *Supervised Hash Learning (*SHL): *SHL can naturally engage supervised, unsupervised and semi-supervised hash learning scenarios. *SHL considers a set of Hamming space codewords that are learned during training in order to capture the intrinsic similarities between the data. The minimization problem of *SHL naturally leads to a set of Support Vector Machine (SVM) problems, which can be efficiently solved by LIBSVM [Chang and Lin, 2011]. Theoretical insight about *SHL s superior performance. Results: We consider 5 benchmark datasets. Compared with 6 other state-of-art methods. The results show *SHL is highly competitive. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

10 Formulation Section 2 Formulation Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

11 Formulation Formulation *SHL utilizes codewords µ g, g N G. Each codeword is associated to a class. Figure: Idea of *SHL Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

12 Formulation Formulation By adjusting its parameters ω, *SHL attempts to reduce the distortion measure: E(ω) ( ) h(xn ), µ ln + min d ( ) h(x n ), µ g g n N U n N L d (1) d is the Hamming distance defined as d(h, h ) b [h b h b ]. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

13 Formulation Formulation Hash code: h(x) sgn f(x) H B for a sample x X. f(x) [f 1 (x)... f B (x)] T, where f b (x) w b, φ(x) Hb + β b with w b Ω wb { w b H b : w b Hb R b, R b > 0 } and β b R for all b N B. H b is a Reproducing Kernel Hilbert Space (RKHS) with inner product, Hb. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

14 Algorithm Section 3 Algorithm Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

15 Algorithm Algorithm Two key observations: Bounds of hash codes: Hinge Hamming distances. d (h(x), µ) = b [µ bf b (x) < 0] d (f, µ) b [1 µ bf b ] +. We have: E(ω) Ē(ω) g n γ g,n d ( ) f(x n ), µ g. Majorization-Minimization (MM): For parameter values ω and ω, we have: Ē(ω) Ē(ω ω ) g n γ d ( ) g,n f(x n ), µ g. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

16 Algorithm Algorithm - First block minimization Minimizing Ē( ω ), one obtains B independent, equivalent problems: inf C w b,m, β b,θ b,µ g,b g n γ g,n m [ ] 1 µ g,b f b (x n ) + 2 w b,m H m θ b,m b N B (2) By considering w b,m and β b for each b as a single block, it leads to the dual form: sup α b Ω ab α T b1 NG 1 2 αt bd b [(1 G 1 T G) K b ]D b α b b N B (3) Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

17 Algorithm Algorithm - First block minimization SVM training problem - LIBSVM. Figure: For each bit, one SVM problem. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

18 Algorithm Algorithm - Second block minimization Second block minimization: inf C γ g,n [1 µ g,bf b (x n )] + w b,m,β b, θ b,µ g,b g n MKL, closed form solution [Kloft et al., 2011]: m w b,m 2 H m θ b,m b N B (4) w b,m 2 p+1 H m θ b,m = ( w m b,m 2p p+1 H m ) 1, m N M, b N B. (5) p Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

19 Algorithm Algorithm - Third block minimization Third block minimization: inf w b,m,β b,θ b, µ g,b C g n γ g,n m [ ] 1 µ g,b f b (x n ) + w b,m 2 H m θ b,m b N B (6) optimize over codewords by substitution: inf µ g,b H γ g,n [1 µ g,b f b (x n )] + g N G, b N B (7) n Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

20 Experiments Section 4 Experiments Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

21 Experiments Supervised Hash Learning Results Methods to compare with: Kernel Supervised Learning (KSH) [Liu et al., 2012]. Binary Reconstructive Embedding (BRE) [Kulis and Darrell, 2009]. Single-layer Anchor Graph Hashing (1-AGH) and its two-layer version (2-AGH) [Liu et al., 2011]. Spectral Hashing (SPH) [Weiss et al., 2008]. Locality-Sensitive Hashing (LSH) [Gionis et al., 1999]. Performance metric: Precision (retrieval accuracy). Precision - Recall (PR) curve. Datasets: Pendigits, USPS, Mnist, PASCAL07, CIFAR-10. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

22 Experiments Pendigits 1 Pendigits 1 Pendigits 1 Pendigits Top s Retrieval Precision (s=10) Our KSH LSH SPH BRE 1 AGH 2 AGH Top s Retrieval Precision Our KSH LSH SPH BRE 1 AGH 2 AGH Precision Our KSH LSH SPH BRE 1 AGH 2 AGH Number of Bits Number of Top s Recall Figure: The top s retrieval results and Precision-Recall curve on Pendigits dataset over *SHL and 6 other hashing algorithms. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

23 Experiments Mnist 1 Mnist 0.95 Mnist 1 Mnist Top s Retrieval Precision (s=10) Our KSH LSH SPH BRE 1 AGH 2 AGH Number of Bits Top s Retrieval Precision Our KSH LSH SPH BRE 1 AGH 2 AGH Number of Top s Precision Our KSH LSH SPH BRE 1 AGH 2 AGH Recall Figure: The top s retrieval results and Precision-Recall curve on Mnist dataset over *SHL and 6 other hashing algorithms. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

24 Experiments CIFAR Top s Retrieval Precision (s=10) Our KSH LSH SPH BRE 1 AGH 2 AGH CIFAR 10 Top s Retrieval Precision CIFAR 10 Our KSH LSH SPH BRE 1 AGH 2 AGH Precision CIFAR 10 Our KSH LSH SPH BRE 1 AGH 2 AGH Number of Bits Number of Top s Recall Figure: The top s retrieval results and Precision-Recall curve on CIFAR-10 dataset over *SHL and 6 other hashing algorithms. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

25 Experiments Qualitative Results Query Image: Car *SHL KSH LSH SPH BRE 1 AGH 2 AGH Figure: Qualitative results on CIFAR-10. Query image is Car. The remaining 15 images for each row were retrieved using 45-bit binary codes generated by different hashing algorithms. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

26 Experiments The Codewords Consider a small set of Mnist, bit length B = 25. Hamming distances between each pair of codewords after training µ 1 µ 2 µ 3 µ 4 µ 5 µ 6 µ 7 µ 8 µ 9 µ 10 µ µ µ µ µ µ µ µ µ µ Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

27 Concentration Guarantees Section 5 Concentration Guarantees Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

28 Concentration Guarantees Concentration Guarantees With high probability, *SHL produces hash codes concentrated around the correct codeword. Images from the same class will be mapped closer to each other, which will benefit precision-recall performance. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

29 Concentration Guarantees Concentration Guarantees Theorem Assume reproducing kernels of {H b } B b=1 s.t. k b(x, x ) r 2, x, x X. Then for a fixed value of ρ > 0, for any f F, any {µ l } G l=1, µ l H B and any δ > 0, with probability 1 δ, it holds that: er (f, µ l ) ^er (f, µ l ) + 2r ρb log ( ) 1 δ R b + N 2N b (8) where er (f, µ l ) 1 B E{d (h, µ l)}, l N G is the true label of x X, ^er (f, µ l ) 1 NB n,b Q ρ (f b (x n )µ ln,b), where { { }} Q ρ (u) min 1, max 0, 1 u ρ. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

30 Summary Section 6 Summary Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

31 Summary Summary A novel hash learning framework - *Supervised Hash Learning (*SHL) is proposed. *SHL is able to address supervised, unsupervised and, even, semi-supervised learning tasks in a unified fashion. Its training algorithm is simple to implement. Experiments on 5 benchmark datasets, compared with 6 other state-of-art methods, show *SHL is highly competitive. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

32 Summary Thank You! Thanks for your time. Codes are available here: yhuang/ Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

33 References Section 7 References Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

34 References References I Chang, C.-C. and Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1 27:27. Software available at cjlin/libsvm. Datta, R., Joshi, D., Li, J., and Wang, J. Z. (2008). Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 40(2):5:1 5:60. Gionis, A., Indyk, P., and Motwani, R. (1999). Similarity search in high dimensions via hashing. In Proceedings of the International Conference on Very Large Data Bases, pages Kloft, M., Brefeld, U., Sonnenburg, S., and Zien, A. (2011). lp-norm multiple kernel learning. Journal of Machine Learning Research, 12: Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

35 References References II Kulis, B. and Darrell, T. (2009). Learning to hash with binary reconstructive embeddings. In Proceedings of Advanced Neural Information Processing Systems, pages Liu, W., Wang, J., Ji, R., Jiang, Y.-G., and Chang, S.-F. (2012). Supervised hashing with kernels. In Proceedings of Computer Vision and Pattern Recognition, pages Liu, W., Wang, J., Kumar, S., and Chang, S.-F. (2011). Hashing with graphs. In Proceedings of the International Conference on Machine Learning, pages 1 8. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

36 References References III Wang, J., Kumar, S., and Chang, S.-F. (2012). Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12): Weiss, Y., Torralba, A., and Fergus, R. (2008). Spectral hashing. In Proceedings of Advanced Neural Information Processing Systems, pages Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

37 Back Up Slides Section 8 Back Up Slides Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

38 Transductive Learning Datasets: Vowel Letter Back Up Slides Vowel Inductive Transductive Letter Inductive Transductive Accuracy Precision Accuracy Precision Number of Bits Number of Bits Figure: Accuracy results between Inductive and Transductive Learning. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

39 Back Up Slides Algorithm Two key observations: Bounds of hash codes: Hinge Hamming distances. d (h(x), µ) = b [µ bf b (x) < 0] d (f, µ) b [1 µ bf b ] +. We have: E(ω) Ē(ω) g n γ g,n d ( ) f(x n ), µ g, by defining: γ g,n { [g = l n ] n N L [ g = arg ming d ( f(x n ), µ g )] n N U (9) Majorization-Minimization (MM): For parameter values ω and ω, we have: Ē(ω) Ē(ω ω ) g n γ d ( ) g,n f(x n ), µ g, where the primed quantities are evaluated on ω. γ g,n { [g = ln ] n N L [ g = arg min g d (f (x n ), µ g )] n N U (10) Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

40 Back Up Slides USPS 1 USPS 1 USPS 1 USPS Top s Retrieval Precision (s=10) Our KSH LSH SPH BRE 1 AGH 2 AGH Number of Bits Top s Retrieval Precision Our KSH LSH SPH BRE 1 AGH 2 AGH Num of Top s Precision Our KSH LSH SPH BRE 1 AGH 2 AGH Recall Figure: The top s retrieval results and Precision-Recall curve on USPS dataset over *SHL and 6 other hashing algorithms. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41

41 Back Up Slides PASCAL07 Top s Retrieval Precision (s=10) Our KSH LSH SPH BRE 1 AGH 2 AGH PASCAL 07 Top s Retrieval Precision PASCAL 07 Our KSH LSH SPH BRE 1 AGH 2 AGH Precision PASCAL 07 Our KSH LSH SPH BRE 1 AGH 2 AGH Number of Bits Number of Top s Recall Figure: The top s retrieval results and Precision-Recall curve on PASCAL07 dataset over *SHL and 6 other hashing algorithms. Huang & Georgiopoulos & Anagnostopoulos Hash Function Learning via Codewords September 09 th, / 41