Biased Support Vector Machine for Relevance Feedback in Image Retrieval

Transcription

1 Biased Support ector Macine for elevance Feedback in Image etrieval Cu-ong oi, Ci-ang Can, Kaiu uang, Micael. Lyu and Irwin King Department of Computer Science and Engineering Te Cinese University of ong Kong Satin, ong Kong SA Abstract elevance feedback as been sown as a powerful tecnique in Content-Based Image etrieval (CBI in te past years. ecently, Support ector Macines (SM ave been engaged in relevance feedback tasks and sown promising results in CBI. Typical approaces apply te SMs in relevance feedback as a strict binary classification task. owever, tese approaces do not consider an important property of relevance feedback, i.e. te imbalanced dataset problem, in wic te negative instances largely overnumber te positive instances. egular relevance feedback by SMs does not consider any bias for tis issue. For solving te imbalance problem, we propose a novel tecnique to formulate te relevance feedback algoritm based on a modified SM called Biased Support ector Macine (BSM. Matematical formulation and explanations are provided for sowing te advantages of our sceme. We conduct a number of experiments to evaluate te performance of our proposed sceme. romising results demonstrate te effectiveness of our tecnique. I. INTODUCTION Content-Based Image etrieval (CBI as been widely studied in te past decade [2. In CBI, low-level visual features representing color, sape and texture are extracted from images to represent teir content. Based on te extracted features, te similarity of two images can be measured by a similarity function. Early approaces for searcing similar images are usually based on te similarity measure on te sum of distances between individual features wit fixed weigts. Tese approaces are not flexible and te retrieval precision may suffer dramatically if a large number of weigted features are irrelevant to te query target. ence, relevance feedback is proposed as an interactive tecnique for flexible feature reweigting and query reformulation [16, [17, [7. elevance feedback as been sown as a powerful tool to improve te retrieval performance of CBI [17, [7, [9. ecently, a lot of classification tecniques ave benn introduced to solve te relevance feedback issues [11, [25, [5, [24 in wic SMs based tecniques are considered as te most promising tecniques. owever, previous studies on relevance feedback by SMs treat te problem as a strict binary classification problem witout noticing an important property of relevance feedback, i.e. te imbalanced dataset problem, in wic te number of negative instances in te negative class are significantly larger tan te positive ones [12. Tis imbalanced dataset problem will lead te positive instances be overwelmed by te negative instances. In order to mitigate tis problem, we propose a modified Support ector Macine [19, [23, [15 called Biased Support ector Macine (BSM wic can well model te relevance feedback and reduce te performance degradation caused by te skewness of datasets. Te rest of te paper is organied as follows. In Section II, we review some related work on relevance feedback and address teir disadvantages. Ten we provide a brief introduction of two-class SMs and one-class SMs in Section III. In Section I, we formulate te BSM algoritm and state its advantages. We ten formulate te relevance feedback tecnique employing BSM and explain te benefits compared wit te conventional tecniques in Section. Experiments, performance evaluation and discussions are given in Section I. Section II concludes our work. II. ELATED WOK In te past years, relevance feedback tecniques ave evolved from early euristic weigt adjustment tecniques to recent various macine learning tecniques [16, [17, [1, [5, [7. In [1, a popular clustering tecnique, SOM (Selforganiing Map, was proposed to construct te relevance feedback algoritm. Besides te SOM, many classic macine learning tecniques are also suggested, suc as Decision Tree [11, Artificial Neural Network [14, and Bayesian learning [4, etc. Moreover, many popular classification tecniques are proposed for solving te relevance feedback recently, suc as Nearest-Neigbor classifiers [26, Bayesian classifiers [22 and Support ector Macines [5, [24, [3, etc. Among tem, SMs based tecniques are te more promising and effective tecniques. Typical relevance feedback approaces by SMs are based on strict binary classifications [5, [24 or one-class classifications [3. owever, te strict binary classifications do not consider te imbalance problem in relevance feedback. Te one-class tecnique seems to avoid te imbalance problem. owever, it cannot work well witout te elp of negative information [27. In order to fuse te negative information, we propose te Biased Support ector Macine derived from te one-class SMs to construct te relevance feedback tecnique in CBI.

2 F e F \ D _ S g III. SUOT ECTO MACINES In te following, we briefly introduce te basic ideas of regular two-class SMs [1 and one-class SMs ( - SM [19, [23, [15. SMs implement te principle of structural risk minimiation by minimiing apnik-cervonenkis dimensions [1. On pattern classification problems, SMs provide very good generaliation performance in empirical applications [1. Let us consider SMs in binary classifications. Generally speaking, a binary classification problem can be formalied as a task to estimate a function based on independent identically distributed (i.i.d. data "!$#&%(' '*+ [18. ere, #-, te training instances are vectors in some space. and is te number of training instances. Te goal of te learning process is to find an optimal decision function wic can classify te unseen data correctly. In teory, te goal is to find te optimal function wit te smallest risk 5476 :9;< 1 32 L 8 (1 were ; is te probability measure for te generation of te training data and L is a loss function. In te simplest form, te goal of learning in SMs is to find te yperplane wit a maximum margin (see Fig. 1. Te vectors closest to te yperplane are called support vectors. (also called -SM [18: LNMO I J K Z [ GQ F Q W \^ S TSU :XW (3 N\ W (4 \^ `_ (5 were W represent te margin errors for te non-separable training data. Wen te margin errors W =, one can sow tat te two classes are separated by a margin wit Sba Q F Q from Eq. (4. By introducing te Lagrange multipliers, te optimiation problem can be sown wit te dual form below [18, [1. I<cEd g g? f gi Z [ j (6 j lk k (7 7`_ (8 j One-class SMs are derived from classical SMs for solving density estimation problems. In typical formulation of -SMs, only positive instances are considered for estimating te density of te data. Tere are two kinds of different formulations of -SMs in te literature [19, [23. ere, we coose to illustrate te spere based approac wit an explicit and good geometric property. Fig. 2 simply illustrates an example of -SMs. Fig. 1. Te linear separating yperplane of SMs for separable data: Te circles and crosses are called positive instances and negative instances, respectively. Te circles and te crosses on te two solid lines are called support vectors. Te dased line between te two solid lines is called te decision yperplane. # More generally, te training data in te original space can be projected to a iger dimensional feature space = wic is spanned by a mapping function. Te mapping function corresponds to a Mercer AB CED wic implicitly computes te dot product in =. ence, te goal of SMs is to find te optimal separating yperplane depicted by a vector F in te feature space = CG C (2 Te task to find te optimal yperplane turns to solving te primal optimiation problem in te form of soft margin SMs Fig. 2. Te spere yperplane in m -SM for constructing te smallest soft spere tat contains most of te positive instances. Te circles are positive instances. Te circles outside of te yperplane are called outliers. Te optimal decision function of te spere based approac of -SMs can be found by solving te optimiation problem as follows [19, [23: I J K W (9 n Mpo fq MO Z [ Q (r Q k W (1 W \s (11!t ere, 2 is a parameter to control te tradeoff between te radius of te yper-spere and te number of positive training samples.

3 ! k e \ ƒ ˆ ƒ \ g I. BIASED SUOT ECTO MACINE In order to incorporate te negative information, we propose te Biased Support ector Macine derived from -SMs for overcoming te imbalance problem of relevance feedback tasks. Our strategy is to describe te data by employing a pair of spere yperplanes in wic te inner one captures most of te positive instances wile te outer one puses out te negative instances. Terefore, te goal of our problem is to find an optimal spere yperplane wic can not only contain most of positive data but also can pus most of negative data out of te spere. Te problem can be visually illustrated in Fig. 3. Te dased spere in te figure is te desired spereyperplane. Te task can be formulated as an optimiation problem and te matematical formulation of our tecnique is given as follows. Fig. 3. Te spere yperplane of BSM. Te circles and te crosses are called positive instances and negative instances, respectively. Te dased spere is te decision yperplane. Let us consider te following training data: u %v' 'w$ (12 were is te number of training instances and x is te dimension of te input space. Te objective function for finding te optimal spereyperplane can be formulated below: n Mpo I J K f y Mpo fq MO S{ W (13 Z [ ~} } (r8} } Nk5 SU W (14 S \^ k W \s (15 were W are te slack r variables for margin error, is te mapping function, is te center of te optimal spereyperplane and is a parameter to control te bias. Te optimiation task can be solved by introducing te Lagrange multipliers: W ~ƒe. G 1 ~} } S{ W (r8} } W ts bs W 2C (16 Let us take te partial derivative of wit respect to, Wp, ƒ and S, respectively. By setting teir partial derivatives to, we obtain te following equations, A. ˆ. ˆ ˆ k Š (17 k Š (18 (19 { (2 By substituting te above derived results to te objective function in Eq. (16, te dual of te primal optimiation can be sown to take te form I<cEd (21 Z [? g g? f gi (22 k k (23 _ (24 Tis dual problem can be solved wit Quadratic rogramming (Q tecniques [13. Ten, te resulting decision function takes te form Œ Ž Z T ~} } (25 C(r8} } were r can be obtained from Eq. (19 and can be solved by support vectors. Based on te decision function, we can know te instances inside te spere yperplane will be predicted as te positive and te negative oterwise.. ELEANCE FEEDBACK USING BSM A. Advantages of BSM in elevance Feedback From te above formulation, one may see tat te optimiation in Eq. (21 is similar to te one in te -SM. Now, we explain te matematical differences compared wit regular SMs and te advantages of our BSM from te geometric perspective for solving te relevance feedback problems. From te results of matematic deduction in te optimiation function, we see tat BSM is wit te following constraint from Eq. (22 (26 Wen replacing wit for te positive class and for te negative one, te constraint can be written as M T M b (27

4 ? were denotes te positive class and N denotes te negative one. owever, in te regular SMs ( -SM, te constraint is wit te form M M (28 Te difference indicates tat te weigt allocated to te positive support vectors in BSM will be larger tan te negative ones wen setting a positive bias factor. Tis can be useful for solving te imbalance dataset problem. owever, regular SMs ( -SM treat te two classes witout any bias wic is not effective enoug to model te relevance feedback problem. Moreover, we can also see te difference from te geometric perspective. Fig. 4 provides te comparison of te decision boundaries of regular SM, -SM and BSM on te syntetic data wit te same kernels (adial Basis Function and parameters ( =. We can see tat te geometric property of BSM is better tan te SM and -SM. BSM can describe te data in a cluster beavior by te spere based boundary and can flexibly control te weigt of te positive class for te imbalanced dataset by adjusting te bias factor. Terefore, compared wit regular SM and -SM, BSM is more reasonable and effective to model te relevance feedback tasks. B. elevance Feedback Algoritm By BSM From te above comparisons, we ave sown te benefits of BSM for solving relevance feedback issues. ere, we describe ow to formulate te relevance feedback algoritm by employing te BSM tecnique. Applying SMs based tecniques in relevance feedback is similar to te classification task. owever, te relevance feedback need to construct te evaluation function to output te relevance value of te retrieval instances. From te decision function in Eq. (3, we build te evaluation function wit te similar form by substituting te equation in Eq. (19 CŽ } } } } C(r8} } C } } (29 were te radius can be solved by a set of support vectors. owever, for te relevance evaluation purpose, constant values can be eliminated. ence, te evaluation function can be sown to take te concise form C? C C (3 Once te parameters are solved in Eq. (21, te evaluation function can be constructed. Consequently, we can rank te C images based on te scores of te te evaluation function. Te images wit iger scores will be more likely to be cosen as te targets. I. EXEIMENTS In te experiments, we compare te performance of tree different algoritms for relevance feedback: regular SM ( - SM, -SM and our proposed BSM. Te experiments are evaluated bot on a syntetic dataset and two real-world image datasets. A. Datasets 1 A Syntetic Dataset: We generate a syntetic dataset to simulate te real-world image dataset. Te dataset consists categories eac of tem contains data points randomly generated by š Gaussians in a -dimensional space. Te means and covariance matrices of te Gaussians for eac category are randomly generated from te range of [,. 2 COEL Image Datasets: Te real-world images are cosen from te COEL image CDs. We organie two datasets containing various images wit different semantic meanings, suc as antique,aviation, balloon, botany, butterfly, car and cat, etc. One of te datasets is wit categories ( -Cat and anoter is wit categories ( -Cat. Eac category includes images belonging to a same semantic class. B. Image epresentation For te real-world image retrieval, te image representation is an important step for evaluating te relevance feedback algoritms. We extract tree different features to represent te images: color, sape and texture. Te color feature engaged is te color moment since it is closer to uman perception naturally. We extract œ moments: color mean, color variance and color skewness in eac color cannel (, S, and, respectively. Tus, 9-dimensional color moment is employed as te color feature in our experiments. We employ te edge direction istogram as te sape feature in our experiments [8. Canny edge detector is applied to obtain te edge images. From te edge images, te edge direction istogram can ten computed. Te edge direction istogram is quantied into ž bins of degrees eac, ence an ž -dimensional edge direction istogram is used to represent te edge feature. We use te wavelet-based texture feature for its effectiveness [21. We perform te Discrete Wavelet Transformation (DWT on te gray images employing a Daubecies- wavelet filter [21. In total, we perform œ -level decompositions and obtain subimages in different scales and orientations. Ten, we coose Ÿ subimages wit most of te texture information and compute te entropy of eac subimage. ence, a Ÿ -dimensional wavelet-based texture feature is obtained to describe te texture information for eac image. C. Experimental esults ere, we present te experimental results by tree different relevance feedback algoritms bot on te syntetic data and te real-world images. For te purpose of objective measure of performance, we assume tat te query judgement is defined on te image categories [24. And te metric of evaluation is te Average recision wic is defined as te average ratio of

5 (a SM ( -SM (b 1-SM (c BSM Fig. 4. circles and crosses represent te positive and negative instances, respectively. Te boundaries of te sadow regions represent te decision boundaries. Decision boundaries of tree classification metods wit te same kernels (BF and parameters ( =ž m : (a -SM, (b m -SM, (c BSM. Te te number of relevant images of te returned images over te number of total returned images. In te experiments, a category is first picked from te database randomly, and tis category is assumed to be te user s query target. Te system ten improves retrieval results by relevance feedbacks. In eac iteration of te relevance feedback process, instances are picked from te database and labelled as eiter positive or negative based on te ground trut of te database. For te first iteration, positive instances and negative instances are randomly picked, and all tree metods are run based on te same set of initial data points. For te iterations afterward, eac metod selects instances closest to te decision boundaries. In te retrieval process, te instances in te positive region are selected and ranked by teir distances from te boundaries. Te precision of eac metod is ten recorded, and te wole process is repeated for times to produce te average precision in eac iteration for eac metod. In te experiments, we implement te algoritms by modifying te codes in te libsvm library [2. We notice tat te experimental settings are important to impact on te evaluation results. To enable an objective measure of performance witout bias, we coose te same kernel and parameters for all metods. Te cosen kernel is based on adial Basis Function (BF wic outperforms tan oter kernels in te experiments. Te first evaluation is on te syntetic dataset. Fig. 5 sows te evaluation results of top-œ returned results. We can observe tat te performance of BSM outperforms te oter approaces. Te -SM acieves te worst performance witout considering te negative information. Te second evaluation is on te real-world datasets. Fig. 6 and Fig. 7 sow te evaluation results on te -Cat dataset and -Cat dataset, respectively. From te results on te real-world datasets, we can see our proposed BSM also outperforms te oter approaces. owever, we notice tat te performances of -SM in te beginning feedback steps are better tan tose of oter approaces. Te reason is tat -SM can reac te enclosed positive region soon, but it Average precision Average precision BSM ν SM 1 SM Number of iterations Fig Fig. 6. Experimental results on te syntetic dataset BSM ν SM 1 SM Number of iterations Experimental results on te -Cat image dataset cannot be furter improved witout te elp of te negative information in furter steps. In order to observe te detailed comparison of tree metods after -iterations, we list te retrieval results in Table. I. From te results, we can also see te similar results matcing te above comparisons. D. Discussions From te experimental results, we see tat our proposed BSM outperforms te regular SM approaces. Typical

6 Average precision Fig. 7. BSM ν SM 1 SM Number of iterations Experimental results on te -Cat image dataset TABLE I AEAGE ECISION AFTE m ITEATIONS Metods Top2@2-Cat Top3@2-Cat Top5@2-Cat -SM ž ž ž m -SM ž žm ž ª ž BSM ž «m ž ª ž Metods Top2@5-Cat Top3@5-Cat Top5@5-Cat -SM ž «ž ž m -SM ž «ž ž BSM ž ª ž žm ž approaces by SMs ( -SM witout considering te bias in te retrieval tasks is not reasonable and good enoug to solve te relevance feedback problem. We also see tat regular one-class SMs do not consider te negative information wic cannot learn te feedback well. Furtermore, we know tere exists oter metods to address te imbalanced dataset problem in literature [12, [6. In te future, we can consider to include oter tecniques in our sceme altoug we ave demonstrated te effectiveness of our proposed BSM tecnique for te relevance feedback problems. II. CONCLUSIONS In tis paper, we investigate SMs tecniques for solving te relevance feedback problems in CBI. We address te imbalanced dataset problem in relevance feedback and propose a novel relevance feedback tecnique wit Biased Support ector Macine. Te advantages of our proposed tecniques are explained and demonstrated compared wit traditional approaces. We perform te experiments bot on syntetic data and real-world image datasets. Te experimental results demonstrate tat our BSM based relevance feedback algoritm is effective and promising for improving te retrieval performance in CBI. EFEENCES [1 C. Burges. A tutorial on support vector macines for pattern recognition. Data Mining and Knowledge Discovery, 2(2: , [2 C.-C. Cang and C.-J. Lin. LIBSM: a library for support vector macines, 21. Software available at ttp: cjlinlibsvm. [3. Cen, X. Zou, and T. uang. One-class svm for learning in image retrieval. In roc. IEEE International Conference on Image rocessing (ICI 1, Tessaloniki, Greece, 21. [4 I. Cox, M. Miller, T. Minka, and. ianilos. An optimied interaction strategy for bayesian relevance feedback. In IEEE Conference on Computer ision and attern ecognition (C 98, pages , Santa Barbara, CA, USA, [5. ong, Q. Tian, and T. uang. Incorporate support vector macines to content-based image retrieval wit relevant feedback. In roc. IEEE International Conference on Image rocessing (ICI, ancouver, BC, Canada, 2. [6 K. uang,. ang, I. King, M. Lyu, and L. Can. Biased minimax probability macine for medical diagnosis. In 8t AI and MAT Symposium, 24. [7 T. uang and X. Zou. Image retrieval by relevance feedback: from euristic weigt adjustment to optimal learning metods. In roc. IEEE International Conference on Image rocessing (ICI 1, Tessaloniki, Greece, Oct. 21. [8 A. K. Jain and A. ailaya. Sape-based retrieval: a case study wit trademark image database. attern ecognition, (9: , [9 I. King and J. Zong. Integrated probability function and its application to content-based image retrieval by relevance feedback. attern ecognition, 36(9: , 23. [1 J. Laaksonen, M. Koskela, and E. Oja. icsom: Self-organiing maps for content-based image retrieval. In roc. International Joint Conference on Neural Networks (IJCNN 99, Wasington, DC, USA, [11 S. MacArtur, C. Brodley, and C. Syu. elevance feedback decision trees in content-based image retrieval. In roc. IEEE Worksop on Content-based Access of lmage and ideo Libraries, pages 68 72, 2. [12 M. Maloof. Learning wen data sets are imbalanced and wen costs are unequal and unknown. In Worksop on ICML 23, 23. [13 O. L. Mangarasian. Nonlinear rogramming. McGraw ill, New ork, [14 F. Qian, M. Li, W.-. Ma, F. Lin, and B. Zang. Alternating feature spaces in relevance feedback. In 3rd Intl Worksop on Multimedia Information etrieval (MI21, 21. [15 G. atsc, S. Mika, B. Scolkopf, and K.-. Muller. Constructing boosting algoritms from svms: an application to one-class classification. IEEE Transactions on attern Analysis and Macine Intelligence, 24(9: , 22. [16. ui, T. uang, and S. Merotra. Content-based image retrieval wit relevance feedback in mars. In roc. IEEE International Conference on Image rocessing (ICI 97, pages , Wasington, DC, USA, Oct [17. ui, T. uang, M. Ortega, and S. Merotra. elevance feedback: A power tool in interactive content-based image retrieval. IEEE Trans. on Circuits and Systems for ideo Tecnology, 8(5: , Sept [18 B. Scolkof, A. J. Smola,. Williamson, and. Bartlett. New support vector algoritms. Neural Computation, 12: , 2. [19 B. Scolkopf, J. latt, J. Sawe-Taylor, A. J. Smola, and. C. Williamson. Constructing boosting algoritms from svms: an application to one-class classification. Neural Computation, 13(7: , 21. [2 A. Smeulders, M. Worring, S. Santini, A. Gupta, and. Jain. Contentbased image retrieval at te end of te early years. IEEE Trans. attern Analysis and Macine Intelligence, 22(12: , 2. [21 J. Smit and S.-F. Cang. Automated image retrieval using color and texture. IEEE Transaction on attern Analysis and Macine Intelligence, Nov [22 Z. Su,. Zang, and S. Ma. elevant feedback using a bayesian classifier in content-based image retrieval. In SIE Electronic Imaging 21, Jan. 21. [23 D. Tax and. Duin. Data domain description by support vectors. In roc. European Symp. Artificial Neural Network (ESANN 99, pages , Bruges, Belgium, [24 S. Tong and E. Cang. Support vector macine active learning for image retrieval. In roc. ACM Multimedia, pages , 21. [25 N. asconcelos and A. Lippman. Bayesian relevance feedback for content-based image retrieval. In roc. IEEE Worksop on Contentbased Access of Image and ideo Libraries, C, Sout Carolina, USA, 2. [26. Wu and B. S. Manjunat. Adaptive nearest neigbour searc for relevance feedback in large image database. In ACM Multimedia conference, 21. [27. an, A. auptmann, and. Jin. Negative pseudo-relevance feedback in content-based video retrieval. In ACM Multimedia (MM 3, Berkeley, CA, USA, 23.