CURL: Co-trained Unsupervised Representation Learning for Image Classification

Transcription

1 CURL: Co-traned Unsupervsed Representaton Learnng for Image Classfcaton Smone Banco, Ganlug Cocca, and Claudo Cusano arxv: v [cs.lg] Sep 0 Abstract In ths paper we propose a strategy for semsupervsed mage classfcaton that leverages unsupervsed representaton learnng and co-tranng. The strategy, that s called CURL from Co-traned Unsupervsed Representaton Learnng, teratvely bulds two classfers on two dfferent vews of the data. The two vews correspond to dfferent representatons learned from both labeled and unlabeled data and dffer n the fuson scheme used to combne the mage features. To assess the performance of our proposal, we conducted several experments on wdely used data sets for scene and object recognton. We consdered three scenaros (nductve, transductve and self-taught learnng) that dffer n the strategy followed to explot the unlabeled data. As mage features we consdered a combnaton of GIST, PHOG, and LBP as well as features extracted from a Convolutonal Neural Network. Moreover, two embodments of CURL are nvestgated: one usng Ensemble Projecton as unsupervsed representaton learnng coupled wth Logstc Regresson, and one based on LapSVM. The results show that CURL clearly outperforms other supervsed and semsupervsed learnng methods n the state of the art. Index Terms Image classfcaton, machne learnng algorthms, pattern analyss, sem-supervsed learnng. I. INTRODUCTION Sem-supervsed learnng [] conssts n takng nto account both labeled and unlabeled data when tranng machne learnng models. It s partcularly effectve when there s plenty of tranng data, but only a few nstances are labeled. In the last years, many sem-supervsed learnng approaches have been proposed ncludng generatve methods [], [], graph-based methods [4], [], and methods based on Support Vector Machnes [6], [7]. Co-tranng s another example of sem-supervsed technque [8]. It conssts n tranng two classfers ndependently whch, on the bass of ther level of confdence on unlabeled data, co-tran each other trough the dentfcaton of good addtonal tranng examples. S. Banco, and G. Cocca are wth the Department of Informatcs Systems and Communcatons, Unversty of Mlano-Bcocca, Mlano, 06 Italy, e-mal: {banco, cocca}@dsco.unmb.t. C. Cusano s wth the Dpartmento d Ingegnera Industrale e dell Informazone, Unversty of Pava, Pava, 700 Italy. e-mal: claudo.cusano@unpv.t The dfference between the two classfers s that they work on dfferent vews of the tranng data, often correspondng to two feature vectors. Poneerng works on co-tranng dentfed the condtonal ndependence between the vews as the man reason of ts success. More recently, t has been observed that condtonal ndependence s a suffcent, but not necessary condton, and that even a sngle vew can be consdered, provded that dfferent classfcaton technques are used [9]. In ths work we propose a sem-supervsed mage classfcaton strategy whch explots unlabeled data n two dfferent ways: frst two mage representatons are obtaned by unsupervsed representaton learnng (URL) on a set of mage features computed on all the avalable tranng data; then co-tranng s used to enlarge the labeled tranng set of the correspondng co-traned classfers (C). The dfference between the two mage representatons s that one s bult on the combnaton of all the mage features (early fuson), whle the other s the combnaton of sub-representatons separately bult on each feature (late fuson). We call the proposed strategy CURL: Co-traned Unsupervsed Representaton Learnng (from the combnaton of C and URL components). The schema of CURL s llustrated n Fg.. In standard co-tranng each classfer s bult on a sngle vew, often correspondng to a sngle feature. However, the combnaton of multple features s often requred to recognze complex vsual concepts [0] []. Both the classfers bult by CURL explot all the avalable mage features n such a way that these concepts can be accurately recognzed. We argue that the use of two dfferent fuson schemes together wth the non-lnear transformaton produced by the unsupervsed learnng procedure, makes the two mage representatons uncorrelated enough to allow an effectve co-tranng of the classfers. The proposed strategy s bult on two base components: URL (the unsupervsed representaton learnng) and C (the classfer used n co-tranng). By changng these two components we can have dfferent embodments of CURL that can be expermented and evaluated. To assess the merts of our proposal we conducted several experments on wdely used data sets: the -

2 Fg.. Schema of the proposed strategy. scene data set, the Caltech-0 object classfcaton data set, and the ILSVCR 0 data set whch contans 000 dfferent classes. We consdered a varety of scenaros ncludng transductve learnng (.e. unlabeled test data avalable durng tranng), nductve learnng (.e. test data not avalable durng tranng), and self-taught learnng (.e. test and tranng data comng from two dfferent data sets). In order to verfy the effcacy of the CURL classfcaton strategy, we also tested two embodments: one that uses Ensemble Projecton unsupervsed representaton coupled wth Logstc Regresson classfcaton, and one based on LapSVM sem-supervsed classfcaton. Moreover dfferent varants of the embodments are evaluated as well. The results show that CURL clearly outperforms other sem-supervsed learnng methods n the state of the art. II. RELATED WORK There s a large lterature on sem-supervsed learnng. For the sake of brevty, we dscuss only the paradgms nvolved n the proposed strategy. More nformaton about these and other approaches to sem-supervsed learnng can be found n the book by Chapelle et al. []. A. Co-tranng Blum and Mtchell proposed co-tranng n 998 [8] and verfed ts effectveness for the classfcaton of web pages. The basc dea s that two classfers are traned on separate vews (features) and then used to tran each other. More precsely, when one of the classfers s very confdent n makng a predcton for unlabeled data, the predcted labels are used to augment the tranng set of the other classfer. The concept has been generalzed to three [] or more vews [4], []. Co-tranng has been used n several computer vson applcatons ncludng vdeo annotaton [6], acton recognton [7], traffc analyss [8], speech and gesture recognton [9], mage annotaton [0], bometrc recognton [], mage retreval [], mage classfcaton [], object detecton [8], [4], and object trackng []. Accordng to Blum and Mtchell, a suffcent condton for the effectveness of co-tranng s that, besde beng ndvdually accurate, the two classfers are condtonally ndependent gven the class label. However, condtonal ndependence s not a necessary condton. In fact, Whang and Zhou [6] showed that co-tranng can be effectve when the dversty between the two classfers s larger than ther errors; ther results provded a theoretcal support to the success of sngle-vew co-tranng varants [7] [9] (the reader may refer to an updated study from the same authors [0] for more detals about necessary and suffcent condtons for co-tranng). B. Unsupervsed representaton learnng In the last years, as a consequence of the success of deep learnng frameworks we observed an ncreased nterest n methods that make use of unlabeled data to automatcally learn new representatons. In fact, these have been demonstrated to be very effectve for the pre-tranng of large neural networks [], []. Restrcted Boltzmann Machnes [] and auto-encoder networks [4] are notable examples of ths knd of methods. The tutoral by Bengo covers n detal ths famly of approaches []. A conceptually smpler approach conssts n usng clusterng algorthms to dentfy frequently occurrng patterns n unlabeled data that can be used to defne effectve representatons. The K-means algorthm has been wdely used for ths purpose [6]. In computer vson ths approach s very popular and lead to the many varants of bag-of-vsual-words representatons [7], [8]. Brefly, clusterng on unlabeled data s used to buld a vocabulary of vsual words. Gven an mage, multple local features are extracted and for each of them the most smlar vsual word s searched. The fnal representaton s a hstogram countng the occurrences of the vsual words. Sparse codng can be seen as an extenson of ths approach, where each local feature s descrbed as a sparse combnaton of multple words of the vocabulary [9] [4]. Another strategy for unsupervsed feature learnng s represented by Ensemble Projecton (EP) [4]. From

3 all the avalable data (labeled and unlabeled) Ensemble Projecton samples a set of prototypes. Dscrmnatve learnng s then used to learn projecton functons tuned to the prototypes. Snce a sngle set of projectons could be too nosy, multple sets of prototypes are sampled to buld an ensemble of projecton functons. The values computed accordng to these functons represent the components of the learned representatons. LapSVM [7] can be seen as an unsupervsed representaton learnng method as well. In ths case the learned representaton s not explct but t s mplctly embedded n a kernel learned from unlabeled data. C. Fuson schemes Combnng multmodal nformaton s an mportant ssue n pattern recognton. The fuson of multmodal nputs can brng complementary nformaton from varous sources, useful for mprovng the qualty of the mage retreval and classfcaton performance [4]. The problem arses n defnng how these modaltes are to be combned or fused. In general, the exstng fuson approaches can be categorzed as early and late fuson approaches, whch refers to ther relatve poston from the feature comparson or learnng step n the whole processng chan. Early fuson usually refers to the combnaton of the features nto a sngle representaton before comparson/learnng. Late fuson refers to the combnaton, at the last stage, of the responses obtaned after ndvdual features comparson or learnng [44], [4]. There s no unversal concluson as to whch strategy s the preferred method for for a gven task. For example, Snoek et al. [44] found that late fuson s better than early fuson n the TRECVID 004 semantc ndexng task, whle Ayache et al. [46] stated that early fuson gets better results than late fuson on the TRECVID 006 semantc ndexng task. A combnaton of these approaches can also be exploted as hybrd fuson approach [47]. Another form of data fuson s Multple Kernel Learnng (MKL). MKL has been ntroduced by Lanckret et al. [48] as extenson of the support vector machnes (SVMs). Instead of usng a sngle kernel computed on the mage representaton as n standard SVMs, MKL learns dstnct kernels. The kernels are combned wth a lnear or non lnear functon and the functon s parameters can be determned durng the learnng process. MKL can be used to learn dfferent kernels on the same mage representaton or by learnng dfferent kernels each one on a dfferent mage representaton [49]. The former corresponds to have dfferent noton of smlarty, and to choose the most sutable one for the problem and representaton at hand. The latter corresponds to have multple representatons each wth a, possbly, dfferent defnton of smlarty that must be combned together. Ths knd of data fuson, n [4], s termed ntermedate fuson. III. THE PROPOSED STRATEGY: CURL In the sem-supervsed mage classfcaton setup the tranng data conssts of both labeled examples {X l, Y} = {(x, y )} L and unlabeled ones X = u = {x } L+U =L+, where x denotes the feature vector of mage, y {,..., K} s ts label, and K s the number of classes. In ths work, for each mage a set of S dfferent mage features x (s), s =,..., S s consdered. Two vews are then generated by usng two dfferent fuson strateges: early and late fuson. In case of Early Fuson (), the mage features are concatenated and then used to learn a new representaton x = ϕ({[x (),..., x (S) ]} n an unsupervsed way, where ϕ( ) s a projecton functon. In case of Late Fuson (), an unsupervsed representaton ϕ s (x (s) ) s ndependently learned for each mage feature and then the representatons are concatenated to obtan x = [ϕ (x () ),..., ϕ S (x (S) )]. Usng the learned and unsupervsed representatons, the two vews are bult: Xl = {x } L =, Xu = {x } L+U and X = {x } L =L+ =, Xu = {x } L+U =L+. l Furthermore, two label sets Y and Y are ntalzed equal to Y. Once the two vews are generated, our method teratvely co-trans two classfers φ and φ on them [8]. SVMs, logstc regressons, or any other smlar technque can be used to obtan them. The dea of teratve co-tranng s that one can use a small labeled sample to tran the ntal classfers over the respectve vews (.e. φ : Xl Y and φ : Xl Y ), and then teratvely bootstrap by takng unlabeled examples for whch one of the classfers s confdent but the other s not. The confdent classfer determnes pseudo-labels [0] that are then used as f they were true labels to mprove the other classfer []. Gven the classfer confdence scores w = φ (x ) and w = φ (x ), the pseudo-labels ŷ and ŷ are respectvely obtaned as: ŷ ŷ = arg max j=,...,k w [j] () = arg max j=,...,k w [j] () In each round of co-tranng, the classfer φ chooses some examples n Xu to pseudo-label for φ, and vce versa. For each class k, let us call X the set of canddate

4 4 unlabeled examples to be pseudo-labeled for φ. Each x X must belong to the unlabeled set,.e. x Xu, has not to be already used for tranng,.e. x / Xl, and ts pseudo-label has to be ŷ = k. Furthermore, φ should be more confdent on the classfcaton of x than φ, and ts confdence should be hgher than a fxed threshold t : x X : w [k] < w [k], w [k] >t () If no x satsfyng Eq. are found, then the constrants are relaxed: x X : w [k] >t, wth t < t (4) Non-maxmum suppresson s appled to add one sngle pseudo-labeled example for each class by extractng the most confdent x X : fnd x X : w [k] = argmax wj [k] () j The selected x and ts correspondng pseudo-label ŷ are added to Xl and Y respectvely. If no x satsfyng Eq. 4 are found, then nothng s added to Xl and Y. Smlarly, the classfer φ chooses some examples n Xu to pseudo-label for φ. At the next co-tranng round, two new classfers φ and φ are traned on the respectve vews, that now contan both labeled and pseudo-labeled examples. The complete procedure of the CURL method s outlned n Algorthms -. IV. EXPERIMENTS CURL s parametrc wth respect to the projecton functon ϕ used n the unsupervsed representaton learnng URL, and the supervsed classfcaton technque C used durng to co-tran φ and φ. As frst embodment of CURL, we used Ensemble Projecton [4] for the former and logstc regresson for the latter. Another embodment, based on LapSVM [7] s presented n Secton V-C. A. Data sets We evaluated our method on two data sets: Scene- (S-) [8], and Caltech-0 (C-0) []. Scene- data set contans 448 mages dvded nto scene categores wth both ndoor and outdoor envronments. Each category has 00 to 400 mages. Caltech-0 contans 8677 mages dvded nto 0 object categores, each havng to 800 mages. Furthermore, we collected a set of random mages by samplng 0,000 mages from the ImageNet data set [] to evaluate our method on the Algorthm : CURL Data: Labeled data {X l, Y}, unlabeled data X u Result: Classfers φ ( ) and φ ( ) begn [Xl,Xu,Xl,Xu ] = computeurl(x l, X u ) Y = Y = Y tran classfer φ : Xl Y tran classfer φ : Xl Y for co-tranng round c = : C do ntalze W = W = Ŷ = Ŷ = foreach x Xu do add w = φ (x ) to W add ŷ = arg max [j] to Ŷ j=,...,k w foreach x Xu do add w = φ (x ) to W add ŷ = arg max [j] to Ŷ j=,...,k w for class number k = : K do for (v, v ) {(,), (,)} do fnd {X, Ŷv } wth X Xu v, X X v l =, Ŷ v s.t.: Ŷv x X and ŷ v Ŷv, hold: ŷ v = k, w v [k] < w v [k], w v [k] >t f X = then fnd {X, Ŷv } wth X Xu v, X X v =, Ŷ v Ŷv s.t.: x X and ŷ v Ŷv ŷ v = k, w v [k] > t l, hold: [X,Ŷv ]=nonmaxsuppr(x,ŷv,w v ) X v l = X v l X Y v = Y v Ŷv tran classfer φ : Xl Y tran classfer φ : Xl Y Algorthm : compute URL Data: Labeled data X l and unlabeled data X u Result: Unsup. representatons Xl, Xu begn Learn representaton ϕ on {[x () Xl = ϕ({[x (),..., x (S) ]} L = ) Xu = ϕ({[x (),..., x (S) ]} L+U =L+ ) Learn representatons ϕ s on {x (s) Xl = {[ϕ (x () ),..., ϕ S (x (S) )]} L = X u = {[ϕ (x () ),..., ϕ S (x (S), X l, X u,..., x (S) ]} L+U = } L+U = )]} L+U =L+

5 Algorthm : non-maxmum suppresson Data: X, Ŷ, W, k Result: X, Ŷ begn fnd {X, Ŷ } wth X X, Ŷ Ŷ s.t.: w [k] = argmax w j [k], wth w j W j task of self-taught mage classfcaton. Snce the current verson of ImageNet has 84 synsets (.e. categores) and a total of more than 4 mllons mages, there s a small probablty that the random mages and mages n the two consdered data sets come from the same dstrbuton. B. Image features In our experments we used the followng three features: GIST [4], Pyramd of Hstogram of Orented Gradents (PHOG) [], and Local Bnary Patterns (LBP) [6]. GIST was computed on the rescaled mages of 6 6 pxels, at scales wth 4, 8 and 8 orentatons respectvely. PHOG was computed wth a -layer pyramd and n 8 drectons. Unform LBP wth radus equal to, and 8 neghbors was used. In Secton V-B we also nvestgate the use of features extracted from a CNN [7] n combnaton wth the prevous ones. C. Ensemble projecton Dfferently from others sem-supervsed methods that tran a classfer from labeled data wth a regularzaton term learned from unlabeled data, Ensemble Projecton [4] learns a new mage representaton from all known data (.e. labeled and unlabeled data), and then trans a plan classfer on t. Ensemble Projecton learns knowledge from T dfferent prototype sets P t = {(s t, ct )}rn =, wth t {,..., T } where s t {,..., L + U} s the ndex of the th chosen mage, c t {,..., r} s the pseudolabel ndcatng to whch prototype s t belong to. r s the number of prototypes n P t, and n s the number of mages sampled for each prototype. For each prototype set, m hypotheses are randomly sampled, and the one contanng mages havng the largest mutual dstance s kept. A set of dscrmnatve classfers φ t ( ) s learned on P t, one for each prototype set, and the projected vectors φ t (x ) are obtaned. The fnal feature vector s obtaned by concatenatng these projected vectors. Followng [4] we set T = 00, r = 0, n = 6, m = 0, usng Logstc Regresson (LR) as dscrmnatve classfer φ t ( ) wth C =. Wthn CURL, Ensemble Projecton s used to learn both Early Fuson and Late Fuson unsupervsed representatons. In the case of Early Fuson (), the feature vector x s obtaned concatenatng the S dfferent features avalable x = [x (),..., x (S) ], s =,..., S. In the case of Late Fuson (), the feature vector x s made by consderng just one sngle feature at tme x = x (s). For both and, the same number T of prototypes s used n order to assure that the unsupervsed representatons have the same sze. D. Expermental settngs We conducted two knds of experments: () comparson of our strategy wth competng methods for sem-supervsed mage classfcaton; () evaluaton of our method at dfferent number of co-tranng rounds. We consdered three scenaros correspondng to three dfferent ways of usng unlabeled data. In the nductve learnng scenaro % of the unlabeled data s used together wth the labeled data for the sem-supervsed tranng of the classfer; the remanng 7% s used as an ndependent test set. In the transductve learnng scenaro all the unlabeled data s used durng both tranng and test. In the self-taught learnng scenaro the set of unlabeled data s taken from an addtonal data set featurng a dfferent dstrbuton of mage content (.e. the 0,000 mages from ImageNet); all the unlabeled data from the orgnal data set s used as an ndependent test set. As evaluaton measure we followed [4] and used the mult-class average precson (MAP), computed as the average precson over all recall values and over all classes. Dfferent numbers of tranng mages per class were tested for both Scene- and Caltech-0 (.e.,,,, 0, and 0). All the reported results represent the average performance over ten runs wth random labeledunlabeled splts. The performance of the proposed strategy are compared wth those of other supervsed and semsupervsed baselne methods. As supervsed classfers we consdered Support Vector Machnes (SVM). As sem-supervsed classfers, we used LapSVM [7], [8]. LapSVM extend the SVM framework ncludng a smoothness penalty term defned on the Laplacan adjacency graph bult from both labeled and unlabeled data. For both SVM and LapSVM we expermented wth the lnear, RBF and χ kernels computed on the concatenaton of the three avalable mage features as

6 6 n [4]. The parameters of SVM and LapSVM have been determned by a greedy search wth a three-fold cross valdaton on the tranng set. We also compared the present embodment of CURL aganst Ensemble Projecton coupled wth a logstc regresson classfer (EP+LR) as n [4]. V. EXPERIMENTAL RESULTS As a frst experment we compared CURL aganst EP+LR, and aganst SVMs and LapSVMs wth dfferent kernels. Specfcally, we tested the two co-traned classfers operatng on early-fused and late-fused representatons, both employng EP for URL and LR as classfer C, that we call CURL-(EP+LR) and CURL-(EP+LR) respectvely. We also ncluded a varant of the proposed method. It dffers n the number of pseudo-labeled examples that are added at each co-tranng round. The varant skps the non-maxmum suppresson step, and at each round, adds all the examples satsfyng Eq.. We denote the two co-traned classfers of the varant as CURL- n (EP+LR) and CURL- n (EP+LR). Fg. shows the classfcaton performance wth dfferent numbers of labeled tranng mages per class, n the three learnng scenaros for both the Scene- and Caltech-0 data sets. For the CURL-based methods we consdered fve co-tranng rounds, and the reported performance correspond to the last round. For SVM and LapSVM only the results usng χ kernel are reported, snce they consstently showed the best performance across all the experments. Detaled results for all the tested baselne methods, and for the CURL varants across the co-tranng rounds are avalable n Tables I, II and III. The behavor of the methods s qute stable wth respect to the three learnng scenaros, wth slghtly lower MAP obtaned n the case of self-taught learnng. It s evdent that our strategy outperformed the other methods n the state of the art ncluded n the comparson across all the data sets and all the scenaros consdered. Among the varants consdered, CURL-(EP+LR) demonstrated to be the best n the case of a small number of labeled mages, whle CURL- n (EP+LR) obtaned the best results when more labeled data s avalable. Classfers obtaned on early-fused representatons performed generally worse than the correspondng ones obtaned on late-fused representatons, but they are stll unformly better than the orgnal EP+LR Ensemble Projecton whch can be consdered as ther non-cotraned verson. SVMs and LapSVMs performed poorly on the Scene- data set, but they outperformed EP+LR and some of the CURL varants on the Caltech-0 data set. TABLE I MEAN AVERAGE PRECISION (MAP) OF THE BASELINE ALGORITHMS, VARYING THE NUMBER OF LABELED IMAGES PER CLASS IN THE THREE LEARNING SCENARIOS CONSIDERED: INDUCTIVE (IND), TRANSDUCTIVE (TRD), AND SE-TAUGHT (ST). Scene- # mg method IND TRD ST SVMln.7.. SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR SVMln SVMrbf.9.. SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR SVMln SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR SVMln... SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR SVMln SVMrbf SVM χ... LapSVMln.4.. LapSVMrbf.4..0 LapSVM χ EP+LR SVMln SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR SVMln SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR Caltech-0 # mg method IND TRD ST SVMln SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR SVMln SVMrbf SVM χ LapSVMln..8. LapSVMrbf.7..4 LapSVM χ EP+LR.6.. SVMln SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ EP+LR..7. SVMln.4.. SVMrbf SVM χ LapSVMln LapSVMrbf LapSVM χ.7..4 EP+LR SVMln SVMrbf SVM χ LapSVMln LapSVMrbf -.6. LapSVM χ EP+LR SVMln SVMrbf -.. SVM χ LapSVMln -.9. LapSVMrbf LapSVM χ EP+LR SVMln -.. SVMrbf SVM χ -.. LapSVMln LapSVMrbf LapSVM χ EP+LR Co-tranng allows to make good use of the early fuson representatons that otherwse lead to worse results than late fuson representatons. In our opnon ths happens because the two vews capture dfferent relatonshps among data. Ths fact s vsble n Fg., whch shows D projectons obtaned by applyng the t-sne [9] method to GIST, PHOG, LBP features, ther concatenaton, and ther learnt early- and late-fused representatons. Unsupervsed representaton learnng allows t-sne to dentfy groups of mages of the same class. Moreover, representatons based on early and late fuson nduce dfferent relatonshps among the classes. For nstance, n the second row of Fg. f the blue and the lght green classes have been placed close to each other on the bottom rght; n Fg. e, nstead, the two classes are well separated. The dfference n

7 7 Scene- nductve Scene- transductve Scene- self-taught Caltech-0 nductve Caltech-0 transductve Caltech-0 self-taught Fg.. Mean Average Precson (MAP) varyng the number of labeled mages per class, obtaned on the Scene- data set (frst row), and on the Caltech-0 data set (second row). Three scenaros are consdered: nductve learnng (left column), transductve learnng (mddle column) and self-taught learnng (thrd column). Note that nductve learnng on the Caltech-0 data set s lmted to labeled mages per class because otherwse for some classes there wouldn t be enough unlabeled data left for both tranng and evaluaton. (a) GIST (b) PHOG (c) LBP (d) concatenaton (e) early fuson (f) late fuson Fg.. t-sne D projectons for the dfferent features used. They are relatve to the Scene- (top row) and Caltech-0 (bottom row) data sets. Dfferent classes are represented n dfferent colors, and the same class wth the same color across the row. the two representatons explans the effectveness of co-tranng and justfes the dfference n performance between CURL-(EP+LR) and CURL-(EP+LR). As further nvestgaton, we also combned the two classfers produced by the co-tranng procedure obtan- ng two other varants of CURL that we denoted as CURL-&(EP+LR) and CURL-&n (EP+LR). However, n our experments, these varants dd not caused any sgnfcant mprovement when compared to CURL-(EP+LR).

8 8 TABLE II MEAN AVERAGE PRECISION (MAP) OF THE CURL VARIANTS, IN THE (EP+LR) EMBODIMENT, VARYING THE NUMBER OF LABELED IMAGES PER CLASS AT THE DIFFERENT CO-TRAINING ROUNDS OBTAINED ON THE SCENE- DATA SET IN THE THREE LEARNING SCENARIOS CONSIDERED: INDUCTIVE (LT), TRANSDUCTIVE (MIDDLE), AND SE-TAUGHT (RIGHT). FOR CLARITY, THE (EP+LR) SUFFIXES HAVE BEEN OMITTED. # co-tran round # mg method 0 4 CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n # co-tran round # mg method 0 4 CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n # co-tran round # mg method 0 4 CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n A. Performance across co-tranng rounds Here we analyze n more detals the performance of our strategy across the fve co-tranng rounds. Results are reported n Fg. 4 wth lnes of ncreasng color saturaton correspondng to rounds one to fve. CURL-(EP+LR) s reported n red lnes, whle CURL- n (EP+LR) n blue. Results are reported n terms of MAP mprovements wth respect to EP+LR, whch, we recall, corresponds to CURL-(EP+LR) wth zero cotranng rounds. For CURL-(EP+LR), performances always ncrease wth the number of rounds. For CURL- n (EP+LR), ths s not true on the Scene- data set wth a small number of labeled examples. In CURL- n (EP+LR) each round of co-tranng adds all the promsng unlabeled samples, wth a hgh chance of ncludng some of them wth the wrong pseudo-label. Ths may result n a concept drft, wth the classfers beng pulled away from the concepts represented by the labeled examples. Ths rsk s lower on the Caltech-0 (whch tends to have more homogeneous classes than Scene-) and when there are more labeled mages. The orgnal CURL-(EP+LR) s more conservatve, snce each of ts co-tranng rounds adds a sngle mage per class. As a result, ncreasng the rounds usually ncreases MAP and never decreases t by an apprecable amount. We observed the same behavor for CURL- (EP+LR) and CURL- n (EP+LR). We omt the relatve fgures for sake of brevty. The plots confrm that CURL-(EP+LR) s better suted for small sets of labeled mages, whle CURL- n (EP+LR) s to be preferred when more labeled examples are avalable. The representaton learned from late fused features explans part of the effectveness of CURL. In fact, even CURL-(EP+LR) wthout cotranng (zero rounds) outperforms the baselne represented by Ensemble Projecton. B. Leveragng CNN features n CURL In ths further experment we want to test f the proposed classfcaton strategy works when more powerful features are used. Recent results ndcate that the generc descrptors extracted from pre-traned Convolutonal Neural Networks (CNN) are able to obtan consstently superor results compared to the hghly tuned state of the art systems n all the vsual classfcaton tasks on varous datasets [7]. We extract a 4096-

9 9 TABLE III MEAN AVERAGE PRECISION (MAP) OF THE CURL VARIANTS, IN THE (EP+LR) EMBODIMENT, VARYING THE NUMBER OF LABELED IMAGES PER CLASS AT THE DIFFERENT CO-TRAINING ROUNDS OBTAINED ON THE CALTECH-0 DATA SET IN THE THREE LEARNING SCENARIOS CONSIDERED: INDUCTIVE (LT), TRANSDUCTIVE (MIDDLE), AND SE-TAUGHT (RIGHT). FOR CLARITY, THE (EP+LR) SUFFIXES HAVE BEEN OMITTED. # co-tran round # mg method 0 4 CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n # co-tran round # mg method 0 4 CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n # co-tran round # mg method 0 4 CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n CURL CURL CURL-& CURL-n CURL-n CURL-&n dmensonal feature vector from each mage usng the Caffe [60] mplementaton of the deep CNN descrbed by Krzhevsky et al. [6]. The CNN was dscrmnatvely traned on a large dataset (ILSVRC 0) wth magelevel annotatons to classfy mages nto 000 dfferent classes. Brefly, a mean-subtracted 7 7 RGB mage s forward propagated through fve convolutonal layers and two fully connected layers. Features are obtaned by extractng actvaton values of the last hdden layer. More detals about the network archtecture can be found n [60], [6]. We leverage the CNN features n CURL usng them as a fourth feature n addton to the three used n Secton IV. The dscrmnatve power of these CNN features alone can be seen n Fg., where ther D projectons obtaned applyng the t-sne [9] method are reported. The expermental results usng the four features, are reported n Fg. 6, for both the Scene- and Caltech- 0 data sets. We report the results n the transductve scenaro only. It can be seen that the results usng the four features are sgnfcantly better than those usng only three features manly due to the dscrmnatve power of the CNN features. Furthermore, the CURL varants acheve better results than the baselnes. Ths suggests that CURL s able to effectvely leverage both low/md level features as LBP, PHOG and GIST, and more powerful features as CNN. C. Second embodment of CURL usng LapSVM In ths Secton we want to evaluate the CURL performance n a dfferent embodment. Specfcally, we substtute the EP and LR components wth LapSVMbased ones. In the LapSVM, frst, an unsupervsed geometrcal deformaton of the feature kernel s performed. Ths deformed kernel s then used for classfcaton by a standard SVM thus by-passng an explct defnton of a new feature representaton. In ths CURL embodment we explot the unsupervsed step as surrogate of the URL component, and SVM as C component. The vew s obtaned concatenatng the GIST, PHOG, LBP and CNN features and generatng the correspondng kernel, whle the one s obtaned by a lnear combnaton of the four kernels computed on each feature. Ths s smlar to what s done n multple kernel learnng [49]. Due to ts performance n the prevous experments, the χ kernel s used for both vews. The expermental

10 0 Scene- nductve Scene- transductve Scene- self-taught Caltech-0 nductve Caltech-0 transductve Caltech-0 self-taught Fg. 4. Performance obtaned by CURL-(EP+LR) and CURL- n(ep+lr) varyng the number of co-tranng rounds. Performance are reported n terms of MAP mprovement wth respect to Ensemble Projecton. Due to the small cardnalty of some classes, nductve learnng on the Caltech-0 has been lmted to fve labeled mages per class. Scene- Caltech-0 Fg.. D projectons for the CNN features on the two data sets used: Scene- (left) and Caltech-0 (rght). Dfferent classes are represented n dfferent colors. results on the Scene- and Caltech-0 data sets n the transductve scenaro, are reported n Fg. 7. We named the varants of ths CURL embodment by addng the suffx (LapSVM). It can be seen that the behavor of the dfferent methods s the same of the prevous plots, wth the LapSVM-based CURL outperformng the standard LapSVM. The plots confrm that CURL-(LapSVM) s better suted for small sets of labeled mages, whle CURL- n (LapSVM) s to be preferred when more labeled examples are avalable. In Fg. 9 and 0 qualtatve results for the Panda class of the Caltech-0 data set are reported: the results are relatve to the case n whch a sngle nstance s avalable for tranng and one sngle example s added at each co-tranng round (.e. each par of rows correspond to CURL-(LapSVM) and CURL-(LapSVM) respectvely). The left part of Fg. 9 contans the tranng examples that are added by the CURL-(LapSVM) and CURL-(LapSVM) at each co-tranng round, whle the rght part and Fg. 0 contan the frst 40 test mages ordered by decreasng classfcaton confdence. Samples belongng to the current class are surrounded by a green boundng box, whle a red one s used for samples belongng to other classes. In the sets of tranng mages, t s possble to see that after the frst co-tranng round, CURL-(LapSVM) selects new examples to add to the tranng set, whle CURL-(LapSVM) adds examples seleted by CURL-

11 Scene- transductve Caltech-0 transductve Fg. 6. Mean Average Precson (MAP) varyng the number of labeled mages per class, obtaned on the Scene- data set (left), and on the Caltech-0 data set (rght). Results are obtaned usng GIST, PHOG, LBP and CNN features. Scene- transductve Caltech-0 transductve Fg. 7. Mean Average Precson (MAP) varyng the number of labeled mages per class, obtaned on the Scene- data set (left), and on the Caltech-0 data set (rght). Results are obtaned usng GIST, PHOG, LBP and CNN features. (LapSVM) n the prevous round. Ths s a pattern that we found to occur also n other categores when very small tranng sets are used. In the sets of test mages, t s possble to see that more and more postve mages are recovered. Moreover, we can see how the mages belongng to the correct class tends to be classfed wth ncreasng confdence and move to the left, whle the confdences of mages belongng to other classes decrease and are pushed to the rght. D. Large scale experment In ths experment we want to test the proposed classfcaton strategy on a large scale data set, namely the ILSVRC 0 whch contans a total of 000 dfferent classes. The experment s run on the ILSVRC 0 valdaton set snce the tranng set was used to learn the CNN features. The ILSVRC 0 valdaton set, whch contans a total of 0 mages for each class, has been randomly dvded nto a tranng and a test set contanng each mages per class. Agan, dfferent numbers of tranng mages per class were tested (.e.,,,, 0, and 0). The second embodment of CURL s used n ths experment. The expermental results are reported n Fg. 8 and represent the average performance over ten runs wth random labeled-unlabeled feature splts. Gven the large range of MAP values, the plot of MAP mprovements wth respect to LapSVM baselne s also reported. It can be seen that the behavor s smlar to that of the prevous plots, wth the LapSVM-based CURL varants outperformng the LapSVM. As for the prevous data sets, the plots show that CURL-(LapSVM) and CURL-(LapSVM) are better suted for small sets of labeled mages, whle CURL- n (LapSVM)and CURL- n (LapSVM) are to be preferred when more labeled examples are avalable. It s remarkable that the proposed classfcaton strategy s able to mprove the results of the LapSVM, snce the CNN features were specfcally

12 learned for the ILSVRC 0. VI. CONCLUSIONS In ths work we have proposed CURL, a semsupervsed mage classfcaton strategy whch explots unlabeled data n two dfferent ways: frst two mage representatons are obtaned by unsupervsed learnng; then co-tranng s used to enlarge the labeled tranng set of the correspondng classfers. The two mage representatons are bult usng two dfferent fuson schemes: early fuson and late fuson. The proposed strategy has been tested on the Scene-, Caltech-0, and ILSVRC 0 data sets, and compared wth other supervsed and sem-supervsed methods n three dfferent expermental scenaros: nductve learnng, transductve learnng, and self-taught learnng. We tested two embodments of CURL and several varants dfferng n the co-traned classfer used and n the number of pseudo-labeled examples that are added at each co-tranng round. The expermental results showed that the CURL embodments outperformed the other methods n the state of the art ncluded n the comparsons. In partcular, the varants that add a sngle pseudo-labeled example per class at each co-tranng round, resulted to perform best n the case of a small number of labeled mages, whle the varants addng more examples at each round obtaned the best results when more labeled data are avalable. Moreover, the results of CURL usng a combnaton of low/md and hgh level features (.e. LBP, PHOG, GIST, and CNN features) outperform those obtaned on the same features by state of the art methods. Ths means that CURL s able to effectvely leverage less dscrmnatve features (.e. LBP, PHOG, GIST) to boost the performance of more dscrmnatve ones (.e. CNN features). RERENCES [] O. Chapelle, B. Schölkopf, A. Zen et al., Sem-supervsed learnng. MIT press, 006. [] K. Ngam, A. McCallum, S. Thrun, and T. Mtchell, Text classfcaton from labeled and unlabeled documents usng em, Machne learnng, vol. 9, no. -, pp. 0 4, 000. [] A. Fujno, N. Ueda, and K. Sato, A hybrd generatve/dscrmnatve approach to sem-supervsed classfer desgn, n Proc. of the Natonal Conf. on Artfcal Intellgence, 00, pp [4] A. Blum and S. Chawla, Learnng from labeled and unlabeled data usng graph mncuts, n Proc. 8th Int l Conf. on Machne Learnng, 00, pp [] O. Chapelle, J. Weston, and B. Schölkopf, Cluster kernels for sem-supervsed learnng, n Advances n neural nformaton processng systems, 00, pp [6] T. Joachms, Transductve nference for text classfcaton usng support vector machnes, n Proc. 6th Int l Conf. on Machne Learnng, vol. 99, 999, pp [7] M. Belkn, P. Nyog, and V. Sndhwan, Manfold regularzaton: A geometrc framework for learnng from labeled and unlabeled examples, The Journal of Machne Learnng Research, vol. 7, pp , 006. [8] A. Blum and T. Mtchell, Combnng labeled and unlabeled data wth co-tranng, n Proc. of the th annual Conf. on Computatonal learnng theory, 998, pp [9] Z.-H. Zhou and M. L, Sem-supervsed learnng by dsagreement, Knowledge and Informaton Systems, vol. 4, no., pp. 4 49, 00. [0] G. Iyengar and H. J. Nock, Dscrmnatve model fuson for semantc concept detecton and annotaton n vdeo, n Proceedngs of the eleventh ACM nternatonal conference on Multmeda, 00, pp. 8. [] P. Gehler and S. Nowozn, On feature combnaton for multclass object classfcaton, n Computer Vson, 009 IEEE th Internatonal Conference on, 009, pp. 8. [] P. Natarajan, S. Wu, S. Vtaladevun, X. Zhuang, S. Tsakalds, U. Park, and R. Prasad, Multmodal feature fuson for robust event detecton n web vdeos, n Computer Vson and Pattern Recognton (CVPR), 0 IEEE Conference on, 0, pp [] Z.-H. Zhou and M. L, Tr-tranng: Explotng unlabeled data usng three classfers, Knowledge and Data Engneerng, IEEE Transactons on, vol. 7, no., pp. 9 4, 00. [4] M. L and Z.-H. Zhou, Improve computer-aded dagnoss wth machne learnng technques usng undagnosed samples, Systems, Man and Cybernetcs, Part A: Systems and Humans, IEEE Transactons on, vol. 7, no. 6, pp , 007. [] Z.-H. Zhou, When sem-supervsed learnng meets ensemble learnng, Fronters of Electrcal and Electronc Engneerng n Chna, vol. 6, no., pp. 6 6, 0. [6] M. Wang, X.-S. Hua, and Y. Da, L-R.and Song, Enhanced sem-supervsed learnng for automatc vdeo annotaton, n IEEE Int l Conf. on Multmeda and Expo, 006, pp [7] S. Gupta, J. Km, K. Grauman, and R. Mooney, Watch, lsten & learn: Co-tranng on captoned mages and vdeos, n Machne Learnng and Knowledge Dscovery n Databases, 008, pp [8] A. Levn, P. Vola, and Y. Freund, Unsupervsed mprovement of vsual detectors usng cotranng, n Proc. of IEEE Int l Conf. on Computer Vson, 00, pp [9] C. Chrstoudas, K. Saenko, L. Morency, and T. Darrell, Coadaptaton of audo-vsual speech and gesture classfers, n Proc. of the Int l Conf. on Multmodal nterfaces, 006, pp [0] H. Feng and T.-S. Chua, A bootstrappng approach to annotatng large mage collecton, n Proc. of the ACM SIGMM Int l Workshop on Multmeda Informaton Retreval, 00, pp. 6. [] H. Bhatt, S. Bharadwaj, R. Sngh, M. Vatsa, A. Noore, and A. Ross, On co-tranng onlne bometrc classfers, n Int l Jont Conf. on Bometrcs, 0, pp. 7. [] S. Tong and E. Chang, Support vector machne actve learnng for mage retreval, n Proc. of ACM Int l Conf. on Multmeda, 00, pp [] M. Gullaumn, J. Verbeek, and C. Schmd, Multmodal semsupervsed learnng for mage classfcaton, n IEEE Conf. on Computer Vson and Pattern Recognton, 00, pp [4] O. Javed, S. Al, and M. Shah, Onlne detecton and classfcaton of movng objects usng progressvely mprovng detectors, n Computer Vson and Pattern Recognton, 00.

13 ILSVRC 0 transductve ILSVRC 0 transductve Fg. 8. Mean Average Precson (MAP) varyng the number of labeled mages per class, obtaned on the ILSVRC 0 data set: MAP values (left) and MAP mprovements over LapSVM baselne (rght). Results are obtaned usng GIST, PHOG, LBP and CNN features. CVPR 00. IEEE Computer Socety Conference on, vol.. IEEE, 00, pp [] F. Tang, S. Brennan, Q. Zhao, and H. Tao, Co-trackng usng sem-supervsed support vector machnes, n Computer Vson, 007. ICCV 007. IEEE th Internatonal Conference on. IEEE, 007, pp. 8. [6] W. Wang and Z. Zhou, Analyzng co-tranng style algorthms, n Proc. of the European Conf. on Machne Learnng, 007, pp [7] S. Goldman and Y. Zhou, Enhancng supervsed learnng wth unlabeled data, n Proc. of the Int l Conf on Machne Learnng, 000, pp [8] M. Chen, Y. Chen, and K. Q. Wenberger, Automatc feature decomposton for sngle vew co-tranng, n Proceedngs of the 8th Internatonal Conference on Machne Learnng (ICML- ), 0, pp [9] W. Wang and Z.-H. Zhou, Co-tranng wth nsuffcent vews, n Asan Conference on Machne Learnng, 0, pp [0] W. Wang and Z. Zhou, A new analyss of co-tranng, n Proc. of the Int l Conf on Machne Learnng, 00, pp. 4. [] G. Hnton, S. Osndero, and Y. Teh, A fast learnng algorthm for deep belef nets, Neural computaton, vol. 8, no. 7, pp. 7 4, 006. [] K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, What s the best mult-stage archtecture for object recognton? n IEEE Int l Conf. on Computer Vson, 009, pp. 46. [] G. Hnton, Tranng products of experts by mnmzng contrastve dvergence, Neural computaton, vol. 4, no. 8, pp , 00. [4] H. Bourlard and Y. Kamp, Auto-assocaton by multlayer perceptrons and sngular value decomposton, Bologcal cybernetcs, vol. 9, no. 4-, pp. 9 94, 988. [] Y. Bengo, Learnng deep archtectures for a, Foundatons and trends n Machne Learnng, vol., no., pp. 7, 009. [6] A. Coates and A. Y. Ng, Learnng feature representatons wth k-means, n Neural Networks: Trcks of the Trade, 0, pp [7] G. Csurka, C. Dance, L. Fan, J. Wllamowsk, and C. Bray, Vsual categorzaton wth bags of keyponts, n ECCV Workshop on statstcal learnng n computer vson, 004, pp.. [8] S. Lazebnk, C. Schmd, and J. Ponce, Beyond bags of features: Spatal pyramd matchng for recognzng natural scene categores, n IEEE Conf. on Computer Vson and Pattern Recognton, vol., 006, pp [9] B. A. Olshausen and D. Feld, Sparse codng wth an overcomplete bass set: A strategy employed by v? Vson research, vol. 7, no., pp., 997. [40] J. Maral, F. Bach, J. Ponce, and G. Sapro, Onlne learnng for matrx factorzaton and sparse codng, The J. of Machne Learnng Research, vol., pp. 9 60, 00. [4] M. Lewck and T. Sejnowsk, Learnng overcomplete representatons, Neural computaton, vol., no., pp. 7 6, 000. [4] D. Da and L. V. Gool, Ensemble projecton for semsupervsed mage classfcaton, n Computer Vson (ICCV), 0 IEEE Internatonal Conference on. IEEE, 0, pp [4] P. K. Atrey, M. A. Hossan, A. El Saddk, and M. S. Kankanhall, Multmodal fuson for multmeda analyss: a survey, Multmeda systems, vol. 6, no. 6, pp. 4 79, 00. [44] C. G. Snoek, M. Worrng, and A. W. Smeulders, Early versus late fuson n semantc vdeo analyss, n Proceedngs of the th annual ACM nternatonal conference on Multmeda. ACM, 00, pp [4] W. S. Noble et al., Support vector machne applcatons n computatonal bology, Kernel methods n computatonal bology, pp. 7 9, 004. [46] S. Ayache, G. Quénot, and J. Gensel, Classfer fuson for SVMbased multmeda semantc ndexng. Sprnger, 007. [47] Z. Wu, L. Ca, and H. Meng, Mult-level fuson of audo and vsual features for speaker dentfcaton, n Advances n Bometrcs. Sprnger, 00, pp [48] G. R. Lanckret, N. Crstann, P. Bartlett, L. E. Ghaou, and M. I. Jordan, Learnng the kernel matrx wth semdefnte programmng, The Journal of Machne Learnng Research, vol., pp. 7 7, 004. [49] M. Gönen and E. Alpaydın, Multple kernel learnng algorthms, The Journal of Machne Learnng Research, vol., pp. 68, 0. [0] D.-H. Lee, Pseudo-label: The smple and effcent semsupervsed learnng method for deep neural networks, n Workshop on Challenges n Representaton Learnng, ICML, 0. [] M.-F. Balcan, A. Blum, and K. Yang, Co-tranng and expanson: Towards brdgng theory and practce, n Advances n neural nformaton processng systems, 004, pp [] L. Fe-Fe, R. Fergus, and P. Perona, Learnng generatve vsual models from few tranng examples: An ncremental bayesan approach tested on 0 object categores, Computer Vson and Image Understandng, vol. 06, no., pp. 9 70, 007.

14 4 round 0 round round round round 4 round Tranng mages Test mages Fg. 9. Qualtatve results of the proposed strategy for the Panda class of the Caltech-0 data set over fve co-tranng rounds. Tran mages are on the left, the frst 7 test mages, ordered by decreasng classfcaton confdence are on the rght. Test mages from 8 to 40 are reported n Fg. 0. [] J. Deng, W. Dong, R. Socher, L.-J. L, K. L, and L. FeFe, Imagenet: A large-scale herarchcal mage database, n Computer Vson and Pattern Recognton, 009. CVPR 009. IEEE Conference on. IEEE, 009, pp. 48. [4] A. Olva and A. Torralba, Modelng the shape of the scene: A holstc representaton of the spatal envelope, Internatonal journal of computer vson, vol. 4, no., pp. 4 7, 00. [] A. Bosch, A. Zsserman, and X. Muoz, Image classfcaton usng random forests and ferns, n Computer Vson, 007. ICCV 007. IEEE th Internatonal Conference on, Oct 007, pp. 8. [6] T. Ojala, M. Petkanen, and T. Maenpaa, Multresoluton gray-scale and rotaton nvarant texture classfcaton wth local bnary patterns, Pattern Analyss and Machne Intellgence, IEEE Transactons on, vol. 4, no. 7, pp , 00. [7] A. S. Razavan, H. Azzpour, J. Sullvan, and S. Carlsson, Cnn features off-the-shelf: an astoundng baselne for recognton, n Computer Vson and Pattern Recognton Workshops (CVPRW), 04 IEEE Conference on. IEEE, 04, pp. 9. [8] V. Sndhwan, P. Nyog, and M. Belkn, Beyond the pont cloud: from transductve to sem-supervsed learnng, n Proceedngs of the nd nternatonal conference on Machne learnng. ACM, 00, pp [9] L. Van der Maaten and G. Hnton, Vsualzng data usng tsne, Journal of Machne Learnng Research, vol. 9, no. 7960, p. 8, 008. [60] Y. Ja, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Grshck, S. Guadarrama, and T. Darrell, Caffe: Convolutonal archtecture for fast feature embeddng, n Proceedngs of the ACM Internatonal Conference on Multmeda. ACM, 04, pp [6] A. Krzhevsky, I. Sutskever, and G. E. Hnton, Imagenet classfcaton wth deep convolutonal neural networks, n Advances n neural nformaton processng systems, 0, pp

15 Test mages Fg. 0. Qualtatve results of the proposed strategy for the Panda class of the Caltech-0 data set over fve co-tranng rounds. The mages are ordered by decreasng classfcaton confdence. Tranng mage, and test mages from to 7 are reported n Fg. 9.