SVD-based Collaborative Filtering with Privacy

Size: px

Start display at page:

Download "SVD-based Collaborative Filtering with Privacy"

Amber Bryant
6 years ago
Views:

1 SVD-based Collaboratve Flterg wth Prvacy Husey Polat Departmet of Electrcal Egeerg ad Computer Scece Syracuse Uversty, 121 L Hall Syracuse, NY , USA Phoe: hpolat@ecs.syr.edu Welag Du Departmet of Electrcal Egeerg ad Computer Scece Syracuse Uversty, 121 L Hall Syracuse, NY , USA Phoe: wedu@ecs.syr.edu ABSTRACT Collaboratve flterg (CF) techques are becomg creasgly popular wth the evoluto of the Iteret. Such techques recommed products to customers usg smlar users preferece data. The performace of CF systems degrades wth creasg umber of customers ad products. To reduce the dmesoalty of flterg databases ad to mprove the performace, Sgular Value Decomposto (SVD) s appled for CF. Although flterg systems are wdely used by E-commerce stes, they fal to protect users prvacy. Sce may users mght decde to gve false formato because of prvacy cocers, collectg hgh qualty data from customers s ot a easy tas. CF systems usg these data mght produce accurate recommedatos. I ths paper, we dscuss SVD-based CF wth prvacy. To protect users prvacy whle stll provdg recommedatos wth decet accuracy, we propose a radomzed perturbatobased scheme. Categores ad Subject Descrptors K.4.4 [Computers ad Socety]: Electroc Commerce Securty; H.2.8 [Database Maagemet]: Database Applcatos Data Mg Geeral Terms Securty, Performace, Expermetato Keywords Prvacy, collaboratve flterg, SVD, radomzato 1. INTRODUCTION Ths wor was supported part by Grat IIS ad IIS from the Uted States Natoal Scece Foudato. Permsso to mae dgtal or hard copes of all or part of ths wor for persoal or classroom use s grated wthout fee provded that copes are ot made or dstrbuted for proft or commercal advatage ad that copes bear ths otce ad the full ctato o the frst page. To copy otherwse, to republsh, to post o servers or to redstrbute to lsts, requres pror specfc permsso ad/or a fee. SAC 05, March 13-17, 2005, Sata Fe, New Mexco, USA. Copyrght 2005 ACM /03/ $5.00. Iformato overload s becomg a major problem for users wth the evoluto of the Iteret. Dfferet approaches are used to separate the terestg ad the valuable formato from the rest order to cope wth ths problem. Collaboratve flterg (CF) s a recet techque that helps users to cope wth formato overload by usg the prefereces of other users. Wth the growth of E-commerce, there s creasg commercal terest CF techology. Some commercal web stes le Amazo.com, CDNow.com, ad MoveFder.com have made successful use of CF. CF systems wor by collectg ratgs for tems ad matchg together users who share the same terest or tastes. Such systems help ew users to better decde whch tems to buy. The goal of CF s to predct how well a user, referred to as the actve user, wll le a tem that he dd ot buy before based o the prefereces of a commuty of users [10]. Whle t was show that the memory-based (correlatobased) CF schemes perform well [10], they suffer from some lmtatos [4, 13]. Therefore, addto to correlatobased CF algorthms, SVD s appled for CF to address such lmtatos [4, 13]. Although t has bee show that CF systems have may successful applcatos several domas, such systems have a umber of dsadvatages [6, 7]. The most mportat s that they are a serous threat to dvdual prvacy. Most ole vedors collect prefereces of ther customers, ad mae efforts to preserve ther customers prvacy. However, several schemes are extremely vulerable ad ca be med for prefereces of users [6]. I addto, customer data s a valuable asset ad t has bee sold whe some E-compaes suffered baruptcy. The courts have supported the rghts of lqudators to sell off data about ther customers persoal formato as a asset. Sce data from users are eeded for CF purposes ad may users have cocers about ther prvacy, provdg prvacy measures s a ey to the success of both data collecto ad producg recommedatos wth decet accuracy. Some people mght be wllg to selectvely dvulge formato f they ca get beeft retur [14]. However, accordg to a survey coducted 1999 [8], a sgfcat umber of people are ot wllg to dvulge ther formato because of prvacy cocers. The challege s how ca users cotrbute ther prvate formato for CF purposes wthout greatly compromsg ther prvacy? Aoymous techques [1, 12] are wdely used to acheve prvacy. Such techques allow users to dvulge ther data wthout dsclosg ther dettes. However, t s dffcult for

2 the database ower to guaratee the qualty of the database because a malcous user could sed radom data ad reder the database useless, or a competg compay could sed a great deal of made-up formato to mae ther products the most favorable oes. It s mportat for the database ower to verfy the dettes of the data cotrbutors to guaratee the qualty. We propose a scheme to allow SVD-based CF wth prvacy. Our goal s to esure users prvacy ad to provde accurate predctos. However, prvacy ad accuracy are coflctg goals; mprovg oe of them decreases the other. We propose a techque to acheve a balace betwee them. We wat to prevet the server from learg whch tems that the users rated before ad how much they le or dsle those rated tems. I our scheme (Fg. 1), each user frst dsguses hs prvate data, ad seds t to the data collector (the server), such that the server caot derve the truthful formato about the user s prvate formato. However, the data dsgusg scheme should stll be able to allow the server to coduct CF from the dsgused data. We use radomzed perturbato (RP) techques [3] to dsguse prvate data. These techques are useful f we are terested aggregate data rather tha dvdual data tems because whe the umber of users ad tems are sgfcatly large, the aggregate formato of these users ca be estmated wth decet accuracy. Sce SVD-based CF s based o aggregate values of a dataset, we hypothesze that by combg the RP techques wth SVD-based CF algorthms, we ca acheve a decet degree of accuracy for SVD-based CF wth prvacy. SVD & Collaboratve Flterg Cetral Database Dsgused Data Actual Data framewor, whch users sed ther data to a server ad they do ot partcpate the CF process; oly the server eeds to coduct the CF. Polat ad Du [11] used radomzed perturbato for prvacy-preservg correlato-based CF. Whle ther wor focuses o correlato-based CF wth prvacy, our wor here focuses o SVD-based CF wth prvacy. I our scheme, the server creates a database (user-tem matrx, A), ad uses SVD to factor A to three matrces that are used for predctos. 3. SVD-BASED CF SVD s a well-ow matrx factorzato techque that factors a m matrx A to three matrces [13] as A = USV T where U ad V are two orthogoal matrces of sze y ad m y, respectvely; y s the ra of the matrx A. S s a dagoal matrx of sze y y havg all sgular values of matrx A as ts dagoal etres. It s possble to reduce the y y matrx S to have oly largest dagoal values to obta a matrx S, < y. Sarwar et al. [13] propose a SVD-based CF algorthm. The sparse user-tem ratgs matrx (A) s flled usg the average ratgs for users to capture a meagful latet relatoshp. The flled matrx s ormalzed by covertg ratgs to z-scores. The ormalzed matrx (A orm) s factored to U, S, ad V usg SVD. The the matrx S s obtaed by retag oly largest sgular values. Accordgly, the dmesos of matrces U ad V are also reduced. The, U S ad S V T are computed. These resultat matrces ca be used to compute the predcto for ay user u o tem q. To compute the predcto, the scalar product of the u th row of U S (deoted as U S (u)) ad the (deoted as S V T (q)) s calculated q th colum of S V T ad the result s deormalzed as follows: p uq = v u + σ u U S (u) S V T (q) (1) USER1 USER2 Data Dsgusg USER 1 USER where v u ad σ u are mea ratg ad stadard devato for user u, respectvely. Sce the user u who s loog for predcto wll do the deormalzato, we ca defe p uq = v u + σ up where Fgure 1: Prvacy preservg CF wth SVD To verfy ths hypothess, we mplemeted the RP techque for the SVD-based CF algorthm [13]. We the coducted a seres of expermets to show how accurate our results are. We measured the overall performace of our scheme based o dsgused data. Our results show that the predctos we have foud o radomzed data are very close to the orgal ratgs. 2. RELATED WORK Cay proposes alteratve models for prvacy-preservg collaboratve flterg (PPCF) whch users cotrol all of ther data [6, 7]. A commuty of users ca compute a publc aggregate of ther data that does ot expose dvdual users data. He teratvely calculates the aggregate requrg oly addto of vectors of user data. He the uses homomorphc ecrypto to allow sums of ecrypted vectors to be computed ad decrypted wthout exposg dvdual data. Our wor here dffers from Cay s wor. Whle hs wor focuses o the P2P framewor, whch users actvely partcpate the CF process, our wor focuses o aother p = U S (u) S V T (q) (2) 4. PRIVACY-PRESERVING SVD-BASED CF Radomzed perturbato techques were frst used by [3] to acheve prvacy. I order to dsguse a umber a, a smple way s to add a radom value r to t. a+r, rather tha a, wll appear the database, where r s a radom value draw from some dstrbuto. Although we caot do aythg to a sce t s dsgused, we ca coduct certa computatos f we are terested the aggregate data rather tha dvdual data tems. The basc dea of radomzato s to perturb the data such a way that the server ca oly ow the rage of the data, ad such rage s broad eough to preserve users prvacy. Wth the prvacy cocers, the server should ot ow the true ratgs of each user ad whch tems that are rated. We created radom umbers usg uform ad Gaussa dstrbutos. I uform dstrbuto, all users create uform radom values from a rage [ α, α] where α s a costat umber. For Gaussa dstrbuto, each user geerates radom values usg ormal dstrbuto wth mea (µ) beg 0 ad stadard devato (σ). Users dsguse ther data before

3 they sed t to the server. The steps of data dsgusg are as follows: 1. The server decdes o the dstrbutos of perturbg data (uform or Gaussa) ad parameters (α, σ, ad µ), ad let each user ow. 2. Each user u flls empty cells of hs ratgs vector usg hs mea vote ad calculates the z-scores. 3. Each user u creates m radom values r uj draw from some dstrbuto, where m s the total umber of tems. The each user u adds those radom values to hs z-score values ad geerates the dsgused z-scores z uj = z uj+r uj for j = 1... m. Fally, each user seds z uj values to the server who creates the dsgused usertem matrx (A ). To provde CF servces, the server frst computes the SVD of matrx A. As explaed before, oce the server computes A T A, t ca fd S ad V matrces based o A T A where S ad V are estmated matrces of S ad V, respectvely. Each etry of A T A s estmated by calculatg the scalar product of rows of matrx A T ad the colums of the matrx A. The etres other tha the dagoal oes are estmated as follows: (A T A ) fg = + z uf r ug + (z uf + r uf )(z ug + r ug) = z ugr uf + r uf r ug z uf z ug z uf z ug (3) where s the total umber of users, f ad g show the row ad colum umbers, respectvely, ad f g. Sce radom values r uf s ad r ug s are depedet ad draw from some dstrbuto wth µ = 0, the expected value È È of r ufr ug s È0. Smlarly, the expected values of z ufr ug ad zugr uf are 0. However, sce the scalar product s computed betwee the same vectors for the dagoal etres (f = g), we ca estmate them as follows: z 2 uf + 2 (A T A ) ff = z uf r uf + (z uf + r uf )(z uf + r uf ) = r 2 uf z 2 uf + r 2 uf (4) Aga, the expected È È value of z ufr uf s 0. However, sce we oly eed È z2 uf values for dagoal etres, we eed to get rd of r2 uf Eq. 4 as follows: (A T A ) ff z 2 uf + r 2 uf σ r 2 z 2 uf (5) where σ r s the stadard devato of radom umbers. After estmatg the matrx A T A, the server ca ow compute the egevalues from A T A, whch are used to fd egevectors that form the matrx V. It the fds the matrx S usg the egevalues estmated from A T A. Fally, the server eeds to calculate the frst y columvectors of U usg b = s 1 Av for = 1... y where v s are colum-vectors of V. Smlarly, b vectors ca be estmated usg A, s, ad v vectors where v s ad s s are estmated from the matrx A T A. The etres of b vectors are estmated as follows: s 1 z jl v l + s 1 b (j) = s 1 (z jl + r jl )v l = r jl v l s 1 z jl v l (6) where j = 1... ad the expected value of È m r jlv l s 0. After estmatg U, S, ad V T from dsgused data, the server forms S ad computes U Ô S ad Ô S V T matrces. To get a predcto for tem q, the user u seds a query (for whch tem he s loog for predcto) to the server who computes p by calculatg the scalar product of the u th row of U Ô Ô S ad the q th colum of S V T ad seds the result to the user u who ca ow calculate the p uq usg Eq EXPERIMENTS We used two datasets our expermets. Jester s a webbased joe recommedato system, developed at Uversty of Calfora, Bereley [9]. The database has 100 joes ad records of 17,988 users. The ratgs rage from -10 to +10, ad the scale s cotuous. MoveLes (ML) data were collected by the GroupLes Research Project at the Uversty of Mesota ( Our ML data cossts of 100,000 ratgs for 1,682 moves by 943 users. Ratgs are made o a 5-star scale. We used the Mea Absolute Error (MAE) ad the stadard devato (σ) as crtera for accuracy aalyss. For prvacy aalyss, we used the method suggested [2], whch taes to accout the dstrbuto of orgal data. The prvacy measure should dcate how closely the orgal value of a tem ca be estmated from the perturbed data. Agrawal ad Aggarwal [2] propose a prvacy measure based o the dfferetal etropy (h) of a radom varable (X). They propose 2 h(x) as a measure of prvacy heret the radom varable X ad deote t by Π(X). The average codtoal prvacy of X gve Z s defed as Π(X Z) = 2 h(x Z) where h(x Z) s the codtoal dfferetal etropy of X gve Z. Ths motvates the metrc P(X Z) = 1 2 I(X;Z), whch s the fracto of prvacy of X lost by revealg Z where I(X; Z) = h(z) h(z X). If the orgal value s X, whch s dsgused by R, after revealg Z (Z = X + R), X has prvacy Π(X Z) = Π(X) 1 P(X Z). 5.1 Methodology We flled the ull etres the user-tem matrx (A) by replacg each ull etry wth the user mea votes for the correspodg rows. We ormalzed matrx A by replacg each etry wth z uj (z uj = (v uj v u)/σ u), where v u s the user average vote ad σ u s the stadard devato of user u. The, we created m radom values r uj usg uform or Gaussa dstrbutos for each user ad added them to the z-score values. Although we used all users ML dataset, we radomly selected 1,000 users for trag from Jester dataset. 10% of the users that we used our expermets were radomly selected as test users. We coducted two classes of expermets terms of avalable umber of ratgs. I the frst class, we wthheld a s-

4 0.8 = 100 Udsgused Data Uform Dstrbuto Gaussa Dstrbuto 4 = 100 Udsgused Data Uform Dstrbuto Gaussa Dstrbuto = 200 Mea Absolute Errors (MAE) = 200 = 943 Mea Absolute Errors (MAE) = 1, Set of Expermets (a) MoveLes dataset (ratg rage: 1 5) Set of Expermets (b) Jester dataset (ratg rage: ) Fgure 2: Number of users ad qualty of predctos gle radomly selected rated tem for each user the test set, ad tred to predct ts value gve all other votes the user rated (All but 1 protocol) [5]. I the secod class, we radomly selected 5 rated tems from each test user as test tems, ad attempted to predct for those tems (All but 5 protocol). We replaced the etres for test tems as ull. The we used our SVD-based scheme to predct ratgs o selected tems for that user. We compared the predctos that we foud based o dsgused data wth the wthheld ratgs. We ra ths procedure 50 tmes for each test user ad foud the MAEs ad the stadard devatos. The we averaged them over all test users. 5.2 Expermetal Results To evaluate the overall performace of our scheme, we coducted several expermets. We hypothesze that prvacy ad accuracy deped o several factors cludg the total umber of users () ad tems (m), the dstrbuto ad the rage of perturbg data, ad the total umber of retaed sgular values (). We foud 10 to be the optmum value of for both datasets Total Number of Users () ad Items (m) To show that our scheme wors better wth creasg, we coducted three dfferet sets of expermets usg both datasets where we fxed the umber of tems to 1,682 ad 100 for ML ad Jester, respectvely ad set = 10 whle varyg. We used All but 5 protocol ad showed our results Fg. 2. We created perturbg data usg uform ad Gaussa dstrbutos wth σ = 1. Fg. 2 shows MAEs for udsgused data, uform ad Gaussa perturbed data for both datasets. As we expected, accuracy mproves wth creasg. I Eq. 3 ad Eq. 4, the scalar products are computed over. I the log ru, the sample mea ad varace of perturbg data wll coverge to ther expected values. Therefore, the accuracy of our scheme s gettg better wth creasg. We coducted expermets to show the effects of dfferet total umbers of tems ad showed our results based o ML data Table 1. We used All but 5 protocol where we fxed whle varyg m. Frst, we used all avalable tems, ad the Table 1: Number of tems - predcto qualty Data Dsguse 943x1, x x100 MAE Uform Gaussa Udsgused σ Uform Gaussa Udsgused we radomly selected 500 ad 100 tems. Whe we selected 100 or 500 tems, we used the users who rated at least two tems amog those tems. Because of that, there are 779 users our thrd group expermets where m = 100. As ca be see from the table, accuracy becomes better wth creasg m because the scalar product betwee A ad v s computed over m Eq. 6. As explaed before, wth creasg m, the sample mea ad varace of perturbg data wll coverge to ther expected values. Therefore, accuracy mproves wth creasg m Level of Perturbato We coducted expermets usg ML data whle varyg the parameters of perturbg data to show how the levels of perturbato affect accuracy. We created radom umbers usg uform ad Gaussa dstrbutos whle varyg the stadard devatos. We used 943x1,682 user-tem matrx. We compared the predctos based o dsgused data usg our scheme wth the predctos o orgal data. Fg. 3 shows how mea absolute errors chage wth creasg level of perturbato. As see from Fg. 3, the level of perturbato s crtcal for accuracy. The results become better wth decreasg levels of perturbato. As we ow, whe the stadard devato s small, the radomess also becomes smaller; thus accuracy ca be mproved Prvacy ad Accuracy To protect the prvate data, the level of perturbato s crtcal. If the level s too low, the perturbed data stll dscloses sgfcat amouts of formato; f t s too hgh, accuracy wll be very low. The greater the level of perturbato, the greater the amout of prvacy we have. For exam-

5 Mea Absolute Errors (MAE) Gaussa Dstrbuto Uform Dstrbuto Stadard Devato of Perturbg Data (σ) Fgure 3: Level of perturbato ad MAEs ple, whle Π(X Z) s whe σ = 0.5, t s whe σ = 1 for uform dstrbuted perturbg data. Wth creasg levels of perturbato, prvacy loss becomes smaller. However, accuracy decreases wth creasg levels of perturbato. We showed the tradeoff betwee prvacy loss ad accuracy Fg. 4 for ML data. Although prvacy levels crease wth creasg levels of perturbato, accuracy becomes worse because accuracy ad prvacy coflct each other. Prvacy Loss Gaussa Dstrbuto Uform Dstrbuto Mea Absolute Errors (MAE) Fgure 4: Prvacy loss vs. accuracy Summary As ca be see from Fg. 2(a), whch we showed results for ML dataset, the MAE s whe we use uform perturbg data for data dsgusg wth σ = 1 ad = 100. However, t s whe we compared the predctos from dsgused data wth the predctos o orgal data where the MAE s Sce the ratg rage for ML dataset s from 1 to 5, MAE = dcates our results are very close to the results geerated from the orgal data. I Fg. 2(b), whch we showed results for Jester dataset, the MAE s for uform perturbg data wth σ = 1 ad = 100 whe we compared the predctos from dsgused data wth the predctos o orgal data. Sce the ratg rage s from -10 to 10 Jester dataset, a error of s equvalet to a 1 5 scale. 6. CONCLUSIONS AND FUTURE WORK We have preseted a soluto to SVD-based CF wth prvacy. Our soluto maes t possble for servers to collect prvate data wthout greatly compromsg users prvacy. Our expermets have show that our soluto ca acheve accurate predctos whle preservg prvacy. We beleve that accuracy of our scheme ca be further mproved f more aggregate formato s dsclosed alog wth the dsgused data, especally those whose dsclosure does ot compromse much of users prvacy. We wll study how these ds of aggregate data dsclosures affect accuracy ad prvacy. 7. REFERENCES [1] Aoymzer.com: [2] D. Agrawal ad C. C. Aggarwal. O the desg ad quatfcato of prvacy preservg data mg algorthms. I Proceedgs of the 20th ACM SIGACT-SIGMOD-SIGART Symposum o Prcples of Database System, Sata Barbara, CA, May [3] R. Agrawal ad R. Srat. Prvacy-preservg data mg. I Proceedgs of the 2000 ACM SIGMOD o Maagemet of Data, pages , Dallas, TX, May [4] D. Bllsus ad M. J. Pazza. Learg collaboratve formato flters. I Proceedgs of the 1998 Worshop o Recommeder Systems, August [5] J. Breese, D. Hecerma, ad C. Kade. Emprcal aalyss of predctve algorthms for collaboratve flterg. I Proceedgs of the 14th Coferece o Ucertaty Artfcal Itellgece, pages 43 52, Madso, WI, July [6] J. Cay. Collaboratve flterg wth prvacy. I IEEE Symposum o Securty ad Prvacy, pages 45 57, Oalad, CA, May [7] J. Cay. Collaboratve flterg wth prvacy va factor aalyss. I Proceedgs of the 25th Aual Iteratoal ACM SIGIR Coferece o Research ad Developmet Iformato Retreval, pages , Tampere, Flad, August [8] L. F. Craor, J. Reagle, ad M. S. Acerma. Beyod cocer: Uderstadg et users atttudes about ole prvacy. Techcal report, AT&T Labs-Research, Aprl Avalable from /lbrary/trs/trs/99/99.4.3/report.htm. [9] D. Gupta, M. Dgova, H. Narta, ad K. Goldberg. Jester 2.0: A ew lear-tme collaboratve flterg algorthm appled to joes. I Worshop o Recommeder Systems Algorthms ad Evaluato, 22d Iteratoal Coferece o Research ad Developmet Iformato Retreval, Bereley, CA, [10] J. L. Herlocer, J. A. Kosta, A. Borchers, ad J. T. Redl. A algorthmc framewor for performg collaboratve flterg. I Proceedgs of the 1999 Coferece o Research ad Developmet Iformato Retreval, August [11] H. Polat ad W. Du. Prvacy-preservg collaboratve flterg usg radomzed perturbato techques. I Proceedgs of the 3rd IEEE Iteratoal Coferece o Data Mg (ICDM 03), Melboure, FL, November [12] M. K. Reter ad A. D. Rub. Crowds: Aoymty for Web trasacto. ACM Trasactos o Iformato ad System Securty, 1(1):Pages 66 92, [13] B. M. Sarwar, G. Karyps, J. A. Kosta, ad J. T. Redl. Applcato of dmesoalty reducto recommeder system-a case study. I ACM WebKDD 2000 Web Mg for E-commerce Worshop, [14] A. F. West. Freebes ad prvacy. Techcal report, Opo Research Corporato, July Avalabe from /sr html.

Short Note: Merging Secondary Variables for Geophysical Data Integration

Short Note: Merging Secondary Variables for Geophysical Data Integration Short Note: Mergg Secodary Varables for Geophyscal Data Itegrato Steve Lyster ad Clayto V. Deutsch Departmet of Cvl & Evrometal Egeerg Uversty of Alberta Abstract Multple secodary data from geophyscal