Privacy preserving data mining multiplicative perturbation techniques

Transcription

1 Privacy preserving data mining multiplicative perturbation techniques Li Xiong CS573 Data Privacy and Anonymity

2 Outline Review and critique of randomization approaches (additive noise) Multiplicative data perturbations Rotation perturbation Geometric Data Perturbation Random projection Comparison

3 Additive noise (randomization) Reveal entire database, but randomize entries User Database x x n + n x 1 xn Add random noise i to each database entry x i For example, if distribution of noise has mean 0, user can compute average of x i slide 3

4 Learning decision tree on randomized data Alice s age 30 70K K Add random number to Age 30 becomes 65 (30+35) Randomizer Reconstruct Distribution of Age Randomizer 65 20K K Reconstruct Distribution of Salary... Classification Algorithm Model

5 Summary on additive perturbations Benefits Easy to apply applied separately to each data point (record) Low cost Can be used for both web model and corporate model user 1 User 2 User n Private info x 1 + x 1 1 xn Web Apps data x n + n

6 Additive perturbations - privacy Need to publish noise distribution The column distribution is disclosed Subject to data value attacks! On the Privacy Preserving Properties of Random Data Perturbation Techniques, Kargupta, 2003a

7 The spectral filtering technique can be used to estimate the original data

8

9 The spectral filtering technique can perform poorly when there is an inherent random component in the original data

10 Randomization data utility Only preserves column distribution Need to redesign/modify existing data mining algorithms Limited data mining applications Decision tree and naïve bayes classifier

11 Randomization approaches Privacy guarantee? Data Utility/ Model accuracy Privacy guarantee Data utility/ Model accuracy Difficult to balance the two factors Low data utility Subject to attacks

12 More thoughts about perturbation 1. Preserve Privacy Hide the original data not easy to estimate the original values from the perturbed data Protect from data reconstruction techniques The attacker has prior knowledge on the published data 2. Preserve Data Utility for Tasks Single-dimensional properties - column distribution, etc. Decision tree, Bayesian classifier Multi-dimensional properties - covariance matrix, distance, etc SVM classifier, knn classification, clustering

13 Multiplicative perturbations Preserving multidimensional data properties Geometric data perturbation (GDP) [Chen 07] Rotation data perturbation Translation data perturbation Noise addition Random projection perturbation(rpp) [Liu 06] Chen, K. and Liu, L. Towards attack-resilient geometric data perturbation. SDM, 2007 Liu, K., Kargupt, H., and Ryan, J. Random projection-based multiplicative data perturbation for privacy preserving distributed data mining. TKDE, 2006

14 Rotation Perturbation G(X) = R*X R m*m - an orthonormal matrix (R T R = RR T = I) X m*n - original data set with n m-dimensional data points G(X) m*n - rotated data set Key features preserves Euclidean distance and inner product of data points preserves geometric shapes such as hyperplane and hyper curved surfaces in the multidimensional space Example: ID age rent tax = * ID age rent tax

15 Illustration of multiplicative data perturbation Preserving distances while perturbing each individual dimensions

16 Data properties A model is invariant to geometric perturbation if distance plays an important role Class/cluster members and decision boundaries are correlated in terms of distance, not the concrete locations Class 1 Class 1 2D Example: Class 2 Rotation and translation Classification boundary Class 1 Classification boundary Class 2 Distance perturbation (Noise addition) Slightly changed Classification boundary Class 2

17 Applicable DM algorithms Models invariant to GDP all Euclidean distance based clustering algorithms Classification algorithms K Nearest Neighbors Kernel methods Linear classifier Support vector machines Most regression models And potentially more

18 When to Use Multiplicative Data Perturbation Data Owner Service Provider/data user G(X)=RX+T+D G(X) F(G(X), ) Mined models/patterns Good for the corporate model or dataset publishing. Major issue!! curious service providers/data users try to break G(X)

19 Attacks! Three levels of knowledge Know nothing naïve estimation Know column distributions Independent Component Analysis Know specific points (original points and their images in perturbed data) distance inference

20 Attack 1: naïve estimation Estimate original points purely based on the perturbed data If using random rotation only Intensity of perturbation matters Points around origin Y Class 1 Class 1 Class 1 Class 2 Class 2 Class 2 X Classification boundary Classification boundary Classification boundary

21 Countering naïve estimation Maximize intensity Based on formal analysis of rotation intensity Method to maximize intensity Fast_Opt algorithm in GDP Random translation T Hide origin Increase difficulty of attacking! Need to estimate R first, in order to find out T

22 Attack 2: ICA based attacks Independent Component Analysis (ICA) Try to separate R and X from Y= R*X

23 Characteristics of ICA 1. Ordering of dimensions is not preserved. 2. Intensity (value range) is not preserved Conditions of effective ICA-attack 1. Knowing column distribution 2. Knowing value range.

24 Countering ICA attack Weakness of ICA attack Need certain amount of knowledge Cannot effectively handle dependent columns In reality Most datasets have correlated columns We can find optimal rotation perturbation maximizing the difficulty of ICA attacks

25 Attack 3: distance-inference attack If with only rotation/translation perturbation, when the attacker knows a set of original points and their mapping Known point image Original Perturbed

26 How is the Attack done Knowing points and their images find exact images of the known points Enumerate pairs by matched distances Less effective for large data we assume pairs are successfully identified Estimation 1. Cancel random translation T from pairs (x, x ) 2. calculate R with pairs: Y=RX R = Y*X calculate T with R and known pairs

27 Countering distance-inference: Noise addition Noise brings enough variance in estimation of R and T Can the noise be easily filtered? Need to know noise distribution, Need to know distribution of RX + T, Both distributions are not published, however. Note: It is very different from the attacks to noise addition data perturbation [Kargupta03]

28 Attackers with more knowledge? What if attackers know large amount of original records? Able to accurately estimate covariance matrix, column distribution, and column range, etc., of the original data Methods PCA,etc can be used What do we do? Stop releasing any kind of data anymore

29 Benefits of Geometric Data Perturbation Privacy guarantee decoupled Data Utility/ Model accuracy Applicable to many DM algorithms -Distance-based Clustering -Classification: linear, KNN, Kernel, SVM, Make optimization and balancing easier! - Almost fully preserving model accuracy - we optimize privacy only

30 A randomized perturbation optimization algorithm Start with a random rotation Goal: passing tests on simulated attacks Not simply random a hillclimbing method 1. Iteratively determine R - Test on naïve estimation (Fast_opt) - Test on ICA (2 nd level) find a better rotation R 2. Append a random translation component 3. Append an appropriate noise component

31 Privacy guarantee:gdp In terms of naïve estimation and ICA-based attacks Use only the random rotation and translation components (R*X+T) Optimized for Naïve estimation only Optimized perturbation for both attacks Worst perturbation (no optimization)

32 Privacy guarantee:gdp In terms of distance inference attacks Use all three components (R*X +T+D) Noise D : Gaussian N(0, 2 ) Assume pairs of (original, image) are identified by attackers no noise addition, privacy guarantee =0 Considerably high PG at small perturbation =0.1

33 Data utility: GDP with noise addition Noise addition vs. model accuracy - noise: N(0, ) Boolean data is more sensitive to distance perturbation

34 Random Projection Perturbation Random projection projects a set of data points from high dimensional space to a lower dimensional subspace F(X) = P*X X is m*n matrix: m columns and n rows P is a k*m random matrix, k <= m Johnson-Lindenstrauss Lemma There is a random projection F() with e is a small number <1, so that (1-e) x-y <= F(x)-F(y) <=(1+e) x-y i.e. distance is approximately preserved.

35 Data Utility: RPP Reduced # of dims vs. model accuracy KNN classifiers SVMs

36 Random projection vs. geometric perturbation Privacy preservation Subject to similar kinds of attacks RPP is more resilient to distance-based attacks Utility preservation(model accuracy) GDP: R and T exactly preserve distances, The effect of D needs experimental evaluation RPP Approximately preserves distances # of perturbed dimensions vs. utility

37 Coming up Output perturbation Cryptographic protocols

38 Methodology of attack analysis An attack is an estimate of the original data Original O(x 1, x 2,, x n ) vs. estimate P(x 1, x 2,, x n ) How similar are these two series? One of the effective methods is to evaluate the variance/standard deviation of the difference [Rakesh00] Var (P O) or std(p-o), P: estimated, O: original

39 Two multi-column privacy metrics q i : privacy guarantee for column i q i = std(p i O i ), O i normalized column values, P i estimated column values Min privacy guarantee: the weakest link of all columns min { q i, i=1..d} Avg privacy guarantee: overall privacy guarantee 1/d q i