Security in Distributed Storage Systems by Communicating a Logarithmic Number of Bits

Transcription

1 Secuity in Distibuted Stoage Systems by Communicating a Logaithmic Numbe of Bits Theodoos K. Dikaliotis Depatment of lectical ngineeing Alexandos G. Dimakis Depatment of lectical ngineeing Tacey Ho Depatment of lectical ngineeing axiv: v1 [cs.cr] 4 May 2010 Califonia Institute of Technology Univesity of Southen Califonia Califonia Institute of Technology mail: tdikal@caltech.edu mail: dimakis@usc.edu mail: tho@caltech.edu Abstact We investigate the poblem of maintaining an encoded distibuted stoage system when some nodes contain advesaial eos. Using the eo-coection capabilities that ae built into the existing edundancy of the system, we popose a simple linea hashing scheme to detect eos in the stoage nodes. Ou main esult is that fo stoing a data object of total size M using an (n,k) MDS code ove a finite field F q, up to t 1 = (n k)/2 eos can be detected, with pobability of failue smalle than 1/M, by communicating only O(n(n k)logm) bits to a tusted veifie. Ou esult constucts small pojections of the data that peseve the eos with high pobability and builds on a pseudoandom geneato that fools linea functions. The tansmission ate achieved by ou scheme is asymptotically equal to the min-cut capacity between the souce and any eceive. I. INTRODUCTION We study the secuity and data integity of distibuted stoage systems that use coding fo edundancy. It is well known that maximum distance sepaable (MDS) codes can offe maximum eliability fo a given stoage ovehead and can be used fo distibuted stoage in data centes and pee-to-pee stoage systems like OceanStoe [1], Total Recall [2], and FS2You [3], that use nodes acoss the Intenet fo distibuted file stoage and shaing. In this pape we ae inteested in dealing with eos in the encoded epesentation. The eos could be intoduced eithe though (unlikely) had dive undetected failues o though a malicious o compomised seve in the stoage netwok. This second theat is much moe eminent when the system uses netwok coding to maintain the edundancy of the encoded system as poposed ecently [4]. To illustate this conside a lage data object that has size M bits. If this object is to be stoed on n seves, depending on the desied edundancy, an (n,k) linea MDS code can be used, dividing the object into k packets of size M/k each, and stoing an encoded packet at each seve. Assuming the code is ove a finite field F q, equiing logq bits to epesent each symbol, each seve will also need to keep a heade denoting the coding coefficients of the linea combinations stoed on the seve (see section II fo the details) and the size of this heade is lage than the size of the useful data if the code is used only once. Fo this eason it was poposed that the same code is used seveal times [5] by dividing each 0 This wok has been suppoted patially by NSF gant CNS , patially by the Ai Foce Office of Scientific Reseach unde gant FA and Caltech s Lee Cente fo Advanced Netwoking

2 Fig. 1. A (4,2) MDS code along with the epai of the fist stoage node. ach node stoes two packets and any two nodes contain enough infomation to ecove all fou data packets. In this example the fist node leaves the system and a new node is fomed by communicating linea combinations f 2,f 3,f 4 which can be used to solve fo x 1,x 2 at the new node. packet into N symbols of logq bits and epeating the same code N times. If N >> n the ovehead of stoing the coefficients becomes negligible. We efe to this as the N extended vesion of an MDS code, shown in Figue 2 fo the (4,2) code used in Figue 1. Obseve that in this example, each node is stoing two linea combinations, (ows) as opposed to one. This sub-packetization is pefomed to facilitate epai though netwok coding as poposed in [4]. The poblem of epai consists of constucting a new encoded node by accessing as little infomation fom existing encoded nodes. In the example of Figue 1, we assume that the fist stoage node failed and the edundancy of the system needs to be efeshed. This is achieved by communicating small linea combinations f 2,f 3,f 4 of the encoded packets fom nodes 2,3, and 4 each of size 1/2 of what each node is stoing, which as poven in [4], is infomation theoetically minimal. As stoage nodes leave the system and new ones ae added, this foms a dynamic stoage netwok that keeps a fixed edundancy and eliability by building new encoded packets fom aleady existing ones. The poblem of secuity should now be clea: even if a single node in this stoage system is compomised and paticipates in this epai pocess, then it can send incoect linea combinations that will ceate eoneous packets at the new nodes. All new nodes using these linea equations will have incoect data and soon the whole system will be contaminated with nodes having eoneous data.

3 Ou contibution: Since the poblem of epaiing a code is equivalent to wieline netwok coding [4], existing techniques fo netwok eo coection can be used to detect and coect the eos [6], [7]. These techniques ae designed to wok fo geneal netwoks and always guaantee a tansmission ate of C 2z, whee C is the min-cut capacity fom the souce to the destination and z is the numbe of links contaminated by the advesay. Ou appoach, that is ceating and communicating small linea hashes which peseve the stuctue of the code, allows the detection of eos and achieves a tansmission ate that can be asymptotically equal to C (by having the eceive connecting to all the non-eoneous nodes) since it takes advantage of the specific stuctue of the netwok and the set of links an advesay can contaminate. To explain ou scheme, conside the (4,2) MDS code of Figue 1 and assume one of the fou nodes contains eos (say in both ows). A tusted veifie that communicates with all fou nodes can find this eo by getting the 8 equations contained in each of the 4 = 6 node pais. Since this is a (4,2) MDS code, the combinations of equations that come fom eo-fee 2 nodes will be full ank and give a consistent solution wheeas the othe sets will give diffeent solutions (o might not even be full ank). This is, of couse, just using the eo-coection capability of the code to detect an eo. Ou contibution involves using this idea to the N-extended vesion of a code, by ceating a linea pojection (hash) of each ow on the same andom vecto. The key obsevation is that if the same andom pojection is used, this ceates an eo-coecting code fo the hashes which can be communicated to the veifie. The benefit is that each hash has size only 1/N of the data in each ow educing the amount of communication to the veifie. One complication is that each node needs to poject its data on the same andom vecto of length N, which equies N logq bits of common andomness. Subsequently the poblem at the veifie is to decode an eo-coecting code unde advesaial eos. This decoding task can be computationally inefficient but we do not addess this issue hee, assuming that the veifie can detect the eos if they ae within the eo coecting capabilities of the code as dictated by the minimum distance (half the minimum distance). Ou analysis investigates unde which conditions the small pojected hash code will detect any eo in the lage amount of data stoed at the nodes. In paticula, we pove the following Theoem 1: In a distibuted stoage system stoing a total ofmbits, using ann extended(n,k) MDS code ovef q, with the n stoage nodes shaing O(M) bits of common andomness, ou andom hashing scheme can detect up to t t 1 (n k)/2 eos by communicating a total of n(n k)(logm+logt 1 ) bits to a veifie, with pobability of failue P[F] 1 M. One impotant weakness of the pevious esult is the lage common andomness equied which is compaable to the total size of the data object stoed (1/k(n k) faction of the M bits). Note that these bits do not have to be a secet, they only need to be ealized afte the eo has been intoduced to the new disk. Thei lage numbe, howeve, makes it impactical to geneate them at one node and then communicate them to the othes. Ou second contibution involves showing how to use only O(log M) bits of common andomness to achieve almost the same pefomance: Theoem 2: In a distibuted stoage system stoing a total of M bits, using an N extended (n,k) MDS code ove F q, with the n stoage nodes shaing O(log M) bits of common andomness, ou pseudoandom hashing scheme can detect up to

4 Fig. 2. Illustation of the 3 extended vesion of the (4,2) MDS code shown in Figue 1. ach of the thee columns stoed on the souce nodes is coded by epeatedly using the (4,2) MDS of Figue 1. Duing veification, each ow is pojected on the vecto T = (1 1 1) and the coesponding poducts S 1,...,S 4 fom a codewod of the initial (4,2) MDS code. Fo example, the eos at the fist ow of the fist node will not be absobed by the pojection as long as (e 11 e 12 e 13 ) (1 1 1) T 0 t 1 = (n k)/2 eos by communicating a total of O(n(n k)logm) bits to the veifie, with pobability of failue P[F ] 1 M. If thee is no common andomness, the veifie can geneate the O(logM) andom bits and communicate these to all the nodes equiing a total of O(n log M) exta communicated bits. Notice that in this case the total numbe of bits communicated scales only logaithmically in M, to achieve a pobability of failue that scales like 1/M. Ou constuction elies on the pseudoandom small-bias geneato used in [8] which can expand logn andom symbols of F q (which equie logn logq andom bits to geneate), into N pseudoandom symbols that can fool any linea function 1. The only modification to ou algoithm is pojecting each stoed ow on this pseudoandom vecto to geneate each hash and this induces only a small addition to the pobability of eo. Notice that ou wok does not ely on any cyptogaphic assumptions and guaantees that eos inseted in the distibuted stoage system will be detected with high pobability if they ae within the capabilities of the code used. Using the eo-coection capability of the code fo distibuted stoage has been suggested befoe as a way to detect eos [10], [11] and identify fee ides within the netwok. A diffeent appoach to find eos injected in distibuted stoage and content distibution systems is the use of signatues and hash functions. Refeence [12] intoduced the use of homomophic hashing functions that enables a nodes to pefom on-the-fly veification of easue-encoded blocks. Gkantsidis et al. [13] used the computationally less expensive secue andom checksums to detect polluted packets in content distibution system that use netwok coding while [14], [15] used a method of subspace signatues based on diffeent cyptogaphic pimitives. See also [16], [17], [18] fo othe elated wok on secuity and distibuted stoage.

5 II. MODL As stated, we conside a data object of size M bits that is divided into k pieces (of size M/k bits each) and these ae coded into n (> k) encoded pieces though a linea (n, k) maximum distance sepaable (MDS) code. These encoded pieces ae stoed on n distinct stoage nodes along with a heade denoting the exact linea combination saved at all the stoage nodes. Since the size of the code (n,k) will be much smalle than N, the ovehead of stoing the code desciption eveywhee (including the veifie) is minimal. This simplifies the model and we can now assume that the eos occu only at the data, since an eo at the heade would be immediately detected. We assume that the oiginal infomation (of size M bits) is oganized into a matix X with k(n k) ows and N columns. The elements of this matix ae elements of the finite field F q, i.e., X F k(n k) N q whee q is a pime o an intege powe of a pime. ach column Xi c F q k(n k) 1 (i {1,...,N}) of matix X will be sepaately encoded with the use of an (n,k) MDS code with geneato matix G F n(n k) k(n k) q and all the columns GXi c F n(n k) 1 q deived by this encoding will be stoed on the n diffeent stoage nodes of the distibuted stoage system. We will call this code applied to the N diffeent columns of matix X as the N extended MDS code. The oveall effect that the N extended MDS code has upon the infomation matix X is captued by the matix multiplication GX. Figue 2 shows such a code fo N = 3 whee the MDS code used is the same as the one shown in Figue 1. The stoage nodes of the distibuted stoage system ae assumed to have limited computational capabilities allowing them only to pefom inexpensive opeations ove the finite field F q. Some of these stoage nodes ae assumed to stoe eoneous infomation, whee these eos might be eithe andom due to hadwae failues o inseted advesaially by a malicious use. The malicious use can be computationally unbounded, have knowledge of all the infomation stoed on the distibuted stoage system and can inset eos to any t of the stoage nodes. We assume the existence of a special node called the veifie that is assigned to check the integity of the data stoed on diffeent stoage nodes. The veifie does not have access to the initial data object (othe than the desciption of the code) and theefoe has to ely on the communicated infomation to check which nodes contain eos. III. RANDOM HASHS A. Illustating example Assume that in the distibuted stoage system shown in Figue 2 with fou stoage nodes it is known that one of them (the fist in this example) stoes eoneous infomation. The goal of the veifie that ovelooks the state of the whole system is fist to find the eoneous disk with the minimum data exchange and second to epai it by using the infomation stoed on the othe disks. Since all thee columns stoed on the distibuted stoage system ae codewods of a (4,2) MDS code with at most one eo (some columns might be eo fee) and minimum distance d = 3, the naïve appoach to find the eoneous disk is to download all data fom diffeent disks and then by using minimum distance decoding on each sepaate column one would be able to find the eoneous disk. 1 Fist intoduced by Nao and Nao in [9] fo linea functions in F 2.

6 The naïve appoach would cetainly find the faulty disk but it would equie the tansfe of double the size of the file stoed ( n km bits of infomation in geneal). So as the size of the file inceases this appoach will become pohibitively expensive in bandwidth. Instead of tansmitting all the infomation stoed on the distibuted stoage system, the cental node could choose a vecto with each component chosen independently and unifomly at andom fom F q and have each stoage node tansmit the inne poduct (called the hash poduct) between the andomly chosen vecto and each of the ows stoed at the disks. In the absence of eos, these hash poducts will fom a codewod of the MDS code used to encode the diffeent columns of the infomation matix. In case thee ae eos, as in the case of the fist node in Figue 2, the multiplication with the andom vecto will not obscue these eos unless S ei = 0 e i1 +e i2 +e i3 = 0, fo i = {1,2}. The eason why the chosen vecto should be andom is so that the advesay can not delibeately choose the eos to make them disappea afte the vecto multiplication. B. Geneal case The initial infomation matix X F n(n k) N q is coded with the use of an N-extended MDS code with geneato matix G F n(n k) k(n k) q. Some of the stoage nodes contain eos and theefoe what is actually stoed on the distibuted stoage system is Y = GX+ whee Y, F n(n k) N q and is the eo matix. The veifie wants to identify all eoneous disks by sending hash poduct equests to all nodes. Then the following theoem holds: Poof of Theoem 1: All stoage nodes shae N logq bits of common andomness and theefoe they can ceate the same andom vecto F N 1 q with each component of vecto dawn unifomly at andom fom F q. Afte the andom vecto is computed, each stoage node calculates the hash poduct inne poduct between the andom vecto and its content on evey ow. These n(n k) hash poducts ae equal to: H = Y = (GX +) H = G(X)+e (1) whee e = F n(n k) 1 q is a column vecto with at most t m non-zeo components epesenting the eoneous disks (these non-zeo components must coespond to the position of at most t m stoage nodes with eos). The key obsevation is that the pojection will not identify an eo patten at a specific ow if vecto is othogonal to that ow of. Intuitively, a andomly selected will be non-othogonal to an abitay ow of with high pobability and this is the pobability we need to analyze. Fom equation (1) it can be seen that the ode of applying the MDS encoding on the diffeent columns of the infomation matix X and the calculation of the hash poducts can be intechanged ((GX) = G(X)) making the pocess of identifying the eoneous disks equivalent to finding the eo positions in a egula MDS code that is guaanteed to succeed if the minimum distance of the code (n k +1) is lage than twice the numbe of eos 2t (that is indeed satisfied by the assumptions of Theoem 1). The set of indices that coespond to the components of vecto e that come fom disk i is R i = {(i 1)(n k)+1,...,i(n k)}. We ae inteested in vecto e since this gives us the positions of the faulty disks. One complication that might aise is the fact that disk i might contain an eo, meaning that ows {j,j R i} of the eo matix ae not all zeo wheeas the

7 coesponding components of vecto e ({e j,j R i }) tun out to be zeo and theefoe ou scheme fails to detect that eo. Assume that the set of eoneous disks is W {1,2,...,n} and define P[F] to be the pobability of failing to detect some eos. We get P[F] = P ( j = 0 ) i W j R i P (j = 0) 1 q n k 2 t 1 q q i W j R i i W (2) whee inequality ( ) holds due to the fact that the pobability that some stoage node with eos poduce zeo hash poducts is less than 1/q f whee f is the numbe of linealy independent eos ows saved at its disk. So by assuming that the advesay has poduced linealy dependent eos would only incease the pobability of failue. If the advesay has saved eo vectos at stoage node i with ank 1 then the pobability P[ (j = 0)] in equation j Ri (2) educes to an equation fo a single ow (assuming ow k): P e kf f = 0 = P f = e kf f = 1 e kf q e kf 0 whee we only took the tems with a non-zeo eo coefficient e kf. The numbes (e kf /e kf ) f (e kf is any non-zeo eo element fom the k th ow) ae independent and unifom ove F q and so is thei sum accoding to Lemma 1. So the last equality e kf 0 holds since two independent unifomly distibuted ove F q andom numbes ae equal with pobability 1/q. When the eos have ank f > 1 then the pobability P[ j Ri ( j = 0)] can be evaluated by disegading the linealy dependent ows. By looking only at the linealy independent ones and by choosing f columns we can fomulate an invetible submatix Êi F q f f and similaly to the pevious analysis we have that P[ j Ri ( j = 0)] = P[Êiˆ =ˆb] whee ˆ,ˆb F f 1 whee ˆ ae the components of the andom vecto that coespond to the columns whee the submatix Êi was fomed. Since ˆb is unifomly andom, due to the pevious analysis P[ ʈ = b] = 1/q f. ach of the n stoage nodes has to convey to the veifie the esult of the hash poduct fom all its (n-k) ows, so that the total size of the hash communicated is H = n(n k)logq, wheeas the size of the file M = k(n k)n logq. By substituting the field q equal to n k 2 M we conclude the poof of Theoem 1. Lemma 1: The sum of any numbe of independent unifomly distibuted andom vaiables gives a unifomly distibuted andom vaiable. Poof: Without loss of geneality we will pove Lemma 1 only fo the case of two andom vaiables. Assume that x,y F q ae two independent and unifomly distibuted andom vaiables. We will pove that x + y is also unifomly distibuted, indeed t 1,t 2 F q : P[x+y = t 1 ] = P[x = t 1 y y = t 2 ]P[y = t 2 ] t 2 F q

8 ( ) = P[x = t 1 t 2 ] 1 q = q 1 q = q 1 q 2 = 1 q t 2 F q t 2 F q 1 whee equality (*) holds due to the independence between x and y. Befoe we continue to pove Theoem 2 we need to give the following definition (extension of Definition 2.1 in [8] to non-pime numbes): Definition 1: a) Let q be a pime o an intege powe of a pime. Fo a andom vaiable X with values in F q, let the bias of X be defined by bias(x) = (q 1)P[X = 0] P[X 0] A andom vaiable X F q is ǫ-biased if bias(x) ǫ. b) The sample space S F q l is ǫ-biased if fo all c F q and each sequence β = (β 1,...,β l ) F q n \{0 l } the following is valid: if a sequence X = (x 1,...,x l ) S is chosen unifomly at andom fom S, then the andom vaiable ( l i=1 β ix i +c) is ǫ-biased. Poof of Theoem 2: All stoage nodes execute the algoithm descibed in Poposition of [8] and poduces a pseudoandom vecto F q N 1 with N components. The quantity m in the algoithm (and consequently the field size F q m too) is chosen so that the bias (q 1)(N 1)/q m is equal to 1 and theefoe q m = (q 1)(N 1) o m = O(logN). The size of the necessay seed that needs to be povided at all the stoage nodes so that they can stat the algoithm is two elements fom F q m chosen unifomly at andom o equivalently 2mlogq O(logN) andom bits. Once all stoage nodes have constucted the same pseudoandom vecto they compute the inne poduct between vecto and the content stoed on each ow of the stoage nodes. These pseudoandom poducts ae all sent to the veifie to identify the eoneous disks. The whole analysis is identical to the poof of Theoem 1 with one majo diffeence in the calculation of failue pobability P[F ]. Fo the case of a pseudoandom vecto, using the same notation as in the poof of Theoem 1: P[F ] = P ( j = 0 ) i W j R i P ( j = 0 ) P ( j = 0 ) i W j R i i W j R i (n k) n k 2 2 q 2(n k)t 1 q whee inequality ( ) holds since P ( j = 0 ) = 2/q. Indeed the bias of the space constucted by the pseudoandom pocedue is 1 that means: (q 1)P ( j = 0 ) P ( j 0 ) 1 (q 1)P ( j = 0 ) [ 1 P ( j = 0 )] 1 2 This algoithm is descibed fo q pime but it is eadily extensible to q equal to an intege powe of a pime.

9 qp ( j = 0 ) 1 1 P ( j = 0 ) 2 q By setting q = 2(n k)t 1 M we conclude the poof. We would like to undeline hee that both theoems above exhibit the same behavio on the pobability. In Theoem 2 the size of the equied common andomness is deceased in the expense of an inceased field size. Moeove the use of pseudoandom geneatos incus the additional computational cost at each stoage node of O(Nm 2 ) o O(MlogM) opeations in F q to geneate the pseudoandom vecto. ACKNOWLDGMNT The authos would like to thank Pofesso Leonad Schulman fo his insights on pseudoandom geneatos. RFRNCS [1] S. Rhea, C. Wells, P. aton, D. Geels, B. Zhao, H. Weathespoon, and J. Kubiatowicz, Maintenance-fee global data stoage, I Intenet Computing, vol. 5, no. 5, pp , [2] R. Bhagwan, K. Tati, Y.-C. Cheng, S. Savage, and G. M. Voelke, Total ecall: System suppot fo automated availability management, in Poc. of the Netwok Systems Design and Implementation, 2004, pp [3] Y. Sun, F. Liu, B. Li, Li, B. Li, and X. Zhang, Fs2you: Pee-assisted semi-pesistent online stoage at a lage scale, in Poc. of the I Confeence on Compute Communications, Apil [4] A. G. Dimakis, P. B. Godfey, Y. Wu, M. O. Wainwight, and K. Ramchandan, Netwok coding fo distibuted stoage systems, Submitted fo jounal publication. Peliminay vesion appeaed in Infocom [5] P. A. Chou, Y. Wu, and K. Jain, Pactical netwok coding, in Poc. of 41st Annual Alleton Confeence on Communication, Contol, and Computing, [6] S. Jaggi, M. Langbeg, S. Katti, T. Ho, D. Katabi, and M. Médad, Resilient netwok coding in the pesence of byzantine advesaies, in Poc. I INFOCOM 2007, Anchoage, AK, May [7] R. Koette and F. R. Kschischang, Coding fo eos and easues in andom netwok coding, I Tansactions on Infomation Theoy, 2007, submitted. [8] C. Betam-Ketzbeg and H. Lefmann, MOD p-tests, almost independence and small pobability spaces, Random Stuctues & Algoithms, vol. 16, no. 4, pp , [9] J. Nao and M. Nao, Small-bias pobability spaces: fficient constuctions and applications, SIAM Jounal on Computing, vol. 22, pp , [10] T. S. J. Schwaz and. L. Mille, Stoe, foget, and check: Using algebaic signatues to check emotely administeed stoage, in Poc. of ICDCS, [11] W. Litwin and T. Schwaz, Algebaic signatues fo scalable distibuted data stuctues, in Poc. of the I Intenational Confeence on Data ngineeing (ICD 04, 2002, pp [12] M. Kohn, M. Feedman, and D. Maziees, On-the-fly veification of ateless easue codes fo efficient content distibution, in Poc. of the I Symposium on Secuity and Pivacy, May 2004, pp [13] C. Gkantsidis and P. Rodiguez, Coopeative secuity fo netwok coding file distibution, in Poc. of I Intenational Confeence on Compute Communications, Apil 2006, pp [14] F. Zhao, T. Kalke, M. Médad, and K. J. Han, Signatues fo content distibution with netwok coding, in Poc. of the I Intenational Symposium on Infomation Theoy (ISIT), [15] D. Chales, K. Jain, and K. Laute, Signatues fo netwok coding, in Annual Confeence on Infomation Sciences and Systems, Mach 2006, pp [16] J. P. Vilela, L. Lima, and J. Baos, Lightweight secuity fo netwok coding, in Poc. of the I Intenational Confeence on Communications, May 2008.

10 [17]. Kehdi and B. Li, Null keys: Limiting malicious attacks via null space popeties of netwok coding, in Poc. of the I Confeence on Compute Communications, Apil [18] A. Le and A. Makopoulou, Locating byzantine attackes in inta-session netwok coding using spacemac, in Poc. of NetCod, June 2010.