Transcription

1 Deaki Research Olie Deaki Uiversiy s isiuioal research reosiory DDeaki Research Olie Research Olie This is he ublished versio (versio of record) of: We, Sheg, Jia, Weijia, Zhou, Wei, Zhou, Walei ad Xu, Chua, CALD : survivig various alicaio-layer DDoS aacks ha mimic flash crowd, i NSS : Proceedigs of he 4h Ieraioal Coferece o Nework ad Sysem Securiy, IEEE, Piscaaway, N.J., h://hdl.hadle.e/536/dro/du: [] IEEE. Persoal use of his maerial is ermied. However, ermissio o reri/reublish his maerial for adverisig or romoioal uroses or for creaig ew collecive works for resale or redisribuio o servers or liss, or o reuse ay coyrighed comoe of his work i oher works mus be obaied from he IEEE. Coyrigh :, by The Isiue of Elecrical ad Elecroics Egieers, Ic. All righs reserved

2 Fourh Ieraioal Coferece o Nework ad Sysem Securiy CALD: Survivig Various Alicaio-Layer DDoS Aacks Tha Mimic Flash Crowd Sheg We, Weijia Jia, Wei Zhou Ceral Souh Uiversiy Chagsha, Hua, Chia, 483 swe@deaki.edu.au Walei Zhou Deaki Uiversiy Melboure, Ausralia, 35 walei@deaki.edu.au Chua Xu Chogqig Uiversiy Chogqig, Sichua, Chia, 444 chuaxu@gmail.com Absrac Disribued deial of service (DDoS) aack is a coiuous criical hrea o he Iere. Derived from he low layers, ew alicaio-layer-based DDoS aacks uilizig legiimae HTTP requess o overwhelm vicim resources are more udeecable. The case may be more serious whe such aacks mimic or occur durig he flash crowd eve of a oular Websie. I his aer, we rese he desig ad imlemeaio of CALD, a archiecural exesio o roec Web servers agais various DDoS aacks ha masquerade as flash crowds. CALD rovides real-ime deecio usig mess ess bu is differe from oher sysems ha use resemblig mehods. Firs, CALD uses a fro-ed sesor o moior he raffic ha may coai various DDoS aacks or flash crowds. Iese ulse i he raffic meas ossible exisece of aomalies because his is he basic roery of DDoS aacks ad flash crowds. Oce abormal raffic is ideified, he sesor seds ATTENTION sigal o acivae he aack deecio module. Secod, CALD dyamically records he average frequecy of each source IP ad check he oal mess exe. Theoreically, he mess exe of DDoS aacks is larger ha he oe of flash crowds. Thus, wih some arameers from he aack deecio module, he filer is caable of leig he legiimae requess hrough bu he aack raffic soed. Third, CALD may divide he securiy modules away from he Web servers. As a resul, i kees maximum erformace o he kerel web services, regardless of he harassme from DDoS. I he exerimes, he records from ad have roved he value of CALD. Keywords-DDoS; Alicaio-layer; Kalma Filer; Iformaio Theory. INTRODUCTION (HEADING ) For may years, disribued deial of service (DDoS) aack has caused severe damage o vicims ad sill cosiues oe of he major hreas i curre iere. A oular form of DDoS oday is he alicaio-layer floods ha overwhelm he Web server wih a large umber of GET requess. To circumve deecio, he aackers icreasigly move away from ure badwidh floods o sealhy DDoS aacks ha masquerade as flash crowds. The successful cases i he early hisory icluded MyDoom [], Code Red [] ad FBI case ivolvig DDoS-for-hire [3]. I rece years, we frequely heard he ews ad comlai abou alicaio-layer DDoS harassme [4-7]. I fac, he siuaio is much worse ha we ca exec because he boe is boomig. The Chia CCTV rogram Ecoomy Half Hour broadcas ha he boe has formed a idusry chai ad he caial cocered we beyod billio Chiese Yua i 9 [8]. Alhough i is o jus DDoS aacks ha are associaed wih boe, curre DDoS aacks are maily lauched by i. For a ivesigaio io DDoS crime, he BBC rogram Click brough a medium sized Websie ad demosraed oly 6 broadbad coecios were eough o make his Websie uusable [9]. The exers i he TV show also said ha he high-raffic sies were oeial vicims for alicaio-layer DDoS aacks. The crimials go io coac wih he Websies ad hreaeed hem i DDoS aacks. All kids of high-raffic Websies ha geerae los of reveue relied o he Websies o be olie, so a lo of he Websies aid u o avoid he DDoS aacks. Couerig alicaio-layer DDoS aack becomes a grea challege because hey ca be moued wih legiimae requess from legiimae coeced machies. The requess origiaig from he comromised comuer are idisiguishable from he requess geeraed by legiimae users. I fac, hey differ from he legiimae oes oly i ie bu o i coe. The malicious requess arrive from a large umber of geograhically disribued machies; hus hey cao be filered o he IP refix. Also, may Websies do o use asswords or logi iformaio, ad eve whe hey do, asswords could be easily sole from he hard disk of a comromised machie. Furher, checkig he siesecific assword requires esablishig a coecio ad allowig uauheicaed clies o access socke buffers ad worker rocesses, makig i easy o mou a aack o he auheicaio mechaism iself. Moreover, he mehod of usig comuaioal uzzles, which requires he clies o recogize a figure ad submi he resul, is o suiable someimes because is demad of arificial egageme aoys eole. Fially, i coras o badwidh aacks [], i is difficul o deec big resources cosumers whe he aack arges higher-layer boleecks such as CPU, daabase ad disk because commodiy oeraig sysems do o suor fie-graied resource moiorig. Furher, a aacker ca resor o muaig aacks which cycle bewee differe boleecks. This aer rooses CALD, a archiecural exesio o roec Web servers agais alicaio-layer DDoS aacks ha masquerade as flash crowds. I is argeed owards largescale olie busiesses as well as o-commercial oral Websie. CALD combies hree fucios: abormal raffic deecio, DDoS aack deecio, filer. a) Abormal raffic deecio: The abormal raffic deecio is a real-ime ime series aalyser. This fucio is deloyed i he fro-ed sesor. Accordig o our idea, he uderlyig assumio is ha regular raffic behaviour is relaed o ormal sysem use, or a leas ha oly chages i ay regular behaviour are ieresig. Thus we aim o deec ay abru chage i he HTTP Ge reques raffic. The basic idea of his fucio is ha oce he modellig is doe, he ouu of he model corresods o ormal sysem / $6. IEEE DOI.9/NSS

3 ATTENTION Or DISMISS DDoS or Flash Crowds? yes PARAMETER yes Ge requess Aomaly Traffic? o Ge requess Aomalous IP? o Ge requess Websie yes Dro Figure : CALD Overview. Noe ha DDoS Aack Deecio comoe is oly acivaed whe fro-ed sesor reors ATTENTION sigal. behaviour, ad he differece bewee observed behaviour ad model ouu gives us he aomalous sigaure. We reor his sigaure as a sigal o DDoS aack deecio comoe ad diversify wheher flash crowd or DDoS really haes. We have see los of successful alicaio of such idea o ework raffic aalysis [, 3]. I his aer, we drew AR model o describe ad redic he Ge reques rae, ad iroduce he dyamical Kalma filer o calibrae he redicio resul because of iesely flucuaio. I curre saus of our research, we oly cosidered saioary AR model which could be exeded o o-saioary rocess i.e. he eve usig Kalma filer for esimaig ime-deede AR coefficies. b) DDoS aack deecio: whe receives ATTENTION sigal from fro-ed sesor, he DDoS aack deecio comoe is acivaed. I races each icomig source IP address as well as each visiig Webage, ad records he average frequecies i a vecor. I a iese raffic, he average frequecy ca be cosidered as he ossibiliy of each source IP address. Based o he vecor, his comoe calculaes he eroy. The eroy called mess exe describes he disribuio of icomig sources ad arge WebPages. We defie he mess exe of source IP addresses as A, he mess exe of arge WebPages as B ad he rae bewee A ad B as R. Theoreically, he varia R i flash crowd is smaller ha he oe i he alicaio-layer DDoS aack. Thus, we are able o se several hresholds ad ick ou he aomalous source IP addresses. c) Filer: whe he abormal raffic is jusified as coaiig DDoS aack, he DDoS aack deecio comoe seds he aomalous source IP addresses o filer so ha i ca release he floodig. Curre alicaio-layer DDoS aack is geerally lauched by more ha, comromised comuers [4]. Imlemeig such a large lis of IP addresses for block is o very easy. Simly searchig mehod will defiiely decrease he efficiecy of he whole sysem. I his aer, we adoed Bloom Filer [5], where we imlemeed wo hash fucios iside ad was caable of limiig he collisio below sixee ehousadh. Fig. summarizes CALD. I has a few imora characerisics. Firsly, CALD addresses various alicaiolayer DDoS aack. Prior work usually has a bias agais he Ge reques floodig he homeage. As will be described i secio, CALD is cocered wih hree yes of DDoS aacks; he laer is sealhier ha he former. They are all easily achieved i he Iere ad CALD ca be caable of dealig wih hem. Nex, he desig of CALD draws esseial roeries agais he alicaio-layer DDoS aack. I he fro-ed sesor, he esseial roery is he large quaiy of Ge reques from he aackers. I he DDoS deecio model, he esseial roery is he differece i he rae of mess exes. No maer which ye of DDoS is lauched by he aackers, CALD is able o race he aack. A las, CALD is simle. The simliciy i he algorihm icreases he efficiecy which is aricular imora i famous Websie. CALD also divides he securiy modules away from he Web servers. As a resul, i kees maximum erformace o he kerel web services, regardless of he harassme from DDoS. Eve he udae ad deloyme become more coveie. The res of he aer is orgaized as follows. We sar by describig he hree yes of hrea models i Secio. Secio 3 describes he deails of CALD. More secially, here we aalyse he esseial roeries of he alicaiolayer DDoS aacks ad flash crowds. Secio 4 elaboraes he exerimes. The las wo secios are cocered wih some discussio ad relaed work abou couerig alicaio-layer DDoS aacks.. THREAT MODEL This aer maily cocers wih hree yes of alicaio-layer DDoS, of which each oe is sealhier ha he former. () The aackers orgaize los of comromised comuer bombig he homeage of he Websie, which we call as reeaed reques DDoS. MyDoom [] ad Code Red [] all belog o he DDoS aack of his kid. () The aackers ick several remium ages ou io a lis ad sochasically choose oe as he arge for each sedig of he HTTP Ge requess. Eve his ye of aack ca erform like Websie Sider [], which i a auomaed maer ad orderly fashio browses he ages of he arge Websie. We have o see ay ews or reors alkig abou such case haeed i he wild as a alicaio-layer DDoS aack. However, i is quie easy o be achieved ad defiiely hard o be racked because he abormal raffic is leveraged io a grou of arges ad acs more like a legiimae 48

4 visiig. We call i recursive reques DDoS. (3) There may exis some large files like image or dowload resources i he Websie. There may also coai some oeraios ha are cocered wih heavy workload like daabase search. The aacker seds Ge reques o aim a hese resources. This reeaed workload DDoS ca be ivoked a a lower reques rae, hereby requirig less work from he aacker ad makig deecio icreasigly difficul. 3. THE DESIGN OF CALD Accordig o our idea, a fro-ed sesor is used o deec he abru chage i he raffic. If abormal raffic exiss, he DDoS aack deecio comoe will be acivaed ad make advaced isecio for a decisio. The DDoS raffic from he malicious IP addresses will be ulimaely blocked ad flash crowded coiues. I his secio, we will describe how he modelig is doe for abormal raffic deecio, how he aomalies are diversified as DDoS or flash crowd, ad how he DDoS raffic is filered. 3. Abormal Traffic Deecio The raffic we will arge is a sream of successive HTTP Ge requess. Traffic iesiy measureme ake a fixed, discree ime iervals from a ime series {y }. The iese chage i he value of measureme meas ossible exisece of alicaio-layer DDoS aacks or flash crowds. Therefore, a isa, he differece bewee measured value y ad model ouu y rereses he abormal cosiue of he raffic. The abormal cosiue is called also he residual series or he model error, ad is defied as d = y -y. As a early sage of our work, we use he saioary AP () model for abormal raffic deecio. The model is defied as: y " a x e () k k k Here y is he redicio. x is he observaio a isa. a k are he saioary model arameer, ad e is observaio error. I oher words, he model uses a weighed sum of revious values o esimae he curre observaio value, he weighs beig he AR coefficies a k, k=... The saioariy meas ha he weighs a k are ime ideede. The idea is ha ormal sysem usage causes sufficiely regular ad smooh raffic, ha he curre value ca be rediced as a liear combiaio of as values. The ar of he raffic behaviour ha cao be rediced i his fashio is sufficiely aomalous o be reored o he ex comoe. I curre sage of desig, we use saioary AR model, which meas he arameers will o ada o chages of he moiored raffic. We simly assig a k o be -k ad he sum of he a k is equal o. I he fuure research, we will exed his comoe. The model degree eeds o be defied. However, is value deeds srogly o he ye of Websies. We will discuss i i he Secio 4. wih he exerimeal seig. The ierval is secod ad he sesor cous he HTTP Ge requess durig he ierval. Based o he revious records, he AR model calculaes he redicio y for he ex se. We eed o calibrae he y because of iese flucuaio i he ework raffic. We fech he deviaio d bewee calibraed value y ad realisic measured value o make decisios. Fig. describes a eisode of he HTTP Ge requess o Aril h. We cosidered he cases i boh Fig. A ad Fig. B as ormal raffic. However, if we se he hreshold of abormal raffic decisio o be a lile small for more sesiiviy, he flucuaio will grealy oise he redicio as we see i Fig. C ad Fig. D. Therefore, we eed o calibrae he value afer redicio. Kalma filer is a adaive ad recursive daa rocessig algorihm ha is suied for o-lie esimaio. Is essece is o comue he covariace ad make a calibraio bewee he measured value ad observed value. Filerig ca be see as a rocess whe a ew observaio arrives. The redicio ad measureme equaios ca be u i vecor form: y " AX e () y MX w (3) Wih he coefficies as A ( a... a ) (,..., ) (4) (5) M ( m... m ) (,...) For rereseig he Kalma filer equaios, we use X - o deoe he esimaed value a ime usig observaios accumulaed a isa -. I addiio, we use C o deoe he covariace a ime. Now he Kalma filer equaios for calibraig he redicio are: x y " AX A( y' (6) C ) C x (7) x x Kg( y x ) (8) Kg( ) C C (9) ( y ) ( Kg( )) C C () Amog he equaios, we fech revious values of x ad iiial he covariace C as: T () C X avg( y ))( X avg( y )) ( Ad he oe-se calibraio resul will be (). As a examle, Fig. E ad Fig. F deic a -ses calibraio. ' () y x Fig. (E) ad Fig. (F) deic a examle of -ses calibraio. Moreover, Kalma filer is a recursive algorihm ha may brig a lile delay. This is because of he N-ses recursio. Thus he oal comlexiy for abormal raffic deecio is N*. I fac, we have o use observaios x -x +N o esimae a imroved redicio. The delay icreases whe N is elarged ad he accuracy iversely decreases. Therefore i should be chose for a desired balace bewee redicio accuracy ad suiable delay. We will discuss his value i Secio 4. ad

5 Requess / s Deviaio Deviaio :3 5: 7:3 : :3 5: 7:3 : :3 (A) 5 Sia Traffic - May 4h, - From :3 To :3 :3 5: 7:3 : :3 5: 7:3 : :3 (C) 5 Sia Deviaio - Wihou Calibraio Sia Deviaio - Kalma Ses Calibraio :3 5: 7:3 : :3 5: 7:3 : :3 (E) Requess / s Deviaio Deviaio :3 5: 7:3 : :3 5: 7:3 : :3 (B) 5 Taobao - May 4h, - From :3 To :3 :3 5: 7:3 : :3 5: 7:3 : :3 (D) 5 Taobao Deviaio - Wihou Calibraio Sia Deviaio - Kalma Ses Calibraio :3 5: 7:3 : :3 5: 7:3 : :3 (F) Figure. Normal Traffic Examle ad Abormal Traffic Deecio. Noe ha his aer uses Kalma Filer o calibrae he deviaio so ha he flucuaio i he ormal raffic will o oise he abormal raffic deecio. 3. Aack Isecio 3.. Acivae & So DDoS Deecio Comoe Whe he abormal raffic is deeced, he fro-ed sesor will sed ATTENTION sigal o he DDoS aack deecio comoe ad acivae his comoe. This meas he DDoS aack deecio does o work for he whole raffic. I oly rus whe abormal raffic arrives. I corresodece, whe he fro-ed sesor fids he raffic has chaged back o ormal series, i will sed DISMISS sigal o he DDoS aack deecio comoe ad so is work. To achieve above fucios, we aggressively choose he hreshold for abormal raffic as: d k (3) d ( ( d avg( d )) ) (4) d / i Afer Kalma filer smoohes he deviaios, he value k i (3) becomes he oly arameer o adjus he sesiiviy for acivaig he DDoS aack deecio. This is aoher balace value because smaller value k will brig more sesiiviy. We will discuss his value i Secio Proeries of DDoS Aacks ad Flash Crowds I his subsecio, we will aalyse he basic roeries of alicaio-layer DDoS aacks ad flash crowds so ha o maer wha sealh mechaisms he aackers ado, he deecio cao be circumveed. As is meioed i he Secio, our work is cocered wih hree yes of aacks. Durig he aalysis o real daase of ad we oiced ha: Reeaed Reques DDoS: his ye of aack maily focuses o he homeage or a ho Webage, so he arges of he raffic coverge o oe or wo oi. We also suose he aack is lauched by a cerai quaiy of bos. Thus he sources of he raffic coverge o a grou of ois. Recursive Reques DDoS: his ye of aack has a characer ha he raffic is scaered io differe WebPages. As a resul, he sources of he raffic coverge o a grou of ois bu he arges of he raffic become disersed i some exe. Reeaed Workload DDoS: his ye of aack is able o use less bos bu eve larger damage o he Websie. However, he raffic will be similar o he case of reeaed reques DDoS. A grou of bos coiually seds requess for a large image or daabase searchig oeraios. Flash Crowd: a uexeced surge i visiors o a Web sie, which is yically because of some ewsworhy eve ha jus ook lace. I may also be due o he aouceme of a ew service or free sofware dowload. The sources of flash crowds are defiiely scaered because hey are legiimae HTTP Ge requess ad from legiimae users. Coversely, he arges of flash crowds coverge o oe or wo ois because he visiors are maily ieresed i oe Webage or resource, which comly wih he defiiio of flash crowd Disiguish DDoS Aacks ad Flash Crowds Because he disersio exe of he alicaio-layer DDoS aacks ad flash crowds are differe i heir arges ad sources, we are able o use his basic roery o disiguish hem. However, such a saisical resemblig mehod souds iracable i real-ime rocess as he saisical algorihms geerally have grea comlexiy i boh sace ad ime. I CALD, we firsly iroduce a recursive mehod o calculae he ossibiliies for each sources ad arges: favg favg (... ) T T T3 T T T ( favg ) /( ) T T (5) / (6) 5

6 Accordig o equaios (5) ad (6), wo dyamic vecors should be imlemeed i DDoS deecio comoe. Whe a ew HTTP Ge reques arrives, CALD will comue he ime differece bewee he wo arrival imes ad udae he average frequecies i he source ad arge vecors. Paricularly, we ideify each arge Webage ad source wih secial serial umber so ha searchig he osiio for udaig or iserig will o cos oo much CPU ime. The comlexiy exe of he whole algorihm kees i O(3), which is acceable for he efficiecy of CALD. I a iese raffic, we simly cosider average frequecy as he ossibiliy. To describe he disersio exe of he alicaio-layer DDoS aacks ad flash crowds, here we iroduce he cocree measureme: Defiiio (Mess Exe): suose here is a se {}, if he elemes i se {} are osiioed disersedly, he he Mess Exe is higher. Oherwise, if he elemes i se {} are coverged i some ois by ay orgaized form, he Mess Exe is lower ad close o. I CALD, he differe mess exes of he sources or arges ossibiliies are caable of describig he righ disribuio of aackers ad arges. I fac, he mess exe deoes he disribuio of coaied iformaio. This is esseially equivale o he relaed coce i iformaio heory. More cocreely, CALD eriodically calculaes he mess exe of a vecor usig Shao formula: ) H (, i,..., ) favg i favg i ) i ( x )log( x ) log( (7) Accordig o he aalysis i Secio 3.., we ca use he rae of mess exes bewee sources ad arges o disiguish various alicaio-layer DDoS aacks ad flash crowds. We deduce he followig coclusio (S is source vecor; T is arge vecor; reeaed reques DDoS as ; recursive reques DDoS as ; reeaed workload DDoS as 3 ad flash crowd as 4): S) T ) S) S) 3 4 (8) T ) T ) 3 T ) 4 i S) I CALD, he DDoS deecio comoe imlemes wo hresholds, oe for mess exe calculaio eriod ad aoher for he boudary bewee DDoS aack ad flash crowd. We u he deails of he former i Secio 4.3 ad he laer i Secio 4.. Whe he DDoS deecio comoe fids a real-ime rae falle i he rage DDoS aack, i will os he source IPs, which has a higher frequecy, o he filer comoe. 3.3 Filers Whe he DDoS deecio comoe has made a decisio ha curre raffic coais alicaio-layer DDoS aack, he DDoS deecio comoe will se he i arameers of he filer for blockig he HTTP Ge requess from hose malicious IP addresses. I fac, curre alicaio-layer DDoS aacks are maily lauched from boe, here is a edecy ha he quaiy of bos cocered i he DDoS aack becomes larger ad larger [4] ha he umber may reach more ha,. As described i Secio 3.(c), we simly choose he source IP addresses ha emerges more ha Q imes (we se Q = i CALD) i secod ierval. However, i is o a easy ask o imleme such a lis which more ha, malicious IP addresses. We are able o fid some relaed work i he echologies of ework rouers for ackes relayig [6, 7]. However, rouers oly eed o look u he refix of a IP address based o mask code. Whe he cases come o he filer comoe of CALD, he looks-u should cover he whole IP address. To avoid he filer becomig a boleeck of he whole sysem, CALD similarly adoed Bloom Filer [5] as he basic IP looks-u ad udae echology. A emy Bloom Filer is a bi array of m bis, all se o. There mus also be k differe hash fucios defied, each of which mas or hashes some se eleme o oe of he m array osiios wih a uiform radom disribuio. Here CALD imlemes wo hash fucios iside (k = ) ad he legh of he bi array m is se o be. Suose he IP address is i doed decimal oaio A.B.C.D, ad he he hash fucios are: Hash fucio : ( A B C D ) mod (9) Hash fucio : ( A * B* C * D) mod () To add a eleme, feed he IP address o each of he wo hash fucios o ge wo array osiios. Se he bis a all hese osiios o. To query for a eleme (es he IP address is i he se), feed i o each of he wo hash fucios o ge wo array osiios. If ay of he bis a hese osiios are, he IP address is o i he se. The reaso is if i were, he all he bis would have bee se o whe i was isered. If all are, he eiher he IP address is i he se, or he bis have bee se o durig he iserig of oher IP addresses. The laer siuaio is called collisio. Because we se he filer wih wo hash fucios, he legh of hash able is ad esimaed quaiy of malicious IP addresses is,, he collisio ossibiliy CP becomes: k k k m k CP lim [ ( ) ] ( e ) m m lim m 4 k k 6 ( ) ( ) 6 4 m () For CALD, he CP of sixee e-housadh is o very big ad he umber of bis meas 8KB memory is eeded for hash able. 4. PERFORMANCE EVALUATION 4. Exerime Evirome Fig. 3 illusraes he key comoe of CALD. Our desig allows differe comoes assembled i differe 5

7 Ge requess / s 5 Sia - Aril h -From 6: To :3 DDoS flash crowds 6: 7: 8: 9: : : : 3: 4: 5: 6: 7: 8: 9: : : : Time Figure 4: The Quaiy of HTTP Ge Requess er Secod. 8 Predicio wihou Calibraio Whe = 5 Deviaio 6 4 6: 7: 8: 9: : : : 3: 4: 5: 6: 7: 8: 9: : : : Time Figure 5: The Deviaio whe = 5. comuers so ha he fucios of CALD will o affec he rocess of Websie. However, i our exerime, we simly iegraed various comoes ogeher wih he Web server. The comuer for exerime was a sadard 3GHz double kerels Peium IV machie wih 4GB memory ad ruig widow visa. We also imlemeed a discree ime series roducer which relayed each HTTP Ge reques of i Aril h. These HTTP Ge requess were colleced from he backboe of Chia Souhwes. The oulaio cocered was more ha 3 millio. The badwidh of he backboe was 4GB. Uder curre codiio of echologies, we were able o collec he requess i GB wherei GB for maageme. Therefore he rae for collecio was oe fifh. As CALD ever cares abou he sequece of HTTP Ge requess, such rough daa is eough for he aalysis. We believe his characer is righ a advaage agais he work of [-3, 6], because someimes we ca hardly gaher he exac HTTP Ge reques series. 4. Sesiiviy ad Modular Delay There are wo asecs ha may ifluece he erformace of CALD: sysem sesiiviy ad modular delay. Sysem sesiiviy decides he abiliy o deec he alicaio-layer DDoS aacks ad modular delay decides he ime for CALD o deec he aacks. There are hree arameers which may affec hese wo kids of erformaces. The firs oe is he arameer of AR model. The radiioal assigme o arameer should is direced by arameer esimaio [7], which is o suiable o be dyamically used i CALD. As will be discussed i Secio 6, CALD does o eed accurae model for racig he raffic. I fac, i curre versio, we se he arameer a k as { -k }, so whe k becomes larger, he effec becomes less, eve early o effec o AR model. Fig. 4 shows a eisode of alicaio-layer DDoS aack from i Aril h. Fig. 4 also describes a flash crowd which haeed i he same day. Fig. 5 shows he corresodig redicio whe we se as 5. The secod oe is he lag of Kalma filer. Larger lag brigs more sesiiviy as log as larger delay for DDoS aacks deecio. Fig. 6 shows -se o 3-ses calibraios which will brig secod o 3 secods delay resecively. I he figures, we oiced a srage heomeo ha -ses calibraio erformace beer amog he hree. The 3-ses calibraio rebouded o be more serraed i he amliude of deviaios. We will fid he reaso i he fuure work. The hird oe is he boudary for disiguishig he aacks ad flash crowds. Seig his hreshold will also affec boh he sesiiviy ad modular delay. I curre work, we have o foud ay reasoable mehods o decide he mos suiable value for his boudary. We simly assiged i as.7 ad successfully deeced he DDoS aack of i he seveh secod afer five secods for AR model reor abormal raffic ad secods for calculaio eriod of mess exe. CALD ever riggered he block of source IPs agais flash crowds. 5. RELATED WORK Jaeyeo Jug ad his collaboraors [8] sudied he roeries of boh alicaio-layer DDoS aack ad flash crowd wih a secial aeio o characerisics ha migh disiguish he wo. Ideifyig hese characerisics allowed a formulaio of a sraegy for Websies o quickly discard malicious requess. However, as we ca say, some of he declared characerisics are a series of heomeo, o he esseial roeries. The aackers were caable of avoidig his aearace by simly adjusig aack scheme. CAPTCHA [9] ad laer aoher imroved versio by Scriah Kadula, ec [], almos erfecly solved he alicaio-layer DDoS, eve he flash crowds, bu his mehod basically required he egageme of clies o recogize a uzzle figure ad iu he resul for a auheicaio. Suraamaya Raja e al. [] roosed a couer-mechaism ha cosiss of a susicio assigme mechaism ad a DDoS-resilie scheduler. The former assiged a susicio measure o a sessio i roorio o is deviaio from legiimae behaviour ad used he laer o decide wheher ad whe he sessio is serviced. Accordig o heir exerimes, he vicims erformace could be imroved from 4 secods o.5 secods. However, we 5

8 Deviaio Deviaio Deviaio 5 Calibreaio se = 6: 7: 8: 9: : : : 3: 4: 5: 6: 7: 8: 9: : : : (A) Calibreaio se = 5 6: 7: 8: 9: : : : 3: 4: 5: 6: 7: 8: 9: : : : (B) Calibreaio se = 3 5 6: 7: 8: 9: : : : 3: 4: 5: 6: 7: 8: 9: : : : (C) Figure 6. The Deviaios whe lag k is se o -3. hik DDoS Shield iroduced i his aer had a drawback ha heir mehod could o acively block he malicious raffic. I oly released he symom of beig aacked vicims. I rece wo years, Dr. Yi Xie adoed Hidde Semi- Markov model for he deecio of alicaio-layer DDoS aack. His mehod recorded he curre user visiig sequeces ad calculaed he diversiies i ossibiliies of he sequeces. The bigges roblem of Hidde Semi-Markov mehod was he algorihm comlexiy. Alhough Dr. Yi Xie imroved his drawback based o M-algorihm i [] ad ideede comoe aalysis i [3], we sill foud ha rackig each user s visiig sequece was o a racical ask. Georgios Oikoomou ad Jelea Mirkovic [4] roosed defeces agais alicaio-layer DDoS aacks via huma behaviour modellig which differeiae DDoS bos from huma users. Their mehod was achieved hrough hree asecs: reques dyamics, reques semaics ad abiliy o rocess visual cues. We foud heir work had several limiaios. For isace, hey ried o build a ossibiliy grah for a Websie, bu as we ca say, for mos large Websies such as wih oo may dyamical Webages iside, hey ca hardly fiish he cosrucio. There are sill oher relaed works abou deecig alicaio-layer DDoS aacks. Simo Byers e.al [4] discussed he dagers ha scalable Iere fucioaliy may rese o he real world. Their work focused uo a aack ha is simle, ye could have grea imac, which hey believed migh occur quie soo. This aer is a early ublished rosecive work abou alicaio-layer DDoS aacks. Paul Barford e.al [5] used wavele o disiguish he flash crowds ad DDoS aacks. However, he mehod of usig wavele was oly a os-morem echology. Takeshi Yaagai e.al [6] roosed HTTP Ge floodig deecio echiques based o aalysis of age access behaviour. They roosed wo deecio algorihms, oe is focusig o a browsig order of ages ad he oher is focusig o a correlaio wih browsig ime o age iformaio size. We hik he drawback of heir work was somewha he same wih Dr. Yi Xie s ad Jaeyeo Jug s, because he roeries for deecio could easily be circumveed by simly modifyig he aack schemes. 6. DISCUSSION I could be argued ha AR model is oo limied for modellig he ye of HTTP Ge raffic. For examle, errors o raid chages i observaios are oe maifesaio of he AR model s limiaios ad o-liear models, for examle, could rovide beer reaciviy o raid chages. We have hree reasos o say wih liear models for fro-ed abormal raffic deecio: () give our uderlyig idea we do o eve wa o model he raffic exacly, esecially he raid chages should be lef ou of he model. We oly ay aeio o he amliude of he chages. () Through he smoohig fucio by N-ses Kalma filer, he oise of raid chages will be miigaed ad he calibraed redicio is able o imrove he accuracy. (3)Simle models mea simler algorihms ad his raslaes o faser ad ligher imlemeaios, which is a imora cosideraio for deloyig he mehod. There is aoher mehodological oi which easily icurs argume. I also derives from he AR liear model. Whe he aackers slowly icrease he abormal raffic, hey are able o circumve he abormal raffic deecio. This is he key weakess of liear redicio model resemblig machie learig echologies. I curre versio of CALD, i fac, we se a hreshold iside. Whe he real-ime raffic rises beyod ha hreshold, o maer wheher abormal raffic deecio fucio is aware of he aomalies, he froed will reor he ATTENTION ad acivae he ex DDoS deecio comoe as well. However, if we iduce he raffic of imor as well as exor, CALD will become caable of defedig such slowly icreasig DDoS aack. We will imrove he CALD i he ex sage of our work. Third, CALD has a few arameers ha we have assiged values based o exeriece. For examle, we se he AR model arameer o be 5 i he exerimes. There is ohig secial abou 5, we oly eed a value ha is eiher oo big or oo small. I fac, he redicio could become 53

9 more log-effecig if we se a bi larger. Similarly, we se he mess exe of flash crowds o be uder. There is also ohig secial abou his hreshold, we jus ge i accordig o our exeriece by aalysig he as records of alicaio-layer DDoS aacks. Fourh, he filer comoe eeds o be flushed eveually sice comromised zombies may ur io legiimae clies. The filer ca be cleaed eiher by reseig all eries simulaeously or by decremeig he various eries a a aricular rae. I he ex sage of our work, we will imleme such fucios i CALD. 7. CONCLUSION I his aer we have reseed a aroach for couerig alicaio-layer DDoS aack ha mimics he ormal users behaviours. The basic assumios are: () abru chages i he raffic mea a ossible exisece of abormal raffic; () he mess exe of various DDoS aacks is larger ha he oe of flash crowds. Based o he firs oi, we iroduce a fro-ed sesor o deec he abormal raffic ad reor is exisece whe abormal raffic arrives. The secod comoe which is used for disiguish DDoS aack ad flash crowd is acivaed or soed by he sigal ATTENTION or DISMISS from he fro-ed sesor. Wih arameers of malicious IP addresses, he filer blocks he abormal raffic ad leaves he Websie o be safe. The aalysis ad exerimes show ha CALD erform well i couerig various alicaio-layer DDoS aacks ad he delay for deecio is ke i a rage of N secods. This aer reses a early sage of our work. I he fuure, we will exed he work i he followig asecs: () imrove he saioary AR o be ime-varyig model where he arameers will deoe he chages of he sysem saus; () as is discussed i Secio 6, CALD is o sesiive o he slowly icreasig DDoS aack. Thus we may cosider boh he ouu ad iu raffic o deal wih his siuaio. ACKNOWLEDGEMENTS We hak Chogqi Uiversiy for suorig he real raffic races o he Iere backboe of Chia. We also hak Professor Guofeg Zhao for givig us los of cosrucive commes o CALD. REFERENCES [] CERT. Icide Noe IN-4- W3/Novarg.A Virus, 4. [] CERT. Icide Noe IN-- Code Red Worm Crashes IIS 4. Servers wih URL Redirecio Eables,. [3] Kevi Poulse. FBI buss alleged DDoS Mafia, h:// 4. [4] J.Leyde. Eas Euroea Gags i Olie Proecio Racke, h:// s_i_olie/, 3. [5] Mary Williams ad Jeremy Kirk. Udaed MyDoom resosible for DDOS aacks, h:// udaed-mydoom-resosible-for-ddos.hml?a=rcb, 9. [6] Chuck Miller. Russia Cofirms ivolveme wih Esoia DDoS Aacks, h:// [7] Dacho Dachev. The DDoS Aack agais CNN.com, h://ddachev.blogso.com/8/4/ddos-aackagais-ccom.hml, 8. [8] SOHU News. Boe has formed idusry chai, h://ews.sohu.com/96/ shml, 9. [9] BBC News. How cyber crimials aack Websies. h://ews.bbc.co.uk//hi/rogrammes/click_olie/ sm, 9. [] B.Krishamurhy ad J.Wag. O Nework-Aware Cluserig of Web Clies, i Proceedigs of he ACM SIGCOMM, Sockholm, Swede,. [] Raul Mahaja e al. Corollig High badwidh Aggregaes i he Nework, ACM Comuer Commuicaio Review,. [] Joui Viiikka, Herve Debar, Ludovic Me e al. Processig Irusio Deecio Alers Aggregaes wih Time Series Modelig, Iformaio Fusio, Elsevier, 9. [3] Wei Lu ad Ali A. Ghorbai. Nework Aomaly Deecio Based o Wavele Aalysis, EURASIP Joural o Advaces i Sigal Processig, Hidawi Publishig Cororaio, 9. [4] Georgios Oikoomou ad Jelea Mirkovic, Modelig Huma Behaviour for Defese agais Flash-Crowd Aacks, IEEE Ieraioal Coferece o Commuicaios, 9. [5] A.Broder ad M.Mizemacher, Nework Alicaios of Bloom Filers: A Survey. Iere Mah, Volume, Number 4, 3. [6] Ioais Ioaidis, Aah Grama ad Mikhail Aallah, Adaive Daa Srucures for IP Lookis. I Proceedigs of he IEEE Comuer ad Commuicaios Socieies (Ifocom), 3. [7] Adrei Broder ad Michael Mizemacher, Usig Mulile Hash Fucios o Imrove IP Lookus. I Proceedigs of he IEEE Comuer ad Commuicaios Socieies (Ifocom),. [8] Jaeyeo Jug, Balachader Krishamurhy ad Michael Rabiovich, Flash Crowds ad Deial of Service Aacks: Characerizaio ad Imlicaios for CDNs ad Web Sies. I Proceedigs of he h Ieraioal World Wide Web Coferece (WWW), Hoolulu, Hawaii, USA,. [9] L.vo Ah, Mauel Blu, Nicholas J.Hoer ad Joh Laford, CAPTCHA: Usig Hard AI Problems for Securiy. I Proceedigs of EUROCRYPT,.94-3, 3. [] Srikah Kadula, Dia Kaabi, Mahias Jacob ad Arhur Berger, Boz-4-Sale: Survivig Orgaized DDoS Aacks Tha Mimic Flash Crowds. I Proceedigs of he d Symosium o Neworked Sysems Desig & Imlemeaio (USENIX NSDI), 5. [] Suraamaya Raja, Ram Swamiaha, Musafa Uysal ad Edward Kighly. DDoS-Resilie Schedulig o Couer Alicaio Layer Aacks uder Imerfec Deecio. I Proceedigs of he IEEE Comuer ad Commuicaios Socieies (Ifocom), 6. [] Yi Xie ad Shu-Zheg Yu, A Large-Sclae hidde Semi-Markov Model for Aomay Deecio o User Browsig Behaviours. IEEE/ACM Trasacios o Neworkig, vol.7, 9. [3] Yi Xie ad Shu-Zheg Yu, Moiorig he Alicaio-Layer DDoS Aacks for Poular Websies. IEEE/ACM Trasacios o Neworkig, vol.7, 9. [4] Simo Byers, Aviel D.Rubi ad David Korma, Defedig Agais a Iere-Based Aack o he Physical World. ACM Trasacios o Iere Techology, vol.4, age.39-54, 4. [5] Paul barford, Jeffery Klie, David Ploka ad Amos Ro, A Sigal Aalysis of Nework Traffic Aomalies. I Proceedigs of ACM SIGCOMM Iere Measureme Worksho,. [6] Takeshi Yaagai, Takamasa Isohara ad Iwao Sasase, Deecio of HTTP-GET Flood Aack Based o Aalysis of Page Access Behaviour. I Proceedigs of IEEE Pacific Rim Coferece o Commuicaios Comuers ad Sigal Processig, 7. [7] George Box, Gwilym M.Jekis ad Gregory C.Reisel, Time Series Aalysis: Forecasig ad Corol, Third Ediio, Preice Hall, 99 54