A Multi-Supports-Based Sequential Pattern Mining Algorithm 1

Transcription

1 A Mult-Supports-Based Sequental Pattern Mnng Algorthm Yun Xong, Yang-yong Zhu Department of Computng and Informaton Technology, Fudan Unversty, Shangha, Chna Abstract Sequental pattern mnng s now wdely used n varous areas, such as the analyss of bologcal sequences, Web access patterns, customer purchase patterns and etc. In ths paper, we propose a new defnton for M-sequences. Also we present multple supports: local support, total support, and dstrbuton support for ther related mnng of local sequental patterns, total sequental patterns and exstence sequental patterns. Based on multple supports, a mult-supports-based sequental pattern mnng algorthm s developed whch can be generally appled to fnd such patterns. Keywords: Data mnng, Sequental pattern mnng, Multple supports. Introducton The problem of mnng frequent patterns n a set of data sequences was frst ntroduced n []. Sequental pattern mnng s to fnd all frequent subsequences,.e., the subsequences whose occurrence frequency n the set of sequences s no less than a user-specfed mn_support threshold []. Prevous studes are carred out from two aspects, one s n sngle sequence [2], the other s n multple sequences wth same type [,3,4,5,6,7,8,9,0]. Regardng the latter case, the current mnng algorthms [-0] just focus on the case whether the pattern occurs n each sequence,.e., support ndcates the number of sequences, so we just nvestgate sequental patterns whch occur n enough number of sequences, nstead of consderng the repettve frequency n each sequence. Correspondngly, the total occurrence frequency n all sequences s not counted n current algorthms. Take web access sequences contanng 00 users for example, comparng two access pattern: home pagenews page and news page-economy page. There are 95 users who vst the former and each vsts t only once, whle just 0 users who vst the latter but each of them 00 tmes. Accordng to current algorthms, what we may get s that the former occurs 95 tmes whle the latter occurs 0 tmes for the support means the number of users contanng pattern. However, obvously we can fnd that the latter access frequency (000 tmes) s larger than the former (95 tmes). Ths result s useful for the web desgners to optmze the webste. Addtonally, tradtonal methods don t take nto account relatonshp among multple sequences. Actually, there should be three patterns should be found n term of multple sequences mnng. The frst s the pattern frequently occurs n one specfc sequence, second s the pattern wth enough total occurrence frequency n all sequences, and the fnal s the pattern whch occurs n enough sequences. The former two patterns are not concerned n recent studes. Correspondng to these, we ll gve three supports: local support, total support and dstrbuton support n ths paper, then we propose a mult-supports-based algorthm for fndng above three sequental patterns. Accordng to t we lst a new defnton for sequence. Addtonally, recent studes classfy sequence nto two categores: tme seres [,2] (lke transacton sequence, stock sequence) and spatal sequence [3,4,5,6] (such as bologcal sequences). In fact, tme seres and spatal sequence are both composed of separate temporal or spatal pont and correspondng value, ther sequental pattern mnng process s consstent n essence. We ntroduce a order to dentfy temporal pont and spatal pont to unfy two knds sequences. The defnton and algorthm n ths paper s generally avalable for tme seres and spatal sequences. The work was supported by 863: the Natonal Hgh Technology Development Program of Chna (200AA38)

2 Ths paper s organzed as follows. In secton 2, a defnton for M-Sequence s gven n secton 2., three knds of supports are defned n secton 2.2, and three sequental patterns are descrbes n secton 2.3. A mult-supports-based sequental pattern mnng algorthm s proposed n secton 3. Secton 4 dscusses the generalty of M-Sequences defnton, meanng of multple supports, and concludes our study n secton Multple supports for sequental pattern mnng 2.. Defnton for M-Sequences Example: Gven M bologcal sequences, denoted as D, shown n Fgure. Regardng one subsequence <ATCCA>, followng three cases are avalable to do sequental pattern mnng: (here gven M s 20) <ACTGT ATCCA GTCTA ACTGT AC TGTAC TGTCC ACTGT T> <GGCTA AGTGT ACGGT CGAGA GCAGA AGTCT TGCTA ACCCT TT> Fgure. Example of bologcal sequences D a. If t occurrence n 5th of sequences s frequent enough (eg. 25 tmes). Such subsequence s the pattern frequently occurs n one specfc sequence of D. It can reflect the personal feature on one sequence. b. If t occurrence n every sequence of D s (20, 32,,2,22) tmes respectvely. Then, total occurrence frequency of subsequence s 325 tmes whch satsfes a certan number, such subsequence s the pattern wth enough total occurrence frequency n all sequences. It can reflect features on the whole set. c. There are 2 sequences contanng t n D. t s the pattern whch occurs n enough sequences. When mnng the sequental pattern, we should take nto consderaton of the gap between two elements also. Hereby, we ntroduce a new attrbute order n the sequence defnton to ndcate correspondng temporal pont or spatal pont of sequence elements. For example the pattern home page-news page s descrbed as <(09:05 30, home page), (09:05 45, news page. Defnton 2.(Sngle sequence): Sngle sequence s a set of pars whose form lke (order, value). A sngle sequence S s denoted by <(X, Y ), (X 2, Y 2 ),, (X n,y n, where (X,Y ) s called an element of sequence, X (<=<=n) s order, t represents temporal or spatal nformaton and X <X j (<j), Y (<=<=n) s a value of sequence element, especally, Y can be a value wth multple objects,.e., (Y,Y 2,,Y k ), furthermore, Y k also would be a complex object. Defnton 2.2(M-Sequences): M-Sequences s a set of sequences contanng M sngle sequences wth same types, t s denoted as: <(X,Y ),(X 2,Y 2 ),.,(X n,y n.. <(X,Y ),(X 2,Y 2 ),.,(X n,y n.. <(X m,y m ),(X m2,y m2 ),.,(X mnn,y mnn Where M s a postve nteger, especally, f M=, t s specfed to a sngle sequence, f M>, t represents multple sequences. <=<=m, n can be equal or unequal to n j. X represents the same type order. Y represents the same type value. For brevty, the order X n would be omtted f obvous enough n context. Defnton 2.3(Sequence element gap): Sequence element gap s the gap between each two elements n sequence,.e., the gap between the order x n and the order x nj (x n <x nj ) correspondng to Y and Y j, denoted as gap(x n,x nj ) = x nj - x n (x n <x nj ). Defnton 2.4(Subsequence): A sequence T = <(x, t ), (x 2, t 2 ),, (x m,t m s a subsequence of one sequence S n M-Sequences, S = <(X,Y ), (X 2,Y 2 ),,, (X n,y n, where m<=n, f, and j m,x j < x (j + ) j m, l < l2 <... < ln m, such that t Y l,t 2 Y l2,,t m Y lm. In ths case, S contan T, S s also called super-sequence. If the value t j (<=j<=m) of the element (x j, t j ) n subsequence T s equal to the value Y nr (<=n r <=n ) of the element (X nr, Y nr ) n super-sequence S, then order x j should be equal to order X nr,.e., x j =X nr. Moreover, T would occur a few tmes n the super-sequence S, so x j n T have a few related orders X nr n S. Defnton 2.5(Equal gap subsequence): Equal gap subsequence s such subsequence whose two neghbor elements sequence element gap,.e., gap(x n,x nj ) s equal when ths subsequence occurs n super-sequence every tme. Especally, f gap(x n,x nj ) =, call t as consecutve subsequence. Defnton 2.6(Unequal gap subsequence): If gap(x n,x nj ) s unequal when ths subsequence occurs n super-sequence each tme, call t unequal gap subsequence. Defnton 2.7(Sequental pattern): Sequental pattern s a subsequence whose occurrence frequency satsfes a certan user-specfed value. Defnton 2.8(Sequental pattern mnng): Sequental pattern mnng s to fnd the complete set of sequental patterns n M-sequences Multple supports of M-Sequences Currently, the support of a subsequence T s defned as the number of sequences n the database contanng 2

3 T [-0]..e., f the subsequence T occurs n one sequence of database then support s added, else unchanged. Under ths defnton, we can fnd the patterns that occur n enough sequences. But the method only usng one knd of support s partal, we can t fnd the patterns that occur frequently n each specfc sequence (e.g. one or part users access custom n web access), or patterns wth enough total occurrence frequency n all sequences (e.g. the pattern A-B possesses maxmum total access frequency n web access sequences). Here, ntroduce three supports: local support, total support and dstrbuton support. Gven a M-Sequences D, sequence S D, subsequence T (T S): Defnton 2.9(Local Support): Subsequence occurrence frequency n any one of sequences n D s called local support. Local support of subsequence T s the number contanng T n S, denoted as Localsupp D (T),.e., Localsupp D (T)= {T T S S D}. Note that the length of each sequence s dfferent, sometmes, we descrbe local support as a percentage type more sutable for some cases. Defnton 2.0(Total Support): Subsequence total occurrence frequency n all sequences n D s called total support. Total support of subsequence T s the sum of subsequence occurrence frequency n each sequence, denoted as Totalsupp D (T),.e., Totalsupp D (T) = {T T S S D}. Defnton 2.(Dstrbuton Support): The number of sequences n M-Sequences contanng subsequence s called dstrbuton support. Dstrbuton support of subsequence T s the number of sequences n D contanng T, denoted as Dstrsupp D (T),.e., Dstrsupp D (T) = {S T S S D}. Hereby, we use a vector to record dstrbuton support: (supp,v,v 2,,v,,v m ), where v (<=<=m) ndcates whether subsequence T occurs n the th sequence or not, t has two values: {0,}, supp s the value of k dstrbuton support, and supp = v. = Generally, each support has a user-specfed mn_support threshold. To fnd varous meanng sequental patterns, the support can be gven separately or n combnng form. Partcularly, when M=, M-sequences represents sngle sequence, only local support s needed Sequental pattern mnng n M-Sequences Gven a M-Sequences D, there are three defntons for sequental patterns correspondng to three supports: Defnton 2.2(Local Sequental Pattern): Local sequental pattern s the pattern occurs frequently n each specfc sequence n M-sequences. A postve nteger ς as the local support threshold, a subsequence T s called a local sequental pattern n M-Sequences f Localsupp D (T) >= ς, the subsequence s local frequent. Defnton 2.3(Total Sequental Pattern): Total sequental pattern s the pattern wth enough total occurrence frequency n all sequences of M-Sequences. A postve nteger ς2 as the total support threshold, a subsequence T s called a total sequental pattern n M- Sequences f Totalsupp D (T) >= ς2, the subsequence s total frequent. Defnton 2.4(Exstence Sequental Pattern): Exstence sequental pattern s the pattern occurs n enough sequences of M-Sequences. A postve nteger ς3 as the dstrbuton support threshold, a subsequence T s called a exstence sequental pattern n M- Sequences f Dstrsupp D (T) >= ς3, the subsequence s exstence frequent. Defnton 2.5(Sequental Pattern mnng n M- Sequences): Sequental pattern mnng n M- Sequences s to fnd local sequental pattern, total sequental pattern and exstence sequental pattern. 3. A mult-supports-based sequental mnng algorthm In ths secton, we wll develop a mult-supportsbased sequental pattern mnng algorthm appled n M-Sequences. Ths algorthm wll fnd not only exstence sequental pattern depend on the sngle support used prevously, but also local sequental pattern and total sequental pattern funded on new multple supports. Data structures and symbol explanaton: L.S m.p L.S m.p k L.S.p L.S.p k s len Fgure 2. Stack t len The order of the element The value of subsequence elements D: M-Sequences (see Defnton 2.2); S: one sequence of M-Sequences, S D. F: A set of frequent subsequences whose length are, F={I,I 2,,I k }; s len : A subsequence whose length s len, the head element of ths subsequence s I (I F). It s stored n t len. len: the length of subsequence; gap j,j+ : gap between the j th and (j+) st elements n s len. t len : A stack correspondng to subsequence s len (shown n Fgure 2). The set of 3

4 poston denoted as L. Each value n L.S.p ndcates the order of each super-sequence S element correspondng to subsequence element. The sum of number n set L.S s local support, and the sum of number n set L s total support. It storng poston n stack makes reduce searchng tme complexty, because we take advantage of dsplacement (p + -p ) to omt rrelevant elements n searchng process. T len : A set of stack t len. The general dea of the algorthm: t based on Apror heurstc [7], f s len s not frequent, then we needn t check s (len+), because occurrence frequency of subsequence s (len+) exceedng s len s mpossble. The process of ths algorthm:. Verfy whether s len s frequent by checkng whether ts each support be no less than each correspondng support threshold. 2. If s len s sequental pattern, then jon s len wth F to create s (len+). 3. Check whether s (len+) s sequental pattern. If true, then create a stack for t. All these stacks put nto a set T (len+). 4. After subsequence n each t len s joned wth F and create new stack, the t len can be deleted. The overall algorthm s shown n Fgure 3. Input: M-Sequences D, local, total, dstrbuton support threshold s ς, ς2, ς3; sequence element gap. Output: All local, total, exstence sequental pattern, and correspondng support. Algorthm: Scan D once to fnd k frequent tems, then put them nto a set F. Each tem n F to form a length- subsequence s. Create a stack t for each s and put them nto set T. len = ; T len+ φ ; whle (T len φ ){ a. for each t len T len { do } s len length len subsequence t len.s from t len } for each s F{ do }s len+ s len jon wth s ; 2}L.S φ ; 3}for each S D{do count = 0; for each p t len.l.s{ do comparng s len+ wth S[p,p+gap,2,..,p+gap len-,len ] n S; f match then{count++; L.S {p}} s len+.locsupport = count;//record t f (s len+.locsupport 0), then {s len+.dssupport++; v = ;} else v = 0; s len+.glosupport = s len+.glosupport + s len+.locsupport;} 4}record the value of three supports of s len+,the vector of dstrbuton support. 5)f s len+ satsfes crteron as stated: { create a new stack t len+ for s len+ : t len+.s s len+ ; t len+.l L;} 6) T len+ T len+ {t len+ };} ) delete t len ;} b. len = len + ;} Fgure 3. Mult-Supports-Based Sequental Pattern Mnng Algorthm If M=, M-sequences s a sngle sequence (usng only local support), and executng once n the step 3). 4. Dscusson In our paper, the defnton for M-Sequences unfes sngle sequence and multple sequences, furthermore, ntegrates tme seres and spatal sequences. In practcal applcatons, M-Sequences can be converted to followng types: f M=, then M-Sequences represents sngle sequence; f M>, then t denotes multple sequences; and that, when the order of sequence elements (<order, value>) s used for temporal pont nformaton, M-Sequences ndcates tme seres; when order s spatal pont, M-Sequences sgnfes spatal sequences. Vrtually, M-Sequences defnes a knd of sequence data source. Usng three new knds of supports presented n ths paper are avalable to fnd more sequental patterns than only a sngle support earler. The users can select a support or supports or combnng form at wll to satsfy more complex applcaton requrements. Example3: Gven a 5-Sequences(M=5) (see table ): Table. a customer transacton 5-sequences Customer d Customer sequences <{ 5}{2}{3}{4}{6}{6}{6}{6}> <{}{2}{3}{4}> <{4}{5}> <{}{3}{4}{3 5}{6}{6}{6}{6}> <{}{3}{5}> Usng algorthm AprorAll, only a support gven(mn_support s 3), we get maxmal large sequences: <34>, <35> and <45> []. But t can t show that the purchased volume of tem {6} s maxmum, because t wll not calculate repettve tmes of {6}, t only counts twce for two customers contan t. Usng total support, the patterns wth enough occurrence frequency n all sequences of M-Sequences can be found. And usng local support, we can analyze frequent patterns n one specfc sequence of M- Sequences. Usng local support combned wth dstrbuton support, we can analyze whether a few sequences have smlar features and generalze them. 4

5 Usng dstrbuton support, patterns whch occur n enough sequences of M-Sequences can be found. The mult-supports-based sequental mnng algorthm enables to fnd all new sequental patterns usng user-specfed support or supports. However, t s just a general algorthm at least and should be mproved n our future work. 5. Conclusons In ths paper, we defne the sequence element as a par (order, value) to unfy temporal and spatal sequences. Furthermore, we gve a new defnton for M-sequences ncludng sngle sequence and multple sequences. Specally, three new knds of supports: local support, total support, and dstrbuton support are ntroduced. Usng these knds of supports we can realze comprehensvely three knds of sequental pattern mnng: local sequental pattern (occurs frequently n each specfc sequence of M-Sequences), total sequental pattern (wth total occurrence frequency n all sequence of M-Sequences), and exstence sequental pattern (whch occurs n enough sequences of M-Sequences) accordng to varous applcatons. Meanwhle, we devse a mult-supportsbased sequental pattern mnng algorthm n relatve to sequental pattern mnng. As future work, we are gong to do more data mnng tasks (such as clusterng, outler detecton) for the sequence data source based on multple supports. 6. References [7] Yang, J., Wang, W., Yu, P. S., Han, J. Mnng long sequental patterns n a nosy envronment. In Proceedngs of the 2002 ACM SIGMOD nternatonal conference on Management of data. ACM Press, (2002), pp [8] Han J, Pe J, Yan X-Feng. From sequental pattern mnng to structured pattern mnng: a pattern-growth approach. J.Comput.Sc.&Technol.Vol.9,No.3,May(2004),pp [9] Zak M. SPADE: An effcent algorthm for mnng frequent sequences. Machne Learnng, (200) : pp [0] Ln, M.-Y.,Lee, S.-Y. Fast dscovery of sequental patterns by memory ndexng. In Proc. of 2002 DaWaK. (2002), pp [] Bettn C, Wang X S, Jajoda S. Mnng temporal relatonshps wth multple granulartes n tme sequences. Data Engneerng Bulletn, (998) 2: pp [2] Han J, Dong G, Yn Y. Effcent mnng of partal perodc patterns n tme seres database. In proc. 999 Int. Conf. Data Engneerng(ICDE 99),Sydney, Australa, Aprl (999), pp [3] Brazman A, Approaches to the automatc dscovery of patterns n bosequences. J.Comp.Bol,5,(998), pp [4]A. Calfano, SPLASH: Structural pattern localzaton analyss by sequental hstograms, Bonformatcs, (2000),6( 4): [5]A.Calfano, Advances n sequence analyss, Current Opnon n Structural Bology (200), : [6]K. Wang, Y. Xu, J. Yu, "Scalable Sequental Pattern Mnng for Bologcal Sequences" Conference on Informaton and Knowledge Management CIKM 2004, Washngton, D.C., USA. Nov, (2004). pp [7] Agrawal R., Imelnsk T., Swam A. Mnng assocaton rules between sets of tems n large databases. In Proc. of the ACM SIGMOD Conf. Management of Data(SIGMOD 93), Washngton, D.C., May (993), pp [] Agrawal R, Strkant R. Mnng sequental patterns. In Proc.995 Int. Conf. Data Engneerng(ICDE 95), Tape Tawan, Mar, (995), pp.3-4. [2] Mannla H, Tovonen H, Verkamo A I. Dscovery of frequent epsodes n event sequences. Data Mnng and Knowledge Dscovery, (997), : pp [3] Srkant R, Agrawal R. Mnng sequental patterns: Generalzatons and performance mprovements. In Proc. 5th Int. Conf. Extendng Database Technology (EDBT 96), Avgnon France, Mar. (996), pp [4] Han J, Pe J, Mortazav-Asl B, chen Q, Dayal U, Hsu M C. Freespan: Frequent pattern-projected sequental pattern mnng. In Proc ACM SIGKDD Int. Conf. Knowledge Dscovery n Databases (KDD 00), Boston MA, Aug. (2000), pp [5] Pe J, Han J, Mortazav-Asl B, Pnto H, Chen Q, Dayal U, Hsu M C. Prefxspan: Mnng sequental patterns effcently by prefx-projected growth. In proc. 200 Int. Conf. Data Engneerng(ICDE 0) (200), pp [6] Masakazu Seno, George Karyps, SLPMner: An Algorthm for Fndng Frequent Sequental Patterns Usng Length-Decreasng Support Constrant In Proceedngs of the 2nd IEEE Internatonal Conference on Data Mnng (ICDM), Maebash, Japan, Dec (2002), pp