Forward and backward DAWG matching. Slobodan Petrović

Size: px

Start display at page:

Download "Forward and backward DAWG matching. Slobodan Petrović"

Antonia Robbins
5 years ago
Views:

1 Forward and backward DAWG matching Slobodan Petrović

2 Contents Introduction Forward DAWG matching (FDM) Backward DAWG matching (BDM) 2/29

3 Introduction A DAWG (Directed Acyclic Word Graph) representation of a string is very convenient for application in search algorithms A DAWG is the most efficient representation of a string in the sense that it is the minimal DFA (Deterministic Finite Automaton) that recognizes all the suffixes of the given string 3/29

4 Introduction Example - the DAWG of the string w=aabbab 4/29

5 Introduction In all applications of DAWG, the search algorithms try to find the longest common substring of two given strings - the search string S and the search pattern w Obviously, if the longest common substring of these two strings is the search pattern itself then the search pattern is contained in the search string and a match is found 5/29

6 Forward DAWG matching (FDM) Finds the longest common substring of two given strings by scanning the search string from left to right Forward matching The search string S is processed on-line, i.e. one symbol at a time The search pattern w is represented by means of its DAWG 6/29

7 Forward DAWG matching (FDM) The FDM algorithm finds the longest substring of the search pattern w ending at each position in the search string S The longest suffix of the current prefix of the search string, which is a substring of the search pattern at the same time 7/29

8 Forward DAWG matching (FDM) Let S be the search string and let w be the search pattern Let DAWG(w) be the DAWG of the search pattern, known in advance Suppose that the FDM algorithm has reached the position i in S and that it has processed the prefix p of S (whose length is i) 8/29

9 Forward DAWG matching (FDM) 9/29

10 Forward DAWG matching (FDM) Since FDM finds the longest suffix of the search string, which is a substring of the search pattern At the position i of the string S it knows the longest suffix s of the prefix p of S, which is at the same time a substring of the search pattern w 10/29

11 Forward DAWG matching (FDM) Let the next symbol of S be t The FDM algorithm now has to find the longest suffix s of p+t, which is at the same time a substring of w Here, the + sign denotes concatenation 11/29

12 Forward DAWG matching (FDM) The FDM algorithm at this step checks whether an edge (x,y) labeled with the symbol t exists in DAWG(w) The vertex x is determined by the previously executed iterations of the algorithm (initially root) If such an edge (x,y) is found then x y and the algorithm starts the next iteration It processes the next symbol t 12/29

13 Forward DAWG matching (FDM) The failure case - an edge (x,y) is not found The property of a DAWG Let u be the concatenation of the labels of edges on any path from the root of DAWG(w) to the vertex x Let v be the concatenation of the labels of edges on the longest path from the root of DAWG(w) to the vertex y u and v are substrings of w If suf(x)=y then v is a suffix of u 13/29

14 Forward DAWG matching (FDM) Example (1) The DAWG of the string w=aabbabb Let x=v 7 and y=v 2 There is a suffix link leading from v 7 to v 2 suf(v 7 )=v 2 Let u=bba (one of the paths to x) and v=a (the longest path to y) Then v is a suffix of u 14/29

15 Forward DAWG matching (FDM) Example (2) 15/29

16 Forward DAWG matching (FDM) By using the property of the DAWG, the FDM algorithm follows the chain of suffix links, starting from the failure point, until it finds an edge (x,y) in the DAWG(w) labeled with the symbol t If such an edge is not found after following the whole suffix link chain to the root then the search is reset, i.e. x is set to the root vertex 16/29

17 Forward DAWG matching (FDM) 17/29

18 Forward DAWG matching (FDM) In the FDM algorithm, the function length(x) returns the length of the longest path from root to the vertex x This value can be stored for each vertex of DAWG(w) during its construction 18/29

19 Backward DAWG matching (BDM) Searches for the longest substring u of a pattern w at the current position of the search string S Very fast on average The average time complexity of the BDM algorithm is O(nlog m/m) The worst-case time complexity is O(nm), where n is the length of the search string S and m is the length of the search pattern w 19/29

20 Backward DAWG matching (BDM) The BDM algorithm uses a window of length m, which moves from left to right The movement of the window is not continuous - in some situations, the window can skip some symbols of the search string, which makes the algorithm faster on average The window is scanned from right to left (backward matching) 20/29

21 Backward DAWG matching (BDM) Instead of the search pattern w, in the BDM search algorithm the reverse pattern w R is used The current longest substring u of w found in S is identified with the vertex in DAWG(w R ) corresponding to u R 21/29

22 Backward DAWG matching (BDM) At the current symbol t from S, the BDM algorithm tries to match the substring t+u with a substring of w The same as finding an edge in DAWG(w R ) labeled with t, which starts from the current DAWG vertex If such an edge is not found, a shift occurs, i.e. some symbols are skipped 22/29

23 Backward DAWG matching (BDM) 23/29

24 Backward DAWG matching (BDM) To facilitate the BDM search, we add to each vertex of the DAWG(w R ) information about whether it lies on the suffix chain starting from the sink vertex of the DAWG Such vertices are called terminal vertices All terminal vertices of the DAWG correspond to suffixes of w R 24/29

25 Backward DAWG matching (BDM) Example (1) 25/29

26 Backward DAWG matching (BDM) Example (2) The terminal vertices in the DAWG(aabbabb) are v 10, v 11, v 6, and v 1, since they lie on the suffix chain starting from the sink vertex v 10 of the DAWG 26/29

27 Backward DAWG matching (BDM) The backward search through the current window ends in two ways Either the whole search pattern w is found or The failure to match t+u has occurred In that case, the algorithm shifts the window to the right by last, where last is the position of the ending symbol of the longest substring u of w found in the current window 27/29

28 Backward DAWG matching (BDM) 28/29

29 Backward DAWG matching (BDM) The variable i points at the symbol just before the window, j is the auxiliary variable used to traverse the window backwards, and last records the position of the recognized substring of w in S 29/29

Conversion Masters in IT (MIT) AI as Representation and Search. (Representation and Search Strategies) Lecture 002. Sandro Spina

Conversion Masters in IT (MIT) AI as Representation and Search (Representation and Search Strategies) Lecture 002 Sandro Spina Physical Symbol System Hypothesis Intelligent Activity is achieved through