Statistical Parsing and CKY Algorithm

Transcription

1 tatistical Parsing and CKY Algorithm Instructor: Wei Xu Ohio tate University Many slides from Ray Mooney and Michael Collins

2 TA Office Hours for HW#2 Dreese 390: - 03/28 Tue 10:00AM-12:00 noon - 03/30 Thu 10:00AM-12:00 noon - 04/04 Tue 10:00AM-12:00 noon Wuwei Lan Readings: - textbook - slide #28,29.30: _slides_spring17/15_more_memm.pdf

3 yntax yntactic Parsing

4 Parsing Given a string of terminals and a CFG, determine if the string can be generated by the CFG: - also return a parse tree for the string - also return all possible parse trees for the string Must search space of derivations for one that derives the given string. - Top-Down Parsing - Bottom-Up Parsing

5 imple CFG for ATI English Grammar Aux Pronoun Proper- Det Nominal Nominal Nominal Nominal Nominal Nominal PP Verb Verb PP PP Prep Lexicon Det the a that this book flight meal money Verb book include prefer Pronoun I he she me Proper- Houston NWA Aux does Prep from to on near through

6 Parsing Example book that flight Verb book Det Nominal that flight

7 Top Down Parsing tart searching space of derivations for the start symbol. Pronoun

8 Top Down Parsing Pronoun X book

9 Top Down Parsing Proper

10 Top Down Parsing Proper X book

11 Top Down Parsing Det Nominal

12 Top Down Parsing Det X book Nominal

13 Top Down Parsing Aux

14 Top Down Parsing Aux X book

15 Top Down Parsing

16 Top Down Parsing Verb

17 Top Down Parsing Verb book

18 Top Down Parsing Verb book X that

19 Top Down Parsing Verb

20 Top Down Parsing Verb book

21 Top Down Parsing Verb book Pronoun

22 Top Down Parsing Verb book Pronoun X that

23 Top Down Parsing Verb book Proper

24 Top Down Parsing Verb book Proper X that

25 Top Down Parsing Verb book Det Nominal

26 Top Down Parsing Verb book Det Nominal that

27 Top Down Parsing Verb book Det Nominal that

28 Top Down Parsing Verb book Det Nominal that flight

29 Bottom Up Parsing tart searching space of reverse derivations from the terminal symbols in the string. book that flight

30 Bottom Up Parsing book that flight

31 Bottom Up Parsing Nominal book that flight

32 Bottom Up Parsing Nominal Nominal book that flight

33 Bottom Up Parsing Nominal Nominal X book that flight

34 Bottom Up Parsing Nominal Nominal PP book that flight

35 Bottom Up Parsing Nominal Nominal PP Det book that flight

36 Bottom Up Parsing Nominal Nominal PP Det Nominal book that flight

37 Bottom Up Parsing Nominal Nominal PP book Det that Nominal flight

38 Bottom Up Parsing Nominal Nominal PP book Det that Nominal flight

39 Bottom Up Parsing Nominal Nominal PP book Det that Nominal flight

40 Bottom Up Parsing Nominal Nominal PP Det Nominal X book that flight

41 Bottom Up Parsing Nominal Nominal PP X book Det that Nominal flight

42 Bottom Up Parsing Verb book Det that Nominal flight

43 Bottom Up Parsing Verb book Det that Nominal flight

44 Bottom Up Parsing Verb book Det that Nominal flight

45 Bottom Up Parsing X Verb book Det that Nominal flight

46 Bottom Up Parsing PP Verb book Det that Nominal flight

47 Bottom Up Parsing PP X Verb book Det that Nominal flight

48 Bottom Up Parsing Verb Det Nominal book that flight

49 Bottom Up Parsing Verb book Det that Nominal flight

50 Bottom Up Parsing Verb book Det that Nominal flight

51 Top Down vs. Bottom Up Top down never explores options that will not lead to a full parse, but can explore many options that never connect to the actual sentence. Bottom up never explores options that do not connect to the actual sentence but can explore options that can never lead to a full parse. Relative amounts of wasted search depend on how much the grammar branches in each direction.

52 yntax CYK Algorithm

53 Dynamic Programming Parsing CKY (Cocke-Kasami-Younger) algorithm based on bottom-up parsing and requires first normalizing the grammar. First grammar must be converted to Chomsky normal form (CNF) in which productions must have either exactly 2 nonterminal symbols on the RH or 1 terminal symbol (lexicon rules). Parse bottom-up storing phrases formed from all substrings in a triangular table (chart).

54 Dynamic Programming a general algorithm design technique for solving problems defined by recurrences with overlapping subproblems first invented by Richard Bellman in 1950s programming here means planning or finding an optimal program, as also seen in the term linear programming Main idea: setup a recurrence of smaller subproblems solve subproblems once and record solutions in a table (avoid any recalculation)

55

56 ATI English Grammar Conversion Original Grammar Aux Pronoun Proper- Det Nominal Nominal Nominal Nominal Nominal Nominal PP Verb Verb PP PP Prep Chomsky Normal Form X1 X1 Aux book include prefer Verb PP I he she me Houston NWA Det Nominal Nominal book flight meal money Nominal Nominal Nominal Nominal PP book include prefer Verb PP PP Prep Note that, although not shown here, original grammar contain all the lexical entires.

57 Exercise

58 CKY Parser i= Book the flight through Houston j= Cell[i,j] contains all constituents (non-terminals) covering words i +1 through j 3 4

59 CKY Parser i= Book the flight through Houston j= Cell[i,j] contains all constituents (non-terminals) covering words i +1 through j 3 4

60 CKY Parser Book the flight through Houston,, Verb, Nominal, None Det Nominal,

61 CKY Parser Book the flight through Houston,, Verb, Nominal, None Det Nominal,

62 CKY Parser Book the flight through Houston,, Verb, Nominal, None Det Nominal,

63 CKY Parser Book the flight through Houston,, Verb, Nominal, None Det Nominal,

64 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Prep

65 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Prep PP Proper

66 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Nominal Prep PP Proper

67 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Nominal Prep PP Proper

68 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Nominal Prep PP Proper

69 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Nominal Prep PP Proper

70 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Nominal Prep PP Proper

71 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Det None Nominal, None Nominal Prep PP Proper

72 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Parse Tree #1 Det None Nominal, None Nominal Prep PP Proper

73 CKY Parser Book the flight through Houston,, Verb, Nominal, None None Parse Tree #2 Det None Nominal, None Nominal Prep PP Proper

74 he announced a program safety in PP trucks and vans he announced a program safety in PP trucks and vans he announced a progra m sa fety in PP trucks and va ns h e announced a program safety in PP trucks and vans he announced a program safe ty in PP trucks and va ns he announced a program safe ty in PP trucks and va ns The Problem with Parsing: Ambiguity IUT: he announced a program to promote safety in trucks and vans POIBLE OUTPUT: + to promote to promote to promote to promote to promote to promote And there are more...

75 yntax Probabilistic Context Free Grammars (PCFG)

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93 split point

94

95 O(n 2 ) for l, i choices O(n 3 N 3 )

96

97 Parsing with Lexicalized CFGs (saw) (man) (saw) DT(the) NN(man) the man (saw) PP(with) Vt(saw) (dog) IN(with) (telescope) saw DT(the) NN(dog) with DT(the) NN(telesc the dog the telescop p(t) = q((saw)! 2 (man) (saw)) q((man)! 2 DT(the) NN(man)) q((saw)! 1 (saw) PP(with)) q((saw)! 1 Vt(saw) (dog)) q(pp(with)! 1 IN(with) (telescope))...

98 yntax Dependency Parsing

99

100

101

102

103

104

105

106

107