10/12/2015. SHRDLU: 1969 NLP solved?? : A sea change in AI technologies. SHRDLU: A demonstration proof. 1990: Parsing Research in Crisis

Transcription

1 SHRDLU: 1969 NLP solved?? : A sea change in AI technologies Example: Natural Language Processing The Great Wave off Kanagawa by Hokusai, ~1830 ] Person: PICK UP A BIG RED BLOCK. Computer: OK. (does it) Person: GRASP THE PYRAMID. Computer: I DON'T UNDERSTAND WHICH PYRAMID YOU MEAN. Person: FIND A BLOCK WHICH IS TALLER THAN THE ONE YOU ARE HOLDING AND PUT IT INTO THE BOX. Computer: BY "IT", I ASSUME YOU MEAN THE BLOCK WHICH IS TALLER THAN THE ONE I AM HOLDING. Computer: OK. (does it) Person: WHAT DOES THE BOX CONTAIN? Computer: THE BLUE PYRAMID AND THE BLUE BLOCK. Person: WHAT IS THE PYRAMID SUPPORTED BY? Computer: THE BOX. CIS Intro to AI 2 SHRDLU: A demonstration proof 1990: Parsing Research in Crisis Informal IBM study in 1990: No existing parsers worked for short sentences from news text. Methodology: Compared a range of best broad coverage parsers in U.S. All but best: < 40% correct (hand checked) for 10 word sentences Best: claimed 60% - (I don t believe it ) How could this be true? Most NLP work in interactive systems, with major user adaptation Proposed Solution: A hand-built corpus of examples (10 6 words of WSJ) Some unknown statistical model trained on that set of examples. CIS Intro to AI 3 CIS Intro to AI 4 The Past: Crucial flaws in the paradigm These and other later systems worked well, BUT 1. Person-years of work to port to new applications 2. Very limited coverage of English Crucially, they worked well because of a magical fact: People automatically adapt and limit their language given a small set of exemplars if the underlying linguistic generalizations are HABITABLE An Early Robust Statistical NLP Application A Statistical Model For Etymology (Church 85) Determining etymology is crucial for text-to-speech Italian English AldriGHetti laugh, sigh IannuCCi accept ItaliAno hate This won t handle pre-existing text! CIS Intro to AI 5 CIS Intro to AI 6 1

2 NLP shift starts here 10/12/2015 An Early Robust Statistical NLP Application The Initial Driver: Success of Statistical Models in Speech Recognition Angeletti 100% Italian Iannucci 100% Italian Italiano 100% Italian Lombardino 58% Italian Asahara 100% Japanese Fujimaki 100% Japanese Umeda 96% Japanese Anagnostopoulos 100% Greek Demetriadis 100% Greek Dukakis 99% Russian Annette 75% French Deneuve 54% French Baguenard 54% Middle French A very simple statistical model (your next homework) solved the problem, despite a wild statistical assumption CIS Intro to AI 7 CIS Intro to AI 8 The State of NLP (and all of AI) NLP Past before 1995: Rich Representations NLP Present: Powerful Statistical Disambiguation Uncertainty & Probability Artificial Intelligence AIMA, Chapter 13 Some slides adapted from CMSC 421 (U. Maryland) by Bonnie Dorr CIS Intro to AI 9 Outline A major problem: uncertainty Probability: the right way to handle uncertainty Frequentists vs. Bayesians Review: Fundamentals of probability theory Joint Probability Distributions Conditional Probability Probabilistic Inference Uncertainty and Truth Values: a mismatch Let action A t = leave for airport t minutes before flight. Will A 15 get me there on time? true/false Will A 20 get me there on time? true/false Will A 30 get me there on time? true/false Will A 200 get me there on time? true/false Problems: The world is not Fully observable (road state, other drivers plans, etc.) Deterministic (flat tire, etc.) Single agent (immense complexity modeling and predicting traffic) And often incoming information is wrong: noisy sensors (traffic reports, etc.) CIS Intro to AI 11 CIS Intro to AI 12 2

3 A purely logical approach: Risks falsehood: A 25 will get me there on time Leads to conclusions that are too weak for decision making: A 1440 will get me there on time almost certainly true but I d have to stay overnight at the airport! A 25 will get me there on time if there is no accident on the bridge and it doesn t rain and my tires remain intact, etc. True but useless Also: Logic represents uncertainty by disjunction A or B might mean A is true or B (or both) is true but I don t know which A or B does not say how likely the different conditions are. CIS Intro to AI 13 AI Methods for handling uncertainty Default or nonmonotonic logic: Assume my car does not have a flat tire Assume A 25 works unless contradicted by evidence Issues: What assumptions are reasonable? How to handle contradiction? Rules with ad-hoc fudge factors: A get there on time Sprinkler 0.99 WetGrass WetGrass 0.7 Rain Issues: Problems with combination, e.g., Sprinkler causes Rain?? Probability Model agent's degree of belief Given the available evidence, A 25 will get me there on time with probability 0.04 Probabilities have a clear calculus of combination CIS Intro to AI 14 Our Alternative: Use Probability Given the available evidence, A 25 will get me there on time with probability 0.04 Probabilistic assertions summarize the ignorance in perception and in models Theoretical ignorance: often, we simply have no complete theory of the domain, e.g. medicine Uncertainty (partial observability): Even if we know all the rules, we might be uncertain about a particular patient Probabilities as Degrees of Belief Subjectivist (Bayesian) (Us) Probability is a model of agent s own degree of belief Frequentist (Not us, many statisticians) Probability is inherent in the process Probability is estimated from measurements Laziness: Too much work to list the complete set of antecedents or consequents to ensure no exceptions CIS Intro to AI 15 CIS Intro to AI 16 Frequentists: Probability as expected frequency Bayesians: Probabilities as Degrees of Belief Frequentist (Not us, many statisticians) Probability is inherent in the process Probability is estimated from measurements P(A) = 1: A will always occur. P(A) = 0: A will never occur. 0.5 < P(A) < 1: A will occur more often than not. CIS Intro to AI 17 Subjectivist (Bayesian) (Us) Probability is a model of agent s own degree of belief P(A) = 1: Agent completely believes A is true. P(A) = 0: Agent completely believes A is false. 0.5 < P(A) < 1: Agent believes A is more likely to be true than false. Increasing evidence strengthens belief therefore changes probability estimate CIS Intro to AI 18 3

4 Frequentists vs. Bayesians I Actual dialogue before election Mitch (Bayesian): I think Obama now has an 80% chance of winning. Sue (Frequentist): What does that mean? It s either 100% or 0%, and we ll find out after election day. Why be subjectivist? Often need to make inferences about singular events How likely is it to rain tomorrow? How likely is it that the Republicans will take the Senate in 2016? De Finetti: It s good business If you don t use probabilities (and get them right) you ll lose bets CIS Intro to AI Probability as a bet If Agent 1 expresses a set of degrees of belief that violate the axioms of probability theory then there is a combination of bets by Agent 2 that guarantees that Agent 1 will lose money every time. de Finetti Making decisions under uncertainty Suppose I believe the following: P(A 25 gets me there on time ) = 0.04 P(A 90 gets me there on time ) = 0.70 P(A 120 gets me there on time ) = 0.95 P(A 1440 gets me there on time ) = Which action to choose? It still depends on my preferences for missing flight vs. time spent waiting, etc. If Agent 1 sets Price(Phillies win)+price(red Sox win) Price(Phillies or RS win), then Agent 2 can always make money. (AIMA 13.2) CIS Intro to AI 22 CIS Intro to AI 23 Decision Theory Decision Theory develops methods for making optimal decisions in the presence of uncertainty. Decision Theory = utility theory + probability theory Utility theory is used to represent and infer preferences: Every state has a degree of usefulness An agent is rational if and only if it chooses an action A that yields the highest expected utility (expected usefulness). Let O be the possible outcomes of A, U(o) be the utility of outcome o, and P A (o) be the probability of o as an outcome for action A, then the Expected Utility of A is EU ( A) PA ( o) U ( o) oo A FAST REVIEW OF DISCRETE PROBABILITY CIS Intro to AI 24 CIS Intro to AI 25 4

5 Standard approach: Sample Spaces More AI-ish description Basic assumption 1: the set of all possible outcomes of an experiment is known Example: Rolling two dice Sample space = D x D, where D = {1,2,3,4,5,6} Each member i of a sample space is called an elementary event Basic assumption 1: The set of all possible complete specifications of all possible states of the world (possible worlds) is known even if an agent is uncertain which describes the actual world E.g., if the world consists of only two Boolean variables Cavity and Toothache, then the sample space consists of 4 distinct elementary events: 1 : Cavity = false Toothache = false 2 : Cavity = false Toothache = true 3 : Cavity = true Toothache = false 4 : Cavity = true Toothache = true Elementary events are mutually exclusive and exhaustive CIS Intro to AI 26 CIS Intro to AI 27 Basic assumption 2: Probabilities of Elementary Events Every i is assigned a probability (1) 0 P( i ) 1 P( i ) is the probability the elementary event i will occur Assuming is finite, we require (2) P( ) 1 P is the probability distribution on Events & Event Spaces An event A is a set of elementary events i, i.e A is any subset of Note: The set of all possible events is the power set of the subsets of The probability of an event A is defined as (3) P( A) P( ) A CIS Intro to AI 28 CIS Intro to AI 29 Probabilities of Events From (1), (2) and 3, it follows, for any finite discrete 0 P( i ) 1 P() = 1 = P(true) P() = 0 = P(false) P(a b) = P(a) + P(b) P(a b) P( A )= 1-P(A) a a b b Discrete random variables A random variable can take on one of a set of different values, each with an associated probability. Its value at a particular time is subject to random variation. Discrete random variables take on one of a discrete (often finite) range of values Domain values must be exhaustive and mutually exclusive For us, random variables will have a discrete, countable (usually finite) domain of arbitrary values. Mathematical statistics usually calls these random elements (Usually, but not here, the domain is R) Example: Weather is a discrete random variable with domain {sunny, rain, cloudy, snow}. Example: A Boolean random variable has the domain {true,false}, CIS Intro to AI 30 CIS Intro to AI 31 5

6 A word on notation Assume Weather is a discrete random variable with domain {sunny, rain, cloudy, snow}. Weather = sunny abbreviated sunny P(Weather=sunny)=0.72 abbreviated P(sunny)=0.72 Cavity = true abbreviated cavity Cavity = false abbreviated cavity Vector notation: Fix order of domain elements: <sunny,rain,cloudy,snow> Specify the probability mass function (pmf) by a vector: P(Weather) = <0.72,0.1,0.08,0.1> CIS Intro to AI 32 Factored Representations: Propositions Elementary proposition constructed by assignment of a value to a random variable: e.g. Weather = sunny e.g. Cavity = false (abbreviated as cavity) Complex proposition formed from elementary propositions & standard logical connectives e.g. Weather = sunny Cavity = false CIS Intro to AI 33 Probability Distribution Joint probability distribution a Probability distribution gives values for all possible assignments: Vector notation: Weather is one of <0.72, 0.1, 0.08, 0.1>, where weather is one of <sunny,rain,cloudy,snow>. P(Weather) = <0.72,0.1,0.08,0.1> Sums to 1 over the domain Practical advice: Easy to check Practical advice: Important to check Probability assignment to all combinations of values of random variables (i.e. all atomic events) The sum of the entries in this table has to be 1 Every question about a domain can be answered by the joint distribution toothache toothache cavity cavity !!! Probability of a proposition is the sum of the probabilities of atomic events in which it holds P(cavity) = 0.1 [add elements of cavity row] P(toothache) = 0.05 [add elements of toothache column] CIS Intro to AI 34 CIS Intro to AI 35 Conditional Probability Conditional Probability (continued) toothache toothache cavity cavity P(cavity)=0.1 and P(cavity toothache)=0.04 are A B both prior (unconditional) probabilities Once the agent has new evidence concerning a previously unknown random variable, e.g., toothache, we can specify a posterior (conditional) probability e.g., P(cavity toothache) P(a b) = P(a b)/p(b) [Probability of a with the Universe restricted to b] So P(cavity toothache) = 0.04/0.05 = 0.8 A B U Definition of Conditional Probability: P(a b) = P(a b)/p(b) Product rule gives an alternative formulation: P(a b) = P(a b) P(b) = P(b a) P(a) A general version holds for whole distributions: P(Weather,Cavity) = P(Weather Cavity) P(Cavity) Chain rule is derived by successive application of product rule: P(X 1,,X n ) = P(X 1,...,X n-1 ) P(X n X 1,...,X n-1 ) = P(X 1,...,X n-2 ) P(X n-1 X 1,...,X n-2 ) P(X n X 1,...,X n-1 ) = n = P(Xi X1,..., X i 1 ) i 1 CIS Intro to AI 36 CIS Intro to AI 37 6