Probability & Expectation. Professor Kevin Gold

Transcription

1 Probability & Expectation Professor Kevin Gold

2 Review of Probability so Far (1) Probabilities are numbers in the range [0,1] that describe how certain we should be of events If outcomes are equally likely (die roll, coin flip), we can calculate probabilities by counting outcomes If drawing multiple items: with replacement means we put each item back every time - we encode the total draw as a sequence and count strings without replacement means we set the item aside and can t redraw it - we encode the total draw as a subset and use combinations

3 Review of Probability So Far (2) Learning information ( Bob didn t get an ace ) can change a probability, but not information that doesn t change the possibilities ( there s some card Bob didn t get ) The axioms of probability include Pr(union of all events) = 1 Pr(A v B) = Pr(A) + Pr(B) - Pr(A^B) Events are independent iff Pr(A^B) = Pr(A)Pr(B); this is generally not true of events that have casual relationships or are informative about each other

4 Doing a Monopoly Problem Using Independence In Monopoly, you have 3 chances to get out of jail without paying a fine, by rolling doubles (same result on 2 6-sided dice). What is the chance of at least one doubles in 3 rolls? Pr (at least one doubles on 3 rolls) = 1 - Pr(no doubles on 3 rolls) = 1 - Pr(no doubles on 1 roll) 3 Using Independence = 1 - (1 - Pr(doubles on 1 roll)) 3 = 1 - (1-1/6) 3 = 1 - (5/6) 3 = 1-125/216 = 91/216 = 0.42

5 A Problem Where Counting Works Better Than Using Independence Four two-card hands are dealt around a poker table. What is the probability that two hands are pocket aces (two aces)? This is difficult to break into individual events, as cards dealt to one person affect another and success is possible in multiple ways Total ways to deal 2 cards to each of 4 people (retaining IDs of people): C(52,2)*C(50,2)*C(48,2)*C(46,2) Ways to split the 4 aces into hand1, hand2: C(4,2) Ways to hand out the two ace hands: C(4,2) {_,_} {_,_} {_,_} Ways to choose hand3, hand4: C(48,2)*C(46,2) {_,_} Overall: C(4,2) 2 C(48,2)C(46,2)/(C(52,2)C(50,2)C(48,2)C(46,2)) =

6 An Extended Poker Example We re playing Texas Hold em, a game where each player is dealt two cards at first. Four people at the table. You re dealt a pair - Jack of Hearts, Jack of Spades. What were the chances you would be dealt a pair?

7 An Extended Poker Example We re playing Texas Hold em, a game where each player is dealt two cards at first. Four people at the table. You re dealt a pair - Jack of Hearts, Jack of Spades. What were the chances you would be dealt a pair? 13*C(4,2)/C(52,2) =78/1326 =0.0588

8 An Extended Poker Example Two players fold (quit), leaving just one opponent. Three cards are dealt face up to the center of the table - the flop. They are JD, 10D,10C. Since you can combine your hand with the cards in the center, you have a full house. But if your opponent is holding the other two tens, they beat you with a four-of-a-kind. What are the chances they have the other two tens?

9 An Extended Poker Example Two players fold (quit), leaving just one opponent. Three cards are dealt face up to the center of the table - the flop. They are JD, 10D,10C. Since you can combine your hand with the cards in the center, you have a full house. But if your opponent is holding the other two tens, they beat you with a four-of-a-kind. What are the chances they have the other two tens? 1 possible two-ten hand C(47,2) possible hands (since 5 cards are out) 1/C(47,2) = 1/1081 =0.0009

10 An Extended Poker Example In Texas Hold Em, two more public cards would be revealed next: the turn and the river. What are the chances either of them is the last Jack??

11 An Extended Poker Example In Texas Hold Em, two more public cards would be revealed next: the turn and the river. What are the chances either of them is the last Jack?? Subset with {JS,?} 46 possibilities for? (after the jack) 46/C(47,2) = 0.043

12 An Extended Poker Example We stay in the game, and find the turn card is QD. If the opponent s cards and the river have between them the Ace of Diamonds and the King of Diamonds, your opponent will have an unbeatable royal flush. What are the chances of that??

13 An Extended Poker Example We stay in the game, and find the turn card is QD. If the opponent s cards and the river have between them the Ace of Diamonds and the King of Diamonds, your opponent will have an unbeatable royal flush. What are the chances of that?? One free card in subset, 8 possibilities used; out of 3 card hands from 52-6=46; 44/C(46,3) =

14 An Extended Poker Example The final card comes, and it s your Jack of Spades! But your opponent stays in? Maybe this makes sense from their perspective. From their perspective, what are the chances you have 4-of-a-kind?

15 An Extended Poker Example The final card comes, and it s your Jack of Spades! But your opponent stays in? Maybe this makes sense from their perspective. From their perspective, what are the chances you have 4-of-a-kind? All possible hands: 52-7 = 45 cards (they know their own); C(45,2) 4-of-a-kind: 10 s (1) or Jacks (1) 2/C(45,2) = 2/990 = 0.002

16 Introduction to Conditional Probability We can deal with the effect of knowledge on a probability in a systematic way with conditional probabilities. We can use to mean given in a probability, as in given that we know the following is true. Pr(sum of die1 and die2 is 7 die1 is a 6) If we are counting to determine probabilities, this restricts the possibilities for both E and S in E / S. Events in E now must satisfy both criteria: what we know is true and what counts as success Outcomes where die1 is a 6: 6 {(6,1), (6,2),,(6,6)} Outcomes where sum is 7 and die1 is a 6: 1 {(6,1)} Pr(die1+die2 = 7 die1 = 6) = 1/6

17 A Mathematical Definition of Conditional Probability Pr(A B) = Pr(A ^ B)/Pr(B). S This equation reflects how knowledge changes the space of possibilities. We now only draw from the space where B is true, and a successful event means both A and B are true. A Pr(A) = A / S

18 A Mathematical Definition of Conditional Probability Pr(A B) = Pr(A ^ B)/Pr(B). S This equation reflects how knowledge changes the space of possibilities. We now only draw from the space where B is true, and a successful event means both A and B are true. A B Pr(A B) = A^B / B

19 Conditional Probability Example Someone at the poker table (2 card hands) says ugh, I m never going to get a card higher than 10 Before even looking at your cards, what is the chance they have a pair, given what they said? And did what they said matter to this probability? Pr(pair not higher than 10) = pair and not higher than 10 / not higher than 10 = 9*C(4,2) / C(9*4, 2) = Earlier we calculated Pr(pair) to be , so this did matter! (fewer possible ranks means more likely to have a pair)

20 Conditional Probability And Independence Rearranging terms of the definition of conditional probability, we have Pr(A^B) = Pr(A B)Pr(B). Compare to the definition of independent events, independent iff Pr(A^B) = Pr(A)Pr(B). The definition of independence holds iff Pr(A B) = Pr(A). This means knowing B shouldn t change our opinion of Pr(A). Similar statements are true of Pr(B A) and Pr(B). Pr(A^B) = Pr(A)Pr(B) does not always hold, but Pr(A^B) = Pr(A B)Pr(B) holds by definition. If you want to multiply but the events aren t independent, you need to know the relevant conditional probabilities.

21 An Example of Multiplying Conditional Probabilities A software engineer has heard: Under normal circumstances, software ships with critical bugs 10% of the time. But 5% of teams have super-buggy programmers, who ensure their teams ship critical bugs 50% of the time. What is the overall probability that software ships with critical bugs, taking super-buggy programmers into account? 0.9 Pr(critical bugs) = Pr(bugs ^ s.b.) + Pr(bugs ^ s.b.) = Pr(critical bugs super-buggy)pr(super-buggy) + Pr(critical bugs not super-buggy)pr(not-super-buggy) = 0.5* *0.95 = = 0.12 (or 12%) superbuggy? bugs?

22 Expectation In making decisions involving randomness, we should be able to weight the outcomes based on how good or bad they are Lottery: small chance of jackpot, large chance of losing a dollar Poker: make different decisions depending on the stakes & how expensive it is to stay in the game The expectation or expected value of a random variable is useful for decision-making - it tells you the value you should get if you average over many trials Lottery: Winning or losing money on average with each ticket? How much? Poker: What is the average payoff for each play in this situation?

23 Mathematical Definition of Expectation The expectation of a random variable X is E[X] = Σ x Pr(X= x)*x - that is, the sum of probability*value over all values. When all values are equiprobable, this is just the average of the values. D = outcome of 6-sided die E[D] = (1/6)(1) + (1/6)(2) + (1/6)(3) + (1/6)(4) + (1/6)(5) + (1/6)(6) = (1/6)( ) = 3.5 But if all values aren t equiprobable, then we factor in the probabilities. L = outcome of lottery with 1 in 10 million chance of winning, $2 million payout E[L] = 2,000,000*(1/10,000,000) + 0*(9,999,999/10,000,000) = $0.20 (not counting the price of the ticket) So at $0.19 per ticket, we should want to buy as many tickets as possible, but at $1, we re probably wasting money

24 Expectation in Computer Science Expectation comes up naturally in the analysis of average case performance and randomized algorithms, where the random variable in question is the running time of the algorithm We may also want to know how full some part of a data structure will get in expectation Not just whether there is a collision, as in the birthday paradox, but how many collisions to expect And another application is that an AI may want to calculate the payoffs for different courses of actions - the self-driving car must quickly calculate whether it s better to slam the brakes

25 Decision-Making and Expectation We re playing a card game where J,Q,K are worth 10 points, A is worth 16, and the other 9 values (2-10) are worth 5. I can pay 6 points before anyone sees any cards to be dealt an extra card at the beginning of the game. Is this worth it? Expected value for a single card: 10*(3/13) + 16*(1/13) + 5*(9/13) = 91/13 = 7 points I will be up a point on average if I take the extra card (-6+7). So I should.

26 Linearity of Expectation Expectation is composed entirely of a weighted sum - no squares etc in the computation, making it linear - and that gives it some handy mathematical properties. One of the most useful of these properties is: given random variables X, Y, and Z, if Z = X + Y, then E[Z] = E[X] + E[Y]. This lets us break expectations of complex random variables down into random variables that are simpler to compute. Example: Let Z be the sum of 2 six-sided dice, and let X and Y be the first and second die rolls. E[Z] = E[X] + E[Y] = = 7.

27 E[Sum of 2 Dice] Without Linearity of Expectation Counts of each roll: E[Z] = (1/36)(2) + (2/36)(3) + (3/36)(4) + (4/36)(5) + (5/36) (6) + (6/36)(7) + (5/36)(8) + (4/36)(9) + (3/36)(10) + (2/36)(11) + (1/36)(12) = 7 Enumerating all possible outcomes of multiple events will almost always take more time than using linearity of expectation

28 Linearity of Expectation Doesn t Require Independence What is the expected number of aces that I will draw in 5 cards (out of 52)? Let Z = the total aces drawn. Then let: V = total aces drawn for 1st card W = total aces drawn for 2nd card X = total aces drawn for 3rd card Y = total aces drawn for 4th card Z = total aces drawn for 5th card (each random variable here is 0 or 1, a so-called indicator variable) Z = V + W + X + Y + Z so E[Z] = E[V] + E[W] + E[X] + E[Y] + E[Z] Any one of these variables is just the chance of drawing a single ace in 52 cards. (These aren t conditioned on the preceding variables - they re each summing over all possibilities for the other variables.) E[Z] = 5*((4/52)*1 + (48/52)*0) = 20/52

29 Why Does Linearity of Expectation Work? A Brief Proof E[X + Y] = Σ Y Σ X (x+y)pr(x=x ^ Y=y) = Σ Y Σ X xpr(x=x ^ Y=y) + Σ Y Σ X ypr(x=x ^ Y=y) (distributive property - this is really the main part of the argument) In the second term, we can sum over x any terms where y remains the same: Σ X ypr(x=x ^ Y=y) = ypr(y=y) We can switch the order of summation and regroup in the first term as well: Σ Y xpr(x=x ^ Y=y) = xpr(x=x) These transformations leave Σ X xpr(x=x) + Σ Y ypr(y=y) = E[X] + E[Y].

30 More Examples of Linearity of Expectation E[sum of sided dice] = E[X 1 + X X 100 ] = E[X 1 ] + E[X 2 ] + + E[X 100 ] = = 100(3.5) = 3500 Playing a game where I need to flip over 3 cards, and 1/4 the deck consists of penalty cards. What s the expected number of penalty cards I flip over? Linearity of Expectation: E[Z] = E[W] + E[X] + E[Y] = 1/4 + 1/4 + 1/4 = 3/4 or 0.75

31 Another Slower Way Computation 3 cards drawn, 1 in 4 bad, what s the expected number of bad cards? Total possibilities for 3 draws: (40 choose 3) Zero bad draws: (30 choose 3)/(40 choose 3) = 4060/9880 = 0.41 One bad draw: (10 choose 1)(30 choose 2)/(40 choose 3) = bad draws: (10 choose 2)(30 choose 1)/(40 choose 3) = bad draws: (10 choose 3)(30 choose 0)/(40 choose 3) = 0.01 E[Z] = 0* * * *0.012 = 0.75

32 Shuffling Invitations Example N invitations get handed out to the same N people at random. What is the expected number that go to the correct person? E[total correct] = E[# to 1st person correct] + + E[# to Nth person correct] E[# to particular person correct] = 1*(1/N) + 0*((N-1)/N) = 1/N E[total correct] = N * (1/N) = 1.

33 Linearity of Expectation in Computer Science Algorithms are often sequences of many steps that can each have some associated randomness The total time spent performing an algorithm is always equal to the total time spent on individual steps Linearity of expectation says that we can break analysis of an algorithm down into the analysis of the time it takes to perform individual steps. It often lets us analyze one simple action, then multiply

34 Waiting for Success Suppose we are trying to accomplish something and have chance p of success. What is the expected number of trials until success? Each failure must have had probability (1-p) Call the number of trials until success X E[X] = Σ X p(1-p) X-1 which simplifies to 1/p X=1 E.g., Six rolls of a six-sided die to get a 6

35 Sample Expected Time Analysis: Coupon Collecting Often in deals or contests, you want to Collect all N of N prizes get one at random each time with replacement So, how many total coupons do we expect to collect, including repeats, until we collect em all? Call the number of coupons we collect in going from i to i+1 unique coupons X i, and the total number of coupons until we have N unique coupons, X n-1 By linearity of expectation, E[X] = Σ E[X i ] i=0 McDonald s Monopoly Pieces

36 Sample Expected Time Analysis: Coupon Collecting So what is E[X i ]? For i=0, it s 1 our first coupon is always progress. Could say N/N = 1. Wait 1. For i = 1, the chance of getting a coupon we want is (N-1)/N just need to avoid the one we got so our expected wait time is N/(N-1) For i = 2, similar logic gets to N/(N-2) And so on until i = N-1, 1/N chance of getting last piece, so wait time N N-1 N Σ 1/i is roughly ln N (think of E[X] = Σ E[X i ] = Σ N/(N-i) = N Σ 1/i N ln N i=1 i=0 i=1 the integral of 1/x) N

37 Other Useful Expectation E[cX] = ce[x] for constants c Facts This is still under the general header of linearity of expectation E[XY] = E[X]E[Y] if X and Y are independent For example, if a lottery with expected payoff E[Y] suddenly only happens at all with probability p, we can take X to be a random variable about whether the lottery happens (0 or 1) E[X] = 0*(1-p) + 1*p = p and the new expected payoff is E[X]E[Y] = pe[y]

38 Probability Lecture 2 Summary Conditional probability lets us talk about Pr(some event something we know) We can multiply conditional probabilities when events aren t independent to get Pr(A^B) Expectation tells us the probability-weighted average result for a random variable Useful for determining which action has the best payoff Linearity of expectation can save some effort by breaking complex variables down to simple ones