18.310 lecture otes September 2, 2013 Sequetial Choice Lecturer: Michel Goemas 1 A game Cosider the followig game. I have 100 blak cards. I write dow 100 differet umbers o the cards; I ca choose ay umbers I wat, ad you do t get to see what umbers I choose. So e.g., I might choose 1, 2, 10000000, π, 10000000, 10 1010, 0.01,... The deck of cards is the shuffled well, so that the cards are ow i uiformly radom order. The deck is placed before you face dow, ad the top card is tured over to show a umber (maybe it s 1000000). You have two choices: you ca say stop, ad choose this card, or skip it. If you skip it, the ext card is tured over, so you ca see the umber, ad agai you may choose to stop ad choose the card, or go oto the ext. The game eds oce you pick a card; you wi if the card you pick was the largest of all the cards. Note that the game would be very easy if you kew what umbers I wrote dow; you d just keep skippig util you see the card with the largest umber. But sice you kow othig about the umbers, it looks pretty hard... It costs $10 to play; if you wi, I ll give you $P. How big should P be before you re willig to play? Surprisigly, you should play eve if P 37! There is a strategy that wis with probability at least 1/e.367.... Let s first see a simpler strategy ad argumet that wis with probability at least 1/4. Here s the strategy: for the first 50 cards, always skip, o matter what the umbers are. After that, stop wheever you see a card that is largest of all the cards you ve see so far. Claim 1. If the largest card is i the 2d half of the deck, ad the secod largest card is i the first half of the deck, the you wi. Proof. After the first 50 cards are overtured, we have see the secod largest card. Thus we will ot stop util we see the largest card (which is i the secod half of the deck, ad so has t bee skipped). Thus Pr(wi) Pr(largest card i 2d half) Pr(2d largest card i 1st half largest card i 2d half) 50 50 100 99 > 1/4. Olie-1
Note that our aalysis was ot completely tight; it s possible to wi eve if the 2d largest card is ot i the first half. For example, if the third largest card is i the first half, ad the largest card appears i the 2d half but before the secod largest card. Now lets try to use the same idea, but optimize thigs. Let be the umber of cards (previously 100, but o reaso to restrict ourselves). Our strategy will be similar, but we will skip over the first t cards (previously, t 50), ad from the o pick a card if it s the largest we ve see so far. Our goal will be to aalyze this strategy precisely for a give k, ad the determie the best choice of k. Our primary iterest will be for large. Theorem 1. The probability of wiig with this strategy is as, Proof. t 1 Pr(wi) ; k 1 k t+1 t Pr(wi) l(t/). Let s split the evet that we wi based o the positio of the largest card: Pr(wi) Pr(pick k th card, ad k th card is the largest) kt+1 Pr(pick k th card k th card is the largest) Pr(k th card is the largest). k t+1 Now Pr(k th card is the largest) 1/, for ay k. O the other had, for k > t, the probability that we pick the k th card, coditioed o it beig the largest, is simply the probability Pr(pick k th card k th card is the largest) T ODO The probability that the largest card rak 1 choice beig the j th oe we look at is 1/. Give that it is the j th possibility we look at, the probability that we successfully choose it is simply k/(j 1), sice we will choose it oly if the best possibility amog the first j 1 is i the first k. Summig o j, we get the success probability, give that we set our threshold to be k, is 1 k k 1 1 P k j 1 j jk+1 Now, we ca use the fact that 1 j1 j l to approximate this sum jk P k k l(k/). Olie-2
2 Itroductio There are times i life whe you will be faced with the problem of choosig amog a large umber alteratives, but you will be forced to decide which choice to make before seeig may or most of them. A typical situatio is whe you are lookig for a apartmet to ret i a market i which really good apartmets at reasoable prices are hard to fid. The, you typically examie potetial places oe by oe. Ad if the oe you curretly see is just right, uless you decide right away to take it, somebody else will, ad you will o loger have this choice available. This problem has traditioally bee called the secretary problem, ad the story attached to it was of a executive iterviewig secretaries. We give a mathematical model of this kid of situatio by imagiig that there are N alteratives, which at the begiig are etirely ukow to you; you view them oe at a time, ad must accept or reject each oe immediately, without seeig ay more. I the first case, we cosider the goal of fidig the very best amog these alteratives. We could also seek to choose oe of the best two, or to fid a strategy to miimize the expected rak of your choice amog the N alteratives (where best is rak 1, ext best rak 2, ad so o,) or make a choice that raks amog the top Q percet of the possible choices. We are goig to assume that you examie the choices i a etirely radom order, so that you have o reaso to believe that the usee choices are better or worse tha those you have already see. Thus, if you have see ad passed k cadidates, we assume that the ext oe to come alog has a equal chace, or probability 1/(k + 1), or lyig i oe of the k + 1 itervals defied by the k choices you have already see. I practical situatios, this assumptio ca be false, so you should be cautious i applyig the aalysis below. For example, a classical strategy o the part of some real estate agets (or asty trick, from the buyer s poit of view) is to first show the buyer five or six really bad houses, ad the show them a reasoable house that (possibly) oe of her frieds has listed. The buyers get a urealistic sese of the quality of the houses o the market, ad are likely to be fooled ad pick the last (ad still average quality) house. 3 Seekig the very best out of possibilities I situatios i which we must make a choice like this, may of us ted to use oe of two quite poor strategies. Either we loath the process of choosig so muech that we take the first possible alterative, or we procrastiate at choosig util there is oly oe possibility left. With either of these strategies, our chace of gettig the best out of the choices we could have see is 1/. We ca do much better. How? Obviously, to do better we must examie the first possibility, ad perhaps the ext few, merely to lear about available choices. At some poit, or threshold, we must decide to accept the ext choice that is the best so far. If our goal is to accept oly the best alterative, we must certaily reject ay that has worst rak tha ay we have see before. We ca aalyze this problem based o the assumptio that the choices are radomly ordered by merit, as follows. Suppose our threshold is the kth possibility, which meas that we defiitely reject the first k choices we see ad pick the ext oe that is the best so far. The we defiitely lose if the rak 1 choice is amog the first k see. Suppose that the rak 1 choice is amog the last k we look at. The, we will reject it if ad oly if the best oe we saw before we looked at the Olie-3
rak 1 choice is amog the first k. If it is, the we would have passed o it, ad sice everythig we saw betwee it ad the rak 1 choice is iferior to the ext best, we would have waited for our rak oe choice. O the other had, if the best choice before the rak 1 choice is after our threshold k, we certaily would have take it, ad ever reached the rak 1 choice. We ca write dow a formula for our chace of success, give that our threshold is k. The probability of our rak 1 choice beig the j th oe we look at is 1/. Give that it is the j th possibility we look at, the probability that we successfully choose it is simply k/(j 1), sice we will choose it oly if the best possibility amog the first j 1 is i the first k. Summig o j, we get the success probability, give that we set our threshold to be k, is P k jk+1 1 k k 1 1 j 1 j Now, we ca use the fact that 1 j1 l to approximate this sum j k P k l(k/). If we differetiate this expressio, we fid that the maximum occurs whe l k/ 1, or k /e. The value of the probability is the k/ 1 e, so if these approximatios are reasoable, our probability of gettig the very best choice is aroud 1/e, which is roughly.37. The results are: with 5, the best k value is 2, which meas you let the first two cadidates go by ad pick the ext better oe; k icreases to 3 whe 8, to 4 whe 11, to 5 whe 13, to 6 whe 16, ad so o. The best value for k is usually the oe earest to /e. The probability of successfully choosig the best cadidate is bigger tha 1/e for small values of. Thus, for 8, choosig k 3 gives a probability.4098 of success. 4 Calculatig probabilities for choosig the very best o a spreadsheet Now, let s look at how we might efficietly calculate the optimum value of the threshold o a spreadsheet; the poit after which we might possibly choose a cadidate. If we are tryig to choose the very best cadidate, the the formula i the previous sectio is fairly efficiet to calculate i a straightforward maer: for each of the possible values of the threshold, we have a ice formula for the probability of success. I fact, we do t eve eed to check all possible thresholds; just the oes ear /e. However, suppose we were tryig to maximize the probability of choosig oe of the best 10 cadidates. The optimum strategy would the look like this: choose 10 thresholds k 1, k 2,..., k 10, ad if you are past the i th threshold, choose the curret cadidate if it is oe of the best i you have see so far. Now, there are roughly choose10 possible values for the 10 thresholds, ad this would make calculatios quite iefficiet if you had to test eve a small fractio of these possible values. There is a clever way to get aroud this. This way works fairly well for ay umber of thresholds, ad also works for quite a few variatios o this problem. What I will explai i these otes is how to do it for the problem where your goal is to choosig the very best possibility, ad I will assig jk Olie-4
some variatio o this for homework. The idea is to work backwords from the last choice, ad at every poit compute the probability of success, give that you have t chose a cadidate before this step. We will let P k be the probability of success, give that we have let the first k cadidates go by. We will use the fact that this probability P k is idepedet of the cadidates we have see so far. If we reach the last possibility without havig chose earlier, we must choose the last, ad the probability that we wi is 1. Thus, P 1 1. Now, let s calculate P 2. suppose that we reach the secod to last possible choice. If this choice is the best oe we ve see so far, we should choose it. The probability that the ext to last is the best oe we ve see so far is 1 1, ad the probability that we wi if we choose it is the probability that the last choice is ot the optimal oe, which is 1. Now, if it s ot best so far (with probability 2 1, we let it go by, ad our chace of wiig is P 1 1. We have to let it go by, ad if we do, our probability of wiig is P 1 1. Thus, we get P 2 Pr(ext to last is best so far) Pr(ext to last is optimal best so far) + Pr(ext to last is ot best so far) P 1 1 1 + 2 1 1 1 2 3 ( 1) Now, suppose that we kow P k, ad wat to calculate P k past the threshold. The we have 1. First, let s assume that we are P k 1 Pr(kth is best so far) Pr(kth is optimal best so far) + Pr(kth is ot best so far) P k. We kow the probability that the kth guy is the best so far is just 1, ad the probability that it s k ot best so far is k 1, so the oly probability we have to calculate to make this recursio work is k the probability that the kth guy is optimal, give that it s the best so far. I thik the easiest way to to thik about this is to realize that the two evets E A ad E B are idepedet, where E A is the evet that the the kth cadidate is the best of the first k cadidates, ad E B is the evet that the optimal cadidate is amog the first k. The, it s easy to check that the probability that we wat to calculate is Pr(E B E A ). Sice these two evets are idepedet, this probability is just Pr(E B ) k. I ve left out a few details here, but you should have leared eough about probability earlier i this course to fill them i fairly easily. Now, we ca compute the probability of wiig, give we choose k as the threshold recursively, as follows P k 1 Pr(kth is best so far) Pr(kth is optimal best so far) + Pr(kth is ot best so far) P k 1 k k + 1 P k. k k It s fairly easy to check that this gives the same formula for P k we saw earlier. But why is this more efficiet? Oe reaso is that this lets us compute all the P k s at oce, ad the we merely eed take Olie-5
the maximum of them. I fact, it becomes very easy to fid the maximum because, startig with k, we ca show that P k icreases at first, ad the decreases, obtaiig a maximum aroud k /e. However, there s aother thig we ca do with this formula which geeralizes more easily to other problems. Let s chage the formula slightly. Let Q k be the probability of wiig with the optimal strategy, give that we have already let the first k choices go by. The we have, by the same reasoig that gave us the formula for P k, Q k 1 Pr(kth is best so far) Pr(optimal strategy kth is best so far) + Pr(kth is ot best so far) Q k But what is the optimal strategy if the kth item is best so far? It is either to choose the kth item or pass o it, depedig o which gives you a larger probability of success. The probability of success if we choose the kth item is just k, ad the probability of success if we pass o it is Q k. Thus, we have the formula ( ) 1 k k Q k 1 max, Q k + 1 Qk. k k Now, how do we geeralize this to the problem where we wat to fid oe of the top L items. We ca write dow a similar formula for Q k 1. For each k, ad for i goig from 1 to L, we have to calculate the probability of success if we choose the k th item, give that the k th item is the i th best so far. The, if the k th item is the i th best so far, we choose it oly if this gives us a higher probability of wiig tha Q k. 5 Seekig oe of the top two (or top L) choices. The problem of fidig the best strategy whe you lower your stadards ad cosider yourself successful if you choose o of the top two choices or oe of the top L choices ca be addressed just as the previous problem, but we have to cosider the possibility of havig more tha oe threshold. Suppose we seek the best or secod best. After the first threshold we accept the best cadidate so far, ad after the secod threshold we take either the best or secod best that comes alog. Whe we seek to get oe of the top L, we similarly wat to cosider L differet thresholds. The calculatios are somewhat more complicated, ad we leave the details for you. The results are: for L 2 ad small values of, we ca achieve a probability of success above 0.6. As icreases, our chace of success dwidles dow to aroud 0.57. 6 Seekig the Best Expected Rak This problem ca also be hadled by the same geeral approach. If there are possible choices all together, we ca compute a threshold for each stage, where we accept a cadidate i some stage if its rak is less tha tha or equal to the threshold for that stage. Call this threshold T (s), where s is the umber of cadidates still usee. Whe s 1, there is oly oe cadidate left after the oe you are curretly examiig. We will be stuck with the rak of the fial cadidate if we pass o the curret choice. This last choice has expected rak ( + 1)/2. Suppose the cadidate we are lookig at at this poit has rak r out of the 1 we have see so far. We ca deduce that it therefore has expected rak r( + 1)/ Olie-6
amog all cdidates, so that we should accept it if r( + 1)/ is better tha ( + 1)/2. we get this by oticig that with probability r/ the last cadidate raises our rak by 1 above its preset value of r. Let us assume that goodess meas low rak, so that rak 1 is best. The our expected rak, before the ( 1)st cgiuce us ( + 1)/4 if we choose the 1st, ad ( + 1)/2 if we do t do so. We have a roughly 50-50 chace of doig either, so our threshold for the ( 2)d choice is roughly 3( + 1)/8. A similar computatio ca be made to deduce the expected rak before seeig the ( 2)d if we have passed over the previous choices. This determies a threshold for our ( 3)rd choice, ad we ca cotiue back to the begiig. Though this souces somewhat complicated, it is t, ad it is very easy for a machie to determie the expected rak at the begiig for ay reasoable. The result is quite surprisig. It turs out that the expected rak at the begiig is always less tha 4, idepedet of. The best strategy appears to be roughly as follows. We let the first quarter of the choices go by. The we take oly the best util the ext quarter of what is left, i.e., choosig the best i the ext (1/4)(3/4) cadidates, ad so o. (Do t cout o this. Figure it out for yourself.) So if you wat to get a expected rak better tha the 4th best, you ca do so eve if there are a billio cadidates. Of course, you have to examie almost a billio of them to do this, which is probably ridiculously impractical. Olie-7
MIT OpeCourseWare http://ocw.mit.edu 18.310 Priciples of Discrete Applied Mathematics Fall 2013 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.