Combinatorics and probability

Transcription

1 Departmet of Mathematics Ma 3/03 KC Border Itroductio to Probability ad Statistics Witer 208 Lecture 3: Combiatorics ad probability Relevat textboo passages: Pitma [2]: Sectios.5.6, pp. 7 77; Appedix, pp Larse Marx []: Sectios 2., 2.5, 2.6, 2.7, pp Sectios The grad coi-flippig experimet This year (208 there were 25 submissios of 28 flips, for a total of 27,520 tosses! You ca fid the data at Recall that I put predictios ito a sealed evelope. Here are the predictios of the average umber of rus, by legth, compared to the experimetal results. (My predictios oly wet through rus of legth 5. Ru Theoretical Wiggle Total Average How well legth average room rus rus did I do? Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Nailed it Did t expect this. a The formula for the theoretical average is the object of a optioal Exercise. b This is based o a Mote Carlo simulatio of the 95% cofidece iterval for a sample size of 20, ot 25. Yes! There are Laws of Chace. How did we do o Heads versus Tails? Out of 27,520 there were: Number Percet Tails 3, Heads 3, If we combie the last six years of 63,56 tosses, the results are: Number Percet Tails 8, Heads 8, How close to 50/50 is this? We ll discuss this i Lecture 0.6. KC Border v ::2.37

2 KC Border Combiatorics ad probability Laplace s model: Uiform probability o fiite sets Recall (Sectio.2 Laplace s [0, pp. 6 7] model of probability as a fractio whose umber is the umber of favorable cases ad whose deomiator is the umber of all cases possible. We formalize this as follows. The uiform probability o a fiite sample space S maes each outcome equally liely, ad every subset of S is a evet. 3.. Theorem (Uiform probability With a uiform probability P o a fiite set S, the for ay subset E of S, P (E E S. Throughout this course ad i daily life, if you come across the phrase at radom ad the sample space is fiite, uless otherwise specified, you should assume the probability measure is uiform. 3.2 Geerally accepted coutig priciples The Uiform Probability (or coutig model was the earliest ad hece oe of the most pervasive probability models. For that reaso it is importat to lear to cout. This is the reaso that probability ad combiatorics are closely related Lists versus sets I fid it very useful to distiguish lists ad sets. Both are collectios of objects, but two lists are differet uless each object appears i the same positio i both lists. For istace, 23 ad 23 are distict lists of three elemets, but the same set. A list is sometimes referred to as a permutatio ad a set is ofte referred to as combiatio Number of lists of legth If I have distict objects, how may distict ways ca I arrage them ito a list (without repetitio? Thi of the objects beig umbered ad startig out i a bag ad havig to be distributed amog umbered boxes There are choices for box, ad for each such choice, there are for positio 2, etc., so all together there are! ( ( 2 2 distict lists of objects. Some of my colleagues reserve the term uiform to refer to a probability space where the sample space is a iterval of real umbers, ad the probability of a subiterval is proportioal to its legth. They eed to expad their horizos. v ::2.37 KC Border

3 KC Border Combiatorics ad probability 3 3 The umber! is read as factorial. By defiitio, ad we have the followig recursio 0!,! (! ( > 0. By covetio, if < 0, the! Number of lists of legth of objects How may distict lists of legth ca I mae with objects? As before, there are choices of the first positio o the lists, ad the choices for the secod positio, etc., dow to ( + choices for the th positio o the list. Thus there are ( ( + }{{} terms distict lists of items chose from items. There is a more compact way to write this. Observe that ( ( + ( ( + ( ( 2 ( ( 2! (! There are! (! distict lists of legth chose from objects. We may write this as (, read order. Note that whe this reduces to! (sice 0!, which agrees with the result i the previous sectio. Whe 0 this reduces to, ad there is exactly oe list of 0 objects, amely, the empty list Number of subsets of size of objects How may distict subsets of size ca I mae with objects? (A subset is sometimes referred! to as a combiatio of elemets. Well there are (! distict lists of legth chose from objects. But whe I have a set of objects, I ca write it! differet ways as a list. Thus each set appears! times i my listig of lists. So I have to tae the umber above ad divide it by! to get the umber of sets. There are! (!! distict subsets of size chose from objects. KC Border v ::2.37

4 KC Border Combiatorics ad probability Defiitio For atural umbers 0 (! (!!, is read as choose For > defie ( 0. It is the umber of distict subsets of size chose from a set with elemets. It is also ow as the biomial coefficiet. Note if >, there are o subsets of size of a set of size, so by covetio we agree that i this case ( 0. Other otatios you may ecouter iclude C(,, C, ad C. (These otatios are easier to typeset i lies of text Some useful idetities ( ( 0 ( ( ( ( ( ( ( Here is a simple proof of (: ( + + is the umber of subsets of size + of a set A with + elemets. So fix some elemet ā A ad put B A \ {ā}. If E is a subset of A of size +, the either (i E B, or else (ii E cosists of ā ad elemets of B. There are ( + subsets E satisfyig (i, ad ( subsets satisfyig (ii. Equatio ( gives rise to Pascal s Triagle, which gives ( as the th etry of the th row (where the umberig starts with 0 ad 0. Each umber is the sum of the two above it: ( 0 0 ( ( 0 ( 2 ( 2 ( ( 3 ( 3 ( 3 ( ( ( ( ( ( 5 ( 5 ( 5 ( 5 ( ( 6 ( 6 ( 6 ( 6 ( 6 ( etc. ( ( 5 5 ( 6 6 Equatio ( also implies (by the telescopig method that ( ( ( ( + + ( etc. ( ( v ::2.37 KC Border.

5 KC Border Combiatorics ad probability Number of all subsets of a set You should already ow the followig, There are 2 distict subsets of a set of objects. The set of subsets of a set is ow as its power set. Let c( deote the cardiality of the power set of a set with elemets. The it is easy to see that c(0, c( 2. More geerally,c( + 2c(: There are two ids of subsets of {x,..., x + }, those that are subsets of {x,..., x } ad those of the form A {x + } where A is a subset of {x,..., x }. So c( Ad so If we sum the umber of sets of size from 0 to, we get the total umber of subsets, so 0 ( 2. This is a special case of the followig result, which you may remember from high school. (The special case is a b Biomial Theorem (a + b 0 ( a b 3.3 A taxoomy of classic experimets May radom experimets ca be reduced to oe of a small umber of classic experimets. This characterizatio is ispired by Ash []. The first id of radom experimet is samplig from a ur. (See Figure 3. Imagie a ur filled with balls of differet colors, or labeled balls. A ball is selected at radom (meaig each ball is equally probable. Note that coi tossig is reducible to samplig from a ur with a equal umber of balls labeled Heads ad Tails. Rollig a die ca be viewed as samplig from a ur with balls labeled,..., 6. Dealig a card is lie samplig from a ur with balls labeled A, A,..., K. For repeated samplig, there are two variatios, samplig with replacemet ad samplig without replacemet. I samplig with replacemet, after beig draw, the ball is replaced i the ur, ad the cotets are remixed. Repeated coi tossig is lie samplig with replacemet, but dealig a a poer or bridge game is lie samplig without replacemet. Aother id of radom experimet is a matchig experimet. I this id of experimet, a radomly selected ball is dropped ito a radomly selected labeled bi. The obvious example of this is roulette. Aother classic textboo questio deals with radomly stuffig letters ito addressed evelopes. KC Border v ::2.37

6 KC Border Combiatorics ad probability 3 6 Figure 3.. The archetypal ur. As aother istace, radioactive decay experimets ca be viewed as havig bis correspodig time itervals, ito which balls symbolizig decays are placed. (This is oe way to thi about Poisso processes, to be discussed i Lecture 3. Aother id of experimet is waitig for somethig special to happe i aother experimet. For istace, we might wat to ow how log it will tae to go broe playig roulette. Or how log betwee radioactive decays. We ca also categorize some of the calculatios we wat to do i coectio with these experimets as to whether order matters or order does t matter. Ofte this comes dow to whether we are iterested i lists or sets. But whatever the experimet or type of result we are iterested i, remember Laplace s maxim. To calculate the probability of the evet E, whe the experimetal outcomes are all equally liely, simply cout the umber of outcomes that belog to E ad divide by the total umber of outcomes i the outcome space S Remar I samplig balls from urs ote that all probabilities derived from Laplace s maxim are ratioal umbers. I thi you ca mae the case that i reality, all measuremets are ratioal How may differet outcomes are there for the experimet of tossig a coi times? Biomial probabilities What is the probability of gettig heads i idepedet tosses of a fair coi? Let s do this carefully. The sample space S is the set of sequeces of legth where each term s i i the sequece is H or T. For each poit s S, let A s {i : s i H}. Sice there are oly two outcomes, if you ow A s, you ow s ad vice versa. Now let E be ay subset of S that has exactly elemets. There is exactly oe poit s S such that A s E. Thus the v ::2.37 KC Border

7 KC Border Combiatorics ad probability 3 7 umber of elemets of S such that A s is precisely the same as the umber of subsets of S of size, amely (. Thus ( Prob(exactly Heads {s S : A s } S Here is a example with 3: s A s HHH {, 2, 3} HHT {, 2} HT H {, 3} HT T {} T HH {2, 3} T HT {2} T T H {3} T T T 2!!(!2. For 2, the set of poits s S with exactly two heads is the set {HHT, HT H, T HH}, which has 3 ( 3 2 elemets, ad probability 3/8. We ca use Pascal s Triagle to write dow these probabilities (Prob of 0 Heads i 0 tosses 2 (Prob of 0, Heads i toss etc. 3 8 (Prob of 0,, 2 Heads i 2 tosses 8 etc How may ways ca a stadard dec of 52 cards be arraged? Here the order matters, so we wat the umber of lists, which is 52! or more precisely: 80,658,75,70,93,878,57,660,636,856,03,766,975,289,505,0,883,277,82,000,000,000,000. This is a astroomically large umber. I fact, sice the uiverse is about 0 20 billio years old ad there are about 7.28 billio people (accordig to Siri, if every perso o the plaet were set to wor arragig a dec i a give order, ad could do so i oe secod, it would tae about lifetimes (to date of the uiverse to go through all the possible arragemets of the dec. For this course we mae the usual assumptio that after shufflig the dec a few times all possible arragemets are equally liely. This is ludicrous. But the assumptio may actually give reasoably good results for typical questios we as about card games, such as those that follow. For more about the distributio of cards after shufflig, see the papers by Bayer ad Diacois [3] ad Assaf, Diacois, ad Soudararaja [2]. A rule of thumb is that it taes at least seve riffle shuffles for the dec to sufficietly mixed up to be able to use the model that all orders of the cards are equally liely provided what you wat to do is to calculate the probabilities of evets typically associated with card games. KC Border v ::2.37

8 KC Border Combiatorics ad probability 3 8 It is importat that whe shufflig the dec the dec, the shuffles have some oise i them. A perfect shuffle is oe where the dec is split perfectly i half, ad the cards from each half are perfectly alterately iterleaved. There are actually two ids of perfect shuffles oe i which the top of the dec remais the top, ad o i which the top card becomes the secod card. The problem with perfect shuffles is that the order of cards is ow. I fact after eight perfect shuffles fixig the top card, the dec is i the same order as it started. If you ca perform perfect shuffles ad have a decet memory (ad I have met such people, the you ca amaze your frieds ad family by aoucig what the sixteeth card i the dec is How may differet five-card draw poer hads are there? I stadard five-card draw poer, each player is dealt five cards before ay bettig occurs. The order i which you receive your cards does ot matter for how you will bet, oly the set of cards i your had. I various forms of stud poer, there is bettig before you receive all their cards, so the order i which you receive cards may ifluece your bets. There are possible five-card hads. ( , 598, How may differet deals? How may distict deals of five-card draw poer hads for seve players are there? (The order of hads matters to the bettig, but the order of cards withi hads does ot. The umber of distict deals is ( ( ( ( ( ( ( }{{} 7 terms Each succeedig had has five fewer cards to choose from, the others beig used by the earlier hads How may five-card poer hads are flushes? To get a flush all five cards must be of the same suit. There are thirtee ras i each suit, so there ( 3 5 distict flushes from a give suit. There are four suits, so there ( 3 58 possible flushes. 5 (This icludes straight flushes. A straight flush is a flush i which the five cards ca be arraged i order. Coutig the Ace as a high card, a straight flush may start o ay of the ie umbers 2, 3,..., 0, so there are 9 36 possible straight flushes. (I iclude royal flushes as straight flushes. A royal flush is a straight flush with a 0, Jac, Quee, Kig, ad Ace. Some players allow a Ace to be either high or low for the purposes of a straight, which adds aother straight flushes for a total of 0. Thus there are flushes that are ot straight flushes (allowig for low Aces. So what is the probability of a flush that is ot a straight flush? 508 2, 598, v ::2.37 KC Border

9 KC Border Combiatorics ad probability Probability of of a id What is the umber of (five-card poer hads that have for of a id (four cards of the same ra? Well, there are 3 choices for the ra of the four-of-a-id, ad 8 choices for the fifth card, so there 3 8 distict hads with four of a id. There are ( 52 5 poer hads, so the probability of four of a id is 3 8 ( Probability of a full house A full house is a poer had with three cards of oe ra ad two cards of a differet ra. How may poer hads are a full houses? Well, there are 3 choices for the ra of the three-ofa-id, ad 2 choices for the ras of the pair, but there are cards of a give ras so there are ( ( 3 sets of three of a id of a give ra. Lie wise there are 2 pairs of a give ras. So there, so there (3 ( ( 3 (2 2 distict full houses. The probability of a full house is 3 ( ( ( Probability of a three-of-a-id had A three-of-a-id poer had is a had with three cards of oe ra r, ad two cards of two differet ras, r 2 ad r 2, where R, R 2, ad r 3 are distict. How may poer hads are a three-of-a-id? Here are two ways to approach the problem. 2. There are 3 ras to choose from, ad for each ra r there are ( 3 ways to choose the three-of-a-id. For each of these 3 52 choices, there are 9 cards left, from which we must choose the remaiig two cards. There are twelve remaiig ras ad we must choose two distict ras there are ( 2 2 ways to do this. Give the choices for the ras, there are choices for each ra, so there are ways to choose the remaiig two cards. There are thus ( 2 2 ( 3 ( 3 distict three-of-a-id hads, ad the probability is (( 2 5, , 92 ( , 5 2. Aother way to reaso about the problem is this. As before, there are 3 ras to choose from, ad for each ra r there are ( 3 ways to choose the three-of-a-id. For each of these 3 52 choices, there are 9 cards left, from which we must choose the remaiig two cards. There are ( ( 9 2 ways to do this. But ot all of 9 2 these lead to three-of-a-id. If oe of these two has the same ra r as our first triple, we ed up with four-of-a id, which is a stroger had. How may ways ca this happe? Well there is oly oe card of ra r left i our remaiig 9 ad 8 ot of ra r, so there are 8 ways to choose the remaiig two to get four of a id. Also, if the two remaiig cards have the same ra, the we get a full house. There are 2 remaiig ras ad for each oe there are ( 2 ways to choose two cards of that KC Border v ::2.37

10 KC Border Combiatorics ad probability 3 0 ra, ad thus ed up with a full house. So the umber of of three-of-a-id hads is 52 ( 9 2 }{{} remaiig pairs which agrees with the aswer above. 8 }{{} four-of-a-id ( 2 5, 92, 2 }{{} full house Deals i bridge I Cotract Bridge, all fifty-two cards are dealt out to four players, so each player has thirtee. The first player ca have ay oe of ( ( 52 3 hads, so the secod may have ay of 39 3 hads, the third may have ay of ( ( 26 3 hads, ad the last player is stuc with the 3 had left over. Thus there are ( ( ( ( distict deals i bridge. After the deal there is a roud of biddig, which results i oe player becomig declarer. The players are divided ito teams of two, ad arraged aroud a four-sided table with sides labeled North, East, South, ad West. The declarer sits at the South positio, ad their parter 2 sits at the North positio. North s card are displayed for all the players to see, but the other players are the oly oes to see their ow hads. South will maes all the plays for North, so North is ow as the emphdummy. This gives the declarer ad advatage because the declarer sees their cards plus the dummy s cards, so the declare ows which cards their team holds, ad by elimiatio ows which 26 cards the oppoets have, but ot how they split up. By cotrast, West or East does ot ow the cards held by their team Splits i bridge Suppose the declarer s oppoets have Clubs betwee them. What is the probability that they are split ( betwee West ad East? This is the probability that West (the player o declarer s left has of the. East will have the remaiig. There are ( , 00, 600 possible hads for West. I order for West s had to have Clubs, they must have oe of the ( subsets of size from the Clubs. The remaiig 3 must be made up from the 26 o-clubs. There are ( 26 3 possibilities. Thus there are ( ( 26 3 hads i which West has clubs, so the probability is ( ( 26 3 that West has clubs. For the case 3 this is /00 for 0, 3, ad 39/00 for, 2. ( Some pedats will claim that the use of they or their as a ugedered sigular proou is a grammatical error. There is a covicig argumet that those pedats are wrog. See, for istace, Huddlesto ad Pullum [9, pp ]. Moreover there is a great eed for a ugedered sigular proou, so I will use they i that role. v ::2.37 KC Border

11 KC Border Combiatorics ad probability Aside: Some practical advice o gamblig Kowig the probabilities for card or dice games is useful, but probably ot the most useful thig to ow about gamblig. For istace, you should ow that it is ever a good idea to carry large amouts of cash ito a bac-alley room full of stragers. Eve if they are ot just goig to rob you at gupoit, they may try to cheat you, so you have to be very silled at detectig card maipulatio. (I have a fried who oce tured up the top card of a dec of cards ad dealt several hads while leavig it i place. Eve though I ew he was ot dealig from the top, I could t see aythig amiss. Eve if o oe is maipulatig the cards, the other players may have sigals to share iformatio ad coordiate bets. Eve if the stragers do t try to cheat you, if you wi too much, they may assume that you are cheatig them, ad upleasatess may esue. (This is true eve i reputable Las Vegas casios. You are probably better off hustlig pool, but that is ot perfectly safe either. 3. Beroulli Trials A Beroulli trial is a radom experimet with two possible outcomes, traditioally labeled Pitma [2]: success ad failure. The probability of success is traditioally deoted p. The probability p. 27 of failure ( p is ofte deoted q. A Beroulli radom variable is simply the idicator of success i a Beroulli trial. That is, { if the trial is a success X 0 if the trial is a failure. 3.5 The Biomial Distributio If there are stochastically idepedet Beroulli trials with the same probability p of success, the probability distributio of the umber of successes is a radom variable, ad its distributio is called the Biomial distributio. To get exactly successes, there must be failures, Pitma [2]: but the order of the successes ad failures does ot matter for the cout. There are 2. (!! (! such outcomes, ad by idepedece each has probability p ( p. (Recall that 0!. For coi tossig, p ( p /2, but i geeral p eed ot be /2. The couts ( are weighted by their probabilities p ( p. Thus ( P ( successes i idepedet Beroulli trials p ( p. Aother way to write this is i terms of the biomial radom variable X that couts success i trials: ( P (X p ( p. Note that the Biomial radom variable is simply the sum of the Beroulli radom variables for each trial. Compare this to the aalysis i Subsectio 3.3.2, ad ote that it agrees because /2 (/2 (/2. Sice p + ( p ad, the Biomial Theorem assures us that the biomial distributio is a probability distributio Example (The probability of heads i 2 coi flips For a fair coi the probability of heads i 2 coi flips is ( ( KC Border v ::2.37

12 KC Border Combiatorics ad probability ( p0.2 p0.5 p Figure 3.2. Biomial Probabilities We ca see what happes to this for large by usig Stirlig s approximatio: Propositio (Stirlig s approximatio where ε 0 as.! e 2π e ε For a proof of Stirlig s approximatio, see, e.g., Robbis [3], Feller [7, p. 52] or [5, 6] or Ash [, pp.3 5], or Diacois ad Freedma [], or the exercises i Pitma [2, p. 36]. Actually Robbis [3] is a little more explicit. He shows that /(2 + < ε < /2. 3 Thus we may write where δ 0 as. (2!!! e 2 (2 2 π e e 2π 2π eε2 2ε 22 π e δ, 3 Robbis s strategy is this: Start by taig logarithms l! l( +. Now l( + is the area of a rectagle of height l( + ad base. Sice l x is a very slow growig fuctio the area of such a rectagle + is approximately l x dx. By carefully eepig trac of the discrepacies, i just uder two pages, oe ca get a good approximatio for l! i terms of l x dx l + l(/e + plus some other small stuff, ad expoetiatig gives Stirlig s result. There are other approaches. Flajolet ad Sedgewic [8] offer five differet proofs. I tha Jim Tao for this referece. v ::2.37 KC Border

13 KC Border Combiatorics ad probability 3 3 So the probability of heads i 2 attempts is 2 2 π 2 2 e δ π e δ 0 as. What about the probability of betwee ad + heads i 2 tosses? Well the probability of gettig j heads i 2 tosses is ( 2 j (/2 2, ad this is maximized at j (See, e.g., Pitma [2, p. 86]. So we ca use this as a upper boud. Thus for P (betwee ad + heads < 2 + π e δ 0 as. So ay reasoable law of averages will have to let grow with. We will come to this i a few more lectures. 3.6 The Multiomial Distributio Larse The multiomial distributio geeralizes the biomial distributio to radom experimets Marx []: with m > 2 types of outcomes. A summary of the outcome of a repeated radom experimet Sectio 0.2, where each experimet has oly two types of outcomes is sometimes represeted by a radom pp variable X, which couts the umber of successes. With more tha two type of outcome, the similar summary would be a radom vector X, where the i th compoet X i of X couts the umber of occurreces of outcome type i. With m possible outcome types where the i th type has probability p i, the i idepedet trials, if + + m, Pitma [2]: P ( i outcomes of type i, i,..., m!! 2! m! p p2 2 pm m Remar If you fid the above claim puzzlig, this may help. Recall that i Subsectio we looed at the umber of sets of size ad showed that there was a oe-to-oe correspodece betwee sets of size ad poits i the sample space with exactly heads. The same sort of reasoig shows that there is a oe-to-oe correspodece betwee partitios of the set of trials, {,..., }, ito m sets E,..., E m with E i i for each i ad the set of poits s i the sample space where there are i outcomes of type i for each i,..., m. Each such sample poit has probability p p2 pm m. How may are there? Well there are ( sets of trials of size. But ow we have to chose a set of size 2 from the remaiig trials, so there are ( ( 2 ways to do this for each of the choices we made earlier. Now we have to choose a set of 3 trials from the remaiig 2 trials, etc. The total umber of possible partitios of the set of trials is thus ( ( ( ( 2 2 m. Expadig this gives 2 3!!(! (! 2!( 2! ( 2! 3!( 2 3! ( 2 m!. m! ( 2 m m! }{{} 0! KC Border v ::2.37 m p. 55

14 KC Border Combiatorics ad probability 3 Now observe that the secod term i each deomiator cacels the umerator i the ext fractio, ad (recallig that 0! we are left with!! 2! m! poits s S, each of which has probability p p2 2 pm m. We ca use radom vectors to describe what is happeig. For each type i,..., m, let X i deote the umber of outcomes of type i. The the radom vector X (X,..., X m has a distributio give by P (X (,..., m!! 2! m! p p2 2 pm m Example Suppose you roll 9 dice. What is the probability of gettig 3 aces (oes ad 6 boxcars (sixes? ( 9 9! 9 3! 0! 0! 0! 0! 6! 6 0, 077, (Recall that 0!. 3.7 Samplig with ad without replacemet Suppose you have a ur holdig N balls, of which B are blac ad the remaiig W N B are white. If the ur is sufficietly well chured, the probability of drawig a blac ball is simply B/N. Now thi of drawig a sample of size N from this uderlyig populatio, ad as what the probability distributio of the compositio of the sample is Samplig without replacemet Samplig without replacemet meas that a ball is draw from the ur ad set aside. The ext time a ball is draw from the ur, the compositio of the balls has chaged, so the probabilities have chaged as well. For b, what is the probability that exactly b of the balls are blac, ad w b? are white? Let s dispose of some obvious cases. I order to have b blac ad w white balls, we must have b mi{b, } ad w mi{w, }. There are ( ( B b sets of size b of blac balls ad W w sets of size w of white balls. Thus there are ( ( B W b w possible ways to get exactly b blac balls ad w white balls i a sample of size w + b, out of ( N possible samples of size. Thus P (b blac & w white ( B ( W b w ( N ( B ( W b w ( B+W. b+w Note that if b > B or w > W, by covetio ( ( B b W w 0 (there are o subsets of size b of a set of size B < b, so this formula wors eve i this case. These probabilities are ow as the hypergeometric distributio. v ::2.37 KC Border

15 KC Border Combiatorics ad probability Samplig with replacemet Samplig with replacemet meas that after a ball is draw from the ur, it is retured, ad the balls are mixed well eough so that each is equally liely. Thus repeated draws are idepedet ad the probabilities are the same for each draw. What is the probability that sample cosists of b blac ad w white balls? This is just the biomial probability P (b blac & w white ( b ( b ( w B W. N N Comparig the two samplig methods Ituitio here ca be cofusig, sice without replacemet every blac ball draw reduces the pool of blac balls maig it less liely to get aother blac ball relative samplig with replacemet, but every white ball draw maes more liely to get a blac ball. O balace you might thi that samplig without replacemet favors a sample more lie the uderlyig populatio. To compare the probabilities of samplig without replacemet to those with replacemet, we ca rewrite the hypergeometric probabilities to mae them loo more lie the biomial probabilities as follows. P (exactly b balls out of are blac ( B ( W b w ( N B! W! b!(b b! w!(w w! N!!(N! or i terms of the order otatio (Subsectio we have P (b blac & w white ( (Bb (W w b (N b terms B! W! (B b! (W w! N! (N! w terms! b!w!, ( {}}{{}}{ B (B (B b + W (W (W w + b N (N (N + }{{} terms for samplig without replacemet versus P (b blac & w white ( b ( b ( w B W N N ( b { b terms }}{ B... B w terms {}}{ W... W. N N }{{} b + w terms for samplig with replacemet. The ratio of the probability without replacemet to the probability with replacemet ca be writte as B B B B B b + B W W W W W w + N W N N N N N +. If b 0, the terms ivolvig B do ot appear, ad similarly for w 0. Whether this ratio is greater or less tha oe is ot obvious. But if we icrease N eepig B/N (ad hece W/N costat, the holdig the sample size fixed, each term i this ratio coverges to for each b. Therefore the ratio coverges to oe. That is, the differece betwee samplig with ad without replacemet holdig the sample size costat becomes isigificat as N gets large, holdig B/N ad W/N fixed. KC Border v ::2.37

16 KC Border Combiatorics ad probability 3 6 But how big is big eough? The oly time samplig with replacemet maes a differece is whe the same ball is chose more tha oce. The probability that all balls are distict is N N N N + N N, so the complemetary probability (of a duplicate is 0( (/N. Now us the Taylor series approximatio that l( x x, to get that log P (duplicate 0 /N ( + /2N. (The probability is less tha oe, so its logarithm is egative. So as Pitma [2, p. 25] asserts, if N, this probability is very small. With moder software, you ca see for yourself how the two samplig methods compare. See Table 3. for a modest example of results calculated by Mathematica. Without With b Replacemet Replacemet Ratio Probability of b blac balls i a sample of size 0 for N 00, B 0, W 90. Without With b Replacemet Replacemet Ratio Probability of b blac balls i a sample of size N 0, 000, B 000, W Table 3.. Samplig without replacemet vs. samplig with replacemet Exchageability This sectio eeds wor. I samplig, a ey feature of the sequeces of outcomes is that they are exchageable. This meas that if I am samplig ad my outcomes are s,..., s i that order, the probability of that order is the same as the sample i ay other order s π(, s π(2,..., s π(, where π is a permutatio of {,..., }. That is why we get expressios p ( p i the biomial distributio. Exchageability is a weaer otio tha idepedece, but applies i may situatios. v ::2.37 KC Border

17 KC Border Combiatorics ad probability 3 7 For istace, cosider samplig without replacemet. The outcomes are ot idepedet. For example, if we are drawig balls without replacemet from a ur with oe white ball ad oe blac ball, the probability that the the first ball is white is /2. The probability that the secod ball is white is the same as the probability that the first ball is blac, so it is also /2. But the probability that both draws are white is zero, ot /2 /2 /. The evets are exchageable, but ot idepedet. Was the example above a flue because there were oly two balls ad both colors are equally liely? No cosider the case with four white balls ad two blac oes. Now draw three balls without replacemet. What is the probability of drawig two white balls ad oe blac ball? There are three such sequeces: Sequece Probability WWB WBW BWW We used this implicitly i figurig out the hypergeometric ad multiomial distributios. A cosequece is that as log as we are ot drawig more tha balls, the probability that the th ball is white is the same as the probability that the first oe is white. 3.8 Matchig There are cosecutively umbered balls ad cosecutively umbered bis. The balls are arraged i the bis (oe ball per bi at radom (all arragemets are equally liely. What is the probability that at least oe ball matches its bi? (See Exercise 28 o page 35 of Pitma [2]. Ituitio is ot a lot of help here for uderstadig what happes for large. Whe is large, there is oly a small chace that ay give ball matches, but there are a lot of them, so oe could imagie that the probability could coverge to zero, or to oe, or perhaps somethig i betwee. Let A i deote the evet that Ball i is placed i Bi i. We wat to compute the probability of A i. This loos lie it might be a job for the Iclusio Exclusio Priciple, sice these i evets are ot disjoit. Recall that it asserts that ( P A i p(a i i i i<j P (A i A j +. i<j< P (A i A j A + (. i <i 2< <i P (A i A i2 A i + ( + P (A A 2... A. Cosider the itersectio A i A i2 A i, where i < i 2 < < i. I order for this evet to occur, ball i j must be i bi i j for j,...,. This leaves balls urestricted, so there KC Border v ::2.37

18 KC Border Combiatorics ad probability 3 8 are (! arragemets i this evet. Ad there are! total arragemets. Thus P (A i A i2 A i (!.! Note that this depeds oly o. Now there are ( size- sets of balls. Thus the term i the formula above satisfies ( (! P (A i A i2 A i.! i <i 2< <i Therefore the Iclusio Exclusio Priciple reduces to ( ( (! P A i ( + i! Here are the values for,..., 0: : Prob(match : 2: : : : : : : : : ( +!. Notice that the results coverge fairly rapidly, but to what? The aswer is ( +!, which you may recogize as (/e. (See the supplemetary otes o series. Pitma [2]: p. 23 Larse Marx []: Waitig: The Negative Biomial Distributio The Negative Biomial Distributio is the probability distributio of the umber of idepedet trials eed for a give umber of heads. What is the probability that the r th success occurs o trial t, for t r? For this to happe, there must be t r failures ad r successes i the first t trials, with a success o trial t. By idepedece, this happes with the biomial probability for r successes o t trials times the probability p of success o trial t: NB(t; r, p ( t r p r ( p t (r p ( t r p r ( p t r (t r. Of course, the probability is 0 for t < r. The special case r (umber of trials to the first success is called the Geometric Distributio: ( t NB(t;, p p 0 ( p t p p( p t (t. 0 v ::2.37 KC Border

19 KC Border Combiatorics ad probability 3 9 Warig: The defiitio of the egative biomial distributio here is the same as the oe i Pitma [2, p. 23] ad Larse Marx [, p. 262]. Both Mathematica ad R use a differet defiitio. They defie it to be the distributio of the umber of failures that occurs before the r th success. That is, Mathematica s PDF[NegativeBiomialDistributio[r, p], t] is our NB(t + r; r, p. Mathematica ad R s defiitio assigs positive probability to 0, ours does ot. Bibliography [] R. B. Ash Basic probability theory. Mieola, New Yor: Dover. Reprit of the 970 editio published by Joh Wiley ad Sos. [2] S. Assaf, P. Diacois, ad K. Soudararaja. 20. A rule of thumb for riffle shufflig. Aals of Applied Probability 2(3: [3] D. Bayer ad P. Diacois Trailig the dovetail shuffle to its lair. Aals of Applied Probability 2(2: [] P. Diacois ad D. Freedma A elemetary proof of Stirlig s formula. America Mathematical Mothly 93(2: [5] W. Feller A direct proof of Stirlig s formula. America Mathematical Mothly 7(0: [6] Correctio to A direct proof of Stirlig s formula. America Mathematical Mothly 75(5:58. [7] A itroductio to probability theory ad its applicatios, 3d. ed., volume. New Yor: Wiley. [8] P. Flajolet ad R. Sedgewic Aalytic combiatorics. Cambridge: Cambridge Uiversity Press. [9] R. Huddlesto ad G. K. Pullum A studet s itroductio to Eglish grammar. Cambridge: Cambridge Uiversity Press. [0] P. S. Laplace A philosophical essay o probability. New Yor: Dover Publicatios. The Dover editio, first published i 995, is a ualtered ad uabridged republicatio of the wor origially published by Joh Wiley ad Sos i 902, ad previously reprited by Dover i 952. The Eglish traslatio by F. W. Truscott ad F. L. Emory, is from the the sixth Frech editio of the wor titled Essai philosophique sur les probabilités, published by Gauthier Villars (Paris as part of the 5-volume series of Laplace s collected wors. The origial Frech editio was published i 8 (Paris. [] R. J. Larse ad M. L. Marx A itroductio to mathematical statistics ad its applicatios, fifth ed. Bosto: Pretice Hall. [2] J. Pitma Probability. Spriger Texts i Statistics. New Yor, Berli, ad Heidelberg: Spriger. [3] H. Robbis A remar o Stirlig s formula. America Mathematical Mothly 62(: KC Border v ::2.37

20