Statistics Intermediate Probability

Session 6 oscardavid.barrerarodriguez@sciencespo.fr April 3, 2018

and Sampling from a Population Outline 1 The Monty Hall Paradox Some Concepts: Event Algebra Axioms and Things About that are True Counting Rules: Permutations and Combination 2 and Sampling from a Population

and Sampling from a Population The Monty Hall Paradox This is a good example of people failing to take information into account when estimating probabilities.

and Sampling from a Population The Monty Hall Paradox Remember the game show "Let s Make a Deal" The contestant is presented with three doors, Door 1, Door 2, Door 3. Behind one door was a fabulous prize and behind the other two doors were gag prizes (goats). Monty Hall tells the contestant to pick one doors. Then MH opens one of the two remaining doors revealing a goat. "you can decide to stay with your choice or switch to the other door" Are you more likely to win by staying with your original choice or are you more likely to win by switching to the other unopened door?

and Sampling from a Population The Monty Hall Paradox Are you more likely to win by staying with your original choice or are you more likely to win by switching to the other unopened door? Usual approach: it doesn t matter whether you stay or switch, because the probability of winning by staying is 0.5 and the probability of winning by switching is 0.5. Unusual approach: The actual probability of winning by staying is 1/3 and the probability of winning by switching is 2/3!! We ll go over the proof of this later.

and Sampling from a Population The Monty Hall Paradox Unusual approach: The actual probability of winning by staying is 1/3 and the probability of winning by switching is 2/3!! We ll go over the proof of this later. for now think about this: the reason people assume the probability of winning by staying and by switching are equal is because people neglect the fact that Monty Hall knows where the prize is!

and Sampling from a Population Some Concepts: Event Algebra As a student of the behavioral sciences you should love probability, You will use them a LOT!! Before getting into more technical aspects of calculating probabilities we need to go through a few basic ideas of where probability comes from.

and Sampling from a Population Some Concepts: Event Algebra Some very important concepts 1. simple random experiment Procedure or operation performed where the outcome is unknown ex-ante 2. Sample Space The collection of all possible outcomes of a simple random experiment Notation: S

and Sampling from a Population Some Concepts: Event Algebra Simple random experiment: Procedure or operation performed where the outcome is unknown ex-ante Sample Space: The collection of all possible outcomes of a simple random experiment Some examples

and Sampling from a Population Some Concepts: Event Algebra 1 Simple random experiment: Procedure or operation performed where the outcome is unknown ex-ante 2 Sample Space: The collection of all possible outcomes of a simple random experiment 3 Event: A unique outcome in the sample space of a simple random experiment 4. Event algebra : Is a method of expressing relationships among events and among combinations of events. The more important relationships between events are Union Intersection Complement Mutually exclusive included

and Sampling from a Population Some Concepts: Event Algebra 4. Event algebra : Union Union: (A B) all outcomes that are associated with event A OR event B OR both Example Say A = {2, 4, 6, 8, 10, 12} and A = {4, 8, 12, 15} (A B) = {2, 4, 6, 8, 10, 12, 15}

and Sampling from a Population Some Concepts: Event Algebra 4. Event algebra : Intersection Intersection: (A B) all outcomes that are associated with event A AND event B. Example Say A = {2, 4, 6, 8, 10, 12} and A = {4, 8, 12, 15} (A B) = {4, 8, 12, }

and Sampling from a Population Some Concepts: Event Algebra 4. Event algebra : Complement Complement: (A c ) all the event that consists of all outcomes that are NOT associated with its own outcomes. A c = {1, 3, 5, 7, 9, 11, 13, 14, 15} Example

and Sampling from a Population Some Concepts: Event Algebra 4. Event algebra : Mutually exclusive Mutually exclusive events: (A B = Φ) Where Φ is impossible. Example Events A and B are not mutually exclusive, because the intersection of events A and B is possible.

and Sampling from a Population Some Concepts: Event Algebra 4. Event algebra : Included Included events: (A B) if each and every outcome in event A is also in event B

and Sampling from a Population Axioms and Things About that are True The probability of the outcomes in a simple random experiment is a function P that assigns a real number, P(E)=a, to each event E in a sample space S, and satisfies the following three axioms: P(E) 0 : probability of an event must be greater than or equal to zero P(S) = 1 : probability of observing something in the sample space is 1 P(A B) = P(A) + P(B) probability of observing at least one of two mutually exclusive events is the sum of their respective probabilities. By using algebra we can derive some other properties:

and Sampling from a Population Axioms and Things About that are True P(E) 0 P(S) = 1 P(A B) = P(A) + P(B) By using algebra we can derive some other properties: For any event, P(A c ) = 1 P(A) For any event, P(A) 1 For all events A and B, P(A B) = P(A) + P(B) P(A B), whether A and B are mutually exclusive or not For all events A and B, P(A) = P(A B) + P(A B c ), whether A and B are mutually exclusive or not.

and Sampling from a Population Axioms and Things About that are True To calculate the probability of an event (A), identify the number of possible outcomes favoring event A and divide that by the number of possible outcomes where event A can occur in the sample space (S): P(A) = A S For example, we want to calculate the probability of selecting a Jack of Clubs from a deck of cards.

and Sampling from a Population Axioms and Things About that are True P(A) = A S For example, we want to calculate the probability of selecting a Jack of Clubs from a deck of cards. P(A) = 1 52 = 0.019 standard deck of 52 cards there is only one jack of clubs, so A = 1 and the number of outcomes where event A can occur is S = 52.

and Sampling from a Population Axioms and Things About that are True P(A) = A S What is the probability of selecting any king?

and Sampling from a Population Axioms and Things About that are True P(A) = A S What is the probability of selecting any king? P(A) = 4 52 = 0.077 There are four possible kings in a deck of cards king of clubs, king of spades, king of diamonds, king of hearts.

and Sampling from a Population Counting Rules: Permutations and Combination 3 Prisoners and 5 Hats There were three prisoners in certain jail, one of whom had normal sight, one of whom had only one eye, and the third of whom was completely blind. All three out and had them stand in a line. He then showed them five hats, three white and two red, which he hid in a bag. From these five hats, he selected three and put one on each of the prisoners heads. None of the men could see what color hat he himself wore. The jailor offered freedom to the sighted man, if he could tell him what color his hat was. (and to prevent guessing, he threatened execution for a wrong answer)

and Sampling from a Population Counting Rules: Permutations and Combination The sighted man could not tell what color hat he wore, and declined to answer. The jailor next offered the same terms to the one-eyed man, who also could not tell what color hat he wore, and also declined to answer. At this point the jailor thought his game was done, but was delighted when the blind prisoner spoke up and asked if he didn t get a turn? Laughing, the jailer offered the blind man the same terms as the others. The blind prisoner smiled and replied, I do not need my sight; From what my friends have said, I clearly see my hat is!

and Sampling from a Population Counting Rules: Permutations and Combination permutations In the behavioral sciences it is often the case that researchers have a certain number of conditions that can be presented to subjects. Definition That is, how many orders in which the conditions can be presented. A permutation is an ordered arrangement of distinct items. The total number of permutation of n distinct items is: n factorial. n!

and Sampling from a Population Counting Rules: Permutations and Combination permutations The total number of permutation of n distinct items is: n! n factorial. The factorial (!) operator, tells you to multiply the number (n) by all numbers less than n not including zero. Examples 2! = 2x1 = 2 3! = 3x2x1 = 6 4! = 4x3x2x1 = 24 Remember 1! = 1 and 0! = 0.

and Sampling from a Population Counting Rules: Permutations and Combination Example more concrete: say an investigator is conducting a taste-preference survey for different types of scotch. The investigator has three types of scotch that can be presented and wants to know all of the permutations that are possible for the three scotches. 3! = 3x2x1 = 6

and Sampling from a Population Counting Rules: Permutations and Combination Permutation 3! = 3x2x1 = 6 The researcher may want to present only a subset (r) of all the possible conditions or outcomes. The number of permutations of r items out of n items is: np r = n! (n r)! In the scotch example: the researcher might want to know the number of permutations of one scotches out of the three scotches: 3P 1 = 3! (3 1)! = 6 2! = 6 2 = 3

and Sampling from a Population Counting Rules: Permutations and Combination Combination Alternatively, a researcher want to know how many possible combinations of subsets are possible from some larger number of conditions. The formula for is: np r = n! (n r)!r! Using the scotch example, assume that the researcher wants to know the number of combinations of 2 out of 3 scotches: 3P 1 = 3! (3 2)! = 6 1!2! = 6 2 = 3 We ll use these ideas of permutations and combinations to calculate probabilities

and Sampling from a Population Counting Rules: Permutations and Combination Combination Example 1: What is the probability of getting exactly one club and exactly one jack in two randomly dealt cards; assuming order does not matter? Event A = one club and one jack There are 13 clubs and 4 jacks, and the number of unordered combinations of clubs and jacks is 13 x 4 =52 Sample Space, S = all unordered two card hands: 52C 2 = 52C 2 = n! (n r)!r! 52! (52 2)!2! = 52! (50)!2! = 52x51 = 1326 2 P(A) = 56 = 0, 042 1326

and Sampling from a Population Counting Rules: Permutations and Combination Combination Example 2: What is the probability that a five-card poker hand contains exactly three clubs? Event A = three clubs and two non-clubs For the three clubs 13C 3 = For the 2 non clubs 13! (13 3)!3! = 13x12x11 = 286 6 Sample space 39C 2 = 39! (37)!2! = 39x38 = 741 2 52C 5 = 52! (47)!5! = 52x51x50x49x48 = 2598960 120

and Sampling from a Population Counting Rules: Permutations and Combination Combination Example 2: What is the probability that a five-card poker hand contains exactly three clubs? Event A = three clubs and two non-clubs Sample space 52C 5 = 52! (47)!5! = 52x51x50x49x48 = 2598960 120 P(A) = 741 286 2598960 = 0.082

and Sampling from a Population Counting Rules: Permutations and Combination Combination Example 3: Event A = all five cards are from the same suit For the 5 cards same suit Sample space 13C 5 = 13! (13 5)!5! = 13x12x11x10x9 = 5148 120 52C 5 = 52! (47)!5! = 52x51x50x49x48 = 2598960 120 P(A) = 5148 2598960 = 0.002

and Sampling from a Population Counting Rules: Permutations and Combination Combination Example 4: Event A = A poker of aces?

and Sampling from a Population One issue with calculating probability is how the selecting of samples from a population can change the probability of future selections. Sample with replacement or sample without replacement? Replacement occurs when after a case is selected it is returned (replaced) to the population. Without replacement occurs when after a case is selected it is not returned to the population. There are pros and cons of each method.

and Sampling from a Population Sampling with replacement the population size (N) never changes, so the probability of an event never changes. =) the population you run the risk of selecting the same case more than once.=( Sampling without replacement Corrects for the possibility of selecting a case multiple times (replace the unit) =) The probability of unselected events being selected increases over time (As N gets lower) =( Most population are large (on the order of millions), altering the number of cases that can be selected should not make much of a difference. Most behavioral scientists use sampling without replacement.

and Sampling from a Population As examples of sampling with and without replacement: N=50 25 psychology majors. 10 neuroscience majors. 10 communications majors 5 counseling majors. Say we select a student at random from the class. The initial probabilities of selecting a student from each major are

and Sampling from a Population Let s say we select a Communications major, but the student was not returned to the class (sampling without replacement). Say that we select a second student, a psychology major, and do not return that student to the class.

and Sampling from a Population Say that we select a second student, a psychology major, and do not return that student to the class. The point is that changing the underlying population size, just like changing information in a situation (e.g., Monty Hall problem), alter probabilities of selecting events:

and Sampling from a Population Probabilities Based on Multiple Variables Assume a researcher is interested in the association between political orientation and sexual behavior. The researcher surveys all incoming freshmen at a university. N = 1000 freshmen and two questions 1. political attitude: liberal and conservative 2. Whether the student has engaged in intercourse: yes and no. Thus, the freshmen can be classified into one of four groups:

and Sampling from a Population Probabilities Based on Multiple Variables Thus, the freshmen can be classified into one of four groups: 1 liberals who have had intercourse 2 liberals who have not had intercourse 3 conservatives who have had intercourse 4 conservatives who have not had intercourse For argument s sake, assume all freshmen are 18 years old and that there are equal numbers of males and females in each group. Contingency table A contingency table is used to record the relationship between two or more variables.

and Sampling from a Population Probabilities Based on Multiple Variables Contingency table A contingency table is used to record the relationship between two or more variables. Now, probabilities that are conditional on (dependent on) some other event can be calculated.

and Sampling from a Population Probabilities Based on Multiple Variables Simple Probabilities: We want to determine the probability of selecting a liberal (conservative) student from among all incoming freshmen. P(L) = numberofliberals N P(C) = numberofconservatives = 700 1000 = 0.7 = 300 1000 = 0.3 N This is the probability of one political orientation without taking into account the variable previously engaged in intercourse.

and Sampling from a Population Probabilities Based on Multiple Variables Student can either be a liberal or can be a conservative, but cannot be a political conservative and a political liberal; this means that the two outcomes for Political Attitude are mutually exclusive. Conditional Probabilities Is the probability of selecting some event (A) given some other event (B) is selected, that is, that some other condition is satisfied. P(A B) = A B B Say the researcher wants to calculate the probability of selecting a student that has engaged in intercourse given the student is also a political conservative.

and Sampling from a Population Probabilities Based on Multiple Variables P(A B) = A B B Say the researcher wants to calculate the probability of selecting a student that has engaged in intercourse given the student is also a political conservative. P(Yes C) = Yes C C = 100 300 = 0.333 33.3 percent chance of selecting a student who has had sexual intercourse, if you have also selected a conservative student.

and Sampling from a Population Probabilities Based on Multiple Variables P(A B) = A B B Let s say that the researcher is now interested in the probability of selecting a liberal student given we have selected a student who has not had intercourse. P(L No) = L No No = 200 400 = 0.5 There is about a 50 % chance that the researcher has selected a liberal freshmen student if he has selected a student who has not engaged in intercourse.

and Sampling from a Population Dependent and independent events Conditional probabilities can be used to determine whether there is a relationship between the two events. Whether the two events are independent or not. If two events are independent it means that selecting one event does not depend on the other event. P(A) = p(a B) two events are not independent, that is, there is a relationship between the events P(A) p(a B) Let s determine whether having previously engaged in intercourse (event A) is independent of being a conservative (event B)...

and Sampling from a Population Dependent and independent events Let s determine whether having previously engaged in intercourse (event A) is independent of being a conservative (event B)... P(Y ) = 600 1000 = 0.6 P(Y C) = 0.33 So P(Y ) p(y C) then some relationship must exist between having engaged in intercourse and being a conservative

and Sampling from a Population Joint Probabilities If you wish to determine the probability of a combination of two events out of all events you must calculate a joint probability. Definition A joint probability is simply the probability of selecting tow events, A and B, together. P(A B) P(A B) = N say that the researcher wants to calculate the probability of selecting a student that has not engaged in intercourse (event A) and is a politically liberal student (event B).

and Sampling from a Population Joint Probabilities P(A B) P(A B) = N Say that the researcher wants to calculate the probability of selecting a student that has not engaged in intercourse (event A) and is a politically liberal student (event B).

and Sampling from a Population Joint Probabilities Say that the researcher wants to calculate the probability of selecting a student that has not engaged in intercourse (event A) and is a politically liberal student (event B). P(N L) = P(N L) N = 200 1000 = 0.2 the probability of selecting a student who has not had intercourse and is a politically liberal student is p = 0.2.

and Sampling from a Population Addition Rule of The addition rule of probability is used to determine the probability of at least one of two different events, or both events, occurring. That is, what is the probability of only event A, only event B, or both events A and B occurring? P(A B) = P(A) + P(B) P(A B) Say that the researcher wants to calculate the probability of selecting a student who has engaged in intercourse (event A) or is politically liberal (event B), or had intercourse and is liberal.

and Sampling from a Population Addition Rule of P(A B) = P(A) + P(B) P(A B) Say that the researcher wants to calculate the probability of selecting a student who has engaged in intercourse (event A) or is politically liberal (event B), or had intercourse and is liberal.

and Sampling from a Population Addition Rule of Say that the researcher wants to calculate the probability of selecting a student who has engaged in intercourse (event A) or is politically liberal (event B), or had intercourse and is liberal. P(Y)= 600/1000 = 0.6. P(L) = 0.7 P(Y L) = 0.5 P(Y L) = 0.6 + 0.7 0.5 = 0.8 there is an 80 percent chance of selecting at least a student how has had intercourse or is a politically liberal student.

and Sampling from a Population Multiplication Rules of Definition The multiplication rule is used to calculate the probability of selecting two events, A and B, like the joint probability. There are two ways that the multiplication rule can be applied: 1 when events are independent 2 when events are not to be independent If two events, A and B, are independent and there is no relationship between the events. Such as: P(A) = p(a B) the following multiplication rule can be applied to events A and B, P(A B) = P(A)P(B)

and Sampling from a Population Multiplication Rules of Example P(A B) = P(A)P(B) Finding the probability of rolling a 2 on a six-sided die (event A) and flipping a tails on a two sided coin (event B). The joint probability of these two events can be found by applying the multiplication rule for independent events: P(2 tails) = P(2)P(tails) = 1 1 6 2 = 1 12 = 0.084

and Sampling from a Population Multiplication Rules of Example 2 P(A B) = P(A)P(B) what is the probability of flipping tails on a two-sided coin (event A) and then flipping heads on the same coin (event B).

and Sampling from a Population Multiplication Rules of If two events A and B are not independent such that P(A) P(A B) then we determine the probability of observing both events as: P(A B) = P(B)P(A B) = P(A)P(B A)

and Sampling from a Population Multiplication Rules of From the contingency table, say the researcher wants to calculate the probability of selecting a student who had intercourse (event A) who is politically conservative (event B). That would be P(A B) = P(B)P(A B) = P(A)P(B A) P(Y C) = P(C)P(Y C) = (0.33)(0.3) = 0.1 Note that this is equal to the joint probability calculated above; that is, P(Yes Conservative) = 100/1000 = 0.1.

and Sampling from a Population Multiplication Rules of Now say that the researcher is interested in the probability of selecting a politically conservative student, not returning that student to the population, and then selecting another politically conservative student? That would be P(A B) = P(B)P(A B) = P(A)P(B A) P(C1 C2) = P(C2)P(C1 C2) = ( 300 1000 )(299 999 ) = 0.0987 Note that this does round to 0.09, but I wanted to show that there

and Sampling from a Population Bayes Theorem From the previous slide we know that: P(B)P(A B) = P(A)P(B A) So P(A B) = P(A)P(B A) P(B) This expression is one form of Bayes Theorem, which is used for computing the conditional probability of an event A given event B.

and Sampling from a Population Bayes Theorem Example, say that we want to calculate the probability of selecting a student who has had intercourse (event A) given they are a conservative student (event B) P(Yes Conservative). from earlier was 100/300 = 0.333. We can also find this probability using Bayes Theorem: P(Y )P(C Y ) P(Y C) = = (0.6)(0.167) = 0.1 P(C) 0.33 0.33 = 0.33 The important point to remember is that you can use relationships between variables and events to calculate probabilities of observing specific outcomes from the combinations of events.

and Sampling from a Population Back to Monty Hall Now that you know a little about probability and understand the most important thing when calculating a probability is to take information into account, let s go back to the Monty Hall problem. Remember: After you make your initial selection and Monty Hall opens a door exposing a goat The probability of winning by staying with your original choice of door is 1/3. The probability of winning through switching to the other unopened door is 2/3 The probabilities change only after Monty Hall opens a door

and Sampling from a Population Back to Monty Hall Winning by staying: Say you initially choose Door 1, which is the door that hides the prize. In this case MH can open Door 2 or Door 3 to expose a goat and the probability that Monty Hall opens either door is 1/2. there is 1/6 chance Monty Hall will open Door 2 and a 1/6 chance he will open Door 3. (1/3)(1/2) = 1/6. In either case, if you stay with Door 1 (the door with the prize) you win if you stay and you lose if you switch.

and Sampling from a Population Back to Monty Hall Winning by switching: say you initially pick Door 2 (which hides a goat) because the initial pick was a door with a goat Monty Hall is forced to open the door that hides the other goat. Remember, Monty Hall knows which door hides the prize! From the 1/3 chance you select Door 2, the probability that Monty Hall will open Door 3 to expose the other goat 1.0.

and Sampling from a Population Back to Monty Hall Winning by switching: say you initially pick Door 3 (which also hides a goat) because the initial pick was a door with a goat Monty Hall is forced to open the door that hides the other goat. From the 1/3 chance you select Door 3, the probability that Monty Hall opens Door 2 is 1.0. In this case if you switch you win, but if you stay you lose.

and Sampling from a Population Back to Monty Hall Indeed, if you look at the statistics from the game show "Let s Make a Deal", people won after a switch about 2/3 of the time, and people won from staying about 1/3 of the time.

and Sampling from a Population Back to Monty Hall The point is that information about events, whether subtle as in the case of Monty Hall or more overt, provides information that can seriously alter a probability structure; and this information should always be taken into account.

and Sampling from a Population Session 6 oscardavid.barrerarodriguez@sciencespo.fr April 3, 2018