Counting & Basic probabilities Stat 430 Heike Hofmann 1
Outline Combinatorics (Counting rules) Conditional probability Bayes rule 2
Combinatorics 3
Summation Principle Alternative Choices Start n1 ways n2 ways... nk ways Stop If a complex action can be performed using k alternative methods and each method can be done in n 1,n 2,...,n k ways, there are a total of N ways that the complex action can be performed, with N = n 1 + n 2 +... + n k 4
Multiplication Principle Sequence of Choices Start n1 ways n2 ways... nk ways Stop If a complex action consists of a series of k actions and each action can be done in n 1,n 2,...,n k ways, there are a total of N ways that the complex action can be performed, with N = n 1 n 2... n k 5
Counting Examples roll two dice Ω 1 = (1, 1), (1, 2)... (1, 6) (2, 1), (2, 2)... (2, 6)...... (6, 1), (6, 2)... (6, 6) 6
e.g. roll two dice (1, 1), (1, 2)... (1, 6) (2, 1), (2, 2)... (2, 6) Ω 1 =...... (6, 1), (6, 2)... (6, 6) Ω 2 = {1, 1}, {1, 2}... {1, 6} {2, 2}... {2, 6}.... {6, 6} 7
e.g. roll two dice (1, 1), (1, 2)... (1, 6) (2, 1), (2, 2)... (2, 6) Ω 1 =...... (6, 1), (6, 2)... (6, 6) 36 elements Ω 2 = {1, 1}, {1, 2}... {1, 6} {2, 2}... {2, 6}.... {6, 6} 7
e.g. roll two dice (1, 1), (1, 2)... (1, 6) (2, 1), (2, 2)... (2, 6) Ω 1 =...... (6, 1), (6, 2)... (6, 6) 36 elements Ω 2 = {1, 1}, {1, 2}... {1, 6} {2, 2}... {2, 6}.... {6, 6} 21 elements 7
e.g. roll two dice (1, 1), (1, 2)... (1, 6) (2, 1), (2, 2)... (2, 6) Ω 1 =...... (6, 1), (6, 2)... (6, 6) order matters Ω 2 = order does not matter {1, 1}, {1, 2}... {1, 6} {2, 2}... {2, 6}.... {6, 6} 36 elements 21 elements 7
e.g. roll three dice order matters order does not matter 8
e.g. roll three dice order matters Ω 1 =6 6 6 = 216 elements order does not matter Ω 2 =... = 55 elements 8
pick four cards from stack of 52 order matters order does not matter 9
pick four cards from stack of 52 order matters Ω 1 = 52 51 50 49 = 6497400 choices order does not matter Ω 2 = 52 = 4 270725 elements 9
Difference between dice & cards example die results are always in {1, 2, 3, 4, 5, 6} (with replacement) once a card is drawn, it is out of the stack (without replacement) 10
Counting Rules Goal: determine overall size of sample space without listing all elements manually Side benefit: usually we can find a mathematical description of the sample space in the process. 11
Combinatorics 12
Counting rules Urn model n numbered objects (balls) in a bag (urn) 1 2 3 13
Ordered samples with replacement Urn model: pick ball from urn, write down number, put ball back,repeat k times Sequential setup fits Multiplication principle: N = n n... n = n k k times 14
Ordered samples without replacement Urn model: pick ball from urn, write down number, put ball not back,repeat k times Sequential k times setup fits Multiplication principle: N = n (n 1)... (n k + 1) k times = n! (n k)! 15
Ordered samples without replacement Urn model: pick ball from urn, write down number, put ball not back,repeat k times Sequential k times setup fits Multiplication principle: N = n (n 1)... (n k + 1) k times this is also called the permutation number = n! (n k)! 15
Unordered samples without replacement Urn model: pick k balls from urn at once Trick: pick balls ordered w/o replacement, then mes remove order: n! N = (n k)! n! n (k!) 1 = (n k)!k! = k 16
Unordered samples with replacement Urn model: pick ball from urn, write down number, put ball back,repeat k times, keep only record of #times each ball is sampled: 1 1 2 4 4 4 7 7 k balls, (n-1) lines, i.e. k+n-1 objects Trick: place k balls among k+n-1 places n + k 1 N = k 17
Example How many ways are there to uniquely rearrange the letters of M I S S I S S I P P I 18
Independence 19
Probability concepts Independence/Dependency If occurrence of event A changes the probability of event B, the events are dependent: P (B A) = if P(A) 0 P (A B) P (A) 20
Independence Events A and B are independent, if P (A B) =P (A) P (B) 21
System reliability Serial System: system works, if all of its components are working Parallel System: system works if at least one component is working 22
Bayes Rule 23
Bayes Rule Definition: cover A set of events B 1,B 2,B 3,... is a cover, if the sets are pairwise disjoint and exhaustive (i.e. they cover the sample space) 24
Bayes Rule Total probability Let B 1,B 2,B 3,... be a cover of the sample space, then P(A) can be computed as k P (A) = P (B i ) P (A B i ). i=1 25
Bayes Theorem Let space, then B 1,B 2,B 3,... P (B j A) = P (B j A) P (A) Bayes Rule = be a cover of the sample P (A B j ) P (B j ) k i=1 P (A B i) P (B i ) 26