Discrete Random Variables: Joint PMFs, Conditioning and Independence

Transcription

1 Discrete Radom Variables: Joit MFs Coditioig ad Ideedece Berli Che Deartmet of Comuter Sciece & Iformatio gieerig Natioal Taiwa Normal Uiversit Referece: - D.. Bertsekas J. N. Tsitsiklis Itroductio to robabilit Sectios.5-.7

2 Motivatio Give a eerimet e.g. a medical diagosis The results of blood test is modeled as umerical values of a radom variable The results of magetic resoace imagig (MRI 核磁共振攝影 ) is also modeled as umerical values of a radom variable We would like to cosider robabilities of evets ivolvig simultaeousl the umerical values of these two variables ad to ivestigate their mutual couligs? robabilit-berli Che

3 Joit MF of Radom Variables Let ad be radom variables associated with the same eerimet (also the same samle sace ad robabilit laws) the joit MF of ad is defied b if evet is the set of all airs that have a certai roert the the robabilit of ca be calculated b Namel ca be secified i terms of ad robabilit-berli Che 3

4 robabilit-berli Che 4 Margial MFs of Radom Variables (/) The MFs of radom variables ad ca be calculated from their joit MF ad are ofte referred to as the margial MFs The above two equatios ca be verified b

5 Margial MFs of Radom Variables (/) Tabular Method: Give the joit MF of radom variables ad is secified i a two-dimesioal table the margial MF of or at a give value is obtaied b addig the table etries alog a corresodig colum or row resectivel robabilit-berli Che 5

6 Fuctios of Multile Radom Variables (/) fuctio Z g of the radom variables ad defies aother radom variable. Its MF ca be calculated from the joit MF Z z g z The eectatio for a fuctio of several radom variables g g Z robabilit-berli Che 6

7 Fuctios of Multile Radom Variables (/) If the fuctio of several radom variables is liear ad of the form Z g a b c Z a b c How ca we verif the above equatio? robabilit-berli Che 7

8 Illustrative amle Give the radom variables ad whose joit is give i the followig figure ad a ew radom variable Z is defied b Z calculate Z Method : [ Z ] Method : Z z z Z Z Z Z Z Z Z Z Z Z [ Z ] robabilit-berli Che 8

9 More tha Two Radom Variables (/) The joit MF of three radom variables ad is defied i aalog with the above as z Z z Z Z The corresodig margial MFs ad z Z z z z Z robabilit-berli Che 9

10 More tha Two Radom Variables (/) The eectatio for the fuctio of radom variables ad Z g Z g z z Z z If the fuctio is liear ad has the form a b cz d a b cz d a b cz d geeralizatio to more tha three radom variables a a a a a a robabilit-berli Che 0

11 Illustrative amle amle.0. Mea of the Biomial. our robabilit class has 300 studets ad each studet has robabilit /3 of gettig a ideedetl of a other studet. What is the mea of the umber of studets that get a? Let i 0 if the ith studet gets a otherwise 300 are beroulli radom variables with commo mea /3 Their sum variable with arameters ( of success i ( ) ad ( /3). That is is the umber 300) ideedet trials ca be iterreted as a biomial radom 300 / i i robabilit-berli Che

12 Coditioig Recall that coditioal robabilit rovides us with a wa to reaso about the outcome of a eerimet based o artial iformatio I the same sirit we ca defie coditioal MFs give the occurrece of a certai evet or give the value of aother radom variable robabilit-berli Che

13 robabilit-berli Che 3 Coditioig a Radom Variable o a vet (/) The coditioal MF of a radom variable coditioed o a articular evet with is defied b (where ad are associated with the same eerimet) Normalizatio roert Note that the evets are disjoit for differet values of their uio is 0 Total robabilit theorem

14 Coditioig a Radom Variable o a vet (/) grahical illustratio Is obtaied b addig the robabilities of the outcomes that give rise to ad be log to the coditioig evet robabilit-berli Che 4

15 Illustrative amles (/) amle.. Let be the roll of a fair si-sided die ad be the evet that the roll is a eve umber roll is eve ad is eve is eve / 3 if 46 0 otherwise robabilit-berli Che 5

16 Let Illustrative amles (/) amle.4. studet will take a certai test reeatedl u to a maimum of times each time with a robabilit of assig ideedetl of the umber of revious attemts. be What is the MF of the umber of attemts give that the studet asses the test? a rereseti fist success geometric g the comes radom umber u variable of with attemts arameter util ( ) Let be the evet that the studet ass the test w ithi attemts ( ) ( ) if m ( ) m 0 otherwise the - - robabilit-berli Che 6

17 Coditioig a Radom Variable o other (/) Let ad be two radom variables associated with the same eerimet. The coditioal MF of give is defied as Normalizatio roert The coditioal MF is ofte coveiet for the calculatio of the joit MF multilicatio (chai) rule ( ) is fied o some value robabilit-berli Che 7

18 robabilit-berli Che 8 Coditioig a Radom Variable o other (/) The coditioal MF ca also be used to calculate the margial MFs Visualizatio of the coditioal MF

19 Illustrative amle (/) amle.4. rofessor Ma B. Right ofte has her facts wrog ad aswers each of her studets questios icorrectl with robabilit /4 ideedetl of other questios. I each lecture Ma is asked 0 or questios with equal robabilit /3. What is the robabilit that she gives at least oe wrog aswer? Let be the umber of questios asked be the umber of questios aswered ( ( ) ( ) ( ( ( 3 ) ( ( 4 ) ( 3 ) ( ) ) ( 4 ) 3 4 ) ( 3 ) ) 4 4 wrog modeled as biomial distributios ) ( 48 k - k ) k robabilit-berli Che 9

20 Illustrative amle (/) Calculatio of the joit MF i amle.4. robabilit-berli Che 0

21 Coditioal ectatio Recall that a coditioal MF ca be thought of as a ordiar MF over a ew uiverse determied b the coditioig evet I the same sirit a coditioal eectatio is the same as a ordiar eectatio ecet that it refers to the ew uiverse ad all robabilities ad MFs are relaced b their coditioal couterarts robabilit-berli Che

22 Summar of Facts bout Coditioal ectatios Let ad be two radom variables associated with the same eerimet The coditioal eectatio of give a evet with is defied b 0 For a fuctio g it is give b g g robabilit-berli Che

23 Total ectatio Theorem (/) The coditioal eectatio of give a value of is defied b We have Let be disjoit evets that form a artitio of the samle sace ad assume that i 0 for all i. The i i i robabilit-berli Che 3

24 robabilit-berli Che 4 Total ectatio Theorem (/) Let be disjoit evets that form a artitio of a evet ad assume that for all. The Verificatio of total eectatio theorem B 0 B i i B B B i i i

25 robabilit-berli Che 5 Illustrative amle (/) amle.7. Mea ad Variace of the Geometric Radom Variable geometric radom variable has MF evet that the be evet that the be Let otherwise 0 (??) otherwise 0 (??) where 0 0 otherwise 0 if ) ( ) ( :.3) amle that (See Note m m

26 robabilit-berli Che 6 Illustrative amle (/) var show that have we set 0 0

27 Ideedece of a Radom Variable from a vet radom variable is ideedet of a evet if ad for all Require two evets ad be ideedet for all If a radom variable is ideedet of a evet ad 0 ad for all robabilit-berli Che 7

28 Illustrative amle amle.9. Cosider two ideedet tosses of a fair coi. Let radom variable be the umber of heads Let radom variable be 0 if the first toss is head ad if the first toss is tail Let be the evet that the umber of head is eve ossible outcomes (TT) (TH) (HT) (HH) / 4 if 0 / / if 0 / 4 if / / / / if if 0 if if if 0 ad are ot ideedet! ad / if 0 / if ad are ideede t! robabilit-berli Che 8

29 Ideedece of a Radom Variables (/) Two radom variables ad are ideedet if or If a radom variable is ideedet of a radom variable for all for all for all with 0 all for all with 0 ad all robabilit-berli Che 9

30 Ideedece of a Radom Variables (/) Radom variables ad are said to be coditioall ideedet give a ositive robabilit evet if Or equivaletl for all for all with 0 ad all Note here that as i the case of evets coditioal ideedece ma ot iml ucoditioal ideedece ad vice versa robabilit-berli Che 30

31 Illustrative amle (/) Figure.5: amle illustratig that coditioal ideedece ma ot iml ucoditioal ideedece For the MF show the radom variables ad are ot ideedet To show ad are ot ideedet we ol have to fid a air of values of ad that For eamle ad are ot ideedet robabilit-berli Che 3

32 Illustrative amle (/) To show ad are ot deedet we ol have to fid all air of values of ad that For eamle ad are ideedet coditioed o the evet / 0 6 / 0 3 / 0 3 / / 0 6 / 0 3 / 0 3 / / 0 / 3 9 / 0 6 / 0 / 3 9 / 0 robabilit-berli Che 3

33 robabilit-berli Che 33 Fuctios of Two Ideedet Radom Variables Give ad be two ideedet radom variables let ad be two fuctios of ad resectivel. Show that ad are ideedet. g h g h the ad Let v u v u V U v h u g v h u g v h u g V U h V g U

34 More Factors about Ideedet Radom Variables (/) If ad are ideedet radom variables the s show b the followig calculatio b ideedece Similarl if ad are ideedet radom variables the g h g h robabilit-berli Che 34

35 robabilit-berli Che 35 If ad are ideedet radom variables the s show b the followig calculatio More Factors about Ideedet Radom Variables (/) var var var var var var

36 More tha Two Radom Variables Ideedece of several radom variables Z Three radom variable ad are ideedet if z z for all Z Z? Comared to the coditios to be satisfied for three ideedet evets ad 3 (i.39 of the tetbook) f g h three radom variables of the form ad are also ideedet Variace of the sum of ideedet radom variables If are ideedet radom variables the var var var var robabilit-berli Che 36

37 Illustrative amles (/3) amle.0. Variace of the Biomial. We cosider ideedet coi tosses with each toss havig robabilit of comig u a head. For each i we let be the Beroulli radom variable which is equal to if the i-th toss comes u a head ad is 0 otherwise. The is a biomial radom variable. i var var i for all i var (Note that ' s are ideedet! ) i i i robabilit-berli Che 37

38 Illustrative amles (/3) amle.. Mea ad Variace of the Samle Mea. We wish to estimate the aroval ratig of a residet to be called B. To this ed we ask ersos draw at radom from the voter oulatio ad we let i be a radom variable that ecodes the resose of the i-th erso: i 0 if if the i - th erso aroves the i B' s - th erso disarove s B' s erformac e erformac e i ssume that ideedet ad are the same radom variable (Beroulli) with the commo arameter ( for Beroulli) which is ukow to us are ideedet ad ideticall distributed (i.i.d.) If the samle mea (is a radom variable) is defied as S i S with arameter robabilit-berli Che 38

39 Illustrative amles (3/3) The eectatio of will be the true mea of S i S i i ( for the Beroulli we assumed here) i The variace of S will aroimate 0 if is large eough lim var S var var i i lim lim lim Which meas that S will be a good estimate of i if is large eough 0 robabilit-berli Che 39

40 Recitatio SCTION.5 Joit MFs of Multile Radom Variables roblems SCTION.6 Coditioig roblems SCTION.6 Ideedece roblems robabilit-berli Che 40