Multivariate Methods. Measurements of p variables on each of n objects. Multivariate data Data display Principal component analysis

Transcription

1 Multiariate Method Multiariate data Data diplay Prinipal omponent analyi Unuperied learning tehnique Diriminant analyi Superied learning tehnique Cluter analyi Unuperied learning tehnique (Read note on thi) 1 Meaurement of p ariable on eah of n objet e.g. length & width of petal & epal of eah of 150 iri flower key feature i that ariable are orrelated & oberation independent Data Diplay Satterplot of pair of omponent Need to hooe whih omponent Matrix plot Star plot et. et. et. None i ery atifatory when p i big Need to elet bet omponent to plot i.e. need to redue dimenionality Digreion on R language detail: Many multiariate routine in library ma So far only onidered data in a dataframe Multiariate method in R often need data in a matrix Ue ommand uh a a.matrix(.) rbind(.) bind(.) Whih reate matrie (ee help) Prinipal Component Analyi (PCA) Tehnique for finding whih linear ombination of ariable ontain mot information. Produe a new oordinate ytem Plot on the firt few omponent are like to how truture in data (i.e. information) Example: Iri data 5 > library(ma) > library(mass) > par(mfrow=(2,2)) > data(iri) > attah(iri) > ir<-bind(sepal.length, Sepal.Width, Petal.Length, + Petal.Width) > ir.pa<-prinomp(ir) > plot(ir.pa) > ir.p<-predit(ir.pa) > plot(ir.pa$ore[,1:2]) > plot(ir.pa$ore[,2:3]) > plot(ir.pa$ore[,3:4]) Thi reate a matrix ir of the iri data, perform pa, ue the generi predit funtion to alulate the oordinate of the data on the prinipal omponent and plot the firt three pair of omponent 6 1

2 7 8 Show importane of eah omponent:- mot information in firt omponent Variane Comp Comp.3 Comp.4 Comp mot information in thi plot Can detet at leat two eparate group in data and maybe Comp Can detet at leat two eparate group in data and maybe one group diide into two?? Can interpret prinipal omponent a refleting feature in the data by examining loading away from the main theme of oure ee example in note. Prinipal omponent analyi i a ueful bai tool for inetigating data truture and reduing dimenionality Diriminant Analyi Key problem i to ue multiariate data on different type of objet to laify future oberation. e.g. the iri flower are atually from 3 different peie (50 of eah) What ombination of epal & petal length & width are mot ueful in ditinguihing between the peie and for laifying new ae e.g. onider a plot of petal length width Firt et up a etor to label the three arietie a or or > ir.peie<-fator((rep("",50), + rep("",50),rep("",50))) Then reate a matrix with the petal meaurement > petal<-bind(petal.length, + Petal.Width) Then plot the data with the label > plot(petal,type="n") > text(petal, + label=a.harater(ir.peie)) 2

3 Petal.Width Informally we an ee ome eparation between the peie and if a new oberation fell in thi region it hould be laified a type & here a type & here a type Petal.Width Howeer, ome milaifiation here between and Petal.Length Petal.Length Thi method ue jut petal length & width make ome mitake Could we do better with all meaurement? linear diriminant analyi (LDA) will gie the bet method when boundarie between region are traight line And quadrati diriminant analyi (QDA) when boundarie are quadratiure How do we ealuate how well LDA perform? Could ue rule to laify data we atually hae Can ue generi funtion predit(.) > ir.lda<- lda(ir,ir.peie) > ir.ld<- predit(ir.lda,ir) > ir.ld$la [1] [38] [75] [112] [149] Leel: Or in a table: > table(ir.peie, ir.ld$la) ir.peie Whih how that Corret laifiation rate 147/150 2 peie C laified a V 1 peie V laified a a C Howeer, thi i optimiti ( heating ) Uing ame data for alulating the lda and ealuating it performane Better would be ro-alidation Omit one oberation Calulate lda Ue lda to laify thi oberation Repeat on eah oberation in turn Alo better would be a randomization tet Randomly permute the peie label 3

4 19 20 Ue ample(ir.peie) to permute label Note:- ampling without replaement > randompeie<-ample(ir.peie) > irrand.lda<-lda(ir,randompeie) > irrand.ld<-predit(irrand.lda,ir) > table(randompeie,irrand.ld$la) randompeie Whih how that only 70 out of 150 would be orretly laified (ompare 147 out of 150) Thi ould be repeated many time and the permutation ditribution of the orret laifiation rate obtained. (or tritly the randomization ditribution ) The obered rate of 147/150 would be in the extreme tail of the ditribution i.e. the obered rate of 147/150 i muh higher than ould our by hane General Comment If we hae high number of dimenion & mall number of point then alway eay to get near perfet dirimination A randomization tet will how if a high laifiation rate i a reult of a real differene between ae or jut geometry. e.g. 2 group, 2 dimenion 3 point alway perfet 4 point 75% hane of perfet dirimination 3 dimenion alway perfet with 2 group 4 point 21 To etimate the true laifiation rate we hould apply the rule to new data e.g. to ontrut the rule on a random ample and apply it to the other oberation > amp<- (ample(1:50,25), + ample(51:100,25), ample(101:150,25)) amp will ontain 25 number from 1 to from 51 to from 101 to > amp 23 [1] [20] [39] [58] So ir[amp,]will hae jut thee ae With 25 from eah peie ir[-amp,]will hae the other Ue ir[amp,] to ontrut the lda and then predit on ir[-amp,] > iramp.lda<lda(ir[amp,],ir.peie[amp]) > iramp.ld<-predit(iramp.lda, ir[- amp,]) > table(ir.peie[-amp], iramp.ld$la) So rule laifie orretly 71 out of 75 Other example in note 24 4

5 25 26 Summary PCA wa introdued Idea of dirimination & laifiation with lda and qda outlined Idea of uing analye to predit illutrated Taking random permutation & random ample illutrated Predition and random ample will be ued in other method for dirimination & laifiation uing neural network et