Paper Presentation. Steve Jan. March 5, Virginia Tech. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

Transcription

1 Paper Presentation Steve Jan Virginia Tech March 5, 2015 Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

2 2 paper to present Nonparametric Multi-group Membership Model for Dynamic Networks, NIPS 13, Myunghwan Kim and Jure Leskovec, Stanford Community Detection in Graphs through Correlation, KDD14, Lian Duan, W. Nick Street, Yanchi Liu, Haibing Lu, New Jersey Institute of Technology, Santa Clara University Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

3 Nonparametric Multi-group Membership Model Social networkis often dynamic in a sense that relations between entities rise and decay over time. Problem: extract a summary of the common structure and dynamic of the underlying relations. Applications: Predict missing relationships, forecast future links, identify clusters and groups of nodes Note It uses lots of statistic techniques to solve this problem. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

4 Dynamic Multi-group Membership Graph Model They pay close attention to the three processes governing network dynamics: Birth and death dynamics of individual groups Evolution of memberships of nodes to groups The structure of network interactions between group members as well as non-members. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

5 Birth and death dynamics of individual groups Why do we know when the groups birth and death? It would be more clear that for the number of groups at each specific time. A group can be be in one of two states:{ active (alive) or inactive (not yet born or dead) }. Figure: Blact: active (alive), White: inactive Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

6 Formal Way of Birth and death dynamics of individual groups It uses distance-dependent Indian Buffet Processes (dd-ibp) to model, which is a time-relate stochastic process. Customers enter an Indian Buffet restaurant and sample some subset of an infinitely long sequence of dishes. In this applications, time t would be customers, they samples a set of active groups K t. Formally speacking, at the first time step t = 1, we have Poisson(λ) number of groups that are initially active, i.e., K 1 Poisson(λ). Poisson(γλ) new groups are also born at time t. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

7 Dynamics of node group memberships Intuition: Nodes joining and leaving groups based on their current status.. They further uses Markov chain to model dynamics of nodes joining and leaving groups. They denote each node i of the network is whether belong to community K at time t by a binary variable zik t {0, 1} where, a k, b k are two parameters and probability. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

8 Relationship between node group memberships and links of the network Intuition: Link netween two nodes based on their current groups They assume there is a connection between nodes memberships to groups and the links of the network. They build on the Multiplicative Attribute Graph model: each group k is associated with a link affinity matrix M R 2 2. These four entries represent groups members, members and non-members, as well as non- members themselves. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

9 Model Inference via MCMC After introducing these three models, then they try to sample these parameters. Sampling node group memberships Z: Use forward-backward recursion algorithm. group membership transition matrix Q: Use a conjugate prior of Bernoulli distribution and some posterior distribution. Sampling link affinities M: Use Metropolis-Hastings and Hybrid Monte Carlo (HMC) sampling. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

10 Experiments Datasets they use: NIPS co-authorships network for T = 17 years (1987 to 2003). DBLP co- authorship network is obtained from 21 Computer Science conferences from 2000 to 2009 (T = 10) INFOCOM dataset represents the physical proximity interactions between 78 students at the 2006 INFOCOM conference, T = 50 Tasks they have: Missing link prediction Future network forecasting Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

11 Missing link prediction Randomly hold out 20% of node pairs throughout the entire time period. Naive: Relationship between each pair of nodes is decided by Bernoulli distribution with Beta(1, 1) prior. LFRM: static networks DRIFT: infinite factorial HMM model. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

12 Future network forecasting Given networks from t = 1,..., T, they want to predict the link of t = T + 1. They train the models on first Tobs networks, fix the parameters, and then for each model they run MCMC sampling one time step into the future. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

13 Conclusion We learn three models for time series. How to sampel these parameters Personally I think this paper is good in terms of the statistics methods they use Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

14 Community Detection in Graphs through Correlation Then, we move to the next paper. This paper is about Community Detection, based on Modularity-based. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

15 Major problem of modularity Resolution problem. K m is an m-clique The detected communities are marked by circles with dash lines. Multi-resolution Further divide each detected community Bias: (the tendency to merge small communities and to split large communities, are introduced.) Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

16 Connection with itemset search Graph communities: number of internal edges is greater than expected under assumption of random partition Correlated itemsets: occur more than expected under the assumption of item independence Connection: modularity = leverage Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

17 Correlated Itemsets Given itemset S = {I 1, I 2,..., I m } with m items in a dataset with n transactions True probability: tp s = P(S) Expected probability ep s = m i=1 P(I i) Correlation measure: M s = f (tp s, ep s ) (tp s ep s ) 2 ep s Chi-square: Probability ratio : tp s /ep s Leverage: tp s ep s Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

18 Correlated itemset example t1: Beef, Chicken, Milk t2:beef, Cheese t3: Cheese, Boots t4: Beef, Chicken, Cheese t5: Beef, Chicken, Clothes, Cheese, Milk For the itemset {Beef, Chicken} tp = 3 5, ep = , Leverage = tp ep = 25 Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

19 Modularity Function Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

20 Transforming modularity function For partition {G 1, G 2,..., G l }on graph G k i : degree of node i k internal : number of nodes in the same group of node i that connect to node. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

21 Transforming modularity function Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

22 Transforming modularity function They found that if translating the undirect-graph modulaity to direct-graph one, they can use itemset criteria to represnt moduality. If we randomly select an edge from the doubly-directed graph: The true probability of the edge in G p : tp = Probability the edge started from G p : Probability the edge ended in G p : i Gpk i 2m j Gpk j 2m i Gpk internal i 2m The expected probability of the edge in G p under the assumption of independence: ep = i Gpk i 2m j Gpk j 2m Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

23 Transforming modularity function Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

24 Transforming modularity function Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

25 Transforming modularity function Connecting correlation with modularity For a given partition G p, partial modularity Q p = tp p ep p For a given itemset S, leverage = tp s ep s Since the other correlation measures are also functions of tp and ep, they can change the partial modularity function Q p by using the formula of other correlation measures. Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

26 Experiments Modify the objective function Greedy search (hierarchical clustering) Baseline: Modularity-based methods (Leverage) Datasets: Real life: 1. Karate club( two equal size communities) 2. College football(12 equal size communities) Evaluation measures: Rand Index (Rand1971), Jaccard, F-measure, Normalized mutual information (Danon 2005) Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

27 Real life datasets Steve Jan (Virginia Tech) Paper Presentation March 5, / 28

28 Summary Connection between community detection and correlation search Modularity is good only when there are large and clear communities Likelihood ratio is robust to any type of communities Probability ratio partitions the whole graph into small communities with 2 or 3 objects Steve Jan (Virginia Tech) Paper Presentation March 5, / 28