Social Influence and Social Prediction mining big social networks. Jie Tang Computer Science, Tsinghua University

Transcription

1 Social Influence and Social Prediction mining big social networks Jie Tang Computer Science, Tsinghua University 1

2 Networked World 1.65 billion MAU 2.5 trillion minutes/month QQ: 800 million MAU WeChat: 800 million MAU 320 million MAU Peak: 143K tweets/s 360 million users influencing our daily life 304 million active users 14 billion items/year ~700 million trans. (alipay) billion on 11/11 2

3 History of Social Network Analysis Computational Social Science [Giles] Computational Social Science [Lazer et al.] Scale Free [Barabási, Albert & Faloutsos et al.] Small World [Watts, Strogatz] HITS [Kleinberg]&PageRank [Brin&Page] Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 3

4 1967: Six Degrees of Separation History of Social Network Analysis Given two individuals selected randomly from the population, what is the probability that the minimum number of intermediaries required to link them is 0, Computational Social Science [Giles] , 2,, k? Computational Social Science [Lazer et al.] 2009 Spread of Obesity, Smoking, Happiness 2008 in Boston. [Christakis, Fowler] 2007 Stanley The Milgram. average The number small of world intermediaries problem. Psychology in the mailing Today, chains 2(1):60 67, was Milgram selected 296 volunteers and to distribute a mail to a stockholder living Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Stanley Hubs Milgram. & Authorities The small [Kleinberg] world problem. Psychology Today, (1):60 67, Cited by 6967 (as of May 2016) Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 4

5 1973: Weak Tie History of Social Network Analysis The weak tie hypothesis argues, if A is linked to both B and C, then there is a greater-than-chance probability that B and C are linked to each other. Computational Social Science [Giles] 2012 Computational Social Science [Lazer et al.] 2009 Essentially Spread of Obesity, form Smoking, A friend s Happiness friend is also 2008 my friend [Christakis, Fowler] 2007 Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] 1998 Granovetter, Mark S. The strength of weak ties. American journal of sociology 78.6 (1973): Cited by (as Hubs & Authorities [Kleinberg] of April 2017) Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

6 1992: Dunbar s Number History of Social Network Analysis Dunbar s number is a suggested cognitive limit to the number of people with whom one can maintain stable social relationships. Computational Social Science [Giles] 2012 Computational Social Science [Lazer et al.] 2009 Spread of Obesity, Smoking, Happiness 2008 relationships. [Christakis, Fowler] 2007 Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, Dunbar proposed that humans can only comfortably maintain 150 stable Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] 1998 Robin I. M. Dunbar. Neocortex size as a constraint on group size in primates. Journal of Human Evolution 22 (6): Hubs & Authorities [Kleinberg] Cited by 1393 (as of May 2016) Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

7 1995: Structural Holes History of Social Network Analysis The position of a bridge between distinct groups allows him or her to transfer valuable information from one group to another. Computational Social Science [Giles] 2012 Computational Social Science [Lazer et al.] 2009 The Spread individual of Obesity, Smoking, can combine Happiness all the ideas 2008 he or she receives from [Christakis, Fowler] 2007 Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, different sources and come up with the most innovative idea among all Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] 1998 Ronald S. Burt. Structural Holes: The Social Structure of Competition. Cambridge: Harvard University Press Hubs & Authorities [Kleinberg] Cited by (as of May 2016) Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

8 History of Social Network Analysis Computational Social Science [Giles] 2012 Computational Social Science The [Lazer et 20 al.] th Century: 2009 Spread of Obesity, Smoking, Happiness 2008 [Christakis, Fowler] 2007 Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, Sociology & Anthropology Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Hubs & Authorities [Kleinberg] Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 8

9 : HITS and PageRank History of Social Network Analysis Computational Social Science [Giles] 2012 Computational Social Science [Lazer et al.] 2009 Stanley Spread Milgram. of Obesity, The Smoking, small world Happiness problem. Psychology 2008 Today, 2(1):60 67, [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] 1995 Structural Hole [Burt] 1. Jon M. Kleinberg. Authoritative sources in a hyperlinked 1992 environment. Dunbar s In Number ACM SODA, [Dunbar] Extended version in Journal of the ACM 46 (1999). Also at IBM Research Report RJ 10076, May Cited by 9896 (as of May 2016) 1973 Weak Tie [Granovetter] 2. Sergey Brin, Lawrence Page. The anatomy of a large-scale hypertextual Web search engine. In WWW 07, Pages Six Degrees of Separation [Milgram] 117,1998. Cited by (as of May 2016) 9 Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] HITS [Kleinberg]&PageRank [Brin&Page] 1997 Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, 1967.

10 1998: Small World History of Social Network Analysis Computational Social Science [Giles] 2012 Computational Small-World Social Science Properties: [Lazer et al.] 2009 Stanley Spread Milgram. of Obesity, The Smoking, small world Happiness problem. Psychology 2008 Today, 2(1):60 67, short average path lengths [Christakis, Fowler] high clustering coefficients Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] 1995 Structural Hole [Burt] Duncan Watts, Steven Strogatz. Collective dynamics 1992 of small-world Dunbar s networks. Number Nature [Dunbar] 393 (6684): Cited by (as of May 2016) Weak Tie [Granovetter] 10 Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] HITS [Kleinberg]&PageRank [Brin&Page] 1997 Stanley Milgram. The small world problem Psychology Today, 2(1):60 67, Six Degrees of Separation [Milgram]

11 1999: Scale Free History of Social Network Analysis A scale-free network is a network whose degree distribution follows a power law. Computational Social Science [Giles] 2012 Computational Social Science [Lazer et al.] 2009 Stanley The Spread Milgram. fraction of Obesity, The P(k) Smoking, small of nodes world Happiness having problem. k Psychology connections 2008 Today, to 2(1):60 67, [Christakis, Fowler] 2007 other nodes: P k ~ k γ 2005 where γ is a parameter whose value is typically in 2003 the range 2 < γ < Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] 1995 Structural Hole [Burt] Albert-LászlóBarabási, Réka Albert. Emergence of 1992 scaling in random Dunbar s networks, Number Science, [Dunbar] 286: , Cited by (as of May 2016) 1973 Weak Tie [Granovetter] Michalis Faloutsos, Petros Faloutsos, Christos Faloutsos. On power-law relationships of the internet topology. In ACM 1967 Six Degrees of Separation [Milgram] SIGCOMM Cited by 5631 (as of May 2016) 11 Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] HITS [Kleinberg]&PageRank [Brin&Page] 1997 Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, 1967.

12 History of Social Network Analysis The Late 20 th Century: Computational Social Science [Giles] 2012 Computational Social Science [Lazer et al.] 2009 Stanley Spread Milgram. of Obesity, The Smoking, small world Happiness problem. Psychology 2008 Today, 2(1):60 67, [Christakis, Fowler] 2007 Computer Science & Physics & Mathematics Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Hubs & Authorities [Kleinberg] Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 12

13 : Community Detection History of Social Network Analysis The property of community structure, in which network nodes are joined Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, together in tightly knit groups, between which there are only looser connections. Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Computational Social Science [Giles] M. Girvan and M. E. J. Newman. Community structure in social and biological networks. PNAS 99(12): , Computational Cited by 8088 (as Social of May Science 2016) [Lazer et al.] Hubs & Authorities [Kleinberg] Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Stanley Milgram. The small world problem Psychology Structural Hole Today, [Burt] 2(1):60 67, Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

14 : Influence Maximization History of Social Network Analysis Minimize marketing cost and more generally to maximize profit, e.g., to get a small number of influential users to adopt a new product, and subsequently trigger a large cascade of further adoptions. Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Computational Social Science [Giles] D. Kempe, J. Kleinberg, E. Tardos. Maximizing the Spread of Influence through a Social Network. In ACM KDD Best Computational Research Paper Social Award Science & Test of [Lazer Time Paper et al.] Award. Cited by 3513 (as of May 2016) Hubs & Authorities [Kleinberg] Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Stanley Milgram. The small world problem Psychology Structural Hole Today, [Burt] 2(1):60 67, Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

15 15 History of Social Network Analysis Given a snapshot of a social network at time t, we seek to accurately predict the Stanley edges Milgram. that will The be small added world to the problem. network Psychology during the interval Today, 2(1):60 67, from time t to a given future time t. Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Computational Social Science [Giles] D. Liben-Nowell, Computational J. Kleinberg. Social Science The Link [Lazer Prediction et al.] Problem for Social Networks. In ACM CIKM, Cited by 2616 (as of May 2016) Hubs & Authorities [Kleinberg] 1997 Stanley Milgram. The small world problem Psychology Structural Hole Today, [Burt] 2(1):60 67, Dunbar s Number [Dunbar] t t 2003: Link Prediction 2009 Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

16 : Network Evolution History of Social Network Analysis Most of graphs densify over time, with the number of edges growing Stanley superlinearly Milgram. in The the small number world of problem. nodes. Psychology Today, 2(1):60 67, The average distance between nodes often shrinks over time. Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Computational Social Science [Giles] J. Leskovec, Computational J. Kleinberg, Social C. Faloutsos. Science [Lazer Graphs et over al.] Time: Densification Laws, Shrinking Diameters and Possible Explanations. In KDD, Best Research Paper Award & Test of Time Award. Cited by 1492 (as of May 2016) Hubs & Authorities [Kleinberg] 1997 Stanley Milgram. The small world problem Psychology Structural Hole Today, [Burt] 2(1):60 67, Dunbar s Number [Dunbar] Densification 2009 Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Weak Tie [Granovetter] Shrinking Diameters Six Degrees of Separation [Milgram]

17 1. Nicholas Christakis, James Fowler. The Spread of Obesity in a Large Social Network Over 32 Years. The New England Journal of Medicine 357 (4): , Cited by 3335 (as of May 2016) 2. Nicholas Christakis, James Fowler. The Collective Dynamics of Smoking in a Large Social Network. The New England Journal of Medicine 358 (21): , Cited by 1232 (as of May 2016) 3. Computational James Fowler, Nicholas Social Christakis. Science [Giles] The Dynamic Spread of Happiness in a Large Social Network. British Medical Computational Journal 337 (768): Social a2338, Science Cited [Lazer by 1178 et al.] (as of May 2016) : Diffusion and Influence History of Social Network Analysis Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] If the husband became obese, the likelihood Hubs & Authorities [Kleinberg] 1997 that his wife would become obese increased Stanley Milgram. The small world problem Psychology Structural Hole Today, [Burt] 2(1):60 67, by 37% Dunbar s Number [Dunbar] by tracking 10,000+ people for 32 years 2009 Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Weak Tie [Granovetter] Six Degrees of Separation [Milgram]

18 : Social Influence Analysis History of Social Network Analysis Social influence is a prevalent, complex and subtle force that governs the Stanley dynamicsof Milgram. all The social small networks. world problem. Psychology Today, 2(1):60 67, Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Jie Tang, Computational Jimeng Sun, Social Chi Wang, Science and Zi [Giles] Yang. Social Influence Analysis in Large-scale Networks. In KDD'09, pages Computational Top cited paper Social in KDD 09. Science [Lazer Cited by et 624 al.] (as of April 2017) How to quantify the social influences from different topics Hubs & Authorities [Kleinberg] 1997 Stanley Milgram. The small world problem Psychology Structural 0.2 Hole Today, [Burt] 2(1):60 67, Topical Affinity Propagation (TAP), Dunbar s Number 0.5 [Dunbar] an efficient model 2009 Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Politics Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 0.74

19 History of Social Network Analysis The Early 21 st Century More Computer & Data Scientists Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, Computational Social Science [Giles] Computational Social Science [Lazer et al.] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Hubs & Authorities [Kleinberg] Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 19

20 Computational Social Science Computational Social Science [Giles] Computational Social Science [Lazer et al.] Spread of Obesity, Smoking, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] A field is emerging that leverages the capacity to collect and analyze data at a scale that may reveal patterns of individual and group behaviors. Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe,Kleinberg,Tardos] Community Detection [Girvan, Newman] 2003 David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Alber-Laszlo Barabasi, et al. from Departments of Sociology, Computer 2002 Science, Physics, Business, Government, etc. at Harvard, MIT, Northeastern, Northwestern, Columbia, Cornell, etc Stanley Milgram. The small world problem. Psychology Today, 2(1):60 67, Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] 1998 Small World [Watts, Strogatz] 1998 Hubs & Authorities [Kleinberg] 1997 Hubs & Authorities [Kleinberg] 1997 Computational Models 1995 Structural Hole [Burt] Big Data Algorithms 1995 Structural Dunbar s Number Hole [Burt] 1992 [Dunbar] Dunbar s Number [Dunbar] Sociology, Physics, Psychology, Business, Management, et al. Weak Tie [Granovetter] 1. Six Degrees of Separation [Milgram] David Lazer et al. Computational Social Science. Science James Giles. Computational Social Science: Making the Links. Nature 2012.

21 History of Social Network Analysis Computational Social Science [Giles] Computational Social Science [Lazer et al.] Scale Free [Barabási,Albert & Faloutsos et al.] Small World [Watts, Strogatz] Hubs & Authorities [Kleinberg] Social Influence Analysis [Tang, Sun] Spread of Obesity, Happiness [Christakis, Fowler] Densification [Leskovec, Kleinberg, Faloutsos] Link Prediction [Liben-Nowell, Kleinberg] Influence Maximization [Kempe, Kleinberg, Tardos] Community Detection [Girvan, Newman] Structural Hole [Burt] Dunbar s Number [Dunbar] Weak Tie [Granovetter] Six Degrees of Separation [Milgram] 21

22 What is a social network? A social network is: a graph made up of : a set of individuals, called nodes, and tied by one or more interdependency, such as friendship, called edges. 22

23 信息时代 1.0: 数据 Web = Information Space? ?? 0.99 数据语义化 hyperlinks between web pages Examples: Google s PageRank Kleinberg s HITS [1] L. Page, S. Brin, R. Motwani, & T. Winograd. (1999). The pagerank citation ranking: bringing order to the web. Stanford University. [2] J. M. Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM) 46.5 (1999):

24 大数据成为 新油田 We generate 2.5x10 18 byte big data per day, and 90% of the data was generated in the past 2 yrs It is becoming the new oil of the 21st century In the past year, 1,477 daily briefings provided to US president are based on big data analysis 80% of stock exchanges are based on big data analysis 导致 云计算时代 传统行业的加入 24

25 信息时代 2.0: 数据 + 用户 Collaborative Web = Info Space + Users 用户语义化 25 Interaction Personalized recommendation Collaborative Filtering [1] P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, & J. Riedl. (1994). GroupLens: an open architecture for collaborative filtering of netnews. CSCW'94. ACM, New York, NY, USA,

26 26 一个例子

27 信息时代 3.0: 融合 + 智能 Social Network = Info. Space + Social Space 信息空间 数据语义 知识 智能 用户空间 交互 用户语义 [1] J. Scott. (1991, 2000, 2012). Social network analysis: A handbook. [2] D. Easley and J. Kleinberg. Networks, crowds, and markets: Reasoning about a highly connected world. Cambridge University Press,

28 大数据需要知识, 需要 hoe 发现和充分利用大数据的价值需要新型数据挖掘与分析方法, 能够从非结构化数据的庞大宝藏中获得知识和洞察力 大数据 + 知识 大数据需要新的工具 : hoe 28

29 数据 -> 知识 -> 智能 知识智能 (Edward Feigenbaum, Tim Berners Lee) Step 1: Where is Knowledge Cyc, Wikipedia, DBPedia, Yago, etc. Step 2: How to Leverage the Knowledge? -> 如何 落地 Edward Feigenbaum 专家系统之父图灵奖获得者 应用驱动 Tim Berners Lee WWW 创始人图灵奖获得者 29

30 算法是核心, 计算 数据是基础 人工智能的核心 : 算法 算法 实现核心 : 机器通过算法实现人工智能 突破方向 : 认知智能是下一个突破方向 突破途径 : 人工智能突破主要通过算法性能的提升, 主要有工程学法和模拟法 方法一 : 工程学方法 (Engineering Approach) 采用传统的编程技术, 利用大量数据处理经验改进提升算法性能 方法二 : 模拟法 (Modeling Approach) 模仿人类或其他生物所用的方法或机理, 提升算法性能, 例如遗传算法和神经网络 基础条件 : 计算和数据 计算能力 现状 : 使用 GPU 并行计算神经网络 作用 : 提升运算速度, 降低计算成本 未来 : 量子计算 速度更快的芯片 数据 现状 : 互联网发展积累了一定数据 作用 : 训练机器, 提升算法性能 未来 : 物联网发展带来环境 行为等全面数据 智能交互驱动 算法分析为核心 数据 / 计算为基础 30

31 一个例子 : 智慧教育 智慧教育 :XuetangX 用户语义化 智能交互 内容语义化 年, 与工程院成立 知识智能 联合研究中心专注研究基于知识的智能转化

32 data Heterog eneous Dynamic Roadmap User Tie Structure Micro Macro tie Influence Big&Big - User Modeling - Demographics - Social Role - Social Tie/Link - Homophily - Social Influence - Triad Formation - Community - Group Behavior Social Theories Graph Theories social BIG Networks 32

33 Computational Models for Social Network Analysis A A + + B + C B _ C (A) (B) friend A A friend friend non-friend B C B C friend non-friend (A) (B) 33 y y 2 =? 4 =? y 2 y 4 h (y 3, y 4, y 5 ) TrFG model y 5 y y 1 1 =1 y 5 =1 Input: social network y 3 h (y 1, y 2, y 3 ) y 3 =0 y 6 y 6 =? f (s v 3, s 3,y 3 ) 3 5 v 6 f (u 5,s 5, y 5 ) 4 f (s 1, u 2,y 1 ) f (u 2, s 2,y 2 ) v f (u 3 4, s 4,y 4 ) 4 6 u 2, s 2 f (s 6, u 6,y 6 ) 2 v 5 (v 2, v 3 ) u 4, s 4 u v 5, s 5 2 u 1, s 1 u 3, s 3 (v 4, v 5 ) (v u 6, s 6 (v 4, v 6 ) 1 v 2, v 1 ) 1 (v 4, v 3 ) (v 6, v 5 ) Observations

34 Related Publications Data Mining IEEE/ACM Trans (22), SIGIR (2) IJCAI/AAAI (16) ACL (3) ICDM (10) JCDL 11 Best Paper Nominated ICDM 12 Challenge Champion Social Network Knowledge Graph SIGKDD (18) ICML WWW (4) ACM Multimedia PKDD 11 Best Stu Paper Runneru SIGKDD Best Poster Award Multimedia 12 Challenge 2nd SIGMOD WWW IJCAI (2) PLOS ONE (2) Journal of Web Semantics ArnetMiner 研究者社会网络分析系统, 吸引全球 220 个国家 832 万独立 IP 访问, 获 2013 年中国人工智能学会科技进步一等奖 34 学堂在线 最大的中文慕课 (MOOC) 平台, 注册人数达到 7,250,000 万, 选课人次超过 10,200,000 万

35 data Heterog eneous Dynamic Roadmap User Tie Structure Micro Macro tie Influence Big&Big - User Modeling - Demographics - Social Role - Social Tie/Link - Homophily - Social Influence - Triad Formation - Community - Group Behavior Social Theories Graph Theories social BIG Networks 35

36 User Opinion and Influence: Love Trump Trump makes USA great again I hate Trump, the worst president ever Trump is fantastic Trump is great! No Trump in 2012! He cannot be the next president! Positive Negative 36

37 Social Interaction: Influence Case 1: Social influence and political mobilization [1] Will online political mobilization really work? A controlled trial (with 61M users on FB) - Social msg group: was shown with msg that indicates one s friends who have made the votes. - Informational msg group: was shown with msg that indicates how many other. - Control group: did not receive any msg. [1] R. M. Bond, C. J. Fariss, J. J. Jones, A. D. I. Kramer, C. Marlow, J. E. Settle and J. H. Fowler. A 61-million-person experiment in social influence and political mobilization. Nature, 489: ,

38 Social Influence Analysis: not just testing 1 Who influenced who? What is the 2 Can we leverage the social influence influence probability? to help practical applications? Trump makes USA great again I hate Trump, the worst president ever Trump is fantastic 0.74 Trump is great! No Trump in 2012! He cannot be the next president! Positive Negative 38

39 Formulation: Topic-based Social Influence Analysis Topics Politics Entertainment Trademarks I love Obama I hate Obama Market Strategy Politics Positive Negative Politics output How to?

40 The Solution: Topical Affinity Propagation Strategy Strategy Market Market Market Basic Idea: If a user is located in the center of a Market community, and is similar to the other users, then she/he would have a strong influence on the other users. Homophily theory Market Strategy [1] Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social Influence Analysis in Large-scale Networks. KDD 09, pages

41 The Solution: Topical Affinity Propagation Define a function to quantify the similarity between neighborhood users Market Strategy Politics How Ada thought he influenced Bob? How Bob thought he was influenced by Ada? Politics Market Strategy Politics Estimate how a user can represent his neighbors Politics Market Strategy The topic information can be obtained by any tagging system or topic modeling approach [1] Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social Influence Analysis in Large-scale Networks. KDD 09, pages

42 Topical Factor Graph (TFG) Model Social link Asymmetric similarity Topological feature or global constraint Nodes that have the highest influence on the current node User-specific attributes Node/user The problem is cast as identifying which node has the highest probability to influence another node on a specific topic along with the edge. 42

43 Topical Factor Graph (TFG) Objective function: 1. How to define? 2. How to optimize? The learning task is to find a configuration for all {y i } to maximize the joint probability. 43

44 How to define (topical) feature functions? Node feature function similarity Edge feature function Global feature function or simply binary 44

45 Model Learning Algorithm Sum-product: - Low efficiency! - Not easy for distributed learning! 45

46 New TAP Learning Algorithm 1. Introduce two new variables r and a, to replace the original message m. 2. Design new update rules: How user i thought he influenced user j? m ij How user j thought he was influenced by user i? [1] Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social Influence Analysis in Large-scale Networks. In KDD, pages ,

47 47 The TAP Learning Algorithm

48 48 More Questions: The essential influence mechanisms are rather complex

49 49 Complex Influence: a. Complex Structural Influence b. Complex Embedding Influence

50 50 Complex Influence: a. Complex Structural Influence b. Complex Embedding Influence

51 Generalized to complex structures [1] Jie Tang, Sen Wu, and Jimeng Sun. Confluence: Conformity Influence in Large Social Networks. KDD'13, pages [2] Jing Zhang, Jie Tang, Yuanyi Zhong, Yuchen Mo, Juanzi Li, Guojie Song, Wendy Hall, and Jimeng Sun. StructInf: Mining Structural Influence from Social Streams. AAAI'17. 51

52 Beyond peer influence Influence is a very complicated mechanism Peer influence Conformity influence Individual influence or? Positive or negative? 52

53 Triadic Influence Learning influence in signed triads And apply the triadic influence for prediction 53

54 Triadic Influence Learning influence in signed triads 54 Most open triads are negatively correlated with user s behavior, implying that user behavior are more strongly influenced by closed circles.

55 Triadic Influence Learning influence in signed triads Strong tie means strong influence 55

56 Challenge in Structural Influence Challenge: how to estimate the influence score? 56

57 Counting in Structural Influence v 1, a 1, t 1 v 1, a 1, t 1 v 1 v 0, a 1, t 2 v 0, a 1, t 2 v 3, a 1, t 2 v 0 v 3 v 3, a 1, t 2 v 2, a 1, t 3 v 3, a 2, t 3 a 1 v 2, a 1, t 3 Active target action v 2 v 5 v 2, a 2, t 4 a 2 v 2, a 2, t 4 v 3, a 2, t 3 Inactive target action v5, a2, t5 Network Action logs Action diffusion graphs 57

58 Fast Sampling StructInf-S1 StructInf-S2 Complete subgraph Incomplete subgraph StructInf-S3 Complete subgraph Incomplete subgraph Complete subgraph Incomplete subgraph 58

59 Variance vs. Sampling Speed Variance: The higher the sampling probability, the smaller the variance will be, while the sampling speed will be slower. Trade off error and time by p and q 59

60 Results Prediction Performance Retweet behavior prediction Variance vs. Speed Basic features: the number of friends, gender, verification status of the user C1 : the number of active neighbors weak : IP < 0:1, moderate : 0.1 < IP < 0.3, strong: IP > 0.3 [1] Jing Zhang, Jie Tang, Yuanyi Zhong, Yuchen Mo, Juanzi Li, Guojie Song, Wendy Hall, and Jimeng Sun. StructInf: Mining Structural 60 Influence from Social Streams. AAAI'17.

61 61 Complex Influence: a. Complex Structural Influence b. Complex Embedding Influence

62 Add Embedding into Factor Graph User embedding y 1 =? y2 =0 y 1 y 2 f(q, e 1, v 1, y 1 ) v 2 v 1 W g(e 2, e 4, y 2, y 4 ) y 3 y 3 =1 y 4 y 4 =? v 3 v 4 e 1 e 2 e 3 e 4 User Combine embedding into factor graph model 62

63 Modeling Influence with GAN Source 63 [1] Yu L, Zhang W, Wang J, et al. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. AAAI. 2017:

64 Problem: Directions Modeling influence&diffusion (action) simultaneously Model time-series user behavior in networks Outlier detection: fraud users Model: Modeling diffusion with TreeGAN Model influence with neural message passing 64

65 We applied the learned influence to help real applications in very big Tencent networks 65

66 Big Data Analytics in Game Data Online gaming is one of the largest industries on the Internet Facebook 250 million users play games monthly 200 games with more than 1 million active users 12% of the company s revenue is from games Tencent (Market Cap: ~150B $) More than 400 million gaming users 50% of Tencent s overall revenue is from games [1] Zhanpeng Fang, Xinyu Zhou, Jie Tang, Wei Shao, A.C.M. Fong, Longjun Sun, Ying Ding, Ling Zhou, and Jarder Luo. Modeling Paying 66 Behavior in Online Social Networks. CIKM'14.

67 Two games: DNF Dungeon & Fighter Online (DNF) A game of melee combat between users and large number of underpowered enemies 400+ million users, the 2 nd largest online game in China Users in the game can fight against enemies by individuals or by groups 67

68 Two games: QQ Speed QQ Speed A racing game that users can partake in competitions to play against other users 200+ million users Users can race against other users by individuals or forma a group to race together Some users may pay 68

69 Task Given behavior log data and paying logs of online game users, predict Free users -> Paying users Will social influence play an important role in this task? 69

70 Test setting Online Test Two groups: test group and control group Send msgs to invite the user to attend a promotion activity. test group2 control group 70

71 Dropout Prediction Dropout prediction using social influence Weekly results accuracy precision recall f1-measure aera under cureve F1 值较之前提高 43.71%

72 From Peer Influence to Triadic Influence [1] Jie Tang, Sen Wu, and Jimeng Sun. Confluence: Conformity Influence in Large Social Networks. KDD'13, pages [2] Jing Zhang, Jie Tang, Yuanyi Zhong, Yuchen Mo, Juanzi Li, Guojie Song, Wendy Hall, and Jimeng Sun. StructInf: Mining Structural Influence from Social Streams. AAAI'17. 73

73 Further beyond peer influence Influence is a very complicated mechanism Peer influence Conformity influence Individual influence or? Positive or negative? 74

74 Results By adding the triadic influence as features for prediction on Weibo and Tencent CF (CrossFire) * LRC Logistic Regression; B Basic features; N Peer influence feature only; T Triadic influence features 75 * On Weibo, we predict who will (re)tweet a message; on Tencent CF, we predict will a free user becomes a paying user.

75 Generalized to other scenarios Generalization: other problem + other dataset Problem: Call back of users Game data: King of Glory ( 王者荣耀 ) With Influence w/o influence Top k #message #success ratio #message #success ratio % % % % % % % % % % 76

76 Group Lifecycle Prediction Group Member Fringe Invite friend ~2.3 million groups generated everyday >25% messages are generated in group chats [1] Jiezhong Qiu, Yixuan Li, Jie Tang, Zheng Lu, Hao Ye, Bo Chen, Qiang Yang, and John Hopcroft. The Lifecycle and Cascade of WeChat 77 Social Messaging Groups. WWW'16, pages

77 Group (or Invitee) Recommendation 78 [1] Yu Han, Jie Tang, Hao Ye, and Bo Chen. Who to Invite Next? Predicting Invitees of Social Groups. IJCAI'17, pages

78 data Heterog eneous Dynamic Roadmap User Tie Structure Micro Macro tie Influence Big&Big - User Modeling - Demographics - Social Role - Social Tie/Link - Homophily - Social Influence - Triad Formation - Community - Group Behavior Social Theories Graph Theories social BIG Networks 79

79 80 Related Publications Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social Influence Analysis in Large-scale Networks. In KDD 09, pages , Chenhao Tan, Jie Tang, Jimeng Sun, Quan Lin, and Fengjiao Wang. Social action tracking via noise tolerant timevarying factor graphs. In KDD 10, pages , Chi Wang, Jiawei Han, Yuntao Jia, Jie Tang, Duo Zhang, Yintao Yu, and Jingyi Guo. Mining Advisor-Advisee Relationships from Research Publication Networks. In KDD'10, pages Jie Tang, Sen Wu, and Jimeng Sun. Confluence: Conformity Influence in Large Social Networks. In KDD 13, pages , Yuxiao Dong, Yang Yang, Jie Tang, Yang Yang, Nitesh V. Chawla. Inferring User Demographics and Social Strategies in Mobile Social Networks. In KDD 14, Yutao Zhang, Jie Tang, Zhilin Yang, Jian Pei, and Philip Yu. COSNET: Connecting Heterogeneous Social Networks with Local and Global Consistency. In KDD'15, pages Jing Zhang, Biao Liu, Jie Tang, Ting Chen, and Juanzi Li. Social Influence Locality for Modeling Retweeting Behaviors. In IJCAI'13, pages , Jing Zhang, Jie Tang, Honglei Zhuang, Cane Wing-Ki Leung, and Juanzi Li. Role-aware Conformity Influence Modeling and Analysis in Social Networks. In AAAI'14, Jing Zhang, Jie Tang, Yuanyi Zhong, Yuchen Mo, Juanzi Li, Guojie Song, Wendy Hall, and Jimeng Sun. StructInf: Mining Structural Influence from Social Streams. In AAAI'17. Yu Han, Jie Tang, Hao Ye, and Bo Chen. Who to Invite Next? Predicting Invitees of Social Groups. IJCAI'17, pages Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. ArnetMiner: Extraction and Mining of Academic Social Networks. In KDD 08, pages , Tiancheng Lou and Jie Tang. Mining Structural Hole Spanners Through Information Diffusion in Social Networks. In WWW'13, pages , Jiezhong Qiu, Yixuan Li, Jie Tang, Zheng Lu, Hao Ye, Bo Chen, Qiang Yang, and John Hopcroft. The Lifecycle and Cascade of WeChat Social Messaging Groups. In WWW'16, pages Jie Tang, Tiancheng Lou, Jon Kleinberg, and Sen Wu. Transfer Learning to Infer Social Ties across Heterogeneous Networks. In TOIS, Vol 34 (2), No. 7, 2016 Jimeng Sun and Jie Tang. A Survey of Models and Algorithms for Social Influence Analysis. Social Network Data Analytics, Aggarwal, C. C. (Ed.), Kluwer Academic Publishers, pages , 2011.

80 Thank you! Collaborators: John Hopcroft, Jon Kleinberg, Chenhao Tan (Cornell) Jiawei Han (UIUC), Philip Yu (UIC) Jian Pei (SFU), Hanghang Tong (ASU) Tiancheng Lou (Pony.ai), Jimeng Sun (GIT) Wei Chen, Ming Zhou, Long Jiang, Chi Wang, Yuxiao Dong (Microsoft) Yutao Zhang, Jing Zhang, Zhanpeng Fang, Zi Yang, Sen Wu, etc. (THU) Jie Tang, KEG, Tsinghua U, Download all data & Codes,