BIG CELLULAR NETWORK DATA. Olof Görnerup IAM Lab SICS Swedish ICT

Transcription

1 BIG CELLULAR NETWORK DATA Olof Görnerup IAM Lab SICS Swedish ICT Cloud and Big Data Day 24 September 2013

2 THE UBIQUITOUS MOBILE DEVICES Penetration percentage of subscriptions in 2013 Central and Eastern Europe: 132% Western Europe: 128% China: 84% Latin America: 114% Africa: 71% Middle East 109% India: 90% North America: 103% Globally: 90% Reference: Ericsson Mobility Report, June 2013.

3 TRULY BIG GETTING BIGGER - FAST 1.6 exabytes* of monthly traffic in Ericsson s networks in Q Amount of traffic doubled Q Q This trend is expected to continue *10 18 bytes = 1000 petabytes = 1 billion gigabytes Reference: Ericsson Mobility Report, June 2013.

4 CHALLENGES WHEN EMPLOYING NETWORK DATA Collecting the data Storing the data (or avoid doing so) Processing the data timely Analyzing the data Preserving privacy Designing business models

5 FIRST EXAMPLE: CRISIS MANAGEMENT Employ cellular networks as large-scale sensor networks Detect accidents and assess their impact Reference: Görnerup, O., Kreuger P. and Gillblad, D., Autonomous accident monitoring using cellular network data. In 10th International Conference on Information Systems for Crisis Response and Management (ISCRAM), 2013.

6 CONSIDERED SCENARIO c 2 c 1 c 0 c 1 c 2

7 CELLS SENSE TRAFFIC FLOW Normalised Inflow Rate Count Time [h] Simulated vehicle flow rate Time [s] Flow rate as seen by a cell

8 ACCIDENT DETECTION APPROACH Define mobility measure that is approximately timeinvariant on short time-scales Estimate and compare short- and long-term dynamics of measure Identify potential accident as divergence between short- and long-term dynamics Measure: Local distribution of vehicles linked to a cell and its neighbouring cells

9 IT WORKS No accident Accident Long-term measure Long term density estimate Long term density estimate Time [s] Time [s] 1.0 Short-term measure Short term density estimate Short term density estimate Time [s] Time [s] Divergence Kullback-Leibler divergence [bits] Time [s] Kullback-Leibler divergence [bits] Accident detected Time [s]

10 SECOND EXAMPLE: ROUTE MINING Use cellular network data to infer common routes and other collective mobility patterns Application areas Urban planning Public transportation traffic management Crisis management Reference: Görnerup, O., Scalable Mining of Common Routes in Mobile Communication Network Traffic Data. Tenth International Conference on Pervasive Computing, 2012.

11 A NEW SUBWAY LINE WHICH IS THE BEST OPTION? Use cellular network data to find common routes Application areas Urban planning (how do )

12 CELL USE CAPTURES ACTUAL MOBILITY PATTERNS A1 A2 B1 B2 A3 A4 A5 A6 B3 B4 B5 Trajectory 1 A7 A8 C1 D1 D2 E1 B6 Trajectory 2 F1 C2 C3 D3 E2 E3 F2 F3 C4 C5 D4 E4 H1 Trajectory 3 F4 F5 F6 G1 G2 H2 H3 F7 F8 G3 G4 G5 G6 G7 G8 A3, A4, A5, C1, D1, D2, B4, B1 F2, F3, F6, G1, C5, D3, E2, E1, B5, B2 F7, F5, F6, G1, G2, D4, E2, E3, B6

13 RELATE CELL SEQUENCES IN A GRAPH S 1 w 13 Sequence similarity S 2 S 3 Cell sequence S 4 S 5 S 6 S 7

14 ESTIMATE SEQUENCE SIMILARITIES STOCHASTICALLY S 1 Jaccard index S 2 w 13 S 3 C i C j C i C j =Prob[H(C i)=h(c j )] Min-hash H(C) =min c C h(c) Cell sequence S 4 S 5 S 6 S 7

15 GROUP SEQUENCES USING GRAPH CLUSTERING S 1 Jaccard index S 2 w 13 S 3 C i C j C i C j =Prob[H(C i)=h(c j )] Min-hash H(C) =min c C h(c) Cell sequence S 4 S 5 S 6 S 7

16 ROUTES ARE REVEALED AS CLUSTERS S 1 S 2 Route 1 S 3 S 4 S 5 Route 2 S 6 S 7

17 EVALUATION Evalutation set 0.5 Relative latitude Relative longitude r Accuracy Runtime Scalability n m

18 SUMMARY Cellular networks generate huge volumes of data with a very high velocity Refining this data is challenging, but enables a multitude of valuable applications Several ongoing activities at SICS in this area So far we have only scratched the surface still a new area with great potential!

19 THANK YOU Contact: