Stroboscope: Declarative Network Monitoring on a Budget

Transcription

1 Stroboscope: Declarative Network Monitoring on a Budget Olivier Tilmans Université catholique de Louvain USENIX NSDI 18 April 11, 2018 Joint work with T. Bühler (ETH Zürich), I. Poese (BENOCS), S. Vissicchio (UCL) and L. Vanbever (ETH Zürich) Adapted from original picture Michael Magg, 2007, CC-BY-SA 3.0

2 Consider this example ISP network topology Border router Router E U Link /24 A B C D Customer peering X Y W Z V 2

3 What is the ingress router for this packet arriving at router D? E U /24 A B C D X Y Packet W Z V 2

4 Which paths does the traffic follow? E U /24 A B C D X Y W Z V 2

5 Which paths does the traffic follow? E Tracking flows network-wide requires to /24 match measurements across multiple vantage points A B C D U X Y W Z V 2

6 Which paths does the traffic follow? E Tracking flows network-wide requires to /24 match measurements across multiple vantage points A B C D NetFlow, ProgME [ToN 11], FlowRadar [NSDI 16] X Y U W Z V 2

7 Is traffic load-balanced as expected? E U /24 A B C D X Y W Z V 3

8 Is the latency acceptable? Packets Time(t) E A B C D U /24 Time(t + 25ms) E U /24 A B C D X Y X Y 3 2 W Z V W Z V 1 3

9 Are there losses? Time(t) 3 2 E U /24 A B C D Time(t + 25ms) E U /24 A B C D X Y X Y 3 W Z V W Z V 1 3

10 Are there losses? Time(t) Time(t + 25ms) E U Fine-grained data-plane performance metrics require packet-level visibility /24 over individual flows A B C D A B C D E U /24 X Y X Y 3 W Z V W Z V 1 3

11 Fined-grained network monitoring is widely researched Gigascope [SIGMOD 03] Planck [SIGCOMM 14] Everflow [SIGCOMM 15] Compiling Path Queries [NSDI 16] Trumpet [SIGCOMM 16] Marple [SIGCOMM 17] 4

12 Fined-grained ISP network monitoring poses unique and unmet challenges Gigascope [SIGMOD 03] No control over end hosts Planck [SIGCOMM 14] Everflow [SIGCOMM 15] Compiling Path Queries [NSDI 16] Trumpet [SIGCOMM 16] Marple [SIGCOMM 17] 4

13 Fined-grained ISP network monitoring poses unique and unmet challenges Gigascope [SIGMOD 03] No control over end hosts Limited data-plane flexibility Planck [SIGCOMM 14] Everflow [SIGCOMM 15] Compiling Path Queries [NSDI 16] Trumpet [SIGCOMM 16] Marple [SIGCOMM 17] 4

14 Fined-grained ISP network monitoring poses unique and unmet challenges Gigascope [SIGMOD 03] No control over end hosts Limited data-plane flexibility Limited monitoring bandwidth Planck [SIGCOMM 14] Everflow [SIGCOMM 15] Compiling Path Queries [NSDI 16] Trumpet [SIGCOMM 16] Marple [SIGCOMM 17] 4

15 Stroboscope: Declarative Network Monitoring on a Budget Collecting traffic slices to monitor networks Adhering to a monitoring budget Using Stroboscope today

16 Consider the following flow of packets E U / A B C D X Y W Z V 6

17 Consider the following flow of packets E U / A 1 B C D X Y W Z V 6

18 Stroboscope activates mirroring for the flow Node mirroring rule E U / A 1 B C D Collector X Y W Z V 6

19 Packets are copied and encapsulated towards the collector E U / A 4 3 B 2 1 C D Collector X Y Mirrored packet W Z V 6

20 The mirroring rule is deactivated after a preset delay E U / A 4 3 B 2 1 C D Collector X Y Mirrored packet W Z V 6

21 Stroboscope stores the traffic slice for analysis E U / A 6 5 B 4 3 C 2 1 D Collector X Y Collected traffic slice W Z V 6

22 Stroboscope periodically toggles the mirroring rule E U / A 7 6 B 5 4 C 3 2 D Collector X Y W Z V 6

23 Stroboscope periodically toggles the mirroring rule E U /24 A 10 B 9 8 C 7 6 D Collector X Y W Z V 6

24 Stroboscope periodically toggles the mirroring rule E U /24 A 10 B 9 8 C 7 6 D Collector X Y W Z V 6

25 Stroboscope collects multiples traffic slices over time E U /24 A 11 B 10 C 9 8 D Collector X Y W Z V 6

26 Stroboscope works with currently deployed routers Most vendors provide traffic mirroring and encapsulation primitives The collector activates mirroring for a flow by updating one ACL Routers autonomously deactivate mirroring rules using timers Traffic slices can be as small as 23 ms on our routers (Cisco C7018) 7

27 Consider the following forwarding path E U /24 A B C D X Y W Z V 8

28 Stroboscope activates mirroring rules along a path MIRROR /24 ON [A B C D] E U /24 A B C D X Y W Z V 8

29 Traffic slices are collected 25 ms E U / X 8 A B C D 3 4 X X W X Z Y V 8

30 A CONFINE query mirrors any packet leaving a region MIRROR /24 ON [A B C D] CONFINE /24 ON [A B E C D] E U /24 A B C D W X Z Y V Confinement region 8

31 A CONFINE query mirrors any packet leaving a region MIRROR /24 ON [A B C D] CONFINE /24 ON [A B E C D] E U /24 A B C D W X Z Y V Confinement region 8

32 Counting packets missing in all last hops of a path estimates loss rates E U / X 8 A B C D 3 4 X X W X Z Y V 8

33 Counting packets partially following the path estimates load-balancing ratios E U / X 8 A B C D 3 4 X X W X Z Y V 8

34 Analyzing matching packets across traffic slices enables fine-grained measurements at scale

35 Analyzing matching packets across traffic slices enables fine-grained measurements at scale Forwarding paths discovery, timestamp reconstruction, payload inspection,...

36 Stroboscope: Declarative Network Monitoring on a Budget Collecting traffic slices to monitor networks Adhering to a monitoring budget Using Stroboscope today

37 Stroboscope defines two types of queries MIRROR CONFINE 11

38 Stroboscope defines two types of queries MIRROR CONFINE 11

39 MIRROR queries reconstruct the path taken by packets MIRROR /24 ON [A B C D] E U /24 A B C D X Y W Z V 12

40 Fewer mirroring rules reduces bandwidth usage MIRROR /24 ON [A B C D] E U /24 A B C D X Y W Z V 12

41 Too few mirroring rules creates ambiguity MIRROR /24 ON [A B C D] E U /24 A B C D X Y W Z V 12

42 Too few mirroring rules creates ambiguity MIRROR /24 ON [A B C D] E The Key-Points Sampling algorithm minimizes mirroring rules and guarantees non-ambiguous reconstructed /24 paths A B C D U X Y W Z V 12

43 Stroboscope defines two types of queries MIRROR CONFINE 13

44 CONFINE queries mirror packets leaving a confinement region CONFINE /24 ON [A B E C D] E U /24 A B C D X Y Edge Mirroring rule W Z V 14

45 Fewer mirroring rules minimizes control-plane overhead CONFINE /24 ON [A B E C D] E U /24 A B C D X Y W Z V 14

46 The lower bound is a multi-terminal node cut CONFINE /24 ON [A B E C D] E U /24 A B C D X Y W Z V 14

47 The lower bound is a multi-terminal node cut CONFINE /24 ON [A B E C D] E A B C D U The Surrounding algorithm minimizes mirroring rules and guarantees to mirror any packet leaving /24 the confinement region X Y W Z V 14

48 Query activations must be scheduled to meet the budget Monitoring queries Where? When? Measurement campaign Key-Points Sampling Surrounding 15

49 Stroboscope divides the monitoring budget in timeslots 150 ms 40 Mbps Timeslot ( 30 ms) 16

50 Stroboscope requires traffic demand estimations? Mbps Q1 Q2 Q3? Mbps Q4 Q5? Mbps Q6 40 Mbps 16

51 Stroboscope conservatively estimates traffic demands Prefix measured in prior iterations? 17

52 Stroboscope conservatively estimates traffic demands Prefix measured in prior iterations? yes Peak observed demand 17

53 Stroboscope conservatively estimates traffic demands Prefix measured in prior iterations? no yes Peak observed demand Prefix with enough NetFlow records? 17

54 Stroboscope conservatively estimates traffic demands Prefix measured in prior iterations? yes Peak observed demand no Prefix with enough NetFlow records? yes NetFlow estimation 17

55 Stroboscope conservatively estimates traffic demands Prefix measured in prior iterations? yes Peak observed demand no Prefix with enough NetFlow records? yes NetFlow estimation no Assume the query requires the full budget 17

56 Confine queries are scheduled in all timeslots 10 Mbps Q1 Q2 Q3 15 Mbps Q4 Q5 20 Mbps Q6 40 Mbps 0* Mbps Confine query 18

57 Stroboscope first approximates a minimal sub-schedule Q4 Q6 Q2 Q1 Q5 Q3 Minimal sub-schedule

58 Stroboscope first approximates a minimal sub-schedule, optionally optimizing for the optimal bin-packing solution Q4 Q2 Q1 Minimal sub-schedule Q2 Q1 Q3 Q5 Q6 Q5 Q3 Q6 Q4 Minimal sub-schedule 19

59 Stroboscope replicates the sub-schedule Q2 Q3 Q2 Q3 Q1 Q5 Q1 Q5 Q6 Q4 Q6 Q4 Minimal sub-schedule

60 Stroboscope replicates the sub-schedule, and minimizes budget leftovers Q2 Q3 Q2 Q3 Budget usage maximization Q2 Q3 Q2 Q3 Q3 Q1 Q5 Q1 Q5 Q1 Q5 Q1 Q5 Q1 Q6 Q4 Q6 Q4 Q6 Q4 Q6 Q4 Q6 Minimal sub-schedule 20

61 Stroboscope achieves deterministic sampling

62 Stroboscope: Declarative Network Monitoring on a Budget Collecting traffic slices to monitor networks Adhering to a monitoring budget Using Stroboscope today

63 Selecting mirroring locations in realistic ISP topologies is fast Time [ms] Key-Points Sampling Input path length Time [ms] Minimal surrounding Node surrounding Region size 23

64 Schedules can be quickly approximated Time [s] Input query count Optimized schedule Approximation 24

65 Stroboscope tracks the rate of mirrored traffic in real time Source A B 1 000Kb/s Destination Collector MIRROR ON [A B] Traffic rate [Kb/s] 5000 budget 2000 mirrored 1000 real Time [s] 25

66 Measurement campaigns are stopped early if the estimated demand are exceeded Source A B Destination Kb/s Collector MIRROR ON [A B] Traffic rate [Kb/s] 5000 budget 2000 mirrored 1000 real Time [s] 25

67 Exceeding the total budget schedules the query once per measurement campaign Source A B 1 000Kb/s Destination Collector MIRROR ON [A B] Traffic rate [Kb/s] 5000 budget 2000 mirrored 1000 real Time [s] 25

68 Stable recorded traffic rates are used for future estimations Source A B 1 000Kb/s Destination Collector MIRROR ON [A B] Traffic rate [Kb/s] 5000 budget 2000 mirrored 1000 real Time [s] 25

69 Stroboscope exceeds the monitoring budget for at most one timeslot Source A B Destination Collector MIRROR ON [A B] Traffic rate [Kb/s] budget 2000 mirrored 1000 real Time [s] 25

70 Stroboscope: Declarative Network Monitoring on a Budget Traffic slicing as a first-class data-plane primitive Strong guarantees on budget compliance and measurement accuracy Measurement analysis decoupled from measurement collection

71

72 Backup slides

73 NetFlow brings a poor visibility over traffic in ISP networks Fraction of BGP prefixes Maximal number of observations 29

74 Stroboscope defines a declarative requirement language MIRROR /24 ON [A B C D], [A E C D] MIRROR /24 ON [A -> D] CONFINE /24 ON [A B E C D] CONFINE /24 [A -> D] E U /24 A B C D MIRROR /24 ON [ -> D] CONFINE /24 ON [ -> D] X Y USING 15 Mbps DURING 500 ms EVERY 5 s W Z V 30

75 The placement algorithms minimize the mirroring rules 1.0 Key-Points Sampling Fraction of experiments Fraction of experiments Optimization gain 90 Minimal surrounding Node surrounding Optimization gain 70 31

76 Time [s] 1000 Optimized schedule Approximation Input query count Fraction of experiments Schedules can be computed by two pipelines Optimization gain