Mining for Statistical Models of Availability in Large-Scale Distributed Systems: An Empirical Study of

Size: px
Start display at page:

Download "Mining for Statistical Models of Availability in Large-Scale Distributed Systems: An Empirical Study of"

Transcription

1 Mining for Statistical Models of Availability in Large-Scale Distributed Systems: An Empirical Study of Bahman Javadi 1, Derrick Kondo 1, Jean-Marc Vincent 1,2, David P. Anderson 3 1 Laboratoire d Informatique de Grenoble, MESCAL team, INRIA, France 2 University of Joseph Fourier, France 3 UC Berkeley, USA IEEE/ACM International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2009) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

2 Introduction and Motivation P2P, Grid, Cloud, and Volunteer computing systems B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

3 Introduction and Motivation P2P, Grid, Cloud, and Volunteer computing systems Main Features: Tens or hundreds of thousands of unreliable and heterogeneous hosts B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

4 Introduction and Motivation P2P, Grid, Cloud, and Volunteer computing systems Main Features: Tens or hundreds of thousands of unreliable and heterogeneous hosts Uncertainty of host availability B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

5 Introduction and Motivation P2P, Grid, Cloud, and Volunteer computing systems Main Features: Tens or hundreds of thousands of unreliable and heterogeneous hosts Uncertainty of host availability B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

6 Introduction and Motivation P2P, Grid, Cloud, and Volunteer computing systems Main Features: Tens or hundreds of thousands of unreliable and heterogeneous hosts Uncertainty of host availability Main Motivation Effective Resource Selection for Stochastic Scheduling Algorithms B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

7 Introduction and Motivation P2P, Grid, Cloud, and Volunteer computing systems Main Features: Tens or hundreds of thousands of unreliable and heterogeneous hosts Uncertainty of host availability Main Motivation Effective Resource Selection for Stochastic Scheduling Algorithms Goal Model of host availability (i.e., subset of hosts with the same availability distribution) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

8 Outline Introduction and Motivation 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

9 Define Availability Measurement CPU availability on each host CPU on host i A1 Time A2 An Length of Availability Intervals: A1, A2,..., An B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

10 Measurement Measurement Method BOINC Middleware for volunteer computing systems Underlying software infrastructure for projects such as B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

11 Measurement Measurement Method BOINC Middleware for volunteer computing systems Underlying software infrastructure for projects such as We instrumented the BOINC client to collect CPU availability traces: Total number of host traces: 226,208 Collection period: April 1, Jan 1, 2009 Total CPU time: 57,800 years Number of intervals: 102,416,434 Assume 100% or 0% availability B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

12 Outline Measurement Remove outliers 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

13 Outliers Measurement Remove outliers Data Set Remove Outliers B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

14 Outliers Measurement Remove outliers Data Set Remove Outliers Check for outliers: Artifacts resulted from a benchmark run periodically every five days B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

15 Outliers Measurement Remove outliers Data Set Remove Outliers Check for outliers: Artifacts resulted from a benchmark run periodically every five days B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

16 Outliers Measurement Remove outliers Data Set Remove Outliers Check for outliers: Artifacts resulted from a benchmark run periodically every five days B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

17 Outline Modelling Process Randomness Tests 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

18 Randomness Tests Modelling Process Randomness Tests Data Set Remove Outliers Randomness Tests Pass iid Hosts Not pass Non-iid Hosts To determine which hosts have truly random availability intervals B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

19 Randomness Tests Modelling Process Randomness Tests Data Set Remove Outliers Randomness Tests Pass iid Hosts Not pass Non-iid Hosts To determine which hosts have truly random availability intervals Four well-known non-parametric tests: Runs test Runs up/down test Mann-Kendall test Autocorrelation function test (ACF) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

20 Modelling Process Randomness Tests Randomness Tests Data Set Remove Outliers Randomness Tests Pass iid Hosts Not pass Non-iid Hosts To determine which hosts have truly random availability intervals Four well-known non-parametric tests: Runs test Runs up/down test Mann-Kendall test Autocorrelation function test (ACF) Test Runs std Runs up/down ACF Kendall All # of hosts Fraction B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

21 Modelling Process Randomness Tests Randomness Tests Data Set Remove Outliers Randomness Tests Pass iid Hosts Not pass Non-iid Hosts To determine which hosts have truly random availability intervals Four well-known non-parametric tests: Runs test Runs up/down test Mann-Kendall test Autocorrelation function test (ACF) Test Runs std Runs up/down ACF Kendall All # of hosts Fraction Result: 34% are i.i.d. hosts (2.2 PetaFLOPS) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

22 Outline Modelling Process Clustering 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

23 Modelling Process Clustering Distribution of Availability Intervals Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

24 Modelling Process Clustering Distribution of Availability Intervals Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts 8 x Frequency Availability (hours) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

25 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

26 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function Method: Hierarchical B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

27 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function Method: Hierarchical Compute all permutations B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

28 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function Method: Hierarchical Compute all permutations Memory intensive B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

29 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function Method: Hierarchical Compute all permutations Memory intensive K-means (fast K-means) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

30 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function Method: Hierarchical Compute all permutations Memory intensive K-means (fast K-means) Fast convergence B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

31 Clustering Method Modelling Process Clustering Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Not pass Non-iid Hosts Generate a few clusters based on availability distribution function Method: Hierarchical Compute all permutations Memory intensive K-means (fast K-means) Fast convergence Dependent on initial centroids B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

32 Distance Metrics Modelling Process Clustering Distance between CDF of two hosts B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

33 Distance Metrics Modelling Process Clustering Distance between CDF of two hosts Kolmogorov-Smirnov: Maximum difference between two CDFs B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

34 Distance Metrics Modelling Process Clustering Distance between CDF of two hosts Kolmogorov-Smirnov: Maximum difference between two CDFs Kuiper: Maximum deviation above and below of two CDFs B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

35 Distance Metrics Modelling Process Clustering Distance between CDF of two hosts Kolmogorov-Smirnov: Maximum difference between two CDFs Kuiper: Maximum deviation above and below of two CDFs Cramer-von Mises: Area between two CDFs B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

36 Modelling Process Clustering Distance Metrics Distance between CDF of two hosts Kolmogorov-Smirnov: Maximum difference between two CDFs Kuiper: Maximum deviation above and below of two CDFs Cramer-von Mises: Area between two CDFs Anderson-Darling: Area between two CDFs, more weight on the tail B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

37 Modelling Process Clustering Distance Metrics Important Challenge: Number of samples in each CDF Few samples > not enough confidence on the result B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

38 Modelling Process Clustering Distance Metrics Important Challenge: Number of samples in each CDF Few samples > not enough confidence on the result Too much samples > the metric will be too sensitive B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

39 Modelling Process Clustering Distance Metrics Important Challenge: Number of samples in each CDF Few samples > not enough confidence on the result Too much samples > the metric will be too sensitive Data Set: different hosts have different number of samples B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

40 Modelling Process Clustering Distance Metrics Important Challenge: Number of samples in each CDF Few samples > not enough confidence on the result Too much samples > the metric will be too sensitive Data Set: different hosts have different number of samples Our solution: randomly select a fixed number of intervals from each host (i.e., 30 samples) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

41 Clustering Results Modelling Process Clustering Dendrogram of hierarchical clustering: 5-10 distinct groups (bootstrap) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

42 Clustering Results Modelling Process Clustering Comparison of distances in clusters (k-means for all iid hosts): B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

43 EDF of clusters Modelling Process Clustering B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

44 Outline Modelling Process Model fitting 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

45 Methods Modelling Process Model fitting Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Model Fitting Not pass Non-iid Hosts B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

46 Modelling Process Model fitting Methods Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Model Fitting Not pass Non-iid Hosts Method: Maximum Likelihood Estimation (MLE) Moment Matching (MM) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

47 Modelling Process Model fitting Methods Data Set Remove Outliers Randomness Tests Pass iid Hosts Clustering Model Fitting Not pass Non-iid Hosts Method: Maximum Likelihood Estimation (MLE) Moment Matching (MM) Target Distributions: Exponential Pareto Weibull Log-normal Gamma B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

48 Graphical Test Modelling Process Model fitting PP-plots: Exponential, Pareto, Weibull, Log-normal, Gamma Cluster 1: B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

49 Graphical Test Modelling Process Model fitting PP-plots: Exponential, Pareto, Weibull, Log-normal, Gamma Cluster 1: Cluster 2: B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

50 Graphical Test Modelling Process Model fitting PP-plots: Exponential, Pareto, Weibull, Log-normal, Gamma Cluster 1: Cluster 2: Cluster 3: Cluster 4: Cluster 5: Cluster 6: B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

51 Modelling Process Model fitting Goodness Of Fit Tests Generate p-values by two GOF tests (average over 1000 runs): Kolmogorov-Smirnov (KS) test Anderson-Darling (AD) test B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

52 Modelling Process Model fitting Goodness Of Fit Tests Generate p-values by two GOF tests (average over 1000 runs): Kolmogorov-Smirnov (KS) test Anderson-Darling (AD) test Exponential Pareto Weibull Log-Normal Gamma Data sets AD KS AD KS AD KS AD KS AD KS All iid hosts Cluster Cluster Cluster Cluster Cluster Cluster B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

53 Modelling Process Model fitting Some properties of clusters Clusters # of hosts % of total avail. mean (hrs) Best fit Parameters shape scale All iid hosts Weibull Cluster Gamma Cluster Gamma Cluster Log-Normal Cluster Gamma Cluster Gamma Cluster Weibull Cluster sizes are different and often significant B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

54 Modelling Process Model fitting Some properties of clusters Clusters # of hosts % of total avail. mean (hrs) Best fit Parameters shape scale All iid hosts Weibull Cluster Gamma Cluster Gamma Cluster Log-Normal Cluster Gamma Cluster Gamma Cluster Weibull Cluster sizes are different and often significant Heterogeneity in distribution parameters (different scale parameters) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

55 Modelling Process Model fitting Some properties of clusters Clusters # of hosts % of total avail. mean (hrs) Best fit Parameters shape scale All iid hosts Weibull Cluster Gamma Cluster Gamma Cluster Log-Normal Cluster Gamma Cluster Gamma Cluster Weibull Cluster sizes are different and often significant Heterogeneity in distribution parameters (different scale parameters) Decreasing hazard rate B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

56 Outline Discussions Significance of Clustering Criteria 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

57 Discussions Significance of Clustering Criteria Significance of Clustering Criteria Could the same clusters have been found using some other static criteria? B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

58 Discussions Significance of Clustering Criteria Significance of Clustering Criteria Could the same clusters have been found using some other static criteria? Cluster by venue: Work, Home, School Cluster by Time zone: 6 different time zones B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

59 Discussions Significance of Clustering Criteria Significance of Clustering Criteria Could the same clusters have been found using some other static criteria? Cluster by venue: Work, Home, School Cluster by Time zone: 6 different time zones 10 5 Expected number per cluster Venue Timezone Reference Actual number per cluster B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

60 Discussions Significance of Clustering Criteria Significance of Clustering Criteria Could the same clusters have been found using some other static criteria? Cluster by venue: Work, Home, School Cluster by Time zone: 6 different time zones Cluster by CPU speed 6 x CPU speed (FLOPS) Cluster index B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

61 Outline Discussions Scheduling Implications 1 Introduction and Motivation 2 Measurement Remove outliers 3 Modelling Process Randomness Tests Clustering Model fitting 4 Discussions Significance of Clustering Criteria Scheduling Implications 5 Related Work 6 Conclusion and Future Work B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

62 Scheduling Implications Discussions Scheduling Implications Scheduling accuracy Global model vs. Individual cluster model B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

63 Scheduling Implications Discussions Scheduling Implications Scheduling accuracy Global model vs. Individual cluster model Ex: Completion probability of a 24-hour task: B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

64 Discussions Scheduling Implications Scheduling Implications Scheduling accuracy Global model vs. Individual cluster model Ex: Completion probability of a 24-hour task: Global model: <20% Cluster 4: 70% B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

65 Discussions Scheduling Implications Scheduling Implications Scheduling accuracy Global model vs. Individual cluster model Ex: Completion probability of a 24-hour task: Global model: <20% Cluster 4: 70% Resource Selection/Replication B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

66 Discussions Scheduling Implications Scheduling Implications Scheduling accuracy Global model vs. Individual cluster model Ex: Completion probability of a 24-hour task: Global model: <20% Cluster 4: 70% Resource Selection/Replication Single job: Prediction of task failure B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

67 Discussions Scheduling Implications Scheduling Implications Scheduling accuracy Global model vs. Individual cluster model Ex: Completion probability of a 24-hour task: Global model: <20% Cluster 4: 70% Resource Selection/Replication Single job: Prediction of task failure Multi-job: How the task size distribution follows the availability distribution B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

68 Related Work Related Work Different from other research Measurement B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

69 Related Work Related Work Different from other research Measurement Resource type: home, work, and school B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

70 Related Work Related Work Different from other research Measurement Resource type: home, work, and school Scale: 200,000 hosts B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

71 Related Work Related Work Different from other research Measurement Resource type: home, work, and school Scale: 200,000 hosts Duration: 1.5 years B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

72 Related Work Related Work Different from other research Measurement Resource type: home, work, and school Scale: 200,000 hosts Duration: 1.5 years Availability : CPU availability B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

73 Related Work Related Work Different from other research Measurement Resource type: home, work, and school Scale: 200,000 hosts Duration: 1.5 years Availability : CPU availability Modelling B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

74 Related Work Related Work Different from other research Measurement Resource type: home, work, and school Scale: 200,000 hosts Duration: 1.5 years Availability : CPU availability Modelling Classification according to randomness tests B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

75 Related Work Related Work Different from other research Measurement Resource type: home, work, and school Scale: 200,000 hosts Duration: 1.5 years Availability : CPU availability Modelling Classification according to randomness tests Cluster-based Model vs Global Model B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

76 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

77 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

78 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

79 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

80 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) Partitioning hosts into subsets by their availability distribution B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

81 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) Partitioning hosts into subsets by their availability distribution Modelling (Apply the methodology for the SETI@home) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

82 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) Partitioning hosts into subsets by their availability distribution Modelling (Apply the methodology for the SETI@home) 34% of hosts have truly random availability intervals B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

83 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) Partitioning hosts into subsets by their availability distribution Modelling (Apply the methodology for the SETI@home) 34% of hosts have truly random availability intervals Six clusters with three different distributions: Gamma, Weibull, and Log-normal B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

84 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) Partitioning hosts into subsets by their availability distribution Modelling (Apply the methodology for the SETI@home) 34% of hosts have truly random availability intervals Six clusters with three different distributions: Gamma, Weibull, and Log-normal B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

85 Conclusion and Future Work Conclusion and Future Work Discovering availability models for host subsets from a distributed system Conclusion Methodology Remove outliers Classification based on the randomness tests (iid vs non-iid) Partitioning hosts into subsets by their availability distribution Modelling (Apply the methodology for the SETI@home) Future Work 34% of hosts have truly random availability intervals Six clusters with three different distributions: Gamma, Weibull, and Log-normal Apply the result for improving makespan of DAG-applications Explore ability of clustering dynamically while the system is on-line B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

86 Conclusion and Future Work Failure Trace Archive Repository of availability traces of parallel and distributed systems, and tools for analysis Facilitate design, validation and comparison of fault-tolerance algorithms and models 15 data sets including data set B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

87 Conclusion and Future Work Failure Trace Archive Repository of availability traces of parallel and distributed systems, and tools for analysis Facilitate design, validation and comparison of fault-tolerance algorithms and models 15 data sets including data set More Details Poster Session at MASCOTS 2009 (Today 19:00-21:00) Website: B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

88 Conclusion and Future Work Thank You Questions? B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

89 Distance Metrics Conclusion and Future Work Distance between CDF of two hosts B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

90 Distance Metrics Conclusion and Future Work Distance between CDF of two hosts Kolmogorov-Smirnov: D n,m = sup F n(x) G m(x) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

91 Distance Metrics Conclusion and Future Work Distance between CDF of two hosts Kolmogorov-Smirnov: D n,m = sup F n(x) G m(x) Kuiper: V n,m = sup F n(x) G m(x) +sup G m(x) F n(x) B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

92 Distance Metrics Conclusion and Future Work Distance between CDF of two hosts Kolmogorov-Smirnov: D n,m = sup F n(x) G m(x) Kuiper: V n,m = sup F n(x) G m(x) +sup G m(x) F n(x) Cramer-von8 Mises: 9 T n,m = nm (n+m) 2 < nx mx = [F n(x i ) G m(x i )] 2 + [F n(y j ) G m(y j )] 2 : ; i=1 j=1 B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

93 Distance Metrics Conclusion and Future Work Distance between CDF of two hosts Kolmogorov-Smirnov: D n,m = sup F n(x) G m(x) Kuiper: V n,m = sup F n(x) G m(x) +sup G m(x) F n(x) Cramer-von8 Mises: 9 T n,m = nm (n+m) 2 < nx mx = [F n(x i ) G m(x i )] 2 + [F n(y j ) G m(y j )] 2 : ; i=1 Anderson-Darling: Q n = R [F(x) Fn(x)]2 ψ(f(x))df ψ(f(x)) = 1 F(x)(1 F (x)) j=1 B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

94 Conclusion and Future Work Fitting with Hyper-Exponential Fitting Method: Expectation Maximization (EM) [using EMpht package] Accurate Flexible Slow B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

95 Conclusion and Future Work Fitting with Hyper-Exponential Fitting Method: Expectation Maximization (EM) [using EMpht package] Accurate Flexible Slow Moment Matching (MM) Less accurate Not flexible Very fast B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

96 Conclusion and Future Work Fitting with Hyper-Exponential Fitting Method: Expectation Maximization (EM) [using EMpht package] Accurate Flexible Slow Moment Matching (MM) Less accurate Not flexible Very fast We used MM for 2-phase hyper-exponential by the first two moments as follows: p = 1 2 (1 CV 2 1 CV 2 +1 ) λ 1 = 2p µ λ 2 = 2(1 p) µ B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

97 PP-Plots Conclusion and Future Work (a) Cluster 1 (b) Cluster 2 (c) Cluster 3 (d) Cluster 4 (e) Cluster 5 (f) Cluster 6 B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

98 Conclusion and Future Work Goodness of Fit Tests Hyper-Exponential (MM) Hyper-Exponential (EM) Data sets Parameters AD KS Parameters AD KS All iid hosts p 1 = λ 1 = p 2 = λ 2 = Cluster 1 p 1 = λ 1 = p 2 = λ 2 = Cluster 2 p 1 = λ 1 = p 2 = λ 2 = Cluster 3 p 1 = λ 1 = p 2 = λ 2 = Cluster 4 p 1 = λ 1 = p 2 = λ 2 = Cluster 5 p 1 = λ 1 = p 1 = λ 2 = Cluster 6 p 1 = λ 1 = p 2 = λ 2 = p 1 = λ 1 = p 2 = λ 2 = p 3 = λ 3 = p 1 = λ 1 = p 2 = λ 2 = p 1 = λ 1 = p 2 = λ 2 = p 1 = λ 1 = p 2 = λ 2 = p 3 = λ 3 = p 1 = λ 1 = p 2 = λ 2 = p 1 = λ 1 = p 2 = λ 2 = p 1 = λ 1 = p 2 = λ 2 = p 3 = λ 3 = B. Javadi (INRIA) Statistical Models of Availability MASCOTS / 34

A quantitative Comparison of Checkpoint with Restart and Replication in Volatile Environments

A quantitative Comparison of Checkpoint with Restart and Replication in Volatile Environments A quantitative Comparison of Checkpoint with Restart and Replication in Volatile Environments Rong Zheng and Jaspal Subhlok Houston, TX 774 E-mail: rzheng@cs.uh.edu Houston, TX, 774, USA http://www.cs.uh.edu

More information

Ismaila Ba MSc Student, Department of Mathematics and Statistics Université de Moncton

Ismaila Ba MSc Student, Department of Mathematics and Statistics Université de Moncton Discrimination between statistical distributions for hydrometeorological frequency modeling Ismaila Ba MSc Student, Department of Mathematics and Statistics Université de Moncton INTRODUCTION The identification

More information

Towards Real-Time Volunteer Distributed Computing

Towards Real-Time Volunteer Distributed Computing Towards Real-Time Volunteer Distributed Computing Sangho Yi 1, Emmanuel Jeannot 2, Derrick Kondo 1, David P. Anderson 3 1 INRIA MESCAL, 2 RUNTIME, France 3 UC Berkeley, USA Motivation Push towards large-scale,

More information

Performance Tuning of Failure Detectors in Wireless Ad-Hoc Networks: Modelling and Experiments

Performance Tuning of Failure Detectors in Wireless Ad-Hoc Networks: Modelling and Experiments Performance Tuning of Failure Detectors in Wireless Ad-Hoc Networks: Modelling and Experiments {Corine.Marchand,Jean-Marc.Vincent}@imag.fr Laboratoire ID-IMAG (UMR 5132), Projet Apache. MIRRA Project:

More information

Modulation Classification based on Modified Kolmogorov-Smirnov Test

Modulation Classification based on Modified Kolmogorov-Smirnov Test Modulation Classification based on Modified Kolmogorov-Smirnov Test Ali Waqar Azim, Syed Safwan Khalid, Shafayat Abrar ENSIMAG, Institut Polytechnique de Grenoble, 38406, Grenoble, France Email: ali-waqar.azim@ensimag.grenoble-inp.fr

More information

Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management

Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management Energy-Efficient Gaming on Mobile Devices using Dead Reckoning-based Power Management R. Cameron Harvey, Ahmed Hamza, Cong Ly, Mohamed Hefeeda Network Systems Laboratory Simon Fraser University November

More information

Fitting Probability Distribution Curves to Reliability Data

Fitting Probability Distribution Curves to Reliability Data AF Fitting Probability Distribution Curves to Reliability Data TransGrid Fitting probability distribution curves to reliability data 31 March 2014 Document information Client: TransGrid Title: Fitting

More information

What is volunteer computing? opportunity for any computer owner to contribute to research in energy, climate change, medicine, astronomy, mathematics

What is volunteer computing? opportunity for any computer owner to contribute to research in energy, climate change, medicine, astronomy, mathematics Volunteer computing Sept. 1, 2017 Kate Cowles 374 SH, 335-0727 kate-cowles@stat.uiowa.edu What is volunteer computing? opportunity for any computer owner to contribute to research in energy, climate change,

More information

UNDERWATER ACOUSTIC CHANNEL ESTIMATION AND ANALYSIS

UNDERWATER ACOUSTIC CHANNEL ESTIMATION AND ANALYSIS Proceedings of the 5th Annual ISC Research Symposium ISCRS 2011 April 7, 2011, Rolla, Missouri UNDERWATER ACOUSTIC CHANNEL ESTIMATION AND ANALYSIS Jesse Cross Missouri University of Science and Technology

More information

Discrete Event Simulation

Discrete Event Simulation Discrete Event Simulation Master 2R SL module MD Jean-Marc Vincent and Arnaud Legrand Laboratory ID-IMAG MESCAL Project Universities of Grenoble {Jean-Marc.Vincent,Arnaud.Legrand}@imag.fr February 2, 2007

More information

Modelling of Real Network Traffic by Phase-Type distribution

Modelling of Real Network Traffic by Phase-Type distribution Modelling of Real Network Traffic by Phase-Type distribution Andriy Panchenko Dresden University of Technology 27-28.Juli.2004 4. Würzburger Workshop "IP Netzmanagement, IP Netzplanung und Optimierung"

More information

This is an author-deposited version published in : Eprints ID : 18274

This is an author-deposited version published in :  Eprints ID : 18274 Open Archive TOULOUSE Archive Ouverte (OATAO) OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. This is an author-deposited

More information

Department of Statistics and Operations Research Undergraduate Programmes

Department of Statistics and Operations Research Undergraduate Programmes Department of Statistics and Operations Research Undergraduate Programmes OPERATIONS RESEARCH YEAR LEVEL 2 INTRODUCTION TO LINEAR PROGRAMMING SSOA021 Linear Programming Model: Formulation of an LP model;

More information

Downlink Erlang Capacity of Cellular OFDMA

Downlink Erlang Capacity of Cellular OFDMA Downlink Erlang Capacity of Cellular OFDMA Gauri Joshi, Harshad Maral, Abhay Karandikar Department of Electrical Engineering Indian Institute of Technology Bombay Powai, Mumbai, India 400076. Email: gaurijoshi@iitb.ac.in,

More information

Optimizing Media Access Strategy for Competing Cognitive Radio Networks Y. Gwon, S. Dastangoo, H. T. Kung

Optimizing Media Access Strategy for Competing Cognitive Radio Networks Y. Gwon, S. Dastangoo, H. T. Kung Optimizing Media Access Strategy for Competing Cognitive Radio Networks Y. Gwon, S. Dastangoo, H. T. Kung December 12, 2013 Presented at IEEE GLOBECOM 2013, Atlanta, GA Outline Introduction Competing Cognitive

More information

Project: IEEE P Working Group for Wireless Personal Area Networks N

Project: IEEE P Working Group for Wireless Personal Area Networks N Project: IEEE P82.15 Working Group for Wireless Personal Area Networks N (WPANs( WPANs) Title: [UWB Channel Model for Indoor Residential Environment] Date Submitted: [2 September, 24] Source: [Chia-Chin

More information

Genbby Technical Paper

Genbby Technical Paper Genbby Team January 24, 2018 Genbby Technical Paper Rating System and Matchmaking 1. Introduction The rating system estimates the level of players skills involved in the game. This allows the teams to

More information

MODELING AND CHARACTERIZATION OF TRAFFIC IN A PUBLIC SAFETY WIRELESS NETWORK

MODELING AND CHARACTERIZATION OF TRAFFIC IN A PUBLIC SAFETY WIRELESS NETWORK MODELING AND CHARACTERIZATION OF TRAFFIC IN A PUBLIC SAFETY WIRELESS NETWORK Božidar Vujičić bvujicic@cs.sfu.ca Communication Networks Laboratory http://www.ensc.sfu.ca/cnl Simon Fraser University Vancouver,

More information

Energy Efficient Soft Real-Time Computing through Cross-Layer Predictive Control

Energy Efficient Soft Real-Time Computing through Cross-Layer Predictive Control Energy Efficient Soft Real-Time Computing through Cross-Layer Predictive Control Guangyi Cao and Arun Ravindran Department of Electrical and Computer Engineering University of North Carolina at Charlotte

More information

Dynamic Data-Driven Adaptive Sampling and Monitoring of Big Spatial-Temporal Data Streams for Real-Time Solar Flare Detection

Dynamic Data-Driven Adaptive Sampling and Monitoring of Big Spatial-Temporal Data Streams for Real-Time Solar Flare Detection Dynamic Data-Driven Adaptive Sampling and Monitoring of Big Spatial-Temporal Data Streams for Real-Time Solar Flare Detection Dr. Kaibo Liu Department of Industrial and Systems Engineering University of

More information

Advanced Techniques for Mobile Robotics Location-Based Activity Recognition

Advanced Techniques for Mobile Robotics Location-Based Activity Recognition Advanced Techniques for Mobile Robotics Location-Based Activity Recognition Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz Activity Recognition Based on L. Liao, D. J. Patterson, D. Fox,

More information

An interval-based target tracking approach for range-only multistatic radar

An interval-based target tracking approach for range-only multistatic radar An interval-based target tracking approach for range-only multistatic radar G.L. Soares, A. Arnold-Bos, L. Jaulin, J.A. Vasconcelos and C.A. Maia UFMG - Belo Horizonte-MG, CEP 31270-010, Brazil ENSIETA

More information

Cherry Picking: Exploiting Process Variations in the Dark Silicon Era

Cherry Picking: Exploiting Process Variations in the Dark Silicon Era Cherry Picking: Exploiting Process Variations in the Dark Silicon Era Siddharth Garg University of Waterloo Co-authors: Bharathwaj Raghunathan, Yatish Turakhia and Diana Marculescu # Transistors Power/Dark

More information

Modeling and Characterization of Traffic in Public Safety Wireless Networks

Modeling and Characterization of Traffic in Public Safety Wireless Networks Modeling and Characterization of Traffic in Public Safety Wireless Networks Božidar Vujičić, Nikola Cackov, Svetlana Vujičić, and Ljiljana Trajković * Simon Fraser University Vancouver, BC, Canada {bvujicic,

More information

Some Parameter Estimators in the Generalized Pareto Model and their Inconsistency with Observed Data

Some Parameter Estimators in the Generalized Pareto Model and their Inconsistency with Observed Data Some Parameter Estimators in the Generalized Pareto Model and their Inconsistency with Observed Data F. Ashkar, 1 and C. N. Tatsambon 2 1 Department of Mathematics and Statistics, Université de Moncton,

More information

Towards Instantaneous Collision and Interference Detection using In-Band Full Duplex

Towards Instantaneous Collision and Interference Detection using In-Band Full Duplex Towards Instantaneous Collision and Interference Detection using In-Band Full Duplex Tom Vermeulen, Mihir Laghate, Ghaith Hattab, Danijela Cabric, and Sofie Pollin Department of Electrical Engineering,

More information

Quantitative Assessment of the Individuality of Friction Ridge Patterns

Quantitative Assessment of the Individuality of Friction Ridge Patterns Quantitative Assessment of the Individuality of Friction Ridge Patterns Sargur N. Srihari with H. Srinivasan, G. Fang, P. Phatak, V. Krishnaswamy Department of Computer Science and Engineering University

More information

Target detection in side-scan sonar images: expert fusion reduces false alarms

Target detection in side-scan sonar images: expert fusion reduces false alarms Target detection in side-scan sonar images: expert fusion reduces false alarms Nicola Neretti, Nathan Intrator and Quyen Huynh Abstract We integrate several key components of a pattern recognition system

More information

Analysis of the electrical disturbances in CERN power distribution network with pattern mining methods

Analysis of the electrical disturbances in CERN power distribution network with pattern mining methods OLEKSII ABRAMENKO, CERN SUMMER STUDENT REPORT 2017 1 Analysis of the electrical disturbances in CERN power distribution network with pattern mining methods Oleksii Abramenko, Aalto University, Department

More information

Quantitative Analysis of Tone Value Reproduction Limits

Quantitative Analysis of Tone Value Reproduction Limits Robert Chung* and Ping-hsu Chen* Keywords: Standard, Tonality, Highlight, Shadow, E* ab Abstract ISO 12647-2 (2004) defines tone value reproduction limits requirement as, half-tone dot patterns within

More information

Volume 29, Issue 4. Detrending and the Distributional Properties of U.S. Output Time Series

Volume 29, Issue 4. Detrending and the Distributional Properties of U.S. Output Time Series Volume 9, Issue 4 Detrending and the Distributional Properties of U.S. Output Time Series Giorgio Fagiolo Sant''Anna School of Advanced Studies, Pisa (Italy). Marco Piazza Sant''Anna School of Advanced

More information

P. Mohana Shankar. Fading and Shadowing. in Wireless Systems. ^ Springer

P. Mohana Shankar. Fading and Shadowing. in Wireless Systems. ^ Springer P. Mohana Shankar Fading and Shadowing in Wireless Systems ^ Springer Contents 1 Overview 1 1.1 Outline 1 References 5 2 Concepts of Probability and Statistics 7 2.1 Introduction 7 2.2 Random Variables,

More information

BENFORD S LAW, FAMILIES OF DISTRIBUTIONS AND A TEST BASIS. This Draft: October 9, 2010 First Draft: August 6, 2006

BENFORD S LAW, FAMILIES OF DISTRIBUTIONS AND A TEST BASIS. This Draft: October 9, 2010 First Draft: August 6, 2006 BENFORD S LAW, FAMILIES OF DISTRIBUTIONS AND A TEST BASIS JOHN MORROW This Draft: October 9, 2010 First Draft: August 6, 2006 Abstract. The distribution of first significant digits known as Benford s Law

More information

Growing an Organic Indoor Location System

Growing an Organic Indoor Location System Growing an Organic Indoor Location System Jun-geun Park MIT CSAIL Joint work with: Ben Charrow (MIT), Dorothy Curtis (MIT), Jonathan Battat (MIT), Einat Minkov (Nokia, Univ. of Haifa), Jamey Hicks (Nokia),

More information

Replacement, Maintenance, and Reliability. Theory and Applications SECOND EDITION. (cfc. Andrew K.S. Jardine Albert H.C. Tsang.

Replacement, Maintenance, and Reliability. Theory and Applications SECOND EDITION. (cfc. Andrew K.S. Jardine Albert H.C. Tsang. Maintenance, Replacement, and Reliability Theory and Applications SECOND EDITION Andrew K.S. Jardine Albert H.C. Tsang (cfc CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint

More information

Detection of Compound Structures in Very High Spatial Resolution Images

Detection of Compound Structures in Very High Spatial Resolution Images Detection of Compound Structures in Very High Spatial Resolution Images Selim Aksoy Department of Computer Engineering Bilkent University Bilkent, 06800, Ankara, Turkey saksoy@cs.bilkent.edu.tr Joint work

More information

Analyzing Data Properties using Statistical Sampling Techniques

Analyzing Data Properties using Statistical Sampling Techniques Analyzing Data Properties using Statistical Sampling Techniques Illustrated on Scientific File Formats and Compression Features Julian M. Kunkel kunkel@dkrz.de 2016-06-21 Outline 1 Introduction 2 Exploring

More information

Fault Location Using Sparse Wide Area Measurements

Fault Location Using Sparse Wide Area Measurements 319 Study Committee B5 Colloquium October 19-24, 2009 Jeju Island, Korea Fault Location Using Sparse Wide Area Measurements KEZUNOVIC, M., DUTTA, P. (Texas A & M University, USA) Summary Transmission line

More information

Project summary. Key findings, Winter: Key findings, Spring:

Project summary. Key findings, Winter: Key findings, Spring: Summary report: Assessing Rusty Blackbird habitat suitability on wintering grounds and during spring migration using a large citizen-science dataset Brian S. Evans Smithsonian Migratory Bird Center October

More information

Statistical tests. Paired t-test

Statistical tests. Paired t-test Statistical tests Gather data to assess some hypothesis (e.g., does this treatment have an effect on this outcome?) Form a test statistic for which large values indicate a departure from the hypothesis.

More information

CS445: Modeling Complex Systems

CS445: Modeling Complex Systems CS445: Modeling Complex Systems Travis Desell! Averill M. Law, Simulation Modeling & Analysis, Chapter 2!! Time-Shared Computer Model Time Shared Computer Model Terminals Computer Unfinished s 2 2... Active

More information

Optimization Techniques for Alphabet-Constrained Signal Design

Optimization Techniques for Alphabet-Constrained Signal Design Optimization Techniques for Alphabet-Constrained Signal Design Mojtaba Soltanalian Department of Electrical Engineering California Institute of Technology Stanford EE- ISL Mar. 2015 Optimization Techniques

More information

A Review of Related Work on Machine Learning in Semiconductor Manufacturing and Assembly Lines

A Review of Related Work on Machine Learning in Semiconductor Manufacturing and Assembly Lines A Review of Related Work on Machine Learning in Semiconductor Manufacturing and Assembly Lines DI Darko Stanisavljevic VIRTUAL VEHICLE DI Michael Spitzer VIRTUAL VEHICLE i-know 16 18.-19.10.2016, Graz

More information

FIFO WITH OFFSETS HIGH SCHEDULABILITY WITH LOW OVERHEADS. RTAS 18 April 13, Björn Brandenburg

FIFO WITH OFFSETS HIGH SCHEDULABILITY WITH LOW OVERHEADS. RTAS 18 April 13, Björn Brandenburg FIFO WITH OFFSETS HIGH SCHEDULABILITY WITH LOW OVERHEADS RTAS 18 April 13, 2018 Mitra Nasri Rob Davis Björn Brandenburg FIFO SCHEDULING First-In-First-Out (FIFO) scheduling extremely simple very low overheads

More information

Mason Chen (Black Belt) Morrill Learning Center, San Jose, CA

Mason Chen (Black Belt) Morrill Learning Center, San Jose, CA Poster ID 12 Google Robot Mason Chen (Black Belt) Morrill Learning Center, San Jose, CA D1 Observations and Research Google Cars stop at the red light and speed up at green light how & why Google Car can

More information

Objectives. Module 6: Sampling

Objectives. Module 6: Sampling Module 6: Sampling 2007. The World Bank Group. All rights reserved. Objectives This session will address - why we use sampling - how sampling can create efficiencies for data collection - sampling techniques,

More information

Possible responses to the 2015 AP Statistics Free Resposne questions, Draft #2. You can access the questions here at AP Central.

Possible responses to the 2015 AP Statistics Free Resposne questions, Draft #2. You can access the questions here at AP Central. Possible responses to the 2015 AP Statistics Free Resposne questions, Draft #2. You can access the questions here at AP Central. Note: I construct these as a service for both students and teachers to start

More information

ΕΠΛ 605: Προχωρημένη Αρχιτεκτονική

ΕΠΛ 605: Προχωρημένη Αρχιτεκτονική ΕΠΛ 605: Προχωρημένη Αρχιτεκτονική Υπολογιστών Presentation of UniServer Horizon 2020 European project findings: X-Gene server chips, voltage-noise characterization, high-bandwidth voltage measurements,

More information

Estimation of speed, average received power and received signal in wireless systems using wavelets

Estimation of speed, average received power and received signal in wireless systems using wavelets Estimation of speed, average received power and received signal in wireless systems using wavelets Rajat Bansal Sumit Laad Group Members rajat@ee.iitb.ac.in laad@ee.iitb.ac.in 01D07010 01D07011 Abstract

More information

Signal segmentation and waveform characterization. Biosignal processing, S Autumn 2012

Signal segmentation and waveform characterization. Biosignal processing, S Autumn 2012 Signal segmentation and waveform characterization Biosignal processing, 5173S Autumn 01 Short-time analysis of signals Signal statistics may vary in time: nonstationary how to compute signal characterizations?

More information

Frugal Sensing Spectral Analysis from Power Inequalities

Frugal Sensing Spectral Analysis from Power Inequalities Frugal Sensing Spectral Analysis from Power Inequalities Nikos Sidiropoulos Joint work with Omar Mehanna IEEE SPAWC 2013 Plenary, June 17, 2013, Darmstadt, Germany Wideband Spectrum Sensing (for CR/DSM)

More information

Scheduling. Radek Mařík. April 28, 2015 FEE CTU, K Radek Mařík Scheduling April 28, / 48

Scheduling. Radek Mařík. April 28, 2015 FEE CTU, K Radek Mařík Scheduling April 28, / 48 Scheduling Radek Mařík FEE CTU, K13132 April 28, 2015 Radek Mařík (marikr@fel.cvut.cz) Scheduling April 28, 2015 1 / 48 Outline 1 Introduction to Scheduling Methodology Overview 2 Classification of Scheduling

More information

Empirical Study on Route-Length Efficiency of Road Networks - Spring 2012 VIGRE Project Report

Empirical Study on Route-Length Efficiency of Road Networks - Spring 2012 VIGRE Project Report Empirical Study on Route-Length Efficiency of Road Networks - Spring 2012 VIGRE Project Report Karthik Ganesan May 11, 2012 I Introduction What does it mean to say that a physical network (such as a road

More information

Appendix III Graphs in the Introductory Physics Laboratory

Appendix III Graphs in the Introductory Physics Laboratory Appendix III Graphs in the Introductory Physics Laboratory 1. Introduction One of the purposes of the introductory physics laboratory is to train the student in the presentation and analysis of experimental

More information

Performance evaluation of several adaptive speckle filters for SAR imaging. Markus Robertus de Leeuw 1 Luis Marcelo Tavares de Carvalho 2

Performance evaluation of several adaptive speckle filters for SAR imaging. Markus Robertus de Leeuw 1 Luis Marcelo Tavares de Carvalho 2 Performance evaluation of several adaptive speckle filters for SAR imaging Markus Robertus de Leeuw 1 Luis Marcelo Tavares de Carvalho 2 1 Utrecht University UU Department Physical Geography Postbus 80125

More information

Correspondence should be addressed to Olga E. Segou;

Correspondence should be addressed to Olga E. Segou; International Journal of Distributed Sensor Networks, Article ID 34, 8 pages http://dx.doi.org/.55/24/34 Research Article Probabilistic Error Modeling and Topology-Based Smoothing of Indoor Localization

More information

Partitions and Permutations

Partitions and Permutations Chapter 5 Partitions and Permutations 5.1 Stirling Subset Numbers 5.2 Stirling Cycle Numbers 5.3 Inversions and Ascents 5.4 Derangements 5.5 Exponential Generating Functions 5.6 Posets and Lattices 1 2

More information

User Research in Fractal Spaces:

User Research in Fractal Spaces: User Research in Fractal Spaces: Behavioral analytics: Profiling users and informing game design Collaboration with national and international researchers & companies Behavior prediction and monetization:

More information

Power Control Optimization of Code Division Multiple Access (CDMA) Systems Using the Knowledge of Battery Capacity Of the Mobile.

Power Control Optimization of Code Division Multiple Access (CDMA) Systems Using the Knowledge of Battery Capacity Of the Mobile. Power Control Optimization of Code Division Multiple Access (CDMA) Systems Using the Knowledge of Battery Capacity Of the Mobile. Rojalin Mishra * Department of Electronics & Communication Engg, OEC,Bhubaneswar,Odisha

More information

INFORMATION AND COMPUTATION HIERARCHY

INFORMATION AND COMPUTATION HIERARCHY INFORMATION AND COMPUTATION HIERARCHY Lang Tong School of Electrical and Computer Engineering Cornell University, Ithaca, NY Acknowledgement: K. Birman, P. Varaiya, T. Mount, R. Thomas, S. Avestimehr,

More information

Design as You See FIT: System-Level Soft Error Analysis of Sequential Circuits

Design as You See FIT: System-Level Soft Error Analysis of Sequential Circuits Design as You See FIT: System-Level Soft Error Analysis of Sequential Circuits Dan Holcomb Wenchao Li Sanjit A. Seshia Department of EECS University of California, Berkeley Design Automation and Test in

More information

Admin Deblurring & Deconvolution Different types of blur

Admin Deblurring & Deconvolution Different types of blur Admin Assignment 3 due Deblurring & Deconvolution Lecture 10 Last lecture Move to Friday? Projects Come and see me Different types of blur Camera shake User moving hands Scene motion Objects in the scene

More information

Control issues in cognitive networks. Marko Höyhtyä and Tao Chen CWC-VTT-Gigaseminar 4th December 2008

Control issues in cognitive networks. Marko Höyhtyä and Tao Chen CWC-VTT-Gigaseminar 4th December 2008 Control issues in cognitive networks Marko Höyhtyä and Tao Chen CWC-VTT-Gigaseminar 4th December 2008 Outline Cognitive wireless networks Cognitive mesh Topology control Frequency selection Power control

More information

Detrending and the Distributional Properties of U.S. Output Time Series

Detrending and the Distributional Properties of U.S. Output Time Series Detrending and the Distributional Properties of U.S. Output Time Series Giorgio Fagiolo Mauro Napoletano Marco Piazza Andrea Roventini. Abstract We study the impact of alternative detrending techniques

More information

Fast Statistical Timing Analysis By Probabilistic Event Propagation

Fast Statistical Timing Analysis By Probabilistic Event Propagation Fast Statistical Timing Analysis By Probabilistic Event Propagation Jing-Jia Liou, Kwang-Ting Cheng, Sandip Kundu, and Angela Krstić Electrical and Computer Engineering Department, University of California,

More information

How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory

How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory Prev Sci (2007) 8:206 213 DOI 10.1007/s11121-007-0070-9 How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory John W. Graham & Allison E. Olchowski & Tamika

More information

Permutations with short monotone subsequences

Permutations with short monotone subsequences Permutations with short monotone subsequences Dan Romik Abstract We consider permutations of 1, 2,..., n 2 whose longest monotone subsequence is of length n and are therefore extremal for the Erdős-Szekeres

More information

Empirical Path Loss Models

Empirical Path Loss Models Empirical Path Loss Models 1 Free space and direct plus reflected path loss 2 Hata model 3 Lee model 4 Other models 5 Examples Levis, Johnson, Teixeira (ESL/OSU) Radiowave Propagation August 17, 2018 1

More information

USING SIMPLE PID CONTROLLERS TO PREVENT AND MITIGATE FAULTS IN SCIENTIFIC WORKFLOWS

USING SIMPLE PID CONTROLLERS TO PREVENT AND MITIGATE FAULTS IN SCIENTIFIC WORKFLOWS USING SIMPLE PID CONTROLLERS TO PREVENT AND MITIGATE FAULTS IN SCIENTIFIC WORKFLOWS Rafael Ferreira da Silva 1, Rosa Filgueira 2, Ewa Deelman 1, Erola Pairo-Castineira 3, Ian Michael Overton 4, Malcolm

More information

Permutation inference for the General Linear Model

Permutation inference for the General Linear Model Permutation inference for the General Linear Model Anderson M. Winkler fmrib Analysis Group 3.Sep.25 Winkler Permutation for the glm / 63 in jalapeno: winkler/bin/palm Winkler Permutation for the glm 2

More information

DyMo: Dynamic Monitoring of Large Scale LTE-Multicast Systems

DyMo: Dynamic Monitoring of Large Scale LTE-Multicast Systems : Dynamic Monitoring of Large Scale LTE-Multicast Systems Yigal Bejerano, Chandrashekhar Raman, Chun-Nam Yu, Varun Gupta, Craig Gutterman, Tomas Young, Hugo Infante, Yousef Abdelmalek, and Gil Zussman

More information

Cognitive Radio Techniques

Cognitive Radio Techniques Cognitive Radio Techniques Spectrum Sensing, Interference Mitigation, and Localization Kandeepan Sithamparanathan Andrea Giorgetti ARTECH HOUSE BOSTON LONDON artechhouse.com Contents Preface xxi 1 Introduction

More information

DyMo: Dynamic Monitoring of Large Scale LTE-Multicast Systems

DyMo: Dynamic Monitoring of Large Scale LTE-Multicast Systems : Dynamic Monitoring of Large Scale LTE-Multicast Systems Yigal Bejerano, Chandru Raman, Chun-Nam Yu, Varun Gupta, Craig Gutterman, Tomas Young, Hugo Infante, Yousef Abdelmalek @ and Gil Zussman Bell Labs,

More information

Is Server Consolidation Beneficial to MMORPG? A Case Study of World of Warcraft Yan Ting Li, Kuan Ta Chen. IIS, Academia Sinica, Taiwan

Is Server Consolidation Beneficial to MMORPG? A Case Study of World of Warcraft Yan Ting Li, Kuan Ta Chen. IIS, Academia Sinica, Taiwan Is Server Consolidation Beneficial to MMORPG? A Case Study of World of Warcraft Yan Ting Li, Kuan Ta Chen MMORPG Massively Multiplayer Online Role Playing Game General property Agenre of computer role

More information

Statistical characterisation of the EM interferences acting on GSM-R antennas fixed above moving trains

Statistical characterisation of the EM interferences acting on GSM-R antennas fixed above moving trains Statistical characterisation of the EM interferences acting on GSM-R antennas fixed above moving trains N. Ben Slimen, V. Deniau, J. Rioult, S. Dudoyer, S. Baranowski To cite this version: N. Ben Slimen,

More information

Communication Theory in the Cloud: The Transformative Power of Cheap Utility Computing

Communication Theory in the Cloud: The Transformative Power of Cheap Utility Computing Communication Theory in the Cloud: The Transformative Power of Cheap Utility Computing Matthew C. Valenti West Virginia University Jan. 30, 2012 This work supported by the National Science Foundation under

More information

Multihop Routing in Ad Hoc Networks

Multihop Routing in Ad Hoc Networks Multihop Routing in Ad Hoc Networks Dr. D. Torrieri 1, S. Talarico 2 and Dr. M. C. Valenti 2 1 U.S Army Research Laboratory, Adelphi, MD 2 West Virginia University, Morgantown, WV Nov. 18 th, 20131 Outline

More information

Mining Phasor Data To Find The Hidden Gems In Your Archive

Mining Phasor Data To Find The Hidden Gems In Your Archive Electric Power Group Presents Phasor Data Mining Application PDMA Mining Phasor Data To Find The Hidden Gems In Your Archive October 16, 2014 Presented by Vivek Bhaman & Frank Carrera Webinar Phone Number:

More information

Edge-Raggedness Evaluation Using Slanted-Edge Analysis

Edge-Raggedness Evaluation Using Slanted-Edge Analysis Edge-Raggedness Evaluation Using Slanted-Edge Analysis Peter D. Burns Eastman Kodak Company, Rochester, NY USA 14650-1925 ABSTRACT The standard ISO 12233 method for the measurement of spatial frequency

More information

DISTRIBUTED DYNAMIC CHANNEL ALLOCATION ALGORITHM FOR CELLULAR MOBILE NETWORK

DISTRIBUTED DYNAMIC CHANNEL ALLOCATION ALGORITHM FOR CELLULAR MOBILE NETWORK DISTRIBUTED DYNAMIC CHANNEL ALLOCATION ALGORITHM FOR CELLULAR MOBILE NETWORK 1 Megha Gupta, 2 A.K. Sachan 1 Research scholar, Deptt. of computer Sc. & Engg. S.A.T.I. VIDISHA (M.P) INDIA. 2 Asst. professor,

More information

OFDM Pilot Optimization for the Communication and Localization Trade Off

OFDM Pilot Optimization for the Communication and Localization Trade Off SPCOMNAV Communications and Navigation OFDM Pilot Optimization for the Communication and Localization Trade Off A. Lee Swindlehurst Dept. of Electrical Engineering and Computer Science The Henry Samueli

More information

Workshop on anonymization Berlin, March 19, Basic Knowledge Terms, Definitions and general techniques. Murat Sariyar TMF

Workshop on anonymization Berlin, March 19, Basic Knowledge Terms, Definitions and general techniques. Murat Sariyar TMF Workshop on anonymization Berlin, March 19, 2015 Basic Knowledge Terms, Definitions and general techniques Murat Sariyar TMF Workshop Anonymisation, March 19, 2015 Outline Background Aims of Anonymization

More information

Path Planning with Fast Marching Methods

Path Planning with Fast Marching Methods Path Planning with Fast Marching Methods Ian Mitchell Department of Computer Science The University of British Columbia research supported by National Science and Engineering Research Council of Canada

More information

Phil Schniter and Jason Parker

Phil Schniter and Jason Parker Parametric Bilinear Generalized Approximate Message Passing Phil Schniter and Jason Parker With support from NSF CCF-28754 and an AFOSR Lab Task (under Dr. Arje Nachman). ITA Feb 6, 25 Approximate Message

More information

WL delft hydraulics. Extreme wave statistics. Rijkswaterstaat, Rijksinstituut voor Kust en Zee (RIKZ) February, 2007.

WL delft hydraulics. Extreme wave statistics. Rijkswaterstaat, Rijksinstituut voor Kust en Zee (RIKZ) February, 2007. Prepared for: Rijkswaterstaat, Rijksinstituut voor Kust en Zee (RIKZ) Extreme wave statistics Report February, 2007 H4803.30 WL delft hydraulics Prepared for: Rijkswaterstaat, Rijksinstituut voor Kust

More information

Closing the loop around Sensor Networks

Closing the loop around Sensor Networks Closing the loop around Sensor Networks Bruno Sinopoli Shankar Sastry Dept of Electrical Engineering, UC Berkeley Chess Review May 11, 2005 Berkeley, CA Conceptual Issues Given a certain wireless sensor

More information

SSB Debate: Model-based Inference vs. Machine Learning

SSB Debate: Model-based Inference vs. Machine Learning SSB Debate: Model-based nference vs. Machine Learning June 3, 2018 SSB 2018 June 3, 2018 1 / 20 Machine learning in the biological sciences SSB 2018 June 3, 2018 2 / 20 Machine learning in the biological

More information

TCP/IP COVERT TIMING CHANNEL: THEORY TO IMPLEMENTATION. Sarah H. Sellke, Chih-Chun Wang Saurabh Bagchi, and Ness B. Shroff

TCP/IP COVERT TIMING CHANNEL: THEORY TO IMPLEMENTATION. Sarah H. Sellke, Chih-Chun Wang Saurabh Bagchi, and Ness B. Shroff 1 TCP/IP COVERT TIMING CHANNEL: THEORY TO IMPLEMENTATION Sarah H. Sellke, Chih-Chun Wang Saurabh Bagchi, and Ness B. Shroff NETWORK COVERT TIMING CHANNELS Confidential Data 1 of RECENT WORK IP Covert Timing

More information

MAKING REMAINING LIFE PREDICTIONS FOR POWER CABLES USING RELIABILITY ANALYSES

MAKING REMAINING LIFE PREDICTIONS FOR POWER CABLES USING RELIABILITY ANALYSES MAKING REMAINING LIFE PREDICTIONS FOR POWER CABLES USING RELIABILITY ANALYSES Jey K. JEYAPALAN, Dr. Jeyapalan & Associates, (USA), jkjeyapalan@earthlink.net ABSTRACT Most underground cable owners would

More information

Achieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters

Achieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters Achieving Desirable Gameplay Objectives by Niched Evolution of Game Parameters Scott Watson, Andrew Vardy, Wolfgang Banzhaf Department of Computer Science Memorial University of Newfoundland St John s.

More information

Scaling Laws for Cognitive Radio Network with Heterogeneous Mobile Secondary Users

Scaling Laws for Cognitive Radio Network with Heterogeneous Mobile Secondary Users Scaling Laws for Cognitive Radio Network with Heterogeneous Mobile Secondary Users Y.Li, X.Wang, X.Tian and X.Liu Shanghai Jiaotong University Scaling Laws for Cognitive Radio Network with Heterogeneous

More information

Chapter 12. Cross-Layer Optimization for Multi- Hop Cognitive Radio Networks

Chapter 12. Cross-Layer Optimization for Multi- Hop Cognitive Radio Networks Chapter 12 Cross-Layer Optimization for Multi- Hop Cognitive Radio Networks 1 Outline CR network (CRN) properties Mathematical models at multiple layers Case study 2 Traditional Radio vs CR Traditional

More information

Distributed Power Control in Cellular and Wireless Networks - A Comparative Study

Distributed Power Control in Cellular and Wireless Networks - A Comparative Study Distributed Power Control in Cellular and Wireless Networks - A Comparative Study Vijay Raman, ECE, UIUC 1 Why power control? Interference in communication systems restrains system capacity In cellular

More information

Data processing framework for decision making

Data processing framework for decision making Data processing framework for decision making Jan Larsen Intelligent Signal Processing Group Department of Informatics and Mathematical Modelling Technical University of Denmark jl@imm.dtu.dk, www.imm.dtu.dk/~jl

More information

NASA Swarmathon Team ABC (Artificial Bee Colony)

NASA Swarmathon Team ABC (Artificial Bee Colony) NASA Swarmathon Team ABC (Artificial Bee Colony) Cheylianie Rivera Maldonado, Kevin Rolón Domena, José Peña Pérez, Aníbal Robles, Jonathan Oquendo, Javier Olmo Martínez University of Puerto Rico at Arecibo

More information

INTELLIGENT SOFTWARE QUALITY MODEL: THE THEORETICAL FRAMEWORK

INTELLIGENT SOFTWARE QUALITY MODEL: THE THEORETICAL FRAMEWORK INTELLIGENT SOFTWARE QUALITY MODEL: THE THEORETICAL FRAMEWORK Jamaiah Yahaya 1, Aziz Deraman 2, Siti Sakira Kamaruddin 3, Ruzita Ahmad 4 1 Universiti Utara Malaysia, Malaysia, jamaiah@uum.edu.my 2 Universiti

More information

Advancing Industry Productivity

Advancing Industry Productivity Advancing Industry Productivity Iddo Hadar Joint Productivity Working Group Session Austin, Texas Thursday, October 12, 2006 F O U N D A T I O N E N G I N E E R I N G G R O U P Safe Harbor Statement This

More information

Demand for Commitment in Online Gaming: A Large-Scale Field Experiment

Demand for Commitment in Online Gaming: A Large-Scale Field Experiment Demand for Commitment in Online Gaming: A Large-Scale Field Experiment Vinci Y.C. Chow and Dan Acland University of California, Berkeley April 15th 2011 1 Introduction Video gaming is now the leisure activity

More information

Towards Application Driven Sensor Network Control. Nael Abu-Ghazaleh SUNY Binghamton

Towards Application Driven Sensor Network Control. Nael Abu-Ghazaleh SUNY Binghamton Towards Application Driven Sensor Network Control Nael Abu-Ghazaleh SUNY Binghamton nael@cs.binghamton.edu Scenario Observer wants to observe something about the phenomenon Track all the lions in this

More information

Chapter 4 SPEECH ENHANCEMENT

Chapter 4 SPEECH ENHANCEMENT 44 Chapter 4 SPEECH ENHANCEMENT 4.1 INTRODUCTION: Enhancement is defined as improvement in the value or Quality of something. Speech enhancement is defined as the improvement in intelligibility and/or

More information