Non-preemptive Coflow Scheduling and Routing

Transcription

1 IEEE Globecom 2016 SAC-ANS 3 Non-preemptive Coflow Scheduling and Routing Ruozhou Yu, Guoliang Xue, and Xiang Zhang Arizona State University Jian Tang Syracuse University 1/22

2 Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 2/22

3 Flows and Coflows q OMG where s my last piece??? Need it now!!! Cloud Task Data piece 1 Data piece 6 Data piece 2 Data piece 5 Data piece 3 Data piece 4 3/22

4 Flows and Coflows q Traditional network scheduling/routing solution v Scheduling/Routing regarding individual flows vgeneral flow: a subset of packet header fields v Fails to account for application-level performance metrics vflow completion time vs. task completion time 4/22

5 Flows and Coflows Task 1 Task 2 You faster, you go ahead J Oh you re so nice!! 5/22

6 Flows and Coflows q Application-aware scheduling/routing: coflows v Flows grouped by application/task v A coflow finishes when all its component flows finish v Advantages: v Captures application-level requirement v Establishes fairness in application-level v Want to do it in a centralized way vnot to leak app privacy to other apps v Or to prevent apps from selfishly congest the network 6/22

7 (Non-)Preemptive Scheduling q Existing coflow scheduling/routing allows preemption! v Pause for the shorter ones! v Advantages: v Better performance and network utilization in theory v Disadvantages: vlarge overhead for flow switching: performance issue for short flows q Switching delays q Switch computations vno ready support in commodity hardware q Standardization on-going: IEEE 802.1Qbu q A long way before commercial-ready q Our stand: non-preemptive scheduling + routing of coflows 7/22

8 Summary of Problem Task 1 Task 2 Now, you go first, this way! Sorry, there s no place. You fired! BOSS You next, that way! Task 4 You this way, free to go! Task 3 8/22

9 Contributions q A first (preliminary) study for Non-preemptive Coflow Scheduling and Routing (NCSR) q An offline scheduling framework: Shortest-Coflow First q A multi-path routing algorithm q A single-path routing algorithm q Performance evaluations 9/22

10 Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 10/22

11 q Network: G = (V, E) System Model q Coflow requests: S = {C 1,, C m } v Each request: C i = {F i,1,, F i,ni } vf i,j = (s i,j, t i,j, d i,j ): source, destination, flow size (demand, in bytes) q Bandwidth allocation vb p i,j(t): bandwidth allocation on path p of flow i, j, at time t vb i,j (t) = sum of bandwidth over all paths at time t 11/22

12 q Flow/coflow completion time v Flow completion time (FCT): System Model v Coflow completion time (CCT): max. FCT of its component flows v Objective: minimize total CCT 12/22

13 Shortest-Coflow First Scheduler q For each coflow: v Compute per-coflow completion time (CCT) v If multi-path enabled, compute using multi-path routing v Otherwise, use single-path routing q Schedule coflows in ascending order of CCT 13/22

14 CCT with Multi-path Routing q Non-linear programming formulation v Sharing among flows within the coflow v CCT as the maximum FCT of component flows q Linearization: let f i = 1 / T i 14/22

15 CCT with Single-path Routing q Additional integer variables to the Multi-path Routing model vx e i,j: link selection for single-path routing q Linear relaxation and deterministic rounding v Relax x e i,j to take continuous values, and solve linear program; v For each flow, find path with maximum minimum x values, and assign; v Re-solve program to obtain bandwidth allocation with fixed path assignments 15/22

16 Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 16/22

17 q Waxman random graphs v 50 nodes v Alpha=0.15, beta=0.2 v Link capacities: [10, 100] Mbps Simulation Setups q Coflows v 25 requests v 1 to 10 flows per request v Flow sizes: [10, 100] Mbps q Comparison: v sscf, mscf: single-path and multi-path SCF algorithm (proposed) v srt, mrt: single-path and multi-path Routing-only algorithm (baseline) v ssff, msff: single-path and multi-path Shortest-Flow First (baseline) 17/22

18 Simulation Results: Average CCT 18/22

19 Simulation Results: Running Time 19/22

20 Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 20/22

21 Conclusions q A first step study on NCSR v Offline optimization model v SCF scheduler for scheduling v Multi-path and single-path routing algorithms q Experiment results v Scheduling more effective than routing: when network congested v Application-awareness brings great advantage q Future work v Enable better sharing/work conservation of resources v Remove the non-sharing rule of coflows 21/22

22 Q&A? THANK YOU VERY MUCH! 22/22