Making sure metrics are meaningful

Transcription

1 Makig sure metrics are meaigful Some thigs are quatifiable, but ot very useful CPU performace: MHz is ot the same as performace Cameras: Mega-Pixels is ot the same as quality Cosistet ad quatifiable metrics provide ope competitio Suppliers of systems always wat to use the best metrics Metrics should be defied by users, ot suppliers People will optimise for metrics (it s what they are for!) Poor metrics lead to poor desig ad optimisatio Part of the specificatio problem HPCE / dt10 / 2015 / 11.1

2 Feasibility: Learig to say o HPCE / dt10 / 2015 / 11.2

3 Feasibility studies People come up with demads I wat real-time spectral aalysis of a 0hz-1GHz sigal We must process HD video withi a latecy of 1ms This base-statio must beam-form 32 chaels Is it feasible to meet those demads? Will it be easy? Will it require optimisatio? Will it require a specialised platform? Is it fudametally impossible? You eed some estimates before you start implemetatio Executio time is the most basic check to be made HPCE / dt10 / 2015 / 11.3

4 Circuit Placemet 1 p 1 p 2 p p 4 p 5 p 6 5 p 7 p 8 p 9 Logical compoets i circuit Physical resources i device Take the graph of circuit, ad fid a valid placemet oto physical resources Make sure that all logical compoets have a uique physical locatio Make sure that all logical coectios map to a physical chael HPCE / dt10 / 2015 / 11.4

5 Circuit Placemet HPCE / dt10 / 2015 / 11.5

6 How difficult is a big problem? HPCE / dt10 / 2015 / 11.6

7 Tryig to measure complexity Wat to capture complexity of a task usig equatios Time complexity: how may steps does it require Space complexity: how much storage does it require We could derive exact equatios for each How may istructios does the task take i total? How may bytes of memory are allocated durig executio Meas we have to worry about lots of details Laguage: did you use C++, Fortra, VHDL? Compiler: what optimisatio flags were used? Architecture: are itegers 32-bit or 64-bit? Exact equatios are sometimes possible, but ofte impractical HPCE / dt10 / 2015 / 11.7

8 Complexity ad Big-O Let s assume the existece of a fuctio g() g() specifies exactly how may steps are take Note: this fuctio does t eed to be stated explicitly! is the size of the problem; e.g. a iput vector of legth Goal: fid a simple fuctio f(), such that g() ϵ O(f()) g( ) O( f ( )) iff 0, m 0: : g( ) m f ( ) c c There must exist a critical value c, ad a positive costat m, such that for all > c the relatio g() m f() holds For icreasig, evetually you ll reach a poit where f() times a costat is always bigger tha g() HPCE / dt10 / 2015 / 11.8

9 Complexity ad Big-O O(f()) is a set of fuctios with the same or lower complexity O() 2 O() O( 2 ) 2 O( 2 ) O() O( 2 ) O( 3 ) It is sort of correct to claim that everythig is O( ) But it s really ot very useful... Try to fid the smallest complexity class cotaiig a fuctio O( 2 ) O( 3 ) O(2 ) Fid the fastest growig compoet, ad choose that HPCE / dt10 / 2015 / 11.9

10 Reductio Rules fuctio F(:iteger) : iteger begi for i = 0../2 acc = acc + iit(i) for j = 64../3 tmp = tmp + fuc(j,acc) ext j ext i retur tmp Executes /2 times: O() Executes (/3-64)*/2 = 2 / 6 32 O( 2 ) O(a g()) O(g()), a is idepet of O(a + g()) O(g()) O( (a ) 2 ) O( 2 ) HPCE / dt10 / 2015 / 11.10

11 Commo complexity classes it SUM(it N) { it acc=0; for(it i=0;i<n;i++) acc=acc+d[i]; retur acc; } fuctio [R]=radMMM(N) A=rad(N,N); B=rad(N,N); R=A*B; it Ack(it N) { it A(it m, it ) { if(m==0) retur +1; if(==0) retur A(m-1,1); retur A(m-1,A(m,-1)); } } fuctio [R]=radFFT(N) A=rad(N,1); B=rad(N,1); R=fft(A,B); retur A(N,N); HPCE / dt10 / 2015 / 11.11

12 Time it SUM(it N) { it acc=0; for(it i=0;i<n;i++) acc=acc+d[i]; retur acc; } 5.E+00 4.E+00 4.E+00 SUM 3.E+00 3.E+00 2.E+00 2.E+00 1.E+00 5.E-01 0.E+00 0.E+00 1.E+07 2.E+07 3.E+07 4.E+07 5.E+07 6.E+07 7.E+07 8.E+07 HPCE / dt10 / 2015 / 11.12

13 Matrix-Matrix Multiply r 1,1 r 1,2 r 1, a 1,1 a 1,2 a 1, b 1,1 b 1,2 b 1, r 2,1 r 2,2 r 2, a 2,1 a 2,2 a 2, = * b 2,1 b 2,2 b 2, r,1 r,2 r, a,1 a,2 a, b,1 b,2 b, fuctio[r]=radmmm(n) A=rad(N,N); B=rad(N,N); R=A*B; fuctio [R]=Mul(A,B) for r=1: for c=1: R(r,c)=sum(A(r,:).*B(:,c)); HPCE / dt10 / 2015 / 11.13

14 Matrix-Matrix Multiply r 1,1 r 1,2 r 1, a 1,1 a 1,2 a 1, b 1,1 b 1,2 b 1, r 2,1 r 2,2 r 2, a 2,1 a 2,2 a 2, = * b 2,1 b 2,2 b 2, r,1 r,2 r, a,1 a,2 a, b,1 b,2 b, fuctio[r]=radmmm(n) A=rad(N,N); B=rad(N,N); R=A*B; fuctio [R]=Mul(A,B) for r=1: for c=1: R(r,c)=sum(A(r,:).*B(:,c)); HPCE / dt10 / 2015 / 11.14

15 Matrix-Matrix Multiply r 1,1 r 1,2 r 1, a 1,1 a 1,2 a 1, b 1,1 b 1,2 b 1, r 2,1 r 2,2 r 2, a 2,1 a 2,2 a 2, = * b 2,1 b 2,2 b 2, r,1 r,2 r, a,1 a,2 a, b,1 b,2 b, fuctio[r]=radmmm(n) A=rad(N,N); B=rad(N,N); R=A*B; fuctio [R]=Mul(A,B) for r=1: for c=1: R(r,c)=sum(A(r,:).*B(:,c)); HPCE / dt10 / 2015 / 11.15

16 Matrix-Matrix Multiply r 1,1 r 1,2 r 1, a 1,1 a 1,2 a 1, b 1,1 b 1,2 b 1, r 2,1 r 2,2 r 2, a 2,1 a 2,2 a 2, = * b 2,1 b 2,2 b 2, r,1 r,2 r, a,1 a,2 a, b,1 b,2 b, fuctio[r]=radmmm(n) A=rad(N,N); B=rad(N,N); R=A*B; fuctio [R]=Mul(A,B) for r=1: for c=1: R(r,c)=sum(A(r,:).*B(:,c)); HPCE / dt10 / 2015 / 11.16

17 Matrix-Matrix Multiply r 1,1 r 1,2 r 1, a 1,1 a 1,2 a 1, b 1,1 b 1,2 b 1, r 2,1 r 2,2 r 2, a 2,1 a 2,2 a 2, = * b 2,1 b 2,2 b 2, r,1 r,2 r, a,1 a,2 a, b,1 b,2 b, fuctio[r]=radmmm(n) A=rad(N,N); B=rad(N,N); R=A*B; fuctio [R]=Mul(A,B) for r=1: for c=1: R(r,c)=sum(A(r,:).*B(:,c)); HPCE / dt10 / 2015 / 11.17

18 Quick-Sort void Qsort(it N, it *data) { partitio(n, data); // O() if(n>1){ Qsort(N/2, data); Qsort(N/2, data+n/2); } } /2 /2 /4 /4 /4 / log 2 HPCE / dt10 / 2015 / 11.18

19 How fast do fuctios grow? O(1) O( 4 ) O() O(1.01 ) O(!) O(2 ) O( log ) HPCE / dt10 / 2015 / 11.19

20 Polyomial Algorithms Polyomial-time algorithms are cosidered easy That s mathematically easy, ot ecesarily practical O(1) Costat time The best complexity class! Not much iterestig i it though... Read a item from RAM O() Liear time Vector additio Search through a u-ordered list O( 2 ) Quadratic time Matrix-vector multiply O( 3 ) Cubic time Dese matrix-matrix multiply Gaussia elimiatio I theory it s lower, but i practise it ofte is t see Strasse s algorithm HPCE / dt10 / 2015 / 11.20

21 Log ad Log-Liear Algorithms Algorithms which recursively sub-divide some space These are actually easy, ot just mathematically O(log ) Logarithmic time Fid a elemet i a sorted list Root fidig through bi-sectio See if a elemet belogs to a set / add a elemet to a set O( log ) Log-Liear time (also called liearithmic) Sortig a vector of items Fast-Fourier-Trasform (FFT) Both are huge improvemets over closest polyomial O(log ) preferred to O() O( log ) much better tha O( 2 ) HPCE / dt10 / 2015 / 11.21

22 Expoetial Algorithms Algorithms which explore some multi-dimesio space Class of algorithms with complexity O(a ) for a>1 Brute-force search of all legth- biary patters : O(2 ) Brute-force search of all legth- decimal strigs : O(10 ) Expoetial time algorithms are geerally very bad Scale extremely poorly with Occasioally useful as log as you are careful O(2 ) algorithm ca be useful for <32 O(a ) algorithm with a close to 1 is sometimes feasible HPCE / dt10 / 2015 / 11.22

23 Combiatorial Algorithms Lookig at permutatios ad combiatios of thigs Lots of specific sub-classes, but geerally O(!) Optimal mappig: bid abstract resources to physical oes Fid the best sub-set from a larger set of resources May iterestig egieerig problems are combiatorial HPCE / dt10 / 2015 / 11.23

24 Avoid aythig more tha polyomial Copyright Fox HPCE / dt10 / 2015 / 11.24

25 Thought Experimet We have four applicatios All curretly ru i oe secod All curretly hadle problems of size 16 Each applicatio has differet complexity Log-liear: O( log ) Cubic: O( 3 ) Expoetial: O(2 ) Combiatorial: O(!) The customer wats to hadle problems of twice the size How much do we eed to accelerate the existig applicatios? HPCE / dt10 / 2015 / 11.25

26 Time Secods log Speedup by about 2.5 times probably use multi-core HPCE / dt10 / 2015 / 11.26

27 Time Secods log Speedup by about 8 times maybe use a GPU? HPCE / dt10 / 2015 / 11.27

28 Time Secods 5 x log Umm... HPCE / dt10 / 2015 / 11.28

29 Time Secods log Speedup by times err, cluster of FPGAs? Cloud? HPCE / dt10 / 2015 / 11.29

30 Time Secods log 3 2! Speedup by times tur every atom i 1kg of iro ito a Petium? HPCE / dt10 / 2015 / 11.30

31 The limits of computatio New systems are ot magic Multi-core CPUs: ~16x speedup GPUs: ~500x speedup FPGAs: ~1000x speedup Cloud: ~10,000x speedup O(!), from =16 to =32 ~10 22 required Lots of problems are completely itractable Travellig Salesma: fid shortest path to visit cities Bi packig: pack objects ito the miimum umber of bis Boolea satisfiability: fid values to make equatio true HPCE / dt10 / 2015 / 11.31

32 How to deal with itractable problems? Measureable metrics! Circuit place-ad-route has ridiculously high complexity But we regularly create desigs with millios of logic gates... Must make decisio about quality versus rutime Wait 1 hour : desig rus at 250MHz Wait 10 hours : desig rus at 310MHz Wait? hours : desig rus at 317 MHZ Some algorithms are progressive ad approximate Quality of solutio improves as more compute time applied Mote-Carlo, Geetic Algorithms, Simulated Aealig,... No guaratee of optimality but at least you get a aswer HPCE / dt10 / 2015 / 11.32

33 Why parallelism fails: Amdahl s Law Split a give compute task X ito two portios A : The parts that caot be easily optimised or accelerated B : The parts that ca be sped up sigificatly Assume we achieve a speed-up of S B times to part B What is the speed-up S X for the etire task? T T X X' T T A A T T B B / S B Origial executio time New executio time S X T X / T TA TB T TB A SB TA TB as T A X' Achieved speedup SB HPCE / dt10 / 2015 / 11.33

34 Maximum Speed-up Maximum speed-up is limited by the serial fractio : T A /(T A +T B ) Need a tiy serial fractio to achieve big speed-ups Are 1000x speed-ups realistic the? T A /(T A +T B ) HPCE / dt10 / 2015 / 11.34

35 Practical example Fiite differece applicatios (fluid-mechaics, physics) Discretise cotiuous space ito cells Discretise cotiuous time ito distict time-steps Goal of acceleratio is to support fier resolutio solutios Usually icrease resolutio of space ad time axis together Let s take as the resolutio alog each axis Tasks withi fiite-differece Iitialisatio: iitialise the items i the first colum Processig: advace each colum through steps i time Collectio: retrieve aswers from fial colum HPCE / dt10 / 2015 / 11.35

36 Space g[1,1]=f1(0) for s=2.. g[s,1]=f1(g[s-1,1]) for t=1..-1 g[1,t+1]=g[1,t] for s=2..-1 g[s,t+1]=f2(g[s-1,t],g[s,t],g[s+1,t]) g[,t+1]=g[,t] Time retur F3(g[1..,t]) HPCE / dt10 / 2015 / 11.36

37 Total work: (+1)(+2) + C O( 2 ) HPCE / dt10 / 2015 / 11.37

38 Total work: (+1)(+2) + C O( 2 ) Critical path Critical path: logest path through depecy graph Assume ifiite processors, ad zero commuicatio overhead HPCE / dt10 / 2015 / 11.38

39 Double the resolutio of the grid Icrease resolutio i both time ad space Model thigs of smaller size Model thigs that happe faster Solutio is better, but more compute itesive Need High-Performace Computig! HPCE / dt10 / 2015 / 11.39

40 Double the resolutio of the grid Icrease resolutio i both time ad space Model thigs of smaller size Model thigs that happe faster Solutio is better, but more compute itesive Need High-Performace Computig! More... HPCE / dt10 / 2015 / 11.40

41 Serial executio: (+2) + (+2) + C Parallel executio: (+2) + + C Speedup: [ C] / [ C] Speedup is O() - icreases liearly with problem size HPCE / dt10 / 2015 / 11.41

42 Why parallelism works: Gustafso s Law Split a task X ito two portios A ad B A caot be accelerated, while B ca be parallelised But ow the executio time of A ad B deps o problem size T A () : time to perform part A for problem of size T T X X' ( ) ( ) T T A A ( ) T ( ) T B B () ( )/ S B Origial executio time New executio time S X ( ) T TA ( ) TB ( ) T T B( ) A( ) S S X B ( )/ T as X' ( ) B if O(T Achieved speedup A ()) O(T B ()) How precise is this? HPCE / dt10 / 2015 / 11.42