, SIAM GS 13 Conference, Padova, Italy

Transcription

1 , SIAM GS 13 Conference, Padova, Italy A Mixed Order Scheme for the Shallow Water Equations on the GPU André R. Brodtkorb, Ph.D., Research Scientist, SINTEF ICT, Department of Applied Mathematics, Norway Andre.Brodtkorb@sintef.no

2 Brief Outline Why are we using GPUs (aka. Why have GPUs become important)? Mixed order scheme on the GPU Summary and future work This is a work in progress Technology for a better society 2

3 Development of the Microprocessor 1942: Digital Electric Computer (Atanasoff and Berry) 1947: Transistor (Shockley, Bardeen, and Brattain) : Integrated Circuit (Kilby) : Microprocessor (Hoff, Faggin, Mazor) More transistors (Moore, 1965) Technology for a better society 3

4 Frequency Scaling : 4004, 2300 trans, 740 KHz 1982: 80286, 134 thousand trans, 8 MHz 1993: Pentium P5, 1.18 mill. trans, 66 MHz 2000: Pentium 4, 42 mill. trans, 1.5 GHz 2010: Nehalem 2.3 bill. trans, 2.66 GHz, multi-core Technology for a better society 4

5 The end of frequency scaling : 29% increase in frequency : Frequency constant : 25% increase in parallelism The power density is proportional to the clock frequency cubed: Parallel CPUs: Multi-core (8-way) Hyper threading (2-way) AVX/SSE/MMX/etc (8-way) A serial program uses <2% of available resources! Technology for a better society 5

6 Massive Parallelism: The Graphics Processing Unit Performance Memory Bandwidth Technology for a better society 6

7 Examples of GPU Use Today Thousands of academic papers Big investment by large software companies Growing use in supercomputers GPU Supercomputers on the Top 500 List 14% 12% 10% 8% 6% 4% 2% 0% aug.2007 jul.2008 jul.2009 jul.2010 jul.2011 jul.2012 Technology for a better society 7

8 Using GPUs for numerical simulation GPUs can give anything from 0.001x to 100x speedup They have a 10x theoretical performance gain over the CPU The interaction between the algorithm and the architecture often determines the end performance Algorithm must suit the GPU and be implemented in a clever way At the end of the day, what counts is the accuracy of obtained results and the cost of computing them. Cost can include time, money, implementation time, and so on. Technology for a better society 8

9 Numerical performance Algorithmic and numerical performance Total performance is the product of algorithmic and numerical performance Your mileage may vary: algorithmic performance is highly problem dependent Explicit stencils Tridiag Sparse linear algebra solvers have low numerical performance Only able to utilize a fraction of the capabilities of CPUs, and worse on GPUs PLU QR Red- Black Explicit schemes with compact stencils can give near-peak numerical performance May give the overall highest performance Multigrid Algorithmic performance Krylov Technology for a better society 9

10 Shallow water on the GPU 1 st order scheme 2 nd order scheme 2013, Adaptive Mesh Refinement Mixed order scheme 2013, Mixed order schemes 2010, Multi-GPU 2012, Sparse simulations 2005, OpenGL prototype 2010, CUDA prototype 2011, Real world physics Technology for a better society 10

11 Shallow water on the GPU 3. Algorithmic developments 1 st order 2 nd order scheme scheme 2013, Adaptive Mesh Refinement Mixed order scheme 2013, Mixed order schemes 2. Computational developments 2010, Multi-GPU 2012, Sparse simulations 1. Prototyping and initial versions 2005, OpenGL prototype 2010, CUDA prototype 2011, Real world physics Technology for a better society 11

12 The Shallow Water Equations Vector of Conserved variables Flux Functions Bed slope source term Bed friction source term Technology for a better society 12

13 Discretization Our grid consists of a set of cells or volumes The bathymetry is a piecewise bilinear function The physical variables (h, hu, hv), are piecewise constants per volume Physical quantities are transported across the cell interfaces Algorithm: 1. Reconstruct physical variables 2. Evolve the solution 3. Average over grid cells Vector of Conserved variables Bed friction source term Bed slope source term Flux Functions Technology for a better society 13

14 Second order numerical scheme Continuous variables Discrete variables Slope reconstruction Flux calculation Evaluate integration points Dry states fix A. Kurganov and G. Petrova, A Second-Order Well-Balanced Positivity Preserving Central-Upwind Scheme for the Saint-Venant System Communications in Mathematical Sciences, 5 (2007), Technology for a better society 14

15 Dry states fix The dry-states fix of the KP scheme alters the slope to guarantee a nonnegative value of h at the integration points The slope is set to interpolate the bathymetry at the negative integration points This has a highly negative impact on the solution in cases where that contain steep partially dry cells Examples: houses, dams, levees, Technology for a better society 15

16 Example of problematic benchmark Front view Experimental setup consists of an instantaneous dam break with a rotated object in front of the breach. Rotated geometry is not possible to represent in Cartesian grid (if not implemented as internal boundary conditions) Non-negative slope fix gives high errors close to the obstacle Top view Soarez-Frazão, Zech, Journal of hydraulic research, 2007 Technology for a better society 16

17 Staircase artifacts The flow onto the obstacle creates very strong staircase artifacts which affect the overall solution negatively Bollermann et al. have published a well balanced reconstruction for 1D, but the approach may be nontrivial to extend to 2D without loss of conservation. We use a mixed order scheme to address the problem Bollermann, Chen, Kurganov, Noelle, A well-balanced reconstruction for wet/dry fronts, 2012 Technology for a better society 17

18 1 st order interfaces Mixed order scheme Main idea: Use first order scheme to compute the flux only for "problematic" interfaces to minimize the impact on the numerical order Use the second order scheme for all other interfaces This requires several steps: We must select the edges that require first order scheme Bed slope source term must be associated with interfaces, not cells Must guarantee conservation with the mixed order scheme Technology for a better society 18

19 Selecting 1 st order interfaces B j w j The interfaces we are interested are partially wet: B j < w j < B j+1 B j > w j > B j+1 B j+1 We still apply the dry states-fix for negative integration points w j This is the same approach as Bollerman et al. and it minimizes the number of changed interfaces B j Bj+1 Technology for a better society 19

20 Bed slope source term The source term is originally computed as cell averages as follows: S j = g B j+1 B j Δx h j 1 2+h j B j h j-1/2 h j+1/2 B j+1 The source term balances the numerical fluxes exactly for lake at rest To compute the edge-based source term, we take half the source term contribution from each of the adjacent cells S j+1 2 = 1 2 S j + S j+1 B j-1 B j B j+1 Technology for a better society 20

21 First order numerical scheme We have simplified the 2 nd order scheme to 1 st order The physical variables are piecewise constants within each cell The bathymetry is piecewise constant within each cell to guarantee non-negative values at integration points The flux is calculated using the central upwind flux function Technology for a better society 21

22 First order source term Our first order source term is as follows S j = g B j+1 B j Δx h j 1 2+h j B j B j+1 Identical to second order source term, but uses first order variables Caveat: Results in a scheme that is not well-balanced! Technology for a better society 22

23 Combining the 1 st and 2 nd order schemes The first order scheme requires no special new variables Implemented as a simple if statement in the code when evaluating the physical variables at each interface The scheme is conservative as both the 1 st order and 2 nd order schemes are conservative 1 st order scheme 2 nd order scheme Mixed order scheme Technology for a better society 23

24 First order results G2 G4 G8 G10 G11 G13 G20 Flow over triangular structure, laboratory dam break (1D EU CADAM) Comparison of second order (line) and first order (dashed line) First order scheme captures the main trends for most gauges, but has a much more smeared solution. Largest difference is at G13 Soarez-Frazão, Journal of hydraulic research, 2007 Technology for a better society 24

25 Mixed order results Second order (top) and mixed order (bottom) Virtually identical results: Maximum absolute difference of 4e-2 over the full simulation period (90s) As expected: the mixed order does not significantly alter the characteristics of the obtained solution Technology for a better society 25

26 Mixed order results First order generates the most smeared solution 1 st order Second order has major problems with the rotated obstacle Mixed order keeps sharp features in the solution, and tackles the rotated geometry as good as the first order scheme 2 nd order Mixed order Technology for a better society 26

27 Summary We have presented An overview of why GPUs are important and suitable for shallow water modeling A mixed order scheme with first and second order central upwind fluxes The use of the mixed order scheme to lessen errors induced by the drystates fix in the original numerical scheme This is a work in progress, and we have several active research topics: 1 st order numerical scheme is not well-balanced: Replace with well-balanced numerical scheme The use of 1 st order numerical schemes for larger timesteps and thus higher performance (e.g., use first order only for interfaces with very high fluxes and low water depths, etc.) Technology for a better society 27

28 Thank you for your attention Some references: A. Brodtkorb, M. L. Sætra, Explicit Shallow Water Simulations on GPUs: Guidelines and Best Practices, CMWR Proceedings, 2012 A. Brodtkorb, M. L. Sætra, M. Altinakar, Efficient Shallow Water Simulations on GPUs: Implementation, Visualization, Verification, and Validation, Computers & Fuids, 55, (2011), pp A. R. Brodtkorb, T. R. Hagen, K.-A. Lie and J. R. Natvig, Simulation and Visualization of the Saint-Venant System using GPUs, Computing and Visualization in Science, 13(7), (2011), pp Contact: André R. Brodtkorb Homepage: Youtube: SINTEF: Technology for a better society 28