From the New York Times Introduction to Concurrency

Transcription

1 From the New York Times Introduction to Concurrency dapted by CP from lectures by Maurice Herlihy at rown SN FRNCISCO, May Intel said on Friday that it was scrapping its development of two microprocessors, a move that is a shift in the company's business strategy Herlihy and Shavit 2 Moore s Law Transistor count still rising On Your Desktop: The Uniprocessor Clock speed flattening sharply cpu memory 2007 Herlihy and Shavit Herlihy and Shavit 4 In the Enterprise: The Shared Memory Multiprocessor (SMP) Your New Desktop: The Multicore processor (CMP) cache cache us cache us ll on the same chip cache cache cache us us shared memory Sun T2000 Niagara shared memory 2007 Herlihy and Shavit Herlihy and Shavit 6 1

2 Multicores re Here Intel ups ante with 4-core chip. New microprocessor, due this year, will be faster, use less electricity... [San Fran Chronicle] MD will launch a dual-core version of its Opteron server processor at an event in New York on pril 21. [PC World] Sun s Niagara will have eight cores, each core capable of running 4 threads in parallel, for 32 concurrently running threads.. [The Inquirer] 2007 Herlihy and Shavit 7 Why do we care? Time no longer cures software bloat The free ride is over When you double the work your program is doing you can t just wait 6 months for it to run the same speed again! Your software must somehow exploit twice as much concurrency 2007 Herlihy and Shavit 8 Traditional Scaling Process Multicore Scaling Process Speedup Speedup User code User code Traditional Uniprocessor Time: Moore s law 2007 Herlihy and Shavit 9 Multicore Unfortunately, not so simple 2007 Herlihy and Shavit 10 Real-World Scaling Process Speedup Sequential Computation thread User code Multicore memory Parallelization and Synchronization require great care 2007 Herlihy and Shavit Herlihy and Shavit 12 2

3 Concurrent Computation threads memory synchrony Sudden unpredictable delays Cache misses (short) Page faults (long) Scheduling quantum used up (really long) 2007 Herlihy and Shavit Herlihy and Shavit 14 Model Summary Multiple threads Sometimes called processes Single shared memory Unpredictable asynchronous delays Road Map Today: background on concurrency Monday: semantics of Haskell s basic concurrency primitives (threads/ MVars) Wednesday: thread programming Following week: Software Transactional Memory (STM) 2007 Herlihy and Shavit Herlihy and Shavit 16 Concurrency Jargon Hardware Processors Software Threads, processes Sometimes OK to confuse them, sometimes not. Parallel Primality Testing Challenge Print primes from 1 to Given Ten-processor multiprocessor One thread per processor Goal Get ten-fold speedup (or close) 2007 Herlihy and Shavit Herlihy and Shavit 18 3

4 1 Load alancing P 0 P 1 P 9 Idea: Split the work evenly Each thread tests range of 10 9 Procedure for Thread i void primeprint { int i = ThreadID.get(); // IDs in {0..9 for (j = i* , j<(i+1)*10 9 ; j++) { if (isprime(j)) print(j); Note Herlihy s slightly awkward pseudocode notation for Haskell 2007 Herlihy and Shavit Herlihy and Shavit 20 Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Need dynamic load balancing Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Need dynamic load balancing rejected 2007 Herlihy and Shavit Herlihy and Shavit 22 Shared Counter Procedure for Thread i each thread takes a number int counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getndincrement(); if (isprime(j)) print(j); 2007 Herlihy and Shavit Herlihy and Shavit 24 4

5 void primeprint { int i = ThreadID.get(); // IDs in {0..9 for (j = i* , j<(i +1)*10 9 ; j++) { if (isprime(j)) print(j); Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getndincrement(); if (isprime(j)) print(j); Shared counter object code Where Things Reside shared counter cache 1 cache us cache us shared memory Local variables 2007 Herlihy and Shavit Herlihy and Shavit 26 Procedure for Thread i Procedure for Thread i Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { Stop when every j = counter.getndincrement(); value taken if (isprime(j)) print(j); 2007 Herlihy and Shavit 27 Counter counter = new Counter(1); void primeprint { long j = 0; while (j < ) { j = counter.getndincrement(); if (isprime(j)) print(j); Increment & return each new value 2007 Herlihy and Shavit 28 Counter Implementation Counter Implementation public long getndincrement() { return value++; public long getndincrement() { return value++; OK for single thread, not for concurrent threads (i.e., not thread safe ) 2007 Herlihy and Shavit Herlihy and Shavit 30 5

6 What It Means What It Means public long getndincrement() { return value++; public long getndincrement() { return value++; temp = value; value = value + 1; return temp; 2007 Herlihy and Shavit Herlihy and Shavit 32 Not so good Is this problem inherent? Value read write read 1 write 2 read 2 write 3 write read read 1 write 2 If we could only glue reads and writes time 2007 Herlihy and Shavit Herlihy and Shavit 34 Challenge Challenge public long getndincrement() { temp = value; value = temp + 1; return temp; 2007 Herlihy and Shavit 35 public long getndincrement() { temp = value; value = temp + 1; return temp; Make these steps atomic (indivisible) 2007 Herlihy and Shavit 36 6

7 Hardware Solution n side: Java public long getndincrement() { temp = value; value = temp + 1; return temp; ReadModifyWrite() instruction 2007 Herlihy and Shavit 37 Mutual Exclusion public long getndincrement() { synchronized { temp = value; value = temp + 1; return temp; 2007 Herlihy and Shavit 38 n side: Java n side: Java Haskell uses slightly different primitives to achieve the same effect public long getndincrement() { synchronized { temp = value; value = temp + 1; return temp; public long getndincrement() { synchronized { temp = value; value = temp + 1; return temp; Synchronized block 2007 Herlihy and Shavit Herlihy and Shavit 40 Mutual Exclusion, or lice & ob share a pond lice has a pet 2007 Herlihy and Shavit Herlihy and Shavit 42 7

8 ob has a pet The Problem The pets don t get along 2007 Herlihy and Shavit Herlihy and Shavit 44 Formalizing the Problem Two types of formal properties in asynchronous computation: Safety Properties Nothing bad happens ever Liveness Properties Something good happens eventually Formalizing our Problem Mutual Exclusion oth pets never in pond simultaneously This is a safety property No Deadlock if only one wants in, it gets in if both want in, one gets in This is a liveness property 2007 Herlihy and Shavit Herlihy and Shavit 46 Simple Protocol Idea Just look at the pond, see if it is empty, and release pet if so Gotcha oth look at the same instant oth release pets ad thing happens in pond 2007 Herlihy and Shavit 47 Telephone Protocol Idea ob calls lice (or vice-versa) Gotcha lice in shower when ob calls ob recharging phone battery when lice calls lice out shopping for pet food when ob calls 2007 Herlihy and Shavit 48 8

9 cola Patient Telephone Protocol Can Protocol Idea ob calls lice (or vice-versa) and lets phone ring until lice answers Gotcha lice goes on vacation for a month... Lesson Need to be able to leave persistent messages (like writing, not speaking) cola cola 2007 Herlihy and Shavit Herlihy and Shavit 50 ob conveys a bit ob conveys a bit cola 2007 Herlihy and Shavit Herlihy and Shavit 52 Can Protocol Flag Protocol Idea Cans on lice s windowsill Strings lead to ob s house ob pulls strings, knocks over cans Gotcha Cans cannot be reused ob runs out of cans 2007 Herlihy and Shavit Herlihy and Shavit 54 9

10 lice s Protocol (roughly) ob s Protocol (roughly) 2007 Herlihy and Shavit Herlihy and Shavit 56 lice s Protocol ob s Protocol Raise flag Wait until ob s flag is down Unleash pet Lower flag when pet returns Raise flag Wait until lice s flag is down Unleash pet Lower flag when pet returns danger! 2007 Herlihy and Shavit Herlihy and Shavit 58 ob s Protocol (2 nd try) ob defers Raise flag to lice While lice s flag is up Lower flag Wait for lice s flag to go down Raise flag Unleash pet Lower flag when pet returns 2007 Herlihy and Shavit 59 The Flag Principle Raise the flag Look at other s flag Flag Principle: If each raises and looks, then Last to look must see both flags up 2007 Herlihy and Shavit 60 10

11 Proof of Mutual Exclusion ssume both pets in pond Derive a contradiction y reasoning backwards Consider the last time lice and ob each looked before letting the pets in Without loss of generality assume lice was the last to look ob last raised flag time ob s last looked Proof lice last raised her flag lice s last look QED 2007 Herlihy and Shavit 61 lice must have seen 2007 ob s Herlihy and Flag. Shavit Contradiction 62 Proof of No Deadlock If only one pet wants in, it gets in. Proof of No Deadlock If only one pet wants in, it gets in. Deadlock requires both continually trying to get in Herlihy and Shavit Herlihy and Shavit 64 Proof of No Deadlock If only one pet wants in, it gets in. Deadlock requires both continually trying to get in. If ob sees lice s flag, he gives her priority (a gentleman ) Remarks Protocol is unfair ob s pet might never get in Protocol uses waiting If ob is eaten by his pet, lice s pet might never get in QED 2007 Herlihy and Shavit Herlihy and Shavit 66 11

12 Moral of Story Mutual Exclusion cannot be solved by transient communication (cell phones) interrupts (cans) It can be solved by one-bit shared variables that can be read or written The Fable Continues lice and ob fall in love & marry 2007 Herlihy and Shavit Herlihy and Shavit 68 The Fable Continues lice and ob fall in love & marry Then they fall out of love & divorce She gets the pets He has to feed them The Fable Continues lice and ob fall in love & marry Then they fall out of love & divorce She gets the pets He has to feed them Leading to a new coordination problem: Producer-Consumer 2007 Herlihy and Shavit Herlihy and Shavit 70 ob Puts Food in the Pond lice releases her pets to Feed mmm mmm 2007 Herlihy and Shavit Herlihy and Shavit 72 12

13 Producer/Consumer lice and ob can t meet Each has restraining order on other So he puts food in the pond nd later, she releases the pets void Releasing pets when there s no food Putting out food if uneaten food remains Producer/Consumer Need a mechanism so that ob lets lice know when food has been put out lice lets ob know when to put out more food 2007 Herlihy and Shavit Herlihy and Shavit 74 Surprise Solution ob puts food in Pond cola cola 2007 Herlihy and Shavit Herlihy and Shavit 76 ob knocks over Can lice Releases Pets yum yum cola cola 2007 Herlihy and Shavit Herlihy and Shavit 78 13

14 lice Resets Can when Pets are Fed Pseudocode cola while (true) { while (can.isup()){; pet.release(); pet.recapture(); can.reset(); lice s code 2007 Herlihy and Shavit Herlihy and Shavit 80 Pseudocode while (true) { while (can.isup()){; ob s code pet.release(); pet.recapture(); can.reset(); while (true) { while (can.isdown()){; pond.stockwithfood(); can.knockover(); lice s code 2007 Herlihy and Shavit 81 Correctness safety Mutual Exclusion Pets and ob never together in pond No Starvation If ob always willing to feed and pets always famished, then pets eat infinitely often. safety Producer/Consumer Pets never enter pond unless there is food, and ob never provides food if there is unconsumed food. liveness 2007 Herlihy and Shavit 82 Could lso Solve Using Flags Waiting oth solutions use waiting while (mumble) { Waiting is problematic If one participant is delayed, so is everyone else! ut delays are common & unpredictable 2007 Herlihy and Shavit Herlihy and Shavit 84 14

15 The Fable drags on ob and lice still have issues The Fable drags on ob and lice still have issues So they need to communicate 2007 Herlihy and Shavit Herlihy and Shavit 86 The Fable drags on illboards are Large ob and lice still have issues So they need to communicate So they agree to use billboards 2007 Herlihy and Shavit C 3 D 2 E 1 Letter Tiles From Scrabble box 2007 Herlihy and Shavit 88 Write One Letter at a Time To post a message W 4 1 S 1 H 4 W 4 1 S 1 H 4 T 1 H 4 E 1 C 3 1 R C 3 D 2 E Herlihy and Shavit 89 whe w 2007 Herlihy and Shavit 90 15

16 Let s send another mesage Uh-Oh L S 1 E 1 L 1 L 1 L 1 V M 3 P 3 S 1 S 1 E 1 L 1 L 1 T 1 H 4 E 1 C 3 1 R 1 L 1 OK 2007 Herlihy and Shavit Herlihy and Shavit 92 Readers/Writers Devise a protocol so that Writer writes one letter at a time Reader reads one letter at a time Reader sees Old message or new message No mixed messages Readers/Writers (continued) Easy with mutual exclusion ut mutual exclusion requires waiting One waits for the other Everyone executes sequentially Remarkably We can solve R/W without mutual exclusion 2007 Herlihy and Shavit Herlihy and Shavit 94 Why do we care? We want as much of the code as possible to execute concurrently (in parallel) larger sequential part implies reduced performance mdahl s law: this relation is not linear Speedup= mdahl s Law of computation given n CPUs instead of Herlihy and Shavit Herlihy and Shavit 96 16

17 mdahl s Law Sequential fraction Speedup= Number of processors 1 1 p + p n Parallel fraction Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? 2007 Herlihy and Shavit Herlihy and Shavit 98 Example Ten processors 60% concurrent, 40% sequential How close to 10-fold speedup? Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Speedup=2.17= 2007 Herlihy and Shavit Herlihy and Shavit 100 Example Ten processors 80% concurrent, 20% sequential How close to 10-fold speedup? Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Speedup=3.57= 2007 Herlihy and Shavit Herlihy and Shavit

18 Example Ten processors 90% concurrent, 10% sequential How close to 10-fold speedup? Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Speedup=5.26= 2007 Herlihy and Shavit Herlihy and Shavit 104 Example Ten processors 99% concurrent, 01% sequential How close to 10-fold speedup? Speedup=9.17= The Moral The small % of a program that is hard to parallelize may have a large impact on overall speedup. :-( 2007 Herlihy and Shavit Herlihy and Shavit