CS 61C: Great Ideas in Computer Architecture Pipelining. Anything can be represented as a number, i.e., data or instrucvons

Transcription

1 CS 61C: Geat Ideas in Compute Achitectue Pipelining 4/8/12 Instucto: David A. Sping Lectue #21 1 Paallel Requests Assigned to compute e.g., Seach Katz Paallel Theads Assigned to coe e.g., Lookup, Ads So+wae Paallel InstucVons >1 one Vme e.g., 5 pipelined instucvons Paallel Data >1 data one Vme e.g., Add of 4 pais of wods Hadwae descipvons All one Vme Pogamming Languages You Ae Hee! Haness Paallelism & Achieve High Pefomance Hadwae Today s Lectue Waehouse Scale Compute Coe Memoy Input/Output InstucVon Unit(s) Main Memoy Coe Smat Phone Logic Gates 2 Compute (Cache) Coe FuncVonal Unit(s) A 0 +B 0 A 1 +B 1 A 2 +B 2 A 3 +B 3 Machine Intepeta4on Levels of RepesentaVon/ IntepetaVon High Level Language Pogam (e.g., C) Compile Assembly Language Pogam (e.g., MIPS) Assemble Machine Language Pogam (MIPS) Hadwae Achitectue DescipCon (e.g., block diagams) Achitectue Implementa4on temp = v[k]; v[k] = v[k+1]; v[k+1] = temp; lw $t0, 0($2) lw $t1, 4($2) sw $t1, 0($2) sw $t0, 4($2) Anything can be epesented as a numbe, i.e., data o instucvons ! Logic Cicuit DescipCon 4/8/12 (Cicuit SchemaCc Diagams) Sping Lectue #21 3 Pipelined ExecuVon Administivia Pipelined Datapath Pipeline Hazads Summay Agenda 4 Review: Pipelining Lessons Pipelining doesn t help latency of single task, it helps thoughput of enve wokload MulVple tasks opeavng simultaneously using diffeent esouces PotenVal speedup = Numbe pipe stages (4 in this case) Time to fill pipeline and Vme to dain it educes speedup: 8 hous/3.5 hous o 2.3X v. potenval 4X in this example Pipeline ate limited by slowest pipeline stage Unbalanced lengths of pipe stages educes speedup 5 Review: Single Cycle Datapath op s t immediate Data Memoy {R[s] + SignExt[imm16]} = R[t] Dst= Rd W= busw npc_sel= 1 clk Rs Rt Rw Rt 0 Ra Rb File imm16 16 clk ExtOp= Extende busa busb inst fetch unit Rs Rt Rd Imm16 zeo ct= Memto= 0 1 Sc= InstucVon<31:0> <21:25> = Data In clk <16:20> MemW= <11:15> WEn Ad <0:15> Data Memoy

2 Steps in ExecuVng MIPS 1) IF: InstucVon Fetch, Incement PC 2) ID: InstucVon Decode, Read istes 3) EX: ExecuVon Memoy- efeence: Calculate Addess AithmeVc- logic: Pefom OpeaVon 4) Mem: Load: Read Data fom Memoy Stoe: Wite Data to Memoy 5) WB: Wite Data Back to iste PC Redawn Single- Cycle Datapath +4 instucvon memoy 1. InstucVon Fetch d s t imm egistes Data memoy 2. Decode/ iste Read 3. Execute 4. Memoy 5. Wite Back 7 8 Pipelined Datapath Moe Detailed Pipeline PC instucvon memoy d s t egistes Data memoy istes named fo adjacent stages, e.g., IF/ID +4 imm 1. InstucVon Fetch 2. Decode/ iste Read 3. Execute 4. Memoy 5. Wite Back Add egistes between stages Hold infomavon poduced in pevious cycle 5 stage pipeline; clock ate potenval 5X faste 4/8/12 9 Sping Lectue #21 10 IF fo Load, Stoe, ID fo Load, Stoe, Highlight combinavonal logic components used + ight half of state logic on ead, lez half on wite

3 EX fo Load MEM fo Load WB fo Load Has bug that was in 1 st edivon of gaduate textbook! Coected Datapath fo Load Wong instucvon supplying wite egiste numbe Coect instucvon supplying egiste numbe Pipelined ExecuVon RepesentaVon Time Evey instucvon must take same numbe of steps, also called pipeline stages, so some will go idle somevmes 17 PC Gaphical Pipeline Diagams +4 instucvon memoy 1. InstucVon Fetch d s t imm egistes Data memoy 2. Decode/ iste Read 3. Execute 4. Memoy 5. Wite Back Use datapath figue below to epesent pipeline 18 3

4 I n s t. O d e Gaphical Pipeline RepesentaVon (In, ight half highlight ead, leu half wite) Time (clock cycles) Load Add Stoe Sub O D$ D$ D$ D$ D$ Pipeline Pefomance Assume Vme fo stages is 100 ps fo egiste ead o wite 200 ps fo othe stages What is pipelined clock ate? Compae pipelined datapath with single- cycle datapath 4/8/12 19 Sping Lectue #21 Inst Inst fetch iste ead op Memoy access iste wite Total time lw 200 ps 100 ps 200 ps 200 ps 100 ps 800 ps sw 200 ps 100 ps 200 ps 200 ps 700 ps R-fomat 200 ps 100 ps 200 ps 100 ps 600 ps beq 200 ps 100 ps 200 ps 500 ps 20 Student Roule@e? Pipeline Pefomance Single- cycle (T c = 800 ps) Pipelined (T c = 200 ps) Administivia Poject 4: Pipelined Cycle Pocesso in Logicsim due 4/15 (No Pat 1) Exta Cedit: Fastest Vesion of Poject 3 Due 4/22 11:59 PM Final Review: Apil 29, 2PM- 5PM, 2050 VLSB All gades finalized: 4/27 Final: Wednesday May 9 11:30-2:30 (TBD) Ge}ng to Know You Pof Agenda Pa@esons ae Scotch- Iish Raised Pesbyteian, which is Chuch of Scotland Came to Pennsylvania ~1800 We wea kilts to weddings Spoan fo stoage Sgian Dubh ( ski en doo ) Tatan Day Apil 5 DeclaaVon of Aboath 10 => DeclaaVon of Independence See movie Baveheat fo, as long as but a hunded of us emain alive, neve will we on any condivons be bought unde English ule. It is in tuth not fo gloy, no iches, no honous that we ae fighvng, but fo feedom fo that alone, which no honest man gives up but with life itself. 23 Pipelined ExecuVon Administivia Pipelined Datapath Pipeline Hazads Summay 24 4

5 Pipeline Speedup If all stages ae balanced i.e., all take the same Vme Time between instucvons pipelined = Time between instucvons nonpipelined Numbe of stages If not balanced, speedup is less Speedup due to inceased thoughput Latency (Vme fo each instucvon) does not decease InstucVon Level Paallelism (ILP) Anothe paallelism fom to go with Request Level Paallelism and Data Level Paallelism RLP e.g., Waehouse Scale CompuVng DLP e.g., SIMD, Map- Reduce ILP e.g., Pipelined InstucAon ExecuAon 5 stage pipeline => 5 instucvons execuvng simultaneously, one at each pipeline stage Hazads SituaVons that pevent stavng the next logical instucvon in the next clock cycle 1. Stuctual hazads Requied esouce is busy (e.g., stashe is studying) 2. Data hazad Need to wait fo pevious instucvon to complete its data ead/wite (e.g., pai of socks in diffeent loads) 3. Contol hazad Deciding on contol acvon depends on pevious instucvon (e.g., how much detegent based on how clean pio load tuns out) 1. Stuctual Hazads Conflict fo use of a esouce In MIPS pipeline with a single memoy Load/Stoe equies memoy access fo data InstucVon fetch would have to stall fo that cycle Causes a pipeline bubble Hence, pipelined datapaths equie sepaate instucvon/data memoies In eality, povide sepaate L1 instucvon cache and L1 data cache I n s t. O d e 1. Stuctual Hazad #1: Single Memoy Time (clock cycles) Load Inst 1 Inst 2 Inst 3 Inst 4 Read same memoy twice in same clock cycle D$ 29 I n s t. O d e 1. Stuctual Hazad #2: istes (1/2) sw Inst 1 Inst 2 Inst 3 Inst 4 Time (clock cycles) Can we ead and wite to egistes simultaneously? D$ 30 5

6 1. Stuctual Hazad #2: istes (2/2) Two diffeent soluvons have been used: 1) File access is VERY fast: takes less than half the Vme of stage Wite to istes duing fist half of each clock cycle Read fom istes duing second half of each clock cycle 2) Build File with independent ead and wite pots Result: can pefom Read and Wite duing same clock cycle 2. Data Hazads An instucvon depends on complevon of data access by a pevious instucvon add $s0, $t0, $t1 sub $t2, $s0, $t3 4/8/12 31 Sping Lectue #21 Fowading (aka Bypassing) Use esult when it is computed Don t wait fo it to be stoed in a egiste Requies exta connecvons in the datapath Coected Datapath fo Fowading? Student Roule@e? Fowading Paths Load- Use Data Hazad Can t always avoid stalls by fowading If value not computed when needed Can t bypass backwad in Vme! Sping 4/8/ Lectue #

7 Stall/Bubble in the Pipeline Stall inseted hee Pipelining and ISA Design MIPS InstucVon Set designed fo pipelining All instucvons ae - bits Easie to fetch and decode in one cycle x86: 1- to 17- byte instucvons (x86 HW actually tanslates to intenal RISC instucvons!) Few and egula instucvon fomats, 2 souce egiste fields always in same place Can decode and ead egistes in one step Memoy opeands only in Loads and Stoes Can calculate addess 3 d stage, access memoy 4 th stage Alignment of memoy opeands Memoy access takes only one cycle Sping 4/8/ Lectue # Why Isn t the DesVnaVon iste Always in the Same Field in MIPS ISA? op s t d shamt funct 31 6 bits 26 5 bits 21 5 bits 16 5 bits 5 bits 6 bits 0 op s t immediate 6 bits 5 bits 5 bits 16 bits Need to have 2 pat immediate if 2 souces and 1 desvnavon always in same place SPUR pocesso (A poject Dave and Randy Katz woked on togethe) Contol Hazads Banch detemines flow of contol Fetching next instucvon depends on banch outcome Pipeline can t always fetch coect instucvon SVll woking on ID stage of banch BEQ, BNE in MIPS pipeline Simple soluvon OpVon 1: Stall on evey banch unvl have new PC value Would add 2 bubbles/clock cycles fo evey Banch! (~ 20% of instucvons executed) 40 I n s t. O d e beq Inst 1 Inst 2 Inst 3 Inst 4 Stall => 2 Bubbles/Clocks Time (clock cycles) D$ I n s t. O d e beq Inst 1 Inst 2 Inst 3 Inst 4 One Clock Cycle Stall Time (clock cycles) D$ Whee do we do the compae fo the banch? 41 Banch compaato moved to Decode stage. 42 7

8 Code Scheduling to Avoid Stalls Reode code to avoid use of load esult in the next instucvon C code fo A = B + E; C = B + F; stall stall lw $t1, 0($t0) lw $t2, 4($t0) add $t3, $t1, $t2 sw $t3, 12($t0) lw $t4, 8($t0) add $t5, $t1, $t4 sw $t5, 16($t0) 13 cycles lw $t1, 0($t0) lw $t2, 4($t0) lw $t4, 8($t0) add $t3, $t1, $t2 sw $t3, 12($t0) add $t5, $t1, $t4 sw $t5, 16($t0) 11 cycles 3. Contol Hazad: Banching OpVmizaVon #1: Inset special banch compaato in Stage 2 As soon as instucvon is decoded (Opcode idenvfies it as a banch), immediately make a decision and set the new value of the PC Benefit: since banch is complete in Stage 2, only one unnecessay instucvon is fetched, so only one no- op is needed Side Note: means that banches ae idle in Stages 3, 4 and Coected Datapath fo BEQ/BNE? Impoved Pipeline Banch Taken 45 Student Roule@e? Chapte 4 The Pocesso Contol Hazads 3. Contol Hazad: Banching OpVon 2: Pedict outcome of a banch, fix up if guess wong Must cancel all instucvons in pipeline that depended on guess that was wong Simplest hadwae if we pedict that all banches ae NOT taken Why? OpVon #3: Redefine banches Old definivon: if we take the banch, none of the instucvons aze the banch get executed by accident New definivon: whethe o not we take the banch, the single instucvon immediately following the banch gets executed (the banch- delay slot) Delayed Banch means we always execute inst a+e banch This opvmizavon is used with MIPS 47 Student Roule@e? 48 8

9 3. Contol Hazad: Banching Notes on Banch- Delay Slot Wost- Case Scenaio: put a no- op in the banch- delay slot Be@e Case: place some instucvon peceding the banch in the banch- delay slot as long as the changed doesn t affect the logic of pogam Re- odeing instucvons is common way to speed up pogams Compile usually finds such an instucvon 50% of Vme Jumps also have a delay slot 49 Example: Nondelayed vs. Delayed Banch Nondelayed Banch Delayed Banch o $8, $9, $10 add $1, $2,$3 add $1, $2, $3 sub $4, $5, $6 sub $4, $5, $6 beq $1, $4, Exit beq $1, $4, Exit o $8, $9, $10 xo $10, $1, $11 xo $10, $1, $11 Exit: Exit: 50 Delayed Banch/Jump and MIPS ISA? Why does JAL put PC+8 in egiste 31? Delayed Banch/Jump and MIPS ISA? Why does JAL put PC+8 in egiste 31? JAL executes following instucaon (PC+4) so should etun to PC+8 51 Student Roule@e? 52 Dynamic Banch PedicVon In deepe and supescala pipelines, banch penalty is moe significant Use dynamic pedicvon Banch pedicvon buffe (aka banch histoy table) Indexed by ecent banch instucvon addesses Stoes outcome (taken/not taken) To execute a banch Check table, expect the same outcome Stat fetching fom fall- though o taget If wong, flush pipeline and flip pedicvon 1- Bit Pedicto: Shotcoming Inne loop banches mispedicted twice! oute: inne: beq,, inne beq,, oute Mispedict as taken on last iteavon of inne loop Then mispedict as not taken on fist iteavon of inne loop next Vme aound Sping 4/8/ Lectue #21 53 Sping 4/8/ Lectue #

10 2- Bit Pedicto Only change pedicvon on two successive mispedicvons The BIG Pictue And in Conclusion, Pipelining impoves pefomance by inceasing instucvon thoughput: exploits ILP Executes mulvple instucvons in paallel Each instucvon has the same latency Subject to hazads Stuctue, data, contol Stalls educe pefomance But ae equied to get coect esults Compile can aange code to avoid hazads and stalls Requies knowledge of the pipeline stuctue Sping 4/8/ Lectue #