Specula1on & Precise Interrupts Fall 2017 Prof. Ron Dreslinski h6p://www.eecs.umich.edu/courses/eecs470 In Order Out of Order In Order Issue Execute Finish Fetch Decode Dispatch Complete Retire Instruction/Decode Buffer Dispatch Buffer Reservation Stations Reorder/ Completion Buffer Store Buffer Many thanks to Prof. Martin and Roth of University of Pennsylvania for most of these slides. Portions developed in part by Profs. Austin, Brehob, Falsafi, Hill, Hoe, Lipasti, Shen, Smith, Sohi, yson, Vijaykumar, and Wenisch of Carnegie Mellon University, Purdue University, University of Michigan, and University of Wisconsin. Slide 1
Announcements Programming assignment #3 posted, due 10/9 Electronic hand-in by 11:59pm HW # 2 due Monday 10/9 11:59pm Final Project Informa1on Wednesday, start thinking about groups Project Descrip1on Posted on Website Slide 2
Readings For oday: Smith & Pleszkun Implemen1ng Precise Interrupts For uesday: H & P Chapter 3.8-3.9 H & P Chapter 3.13 Have you read yet? D. Sima Design Space of Register Renaming echniques Slide 3
omasulo Example Map able Regfile value Fetched insns R op Reservation Stations 1 2 CDB. V1 FU V2 CDB.V Slide 4
Value/Copy-Based Register Renaming omasulo-style register renaming Called value-based or copy-based Names: architectural registers Storage locahons: register file and reserva1on sta1ons Values can and do exist in both Register file holds master (i.e., most recent) values + RS copies eliminate WAR hazards Storage loca1ons referred to internally by RS# tags Register table translates names to tags ag == 0 value is in register file ag!= 0 value is not ready and is being computed by RS# CDB broadcasts values with tags agached So insns know what value they are looking at Slide 5
Are we done? When can omasulo s machine go wrong? Control transfers! Branch instruchons Need a branch predictor to predict what to fetch next More on branch predic1on later Need specula:ve execu:on to clean up state when we guess wrong Interrupts & ExcepHon (today s focus) Need to handle uncommon execu1on cases Branch to a specialized sokware handler Both face similar problem Don t know rela1ve order of instruc1ons in RS Slide 6
Interrupts An unexpected transfer of control flow Pick up where you lek off once handled (restartable) ransparent to interrupted program Kinds: Asynchronous I/O device wants agen1on Can defer interrupt un1l convenient Synchronous (aka excep1ons, traps) Unusual condi1on for some instruc1on Examples? Explicit calls to opera1ng system sokware i 1 i 2 i 3 restart H 1 H 2. H n Slide 7
Precise Interrupts Sequen1al Code Seman1cs Overlapped Execu1on i1: xxxx i1 i2: xxxx i2 i1: i2: i3: i3: xxxx i3 Precise interrupt appears to happen between two instruc:ons Slide 8
Speculation and Precise Interrupts Why are we discussing these together? Sequen1al (vn) seman1cs for interrupts All insns before interrupt should be complete All insns aker interrupt should look as if never started (abort) Basically want same thing for mis-predicted branch What makes precise interrupts difficult? OoO comple1on must undo post-interrupt writebacks Same thing for branches In-order branches complete before younger insns writeback OoO not necessarily Precise interrupts, mis-specula1on recovery: same problem Same problem same soluhon Slide 9
Precise State Specula1ve execu1on requires (Ability to) abort & restart at every branch Abort & restart at every load useful for load specula1on (later) And for shared memory mul1processing (much later) Precise synchronous (program-internal) interrupts require Abort & restart at every load, store,?? Precise asynchronous (external) interrupts require Abort & restart at every?? Bite the bullet Implement abort & restart at every insn Called precise state Slide 10
Precise State Options Imprecise state: ignore the problem! Makes page faults (any restartable excep1ons) difficult Makes specula1ve execu1on almost impossible IEEE standard strongly suggests precise state Compromise: Alpha implemented precise state only for integer ops Force in-order comple1on (W): stall pipe if necessary Slow Precise state in sokware: trap to recovery rou1ne Implementa1on dependent rap on every mis-predicted branch (you must be joking) Precise state in hardware + Everything is beger in hardware (except policy) Slide 11
he Problem with Precise State insn buffer regfile I$ B P D S D$ Problem: writeback combines two separate func1ons Forwards values to younger insns: OK for this to be out-of-order Write values to registers: would like this to be in-order Similar problem (decode) for OoO execu1on: solu1on? Split decode (D) in-order dispatch (D) + out-of-order issue (S) Separate using insn buffer: scoreboard or reserva1on sta1on Slide 12
Re-Order Buffer () Reorder buffer () regfile I$ B P D$ W1 W2 Insn buffer re-order buffer () Buffers completed results en route to register file May be combined with RS or separate Combined in picture: register-update unit RUU (Sohi s method) Separate (more common today): P6-style Split writeback (W) into two stages Why is there no latch between W1 and W2? Slide 13
Complete and Retire Reorder buffer () regfile I$ B P D$ C R Complete (C): second part of writeback Completed insns write results into + Out-of-order: wait doesn t back-propagate to younger insns ReHre (R): aka commit, graduate writes results to register file In order: stall back-propagates to younger insns Slide 14
Load/Store Queue (LSQ) makes register writes in-order, but what about stores? As usual, i.e., to D$ in X stage? Not even close, imprecise memory worse than imprecise registers Load/store queue (LSQ) Completed stores write to LSQ When store re1res, head of LSQ wrigen to D$ When loads execute, access LSQ and D$ in parallel Forward from LSQ if older store with matching address More modern design: loads and stores in separate queues More on this later Slide 15
+ LSQ regfile I$ B P C R load/store store data LSQ load data addr D$ Modulo gross simplifica1ons, this picture is almost realis1c! Slide 16
P6 P6: Start with omasulo s algorithm add Separate and RS Simple-P6 Our old RS organiza1on: 1 ALU, 1 load, 1 store, 2 3-cycle FP Slide 17
P6 Data Structures Reserva1on Sta1ons are same as before head, tail: pointers maintain sequen1al order R: insn output register, V: insn output value ags are different omasulo: RS# P6: # Map able is different +: tag + ready-in- bit ==0 Value is ready in regfile!=0 Value is not ready!=0+ Value is ready in the Slide 18
P6 Data Structures Map able + Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch Insn fields and status bits ags Values Slide 19
P6 Data Structures ht # Insn R V S X C 1 ldf X(r1),f1 2 mulf f0,f1,f2 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 f2 r1 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S no 4 FP1 no 5 FP2 no Slide 20
P6 Pipeline New pipeline structure: F, D, S, X, C, R D (dispatch) Structural hazard (/LSQ/RS)? Stall Allocate /LSQ/RS Set RS tag to # Set Map able entry to # and clear ready-in- bit Read ready registers into RS (from either or Regfile) X (execute) Free RS entry Use to be at W, can be earlier because RS# are not tags Slide 21
P6 Pipeline C (complete) Structural hazard (CDB)? wait Write value into entry indicated by RS tag Mark entry as complete If not overwrigen, mark Map able entry ready-in- bit (+) R (rehre) Insn at head not complete? stall Handle any excep1ons Write head value to register file If store, write LSQ head to D$ Free /LSQ entries Slide 22
P6 Dispatch (D): Part I Map able + Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch RS/ full? stall Allocate RS/ entries, assign # to RS output tag Set output register Map able entry to #, clear ready-in- Slide 23
P6 Dispatch (D): Part II Map able + Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch Read tags for register inputs from Map able ag==0 copy value from Regfile (not shown) ag!=0 copy Map able tag to RS ag!=0+ copy value from Slide 24
P6 Complete (C) Map able + Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch Structural hazard (CDB)? Stall : broadcast <value,tag> on CDB Write result into, if s1ll valid clear Mapable ready-in- bit Match tags, write CDB.V into RS slots of dependent insns Slide 25
P6 Retire (R) Map able Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch head not complete? stall : free entry Write head result to Regfile If s1ll valid, clear Map able entry Slide 26
P6 Data Structures Map able + Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch Insn fields and status bits ags Values Slide 27
P6 Data Structures ht # Insn R V S X C 1 ldf X(r1),f1 2 mulf f0,f1,f2 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 f2 r1 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S no 4 FP1 no 5 FP2 no Slide 28
P6: Cycle 1 ht # Insn R V S X C ht 1 ldf X(r1),f1 2 mulf f0,f1,f2 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) f1 Map able Reg + f0 f1 #1 f2 r1 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD yes ldf #1 [r1] 3 S no 4 FP1 no 5 FP2 no set # tag allocate Slide 29
P6: Cycle 2 ht # Insn R V S X C h 1 ldf X(r1),f1 f1 c2 t 2 mulf f0,f1,f2 f2 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 #1 f2 #2 r1 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD yes ldf #1 [r1] 3 S no 4 FP1 yes mulf #2 #1 [f0] 5 FP2 no set # tag allocate Slide 30
P6: Cycle 3 ht # Insn R V S X C h 1 ldf X(r1),f1 f1 c2 c3 2 mulf f0,f1,f2 f2 t 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 #1 f2 #2 r1 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #3 #2 [r1] 4 FP1 yes mulf #2 #1 [f0] 5 FP2 no free allocate Slide 31
P6: Cycle 4 ht # Insn R V S X C h 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 c4 3 stf f2,z(r1) t 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) r1 Map able Reg + f0 f1 #1+ f2 #2 Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU yes add #4 [r1] 2 LD no 3 S yes stf #3 #2 [r1] 4 FP1 yes mulf #2 #1 [f0] CDB.V 5 FP2 no CDB V #1 [f1] r1 #4 ldf finished 1. set ready-in- bit 2. write result to 3. CDB broadcast allocate #1 ready grab CDB.V Slide 32
P6: Cycle 5 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 h 2 mulf f0,f1,f2 f2 c4 c5 3 stf f2,z(r1) 4 addi r1,4,r1 r1 c5 t 5 ldf X(r1),f1 f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 #5 f2 #2 CDB V r1 #4 ldf retires 1. write result to regfile Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU yes add #4 [r1] 2 LD yes ldf #5 #4 3 S yes stf #3 #2 [r1] 4 FP1 no 5 FP2 no allocate free Slide 33
P6: Cycle 6 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 h 2 mulf f0,f1,f2 f2 c4 c5+ 3 stf f2,z(r1) 4 addi r1,4,r1 r1 c5 c6 5 ldf X(r1),f1 f1 t 6 mulf f0,f1,f2 f2 7 stf f2,z(r1) Map able Reg + f0 f1 #5 f2 #6 r1 #4 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD yes ldf #5 #4 3 S yes stf #3 #2 [r1] 4 FP1 yes mulf #6 #5 [f0] 5 FP2 no free allocate Slide 34
P6: Cycle 7 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 h 2 mulf f0,f1,f2 f2 c4 c5+ 3 stf f2,z(r1) 4 addi r1,4,r1 r1 [r1] c5 c6 c7 5 ldf X(r1),f1 f1 c7 t 6 mulf f0,f1,f2 f2 7 stf f2,z(r1) Map able Reg + f0 f1 #5 f2 #6 r1 #4+ CDB stall D (no free S RS) V #4 [r1] Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD yes ldf #5 #4 CDB.V 3 S yes stf #3 #2 [r1] 4 FP1 yes mulf #6 #5 [f0] 5 FP2 no #4 ready grab CDB.V Slide 35
P6: Cycle 8 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 h 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 3 stf f2,z(r1) c8 4 addi r1,4,r1 r1 [r1] c5 c6 c7 5 ldf X(r1),f1 f1 c7 c8 t 6 mulf f0,f1,f2 f2 7 stf f2,z(r1) Map able Reg + f0 f1 #5 f2 #6 r1 #4+ Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #3 #2 [f2] [r1] 4 FP1 yes mulf #6 #5 [f0] 5 FP2 no CDB V #2 [f2] stall R for addi (in-order) #2 invalid in Mapable don t set ready-in- #2 ready grab CDB.V Slide 36
P6: Cycle 9 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 h 3 stf f2,z(r1) c8 c9 4 addi r1,4,r1 r1 [r1] c5 c6 c7 5 ldf X(r1),f1 f1 [f1] c7 c8 c9 6 mulf f0,f1,f2 f2 c9 t 7 stf f2,z(r1) Map able Reg + f0 f1 #5+ f2 #6 r1 #4+ retire mulf CDB V #5 [f1] all pipe stages active at once! Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #7 #6 #4.V 4 FP1 yes mulf #6 #5 [f0] CDB.V 5 FP2 no free, re-allocate #5 ready grab CDB.V Slide 37
P6: Cycle 10 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 h 3 stf f2,z(r1) c8 c9 c10 4 addi r1,4,r1 r1 [r1] c5 c6 c7 5 ldf X(r1),f1 f1 [f1] c7 c8 c9 6 mulf f0,f1,f2 f2 c9 c10 t 7 stf f2,z(r1) Map able Reg + f0 f1 #5+ f2 #6 r1 #4+ CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #7 #6 #4.V 4 FP1 no 5 FP2 no free Slide 38
P6: Cycle 11 ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5 c8 3 stf f2,z(r1) c8 c9 c10 h 4 addi r1,4,r1 r1 [r1] c5 c6 c7 5 ldf X(r1),f1 f1 [f1] c7 c8 c9 6 mulf f0,f1,f2 f2 c9 c10 t 7 stf f2,z(r1) Map able Reg + f0 f1 #5+ f2 #6 r1 #4+ retire stf CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #7 #6 #4.V 4 FP1 no 5 FP2 no Slide 39
Precise State in P6 Point of is maintaining precise state How does that work? Easy as 1,2,3 1. Wait un1l last good insn re1res, first bad insn at head 2. Clear contents of, RS, and Map able 3. Start over Works because zero (0) means the right thing 0 in /RS entry is empty ag == 0 in Map able register is in regfile and because regfile and D$ writes take place at R Example: page fault in first stf Slide 40
P6: Cycle 9 (with precise state) ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 h 3 stf f2,z(r1) c8 c9 4 addi r1,4,r1 r1 [r1] c5 c6 c7 5 ldf X(r1),f1 f1 [f1] c7 c8 c9 6 mulf f0,f1,f2 f2 c9 t 7 stf f2,z(r1) Map able Reg + f0 f1 #5+ f2 #6 r1 #4+ CDB PAGE FAUL V #5 [f1] Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #7 #6 #4.V 4 FP1 yes mulf #6 #5 [f0] CDB.V 5 FP2 no Slide 41
P6: Cycle 10 (with precise state) ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 f2 r1 CDB V faulting insn at head? CLEAR EVERYHING Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S no 4 FP1 no 5 FP2 no Slide 42
P6: Cycle 11 (with precise state) ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 ht 3 stf f2,z(r1) 4 addi r1,4,r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 f2 r1 CDB V SAR OVER (after OS fixes page fault) Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU no 2 LD no 3 S yes stf #3 [f4] [r1] 4 FP1 no 5 FP2 no Slide 43
P6: Cycle 12 (with precise state) ht # Insn R V S X C 1 ldf X(r1),f1 f1 [f1] c2 c3 c4 2 mulf f0,f1,f2 f2 [f2] c4 c5+ c8 h 3 stf f2,z(r1) c12 t 4 addi r1,4,r1 r1 5 ldf X(r1),f1 6 mulf f0,f1,f2 7 stf f2,z(r1) Map able Reg + f0 f1 f2 r1 #4 CDB V Reserva1on Sta1ons # FU busy op 1 2 V1 V2 1 ALU yes addi #4 [r1] 2 LD no 3 S yes stf #3 [f4] [r1] 4 FP1 no 5 FP2 no Slide 44
P6 Performance In other words: what is the cost of precise state? + In general: same performance as plain omasulo is not a performance device Maybe a ligle beger (RS freed earlier fewer struct hazards) Unless is too small In which case struct hazards become a problem Rules of thumb for size At least N (width) * number of pipe stages between D and R At least N * t hit-l2 Can add a factor of 2 to both if you want What is the ra1onale behind these? Slide 45
P6 (omasulo+) Redux Popular design for a while (Rela1vely) easy to implement correctly Anything goes wrong (mispredicted branch, fault, interrupt)? Just clear everything and start again Examples: Intel Pen1umPro, IBM/Motorola PowerPC, AMD K6 Actually making a comeback Examples: Intel Pen1umM But went away for a while, why? Slide 46
he Problem with P6 Map able + Regfile value R value Head Retire Dispatch op RS 1 2 CDB. V1 FU V2 CDB.V ail Dispatch Problem for high performance implementa1ons oo much value movement (regfile/ RS regfile) Mul1-input muxes, long buses complicate rou1ng and slow clock Slide 47