MULTISCALAR PROCESSORS

Transcription

1 MULTISCALAR PROCESSORS

2 THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE

3 MULTISCALAR PROCESSORS by Manoj Franklin University of Maryland, US.A. SPRINGER SCIENCE+BUSINESS MEDIA, LLC

4 Library of Congress Cataloging-in-Publication Data A C.I.P. Catalogue record for this book is available from the Library of Congress. Franklin, Manoj MULTISCALAR PROCESSORS ISBN DOI / ISBN (ebook) Copyright 2003 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 2003 Softcover reprint of the hardcover 1 st edition 2003 All rights reserved. No part ofthis work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without the written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Permission for books published in Europe: permissions@wkap.nl Permissions for books published in the United States of America: permissions@wkap.com Printed an acid-free paper.

5 Foreword The revolution of semiconductor technology has continued to provide microprocessor architects with an ever increasing number of faster transistors with which to build microprocessors. Microprocessor architects have responded by using the available transistors to build faster microprocessors which exploit instruction-level parallelism (ILP) to attain their performance objectives. Starting with serial instruction processing in the 1970s microprocessors progressed to pipelined and superscalar instruction processing in the 1980s and eventually (mid 1990s) to the currently popular dynamically-scheduled instruction processing models. During this progression, microprocessor architects borrowed heavily from ideas that were initially developed for processors of mainframe computers and rapidly adopted them for their designs. In the late 1980s it was clear that most of the ideas developed for high-performance instruction processing were either already adopted, or were soon going to be adopted. New ideas would have to be developed to continue the march of microprocessor performance. The initial multi scalar ideas were developed with this background in the late 1980s at the University of Wisconsin. The objective was to develop an instruction processing paradigm for future microprocessors when transistors were abundant, but other constraints such as wire delays and design verification were important. The multiscalar research at Wisconsin started out small but quickly grew to a much larger effort as the ideas generated interest in the research community. Manoj Franklin's Ph.D thesis was the first to develop and study the initial ideas. This was followed by the Wisconsin Ph.D theses of Scott Breach, T.N. Vijaykumar, Andreas Moshovos, Quinn Jacobson and Eric Rotenberg which studied various aspects of the multi scalar execution models. A significant amount of research on processing models derived from multi scalar was also carried out at other universities and research labs in the 1990s. Today variants of the basic multiscalar paradigm and other follow-on models continue to be the focus of significant research activity as researchers continue to build the knowledge base that will be crucial to the design of future microprocessors.

6 vi This book provides an excellent synopsis of a large body of research carried out on multiscalar processors in the 1990s. It will be a valuable resource for designers of future microprocessors as well as for students interested in learning about the concepts of speculative multithreading. GURI SOH! UNIVERSITY OF WISCONSIN-MADISON

7 Soli Deo Gloria

8 Contents Foreword by Guri Sohi Preface Acknowledgments v xv xix 1 INTRODUCTION Technology Trends Sub-Micron Technology Implications of Sub-Micron Technology Instruction-Level Parallelism (ILP) Extracting ILP by Software Extracting ILP by Hardware Thread-Level Parallelism (TLP) Speculative TLP Challenges for TLP Processing The Multiscalar Paradigm The Multiscalar Story Developing the Idea Multi-block based Threads and the ARB Maturing of the Ideas Other Speculative Multithreading Models The Rest of the Story 20 2 THE MULTISCALAR PARADIGM Ideal TLP Processing Paradigm-The Goal Multiscalar Paradigm-The Basic Idea Multiscalar Execution Example Control Dependences 30

9 x Register Data Dependences Memory Data Dependences Interesting Aspects of the Multiscalar Paradigm Comparison with Other Processing Paradigms Multiprocessing Paradigm Superscalar Paradigm VLIW Paradigm The Multiscalar Processor Summary 40 3 MULTISCALAR THREADS-STATIC ASPECTS Structural Aspects of Multiscalar Threads Definition Thread Spawning Model Thread Flow Graph Thread Granularity Thread Size Variance Thread Shape Thread Entry Points Thread Exit Points Data Flow Aspects of Multiscalar Threads Shared Name Spaces Inter-Thread Data Dependence Program Partitioning Compiler-based Partitioning Hardware-based Partitioning Static Thread Descriptor Nature of Information Compatibility Issues and Binary Representation Concluding Remarks 62 4 MULTISCALAR THREADS-DYNAMIC ASPECTS Multiscalar Microarchitecture Circular Queue Organization of Processing Units PU Interconnect Thread Processing Phases Spawn: Inter-Thread Control Prediction Activate Execute 70

10 Contents xi Resolve Commit Squash Thread Assignment Policies Number of Threads in a PU Thread-PU Mapping Policy Thread Execution Policies Intra-PU Thread Concurrency Policy: TLP Intra-Thread Instruction Concurrency Policy: ILP Recovery Policies Thread Squashing Basic Block Squashing Instruction Re-execution Exception Handling Exceptions Interrupt Handling Concluding Remarks 80 5 MULTISCALAR PROCESSOR-CONTROL FLOW Inter-Thread Control Flow Predictor Dynamic Inter-Thread Control Prediction Control Flow Outcome Thread History Prediction Automata History Updates Return Address Prediction Intra-Thread Branch Prediction Problems with Conventional Branch Predictors Bimodal Predictor Extrapolation with Shared Predictor Correlation with Thread-Level Information to Obtain Accurate History Hybrid of Extrapolation and Correlation Intra-Thread Return Address Prediction Private RASes with Support from Inter-Thread RAS Detailed Example Instruction Supply Instruction Cache Options 101

11 xii A Hybrid Instruction Cache Organization for Multiscalar Processor Static Thread Descriptor Cache (STDC) Concluding Remarks MULTISCALAR PROCESSOR-REGISTER DATA FLOW Nature of Register Data Flow in a Multiscalar Processor Correctness Issues: Synchronization Register Data Flow in Example Code Performance Issues Decentralized Register File Multi-Version Register File-Basic Idea Local Register File Performing Intra-Thread Register Data Flow Performing Inter-Thread Register Data Flow Inter-Thread Synchronization: Busy Bits How are Busy Bits Set? Forwarding of Create Mask How are Busy Bits Reset? Forwarding of Register Values Strategies for Inter-Thread Forwarding Multi-Version Register File-Detailed Operation Algorithms for Register Write and Register Read Committing a Thread Squashing a Thread Example Data Speculation: Relaxing Inter-Thread Synchronization Producer Identity Speculation Producer Result Speculation Consumer Source Speculation Compiler and ISA Support Inter-Thread Data Flow Information Utilizing Dead Register Information Effect of Anti-Dependences Concluding Remarks MULTISCALAR PROCESSOR-MEMORY DATA FLOW Nature of Memory Data Flow in a Multiscalar Processor Example Performance Issues 154

12 Contents xiii 7.2 Address Resolution Buffer (ARB) Basic Idea liardvvare Structure liandling of Loads and Stores Committing or Squashing a Thread Reclaiming the ARB Entries Example Tvvo-LevellIierarchical ARB Novel Features of ARB ARB Extensions Memory Dependence Table: Controlled Dependence Speculation Multi-Version Cache Local Data Cache Performing Intra-Thread Memory Data Flovv Performing Inter-Thread Memory Data Flovv Detailed Working Comparison vvith Multiprocessor Caches Speculative Version Cache Concluding Remarks MULTISCALAR COMPILATION Role of the Compiler Correctness Issues Performance Issues Compiler Organization Program Partitioning Criteria Thread Size Criteria Control Flovv Criteria Data Dependence Criteria Interaction Among the Criteria Program Partitioning lieuristics Basic Thread Formation Process Control Flovv lieuristic Data Dependence lieuristics Loop Recurrence lieuristics Implementation of Program Partitioning Program Profiling Optimizations 195

13 xiv Code Replication Code Layout Intra-Thread Static Scheduling Identifying the Instructions for Motion Cost Model Code Transformations Scheduling Loop Induction Variables Controlling Code Explosion Crosscutting Issues Concluding Remarks RECENT DEVELOPMENTS Incorporating Fault Tolerance Where to Execute the Duplicate Thread? When to Execute the Duplicate Thread? Partitioning of PUs Multiscalar Processor with Trace-based Threads Implementation Hurdles of Complex Threads Tree-Like Threads Instruction Cache Organization Advantages Trace Processors Hierarchical Multiscalar Processor Microarchitecture Inter-Superthread Register Data Flow Inter-Superthread Memory Data Flow Advantages of Hierarchical Multiscalar Processing Compiler-Directed Thread Execution Non-speculative Inter-Thread Memory Data Flow Thread-Level Pipelining Increased Role of Compiler A Commercial Implementation: NEC Merlot 223 Index 235

14 Preface Semiconductor technology projections indicate that we are on the verge of having billion-transistor chips. This ongoing explosion in transistor count is complemented by similar projections for clock speeds, thanks again to advances in semiconductor process technology. These projections are tempered by two problems that are germane to single-chip microprocessors-on-chip wire delays and power consumption constraints. Wire delays, especially in the global wires, become more important, as only a small portion of the chip area will be reachable in a single clock cycle. Power density levels, which already exceed that of a kitchen hot plate, threaten to reach that of a nuclear reactor! Looking at software trends, sequential programs still constitute a major portion of the real-world software used by various professionals as well as the general public. State-of-the-art processors are therefore designed with sequential applications as the primary target. Continued demands for performance boost have been traditionally met by increasing the clock speed and incorporating an array of sophisticated microarchitectural techniques and compiler optimizations to extract instruction level parallelism (ILP) from sequential programs. From that perspective, ILP can be viewed as the main success story form of parallelism, as it was adopted in a big way in the commercial world for reducing the completion time of ordinary applications. Today's superscalar processors are able to issue up to six instructions per cycle from a sequential instruction stream. VLSI technology may soon allow future microprocessors to issue even more instructions per cycle. Despite this success story, the amount of parallelism that can be realistically exploited in the form of ILP appears to be reaching its limits, especially when the hardware is limited to pursuing a single flow of control. Limitations arise primarily from the inability to support large instruction windows--due to wire delay limitations and complex program control flow characteristics-and the ever-increasing latency to memory.

15 xvi Research on the multiscalar execution model started in the early 1990s, after recognizing this inadequacy of just relying on ILP. The goal was to expand the "parallelism bridgehead" established by ILP by augmenting it with the "ground forces" of thread-level parallelism (TLP), a coarser form of parallelism that is more amenable to exploiting control independence. Many studies on parallelism indeed confirm the significant performance potential of paralleuy executing multiple threads of a program. The difficulties that have been plaguing the parallelization of ordinary, non-numeric programs for decades have been complex control flows and ambiguous data dependences through memory. The breakthrough provided by the multiscalar execution model was the use of "sequential threads," i.e., threads that form a strict sequential ordering. Multiscalar threads are nothing but subgraphs of the control flow graph of the original sequential program. The sequential ordering of threads dictates that control passes from a thread to exactly one successor thread (among different alternatives). At run-time, the multiscalar hardware exploits TLP (in addition to ILP) by predicting and executing a dynamic sequence of threads on multiple processing units (PUs). This sequence is constructed by performing the required number of thread-level control predictions in succession. Threadlevel control speculation is the essence of multiscalar processing; sequentially ordered threads are executed in parallel in a speculative manner on independent PUs, without violating sequential program semantics. In case of misspeculation, the results of the incorrectly speculated thread and subsequent threads are discarded. The collection of PUs is built in such a way that (i) there are only a few global wires, and (ii) very little communication occurs through global wires. Localized communication can be done using short wires, and can be expected to be fast. Thus the use of multiple hardware sequencers (to fetch and execute multiple threads)-besides making judicious use of the available transistor budget increase-fits nicely with the goal of reducing on-chip wire delays through decentralization. Besides forming the backbone of several Ph.D. theses, the multiscalar model has sparked research on several other speculative multithreading modelssuperthreading, trace processing, clustered multithreading, and dynamic multithreading. It has become one of the landmark paradigms, with appearances in the Call for Papers of important conferences such as [SCA and Micro. It has been featured in an article entitled "What's Next for Microprocessor Design?" in the October 2, 1995 issue of Microprocessor Report. Recently multiscalar ideas have found their way into a commercial implementation from NEe called Merlot, furthering expectation for this execution model to become one of the "paradigms of choice" for future microprocessor design. A detailed understanding of the software and hardware issues related to the multi scalar paradigm is of utmost importance to researchers and graduate students working in advanced computer architecture. The past few years have

16 PREFACE xvii indeed seen many publications on the multiscalar paradigm, both from the academia and the industry. However, there has been no book that integrates all of the concepts in a cohesive manner. This book is intended to serve computer professionals and students by providing a comprehensive treatment of the basic principles of multi scalar execution as well as advanced techniques for implementing the multi scalar concepts. The presentation benefits from the many years of experience the author has had with the multi scalar execution model, both as Ph.D. dissertation work and as follow up research work. The discussion within most of the sections follows a top-down approach. This discussion is accompanied by a wealth of examples for clarity and ease of understanding. For each major building block, the book presents alternative designs and discusses design trade-offs. Special emphasis is placed on highlighting the major challenges. Of particular importance is deciding where a thread should start and end. Another challenge is enforcing proper synchronization and communication of register values as well as memory values from an active thread to its successors. The book provides a comprehensive coverage of all topics related to multiscalar processors. It starts with an introduction to this topic, including technology trends that provided an impetus to the development of multi scalar processors and are likely to shape the future development of processors. It ends with a review of the recent developments related to multiscalar processors. We have three audiences in mind: (1) designers and programmers of next-generation processors, (2) researchers in computer architecture, and (3) graduate students studying advanced computer architecture. The primary intended audience are computer engineers and researchers in the field of computer science and engineering. The book can also be used as a textbook for advanced graduate-level computer architecture courses where the students have a strong background in computer architecture. This book would certainly engage the students, and would better prepare them to be effective researchers in the broad areas of multithreading and parallel processing. MANO] FRANKLIN

17 Acknowledgments First of all, I praise and thank my Lord JESUS CHRIST-to whom this book is dedicated-for HIS love and divine guidance all through my life. Everything that I am and will ever be will be because of HIM. It was HE who bestowed me with the ability to do research and write this book. Over the years, I have come to realize that without such an acknowledgement, all achievements are meaningless, and a mere chasing after the wind. So, to HIM be praise, glory, and honor, for ever and ever. I thank my family and friends for their support and encouragement throughout the writing of this book. I like to acknowledge my parents Prof. G. Aruldhas and Mrs. Myrtle Grace Aruldhas who have been a constant inspiration to me in intellectual pursuits. My father has always encouraged me to strive for insight and excellence. Thanks to my wife, Bini, for her companionship, love, understanding, and undying support. And thanks to my children, Zaneta, Joshua, and Tesiya, who often succeeded in steeling my time away from this book and have provided the necessary distraction. Prof. Guri Sohi, my Ph.D. advisor, was instrumental in the development and publicizing of the multiscalar paradigm. He provided many insightful advice while I was working on the multiscalar architecture for my Ph.D. Besides myself, Scott Breach and T. N. Vijaykumar also completed Ph.D. theses on the multi scalar paradigm. Much of the information presented in this book has been assimilated from our theses and papers on the multiscalar paradigm. The National Science Foundation, DARPA, and IBM have been instrumental in funding the research on the multiscalar architecture at University of Wisconsin-Madison, University of Minnesota, and University of Maryland. Without their support, multi scalar research would not have progressed very far. Finally, I thank Susan Lagerstrom-Fife and Sharon Palleschi of Kluwer Academic Publishers for their hard work in bringing this manuscript to publication.