THE CHANGING GOVERNANCE OF THE SCIENCES

Transcription

1

2 THE CHANGING GOVERNANCE OF THE SCIENCES

3 Sociology of the Sciences Yearbook VOLUME XXVI Managing Editor: Peter Weingart, Universität Bielefeld, Germany Editorial Board: Yaron Ezrahi, The Israel Democracy Institute, Jerusalem, Israel Ulrike Felt, Institute für Wissenschaftstheorie und Wissenschaftsforschung, Vienna, Austria Michael Hagner, Max-Planck-Institut f ür Wissenschaftsgeschichte, Berlin, Germany Stephen H. Hilgartner, Cornell University, Ithaca, U.S.A. Sheila Jasanoff, Harvard University, Cambridge, MA, U.S.A. Sabine Maasen, Wissenschaftsforschung/Wissenschaftssoziologie, Basel, Switzerland Everett Mendelsohn, Harvard University, Cambridge, MA, U.S.A. Helga Nowotny, ETH Z ürich, Z ürich, Switzerland Hans-Jorg Rheinberger, Max-Planck Institut f ür Wissenschaftsgeschichte, Berlin, Germany Terry Shinn, GEMAS Maison des Sciences de l Homme, Paris, France Richard D. Whitley, Manchester Business School, University of Manchester, United Kingdom Björn Wittrock, SCASSS, Uppsala, Sweden The titles published in this series are listed at the end of this volume.

4 THE CHANGING GOVERNANCE OF THE SCIENCES The Advent of Research Evaluation Systems Edited by RICHARD WHITLEY University of Manchester, UK and JOCHEN GLÄSER University of Lancaster, UK

5 Library of Congress Control Number: ISBN (HB) ISBN (e-book) Published by Springer, P.O. Box 17, 3300 AA Dordrecht, The Netherlands. Printed on acid-free paper All Rights Reserved 2007 Springer Science+Business Media B.V. No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work.

6 PREFACE This book arose from a conversation between Lars Engwall and Richard Whitley during the 2003 European Group for Organization Studies Colloquium in Copenhagen about important topics for future research. We agreed then, and this has been amply confirmed by subsequent events, that the proliferation of research evaluation schemes, especially in Europe, was an important feature of the changing relationships between the state, universities and scientific research more generally, which needed systematic and comparative analysis. With the support of the Bank of Sweden Tercentenary Foundation, this discussion led to workshops in Uppsala and Manchester involving colleagues from Australia, Germany and the Netherlands at which the main framework for such an analysis was developed. This framework then formed the basis for the conference at Bielefeld in 2005 at which earlier versions of most of the papers in this volume were presented and extensively discussed. We are very grateful for the support of the PRIME network of the European Commission and the Max Planck Institute for the History of Science for this conference, as well as the invaluable efforts of Peter Weingart of the University of Bielefeld. We are also greatly indebted to the members of the Editorial Board who commented on various drafts of these papers as well as to Aant Elzinga, Stefan Kuhlmann, Philippe Laredo, Arie Rip and Nic Vonortas who acted as referees. In preparing the volume for publication we have been greatly assisted by Kathryn Morrison and would also like to acknowledge the help of Kate Barker in liaising with the PRIME network, authors, and referees. Richard Whitley Jochen Gläser v

7 TABLE OF CONTENTS List of Contributors...xiii Introduction Chapter 1 Changing Governance of the Public Sciences: The Consequences of Establishing Research Evaluation Systems for Knowledge Production in Different Countries and Scientific Fields... 3 Richard Whitley Introduction... 3 Characteristics of Research Evaluation Systems and their Consequences... 6 The Consequences of Strong Research Evaluation Systems in Different Public Science Systems The Effects of Strong Research Evaluation Systems on Different Kinds of Scientific Fields Conclusions References Universities as Strategic Actors: State Delegation and Changing Patterns of Evaluation Chapter 2 The Visible Hand versus the Invisible Hand: The Allocation of Research Resources in Swedish Universities Lars Engwall and Thorsten Nybom Introduction Institutional Control Entry Control Institutional Rules Conclusions Input Control Screening of Candidates Allocation Procedures Conclusions Output Control Conclusions References vii

8 viii Chapter 3 TABLE OF CONTENTS Prospective and Retrospective Evaluation Systems in Context: Insights from Japan: Robert Kneller Introduction University Research and Prospective Peer Review The University Research System The Organisation of University Research and the Academic Reward Systems Research Funding and Prospective Peer Review The Advent of Retrospective Research Evaluation Final Remarks References Appendix Chapter 4 Elite Through Rankings The Emergence of the Enterprising University Peter Weingart and Sabine Maasen Enterprising University Ahead? Universities: Elements of Organisational Actorhood Becoming an Entrepreneurial Actor: Elite Universities Displaying the Entrepreneurial Actor: University Rankings Enterprising University in the Audit Society References Chapter 5 The Social Construction of Bibliometric Evaluations Jochen Gläser and Grit Laudel A Constructivist Approach to Bibliometric Evaluation Tools The Modalities of Bibliometric Evaluation Methods Marketing by Dropping Modalities The Demand for Modality-free Bibliometric Evaluations Working on Modalities Amateur Bibliometrics Conclusions References Weak and Strong Research Evaluation Systems Chapter 6 Evaluation Without Evaluators: The Impact of Funding Formulae on Australian University Research Jochen Gläser and Grit Laudel The Elusive Effects of Research Evaluation Systems Approach

9 TABLE OF CONTENTS ix Research Strategy the Challenge of Causal Attribution Methodology and Methods the Challenge of Empirical Identification Adaptations to a Strong Research Evaluation System The Situation of Universities Adaptation to the Research Funding Environment Consequences for Academics Adaptation by Academics Changes in Knowledge Production Intended and Unintended Effects of Strong RES References Chapter 7 The Basic State of Research in Germany: Conditions of Knowledge Production Pre-Evaluation Stefan Lange Introduction The Changing Governance of Academic Research in Germany University and Faculty Evaluation and Steering of Research Academic Responses to the Changing Context of Research Time Funding Evaluations and Incentive Schemes Profile-building Adaptation Patterns Concluding Remarks References Chapter 8 Research Evaluation as Organisational Development: The Work of the Academic Advisory Council in Lower Saxony (FRG) Christof Schiene and Uwe Schimank Introduction Procedure Research Ideals Effects References Chapter 9 Interfering Governance and Emerging Centres of Control: University Research Evaluation in the Netherlands Barend van der Meulen Introduction Episode 1: The Construction of an Evaluation Episode 2: Construction of an Evaluation Object and Centres of Control

10 x TABLE OF CONTENTS Episode 3: Standardisation, Comparability and Second Order Dynamics Conclusions References Appendix Chapter 10 Research Evaluation in Transition: Individual versus Organisational Assessment in Spain Laura Cruz-Castro and Luis Sanz-Menéndez Introduction The Academic System: Governance and Funding Size and Governance Funding Institutionalisation Processes and Characteristics of the Spanish RES Institutional Arrangements of the Research Evaluation System (RES) The National Agency for Evaluation and Foresight The National Commission for the Evaluation of Research Activity New Developments in the Spanish RES Feedback Effects Between Evaluation and Research Systems References Chapter 11 Death by Peer Review?: The Impact of Results-Oriented Management in U.S. Research Susan E. Cozzens Analytic Frameworks The U.S. Public Research System Forms of Funding Types of Research Evaluation Systems Mission Agency Example Basic Research Agency Changes Consequences The Benefits of Competition Interdisciplinarity Short-termism Strategic Alignment Open Windows Conclusions References

11 TABLE OF CONTENTS xi Concluding Reflections Chapter 12 The Social Orders of Research Evaluation Systems Jochen Gläser A Sociological Approach to Research Evaluation The Problem of Governing Science The Place of RES in the Governance by Social Orders Three Rationales for RES Intrusive RES Competitive RES Responses by Universities The Interactions of Hierarchies and Competitions with Scientific Communities Success and Possible Failures of RES Competition Failure of Quasi-market RES Hierarchy Failure of Intrusive RES Incentives for Future Research References Author Index

12 LIST OF CONTRIBUTORS Cozzens, Susan, Professor Dr., Technology Policy and Assessment Center, School of Public Policy, Georgia Institute of Technology, Atlanta, GA , USA. Cruz-Castro, Laura, Dr., Instititue of Public Goods and Policies, Consejo Superior de Investigaciones Cientificas (CSIC), Albasanz, D14, E Madrid, Spain. Engwall, Lars, Professor Dr., Department of Business Studies, Uppsala University, Box 513, Uppsala, Sweden. Gläser, Jochen, Dr., Research School of Social Sciences, The Australian National University, Canberra ACT 0200, Australia. Kneller, Robert, Professor Dr., University of Tokyo, RCAST, 4-6-1, Komaba, Meguro-ku, Tokyo , Japan. Lange, Stefan, Dr., Stiftungslehrstuhl Wissenschaftsmanagement, Deutsche Hochschule für Verwaltungswissenschaften Speyer, Postfach 1409, D Speyer, Germany. Laudel, Grit, Dr., Research School of Social Sciences, The Australian National University, Canberra ACT 0200, Australia. Maasen, Sabine, Professor Dr., Programm für Wissenschaftsforschung, Universität Basel, Missionsstrasse 21, CH-4003 Basel, Switzerland. Nybom, Thorsten, Professor Dr., Department of Humanities, Örebro University, SE Örebro, Sweden. Sanz-Menendez, Luis, Professor Dr., Institute of Public Goods and Policies, Consejo Superior de Investigaciones Cientificas (CSIC), Albasanz, D13, E Madrid, Spain. Schiene, Christof, Dipl.-Soz., Referat 21: Hochschulentwicklung und controlling, Niedersächsisches Ministerium für Wissenschaft und Kultur, Leibnizufer 9, D Hannover, Germany. Schimank, Uwe, Professor Dr., Institut für Soziologie, FernUniversität in Hagen, D Hagen, Germany. Van der Meulen, Barend, Professor Dr., MB-STeHPS, University of Twente, PO Box 217, 7500 AE Enschede, The Netherlands. Weingart, Peter, Professor Dr., Institut für Wissenschafts- und Technikforschung, Universität Bielefeld, Postfach , D Bielefeld, Germany. Whitley, Richard, Professor Dr., Manchester Business School, University of Manchester, Booth Street West, GB-Manchester M15 6PB, UK. xiii

13 Introduction

14 CHAPTER 1 RICHARD WHITLEY CHANGING GOVERNANCE OF THE PUBLIC SCIENCES The Consequences of Establishing Research Evaluation Systems for Knowledge Production in Different Countries and Scientific Fields INTRODUCTION Many recent discussions of state science and technology policies in OECD economies since the end of the Second World War have distinguished a number of distant phases or paradigms (Ruivo 1994) in state-science relations, which reflected both different perceptions of the role of scientific research in industrialised societies and changes in the size and complexity of the public science systems (see e.g. Brooks 1990; Freeman and Soete 1997: ; Martin 2003). From viewing academic and other research on fundamental processes as producing knowledge that would contribute to the achievement of economic and social goals in a predominantly linear fashion, many political and bureaucratic elites have come to see the connections between scientific research and public purposes as being more complex and interactive, and to believe that realising social and economic benefits involves a more active steering of research than was commonly thought in the 1950s and 1960s (Braun 2003). Increasingly, research conducted in universities and similar organisations concerned with understanding basic processes in the material, life and social worlds is being viewed by politicians, business elites and other influential actors as a strategically important activity for national and regional development that can and should be managed for public policy purposes. Not only have the public sciences, i.e. those reputationally governed work organisations producing knowledge for publication in peer reviewed journals and books (Dasgupta and David 1994; Whitley 2000), become too large and expensive to be left to scientific elites to manage, but their outputs have become strategic assets in the international competition to dominate technological development in new industries and ensure continued economic growth. As such, their development, organisation and governance are now R. Whitley and J. Gläser (eds.), The Changing Governance of the Sciences, Springer Science+Business Media B.V

15 4 RICHARD WHITLEY the focus of state policies and procedures in most OECD countries, with many states establishing ministries of science and technology and specific agencies and programmes to steer research in desired directions (Drori et al. 2003). This steering often involves the incorporation of public policy goals and evaluation standards into the selection and monitoring processes of research funding agencies, such that project proposals have to show how the research contribute to these objectives if they were to be funded (Braun 1998). Problem focused or strategic funding agencies, such as the British Medical Research Council and the US National Institutes of Health, are often charged with supporting the best research that will contribute to their overall strategic objectives, such as solving specific medical problems, so that researchers are encouraged to study phenomena that will generate useful results. Much of this work falls into what Stokes (1997: 71-74) has termed Pasteur s quadrant, i.e. research seeking to understand the basic mechanisms producing particular effects that can be used to deal with practical problems, such as, in the case of Pasteur, spoilage in fermentation processes. The role of these kinds of agencies, and similar attempts to steer current research goals and careers towards public policy uses, has become more significant as the provision of state funding for research stabilises or declines relative to the number of researchers needing external resources to conduct projects, and alternative sources are few in number and/or limited in their resources (Braun, 1998). Most of these attempts have focused on the conditional selection and monitoring of research projects according to their likely contribution to strategic objectives as claimed in project proposals. They have relied on the prospective evaluation of new proposals and their applicants to guide research in particular directions, as well as maintain high quality through competitive bidding for resources allocated by variously organised peer review procedures. In most countries, state investment in this prospective evaluation of applications for funds and guiding resource allocation procedures towards the achievement of public policy goals has been considerably greater than that provided for the retrospective evaluation of the results. As Cozzens (1990: 283) puts it: The US funding system places overwhelming emphasis on pre-performance evaluation (in the form of the peer review system) and gives minimal attention to programme evaluation. Thus major US funding agencies devote most of their efforts to giving out grants, not to finding out what happened once they were made. However, as Cozzens presciently suggested in the same article (ibid.: 282): Institutions and nations that are not currently evaluating, regularly and with all the skill they can muster, will probably be doing so ten years hence, partly to ensure the high quality of research outputs, and partly to satisfy potential and actual demands for increased political accountability of the greatly enlarged public science system (see also Cozzens and Kneller s chapters in this volume). As developments in the 1990s and the contributions to this volume attest, retrospective evaluation systems for assessing the quality of research conducted in universities and similar research organisations have indeed proliferated across many OECD economies, such that the ex ante evaluation of proposals for external project