Indicators on Science, Technology and Innovation History and new Perspectives

Size: px

Start display at page:

Download "Indicators on Science, Technology and Innovation History and new Perspectives"

Junior Perry
6 years ago
Views:

Indicators on Science, Technology and Innovation History and new Perspectives An international Conference Lugano, Università della Svizzera italiana 16-18 November 2006 Rémi Barré, Ghislaine

1 Indicators on Science, Technology and Innovation History and new Perspectives An international Conference Lugano, Università della Svizzera italiana November 2006 Rémi Barré, Ghislaine Fillatrieau, Benoît Godin, Benedetto Lepori 1 Introduction The European Network of Excellence on Policies for Research and Innovation in Europe (PRIME) will organize an international conference on Indicators on Science, Technology and Innovation in the framework of its European Network of Indicator Producers project (ENIP). The conference follows the European Workshop on New Perspectives on Science and Technology Indicators organized by ENIP in September 2005 in Lisbon (see It will focus on two closely related issues: Firstly, the historical roots of the today s STI indicators, both concerning concerning conceptual aspects and the organization of their production. Secondly, new perspectives and developments on STI indicators, centered around the notion of positioning indicators and the aim of building an European platform for these indicators. The conference will take place at the University of Lugano, Switzerland, from the 16 th to the 18 th November This introduction outlines the main themes and reflections behind the conference, while the rest of the documents set its organization framework. 1.1 One hundred years of STI indicators 2006 marks a centennial: that of statistics on science, technology and innovation (see In 1906 James McKeen Cattell, an American psychologist and editor of Science for fifty years ( ), published the first edition of a directory of scientists entitled American Men of Science. Based on the directory, Cattell published regular

2 statistical analyses for thirty years on the demography, geography and what he called the performance of scientists. At the same time (early 1900s), psychologists developed bibliometrics (counting scientific papers) as a tool for measuring the advancement of psychology as a science. Since these very first exercises, the measurement of science, technology and innovation has changed considerably (Godin 2005). At its very beginning, statistics was concerned with measuring the size of the scientific community (counting the number of men of science ) and scientists activities (counting papers). The measurements were conducted by scientists themselves, among them psychologists and geographers. In the early 1920s, these statistics and their sources became institutionalized and, from the 1940s to the 1950s, new ones were constructed. It was no longer scientists, but government departments and national bureaus of statistics, those on which most of us rely today. The main works conducted by these public institutions, unlike previous measurements, dealt with measuring a national budget for science by counting the money devoted to R&D. The focus was no longer exclusively on universities, as Cattell s had been, but on all economic sectors: industry, government, university, and non-profit. The focus was no longer on men of science either, but on organizations and their R&D activities. Above all, the focus was on measuring the efficiency or productivity of the science system, defined as the output arising from research activities. Over the past hundred years, thanks to national statistical offices, we have gained in terms of the diversity, quality and robustness of statistics. However, we have also lost some fundamentals. The very first measurements on science centered on counting men of science, because human resources was considered to be what an influential American report of 1947 identified as the ultimate resource: the ceiling on research and development activities is fixed by the availability of trained personnel, rather than by the amounts of money available. The limiting resource is manpower. Today, the statistics on human resources in science and technology is one of the poorest sets of statistics that we possess (reference). National as well as international organizations are aware of this fact, and are trying to remedy the deficiency. But progress is slow. Another lost opportunity of the last hundred years, with regard to statistics, is measuring outcomes. If one distinguishes output from outcomes, he observes that there exist very few indicators on the outcomes of research, and those that exist are all of an economic type, among them productivity indicators. Representatives of the indicators of output and outcomes of an economic type include: patents, innovations, high-technology trade, and the technological balance of payments. Generally, the measurement of the social impacts of science is totally missing: education, health, environment, quality of life, etc. These were precisely the outcomes that Cattell identified as arising from science. To him, industry was a means, not an end. Today it is the market and the priorities of governments with regard to the market that entirely measurements. Admittedly, the challenges are many for anyone concerned with measuring intangible outcomes of a social type. There are currently several initiatives in many countries looking precisely at how to measure outcomes other than strictly economic outcomes. Unfortunately, these initiatives are not conducted by statistical offices, but by government departments, for their own ad hoc needs and not necessarily for developing national indicators of a systematic nature. Developing such indicators is nevertheless as important as measuring social capital or knowledge management. The past hundred years has been a very productive period with regard to statistics on science. Since the 1950s, we owe this progress mainly to governments and their statistical offices. Most of the time, academics are actually users of statistics produced by the state. Statistics on science are now at a crossroads. Users of statistics are asking for a lot more information than before because their analyses and/or decisions are more fine-grained. National aggregates are no longer enough, and neither are standard classifications.

3 1.2 The new challenge: positioning indicators In the past, most of the work on STI indicators has been based on an Input/output model (Godin 2006), where the indicators are structured along the categories of input, ouput and outcome, following a production function logic, in a national accounts paradigm. As a consequence the level of analysis is the country and the indicators are built as aggregates at this level, the individual agents disappear and the national innovation systems is reduced to a single point, described in its different aspects. Good examples of this kind of indicators production are R&D statistics based on the Frascati manual (OECD 2002), human resources statistics based on the Canberra manual (reference) and innovation statistics and surveys based on the Olso manual. Despite some well-know shortcomings in the quality and comparability of data (Godin 2005; Lepori 2005), this activity has reached a rather mature stage, with a clear division of work between international agencies like OECD and Eurostat playing a role of harmonization and of production of international comparative statistics, while national statistical agencies are in charge of collecting and elaborating data at national level (Esterle and Theves 2005). However, the last two decades have seen the emergence of a set of indicators based on a different model, which we could call positioning indicators (Barré 2005): in this model, the indicators aim at characterising the elements of national innovation system as such, considering it is made of differentiated, autonomous and strategic agents. It considers important to describe their interactions, linkages as well as their complementarities, competition and cooperation, to figure out the different types and categories they belong to. The indicators focus on the description of the actors within the system, the key notion being that of a distributed intelligence, where the various actors have a strategy of their own. Positioning indicators have a very different status, being an emerging activity, performed in an adhoc way, by a variety of institutions. The actors producing such positioning indicators can be University research teams (Manchester university, Leiden University ), special units within National statistical Offices (Norway), Contract research organisation (FhG-ISI, TNO ), private consultancies (Technopolis ), dedicated institutions with a coordinating mission (OST in France and its positioning indicators cooperative, the Dutch indicators consortium ) or professional associations. An illustration of the situation is the Report on European S&T Indicators (REIST) issued by the Commission in 2003, where a good part of the indicators are positioning indicators based on ad-hoc treatment of data; these indicators are produced by a large variety of institutions. PRIME promoted during the last two years the development of specific positioning indicators concerning two central domains for the European Research Area: firstly, indicators on individual higher education institutions (AQUAMETH and Observatory of European Universities; CHINC project funded by IPTS; Bonaccorsi, Daraio and Lepori 2005) and, secondly, indicators concerning public funding of research activities and, especially, project funding (Lepori et al. 2005). Both activities proved the feasibility of the production of such indicators by collection and harmonization of data from various national sources: examples of interesting applications of this approach are comparative work on efficiency of higher education institutions in Europe (Bonaccorsi and Daraio 2006), the analysis of the evolution of higher education funding in Europe during the last ten years (Lepori et al. 2005) and the first quantitative comparison of project funding systems in European countries (Lepori et al. 2005a). insert other examples OEU? Nanotech? Hence a session in the conference will be devoted to the presentation of examples of new indicators indicators and of their application for policy analysis, both from projects inside and outside PRIME. 1.3 Towards an European platform for Science and Technology Indicators (ESTIP) Despite this considerable progress, PRIME activities showed a number of limitations in the development of positioning indicators, especially concerning methodology and comparability of

4 data (Bonaccorsi, Daraio and Lepori 2005; Slipersaeter et al. 2005); thus, if their development has to go beyond the experimental phase, there is a need for the establishment of more systematic procedures for data collection and validation, requiring a durable institutionalization beyond individual projects. This is a pressing issue also for policy needs since positioning indicators are a key element for the development of the European Research Area. We underline that the situation in this context is much more complex in Europe than in the USA, because we are (still) confronted to 26 different research and higher education systems: these differences have a direct impact on the type and quality of the available data, but also on their meaning and interpretation; hence the need of always contextualizing the produced indicators in respect to the national system considered. To address these issues, PRIME launched an exploratory activity to develop scenarios for a future European STI Indicators Platform (ESTIP), chaired by Rémi Barré and Philippe Larédo; a preliminary discussion together with the main stakeholders has been organized at the Lisbon conference, while a working document will be presented and discussed at the PRIME annual conference in Paris in February Hence the final session of the Lugano conference will be devoted to the presentation and discussion of the results of this activity in view also of the launch of the 7 th Framework Programme. 2 Program of the Conference Part I. Lessons from the History This part of the conference looks at history in order to identify what lessons have been or could be learned from past practices. It will study the origins of S&T statistics, their characteristics, the rationales for their development and the uses that have been made of them. Specific questions addressed are: - What explains the origins of statistics on science, technology and innovation? - Who were the forerunners, the pioneers and the followers? - What were the motives of early statisticians? - What type of statistics was developed? - What uses were made of the statistics developed? - What was the politics of measurement? - How to make a sociology of statistics? It will be organized in two sessions, the first one being devoted to keynote speeches from leading people in the field, the second one to papers selected through a call for papers. Session 1. Historical overview Thursday 16 November 2006, Suggested presentations could be: - B. Godin, A Hundred Years of Statistics on S&T, C. Freeman (or A. Young) on R&D - E. Garfield (or T. Braun) on bibliometrics - K. Smith on innovation - XXX on patents Benoit Godin to propose and contact potential keynote speakers Session 2. Paper presentations on history of STI indicators Thursday 16 November 2006,

5 Papers selected with the call for papers. 2 sessions with 3-4 papers each (if needed we could have to parallel sessions). Part II. New perspectives for STI indicators This session looks at the future and examines the conditions required to produce new and better measurements. Topics discussed are: - The requirements and needs of users, both institutional and academic. - The misses and holes in current statistics. - The methodological challenges for producing renewed statistics. - The new and promising avenues. - Examples of recent innovative works. Session 1. Positioning indicators Friday 17 November 2007, This session will be devoted to the presentation of innovative work on positioning indicators, both from PRIME and outside, selected through a call for papers. We foresee 4 sessions with 3-4 papers each (parallel sessions if needed). Papers could either present methodological advance in the fields, as well as example of development of new indicators and their use for policy analysis. Examples of possible themes are the following (revise and complete list): Indicators for higher education institutions. Indicators on funding of research activities. Complete list with 3-4 selected subjects (avoiding superposition with Leiden) Session 2. Scenarios and institutional perspectives Saturday 18 November 2006, Focusing on institutional case studies of development of positioning indicators + presentation of the platform concept from PRIME Discuss with Rémi Barré 3 Organization Honour Committee Gabriele Gendotti, Minister of Education of Canton Ticino Charles Kleiber, Swiss Secretary of State for Education and Research Somebody from European Commission Somebody from OECD Others? Conference Chair Benoît Godin Benedetto Lepori, Università della Svizzera italiana, Lugano Scientific Committee Rémi Barré Andrea Bonaccorsi, Università di Pisa Ghislaine Filliatreau, Observatoire des Sciences et Techniques, Paris. Jean-Philippe Leresche, Observatoire Science Politique Société, Université de Lausanne. Philippe Larédo (to be completed)

6 4 Practical issues and venue Date 16-18th November Venue Università della Svizzera italiana, Lugano, Switzerland ( Check for room availability Conference Website On including Presentation and program. Call for papers; paper submission form. Abstracts and full papers (later on). Practical information. To be put on-line 1 st January Hotels Prepare a list of hotels possibly block a number of rooms with 2 hotels of different categories (high / low). Participants reserve directly their hotel and pay it directly ( Meals Organize catering for lunches (3) / coffee breaks Organize conference dinner (Thursday evening) Friday evening: free dinner? Participation fee Hotel and travel paid directly by participants (except for keynote speakers). Participation fee should include: Lunches + coffee breaks Conference dinner. Contribution for organization costs. 5 Budget and funding See attached excel sheet. 6 Timetable ASAP. Finalization of the concept by BG and BL. Circulation to some people to be involved (Barré, Bonaccorsi, Larédo) 15 th December: Discussion at the ENIP coordination and finalization of the concept for circulation. 1 st January Preliminary announcement and issue of the Call for papers. 1 st April Deadline for submission of abstracts and sessions proposals. 1 st June Answers from the scientific committee. 1 st July Final program available on-line.

7 1 st October Deadline for submission of full papers October Conference. 7 References insert Godin B. (2005), Bonaccorsi A., Daraio C., Lepori B. (2006)

8 Call for papers Indicators on Science, Technology and Innovation. History and new Perspectives An international Conference Lugano, Università della Svizzera italiana November 2006 Short introduction on the conference PRIME/ENIP Themes We invite presentations on the following subjects. 1) Historical development of S&T indicators. Papers for the indicators session: development positioning indicators + institution building Sessions on positioning indicators (additionally to those already indicated in the call) Submission Extended abstracts for papers should be submitted electronically to blepori@unisi.ch until 1 st April Abstract should indicate name and affiliation of the authors, research subject and methodology as well as (preliminary) results and conclusions. Selected references should be provided. Abstracts should not exceed three pages ( signs). Answer on acceptance will be given to the authors not later than June 1 st For additional information please refer to Conference Website: Benoit Godin Conference Chair Benedetto Lepori Conference Chair

Statistics and Science, Technology and Innovation Policy: How to Get Relevant Indicators

Statistics and Science, Technology and Innovation Policy: How to Get Relevant Indicators Benoît Godin 385, rue Sherbrooke Est Montreal, Quebec Canada H2X 1E3 benoit.godin@ucs.inrs.ca OECD Blue Sky II Conference