December Network Analysis of Civil Society Organisations participation in the EU Framework Programmes

Transcription

1 December 2016 Network Analysis of Civil Society Organisations participation in the EU Framework Programmes

2 EUROPEAN COMMISSION Directorate-General for Research and Innovation Directorate B Open Innovation and Open Science Unit B.7 - Science with and for society Contact: Constantin-Ovidiu Dumitrescu RTD-SIS7-CSO-SIMULATION@ec.europa.eu constantin-ovidiu.dumitrescu@ec.europa.eu RTD-PUBLICATIONS@ec.europa.eu European Commission B-1049 Brussels

3 EUROPEAN COMMISSION Network Analysis of Civil Society Organisations participation in the EU Framework Programmes edited by WU Vienna FAS.research De Montfort University Directorate-General for Research and Innovation 2017 Science in Society

4 EUROPE DIRECT is a service to help you find answers to your questions about the European Union Freephone number (*): (*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you) LEGAL NOTICE Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of the following information. The views expressed in this publication are the sole responsibility of the author and do not necessarily reflect the views of the European Commission. More information on the European Union is available on the internet ( Luxembourg: Publications Office of the European Union, PDF ISBN doi: /71466 KI EN-N European Union, Reproduction is authorised provided the source is acknowledged.

5 This report represents Deliverable D7 Final report of the project Study on network analysis of Civil Society Organisations participation in EU framework programmes commissioned by DG Research & Innovation of the European Commission (contract No. RTD-B SI ). It has been prepared by WU Vienna (Austria) together with FAS.research (Vienna, Austria) and De Montfort University (Leicester, UK). Authors: Project leader: André Martinuzzi (WU) Coordination of work: Markus Hametner (WU) Team leaders: Harald Katzmair (FAS), Bernd Stahl (DMU) Contributing authors: Asya Dimitrova, Wolfgang Lorenz, Eva More-Hollerweger, Gabriel Wurzer (WU); Christine Chung, Christian Gulas, Georg Schroll, Andrea Werdenigg (FAS); Stephen Rainey, Kutoma Wakunuma (DMU). Suggested citation: Martinuzzi, A., Hametner, M., Katzmair, H., Stahl, B., Dimitrova, A., Lorenz, W., More- Hollerweger, E., Wurzer, G., Chung, C., Gulas, C., Schroll, G., Werdenigg, A., Rainey, S., Wakunuma, K., Network Analysis of Civil Society Organisations participation in EU Framework Programmes, Vienna & Leicester, Vienna & Leicester, December 2016 Disclaimer: The information and views set out in this report are those of the authors and do not necessarily reflect the official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this study. Neither the Commission nor any person acting on the Commission s behalf may be held responsible for the use which may be made of the information contained therein.

6

7 Table of contents Table of contents... 3 List of tables... 5 List of figures... 6 Abstract... 7 Executive summary... 8 Résumé Résumé Analytique Kurzfassung Introduction Literature review of network analysis approaches Critical analysis of previous SNA approaches and studies in R&I evaluation and measurement of impact in general Critical analysis of previous SNA studies in FP1-7 in regard of effects on R&I, CSO participation and impact on European society Critical analysis of R&I related SNA Literature in the context agent based simulation modelling, p* modelling and systems dynamics Development of Multi Theory Multi Level SNA methodology (MTML) Literature review on CSOs in EU and in R&I Background Methodology Summary of findings CSO definition in different bodies of literature Action Logic of CSOs Refinement of proposed CSO typology Compilation of project dataset Survey with organisations categorised as OTH Refined CSO typology and additional screening of OTH organisations Finalisation of project dataset Case studies of networks involving CSOs Guiding Questions Methodology CONSIDER re-analysis Additional case studies Selection of cases Findings Summary of findings additional cases Summary of findings CONSIDER cases Conclusions

8 6 Role and position of CSOs in FP networks Network Analysis Data Base Data Management and Network Generation CSO Participation General Statistics Network Parameters Key Network Analysis The Correlation of Network Parameters and CSO Participation Key Actor Analysis Key Link Analysis Connections of CSOs Publication Analysis Data Base Core Areas of Scientific Output Scientific Output and CSO Participation Media Analysis Background and Data Base Types of institutions mentioned in media reports Topics of Media Reports Types of Institutions and Topics Simulation of policy interventions Overview of the simulation model Scope of the model Organisations Skill vectors Call design Participation Simulation Output Calibration and validation of the model Simulated scenarios Analysis of scenarios Overview across different scenarios Summary of findings per scenario Effects of the main scenarios on network parameters Differences between the main scenarios in terms of concentration effects Differences between the main scenarios in terms of capacity building ( learning ) Conclusions & recommendations CSO participation in the FPs Benefits related to CSO participation in the FPs How to increase CSO involvement in the FPs Recommendations for the upcoming FP Research outlook Bibliography

9 List of tables Table 1: Key search strings used...42 Table 2: Working CSO definitions...44 Table 3: Preoccupations in literature reviews...45 Table 4: Preliminary CSO typology from the literature review...46 Table 5: Numbers of CSOs included in the final project dataset...51 Table 6: Main findings from additional cases...54 Table 7: Main findings from re-analysis of CONSIDER cases...54 Table 8: FP6, FP7, and H2020* types of institutions...59 Table 9: Funding distribution among organisation types...59 Table 10: Percentages of funding by CSO type, programme, and thematic area (1)..61 Table 11: Percentages of funding by CSO type, programme, and thematic area (2)..62 Table 12: FP7 participation of institution types in funding schemes...63 Table 13: Percentages of organisation types within the core, semi-periphery, and periphery of the FP networks (FP6 and FP7)...69 Table 14: Selected network parameters: FP7 without universities and without CSOs.70 Table 15: FP7 coordination and participation of institution types (absolute numbers).72 Table 16: Table 17: Table 18: Table 19: Table 20: Scientific disciplines and types of institutions by number of publications (absolute numbers) Scientific disciplines and types of institutions by number of publications (row percentages) Scientific disciplines and types of institutions by number of publications (column percentages) Numbers of projects with/without CSO participation (all types) and with/without publications...81 Numbers of institutions mentioned in media reports about EU funded projects (FP related or not) Table 21: Parameters for scenarios...90 Table 22: Comparison of baseline scenario with FP7 data...92 Table 23: Effects of main scenarios on the overall network...99 Table 24: Table 25: Table 26: Summary of the four main scenarios and their effects on the FP network (relative to the baseline scenario) Summary of findings in relation to the study's research questions on CSO benefits Summary of findings in relation to the study's research questions on how to increase CSO involvement Table 27: Characteristics of the Horizon 2020 programme structure

10 List of figures Figure 1: MLMT approach...40 Figure 2: Levels of "CSO-ness"...46 Figure 3: Operationalised typology ("decision tree") for categorising CSOs...48 Figure 4: Transforming CORDIS project data into network data Figure 5: Core of FP7 cooperation network...58 Figure 6: Survival rate of institution types from FP6 to FP7 in % of organisations..60 Figure 7: Correlations between network parameters (FP6)...66 Figure 8: Figure 9: FP7 average duration of projects and relative amount of CSO participations (all types)...71 FP7 thematic areas by CSO participation and percentages of publications within OpenAIRE data Figure 10: Combining different data sources to generate FP7 publication networks...75 Figure 11: Network of FP7 thematic areas and ASJC disciplines...76 Figure 12: Network of ASJC-disciplines (All Journal Science Classification) which are connected through EU funded publications...80 Figure 13: Topics of projects with CSO participation in media reports comparison inside/outside FPs...85 Figure 14: Top 5 topics of each institution type in media reports about EU funded projects (FP and non FP related)...86 Figure 15: Budget distribution and evaluation weights for main scenarios...94 Figure 16: Budget shares of different organisation types across the main scenarios...95 Figure 17: Budget to the different organisation types across the main scenarios...96 Figure 18: Shares of different organisation types across the main scenarios...96 Figure 19: Numbers of different organisation types across the main scenarios...97 Figure 20: Differences in shares of organisations by size (big, medium, small) across the main scenarios relative to the baseline scenario Figure 21: Budget share of the top-10 organisations across the main scenarios Figure 22: Figure 23: Differences in regional budget distribution across the main scenarios relative to the baseline scenario "Learning" effects of organisations across the main scenarios relative to the baseline scenario

11 Abstract This study examines the participation of Civil Society Organisations (CSO) in the EU research framework programmes (FP) through literature reviews, a series of case studies, social network analyses and agent-based modelling. It revealed that CSOs have played a marginal role in FP6 and FP7, and had very limited effects on network performance and research output. This can be explained by different logics: up to now, the FPs have been mainly shaped by researchers, focusing on scientific excellence, and the business sector, focusing on profits and competitiveness, when CSOs follow a different logic, focusing on societal impacts. Therefore, in addition to involving CSOs as partners, future FPs could optimise the use of CSOs potential by (1) involving them more significantly in other roles (e.g. agenda setting, proposal evaluation), (2) putting stronger emphasis on the societal impacts of the whole programme, and (3) funding more and smaller projects to counter concentration effects and reduce entry barriers. Our simulation showed that such a scenario would substantially increase the FPs contributions to high level policy objectives (such as the UN Sustainable Development Goals), enhance the respective competencies of different types of organisations (universities, businesses and CSOs) and bring science and citizens closer without diluting scientific excellence. 7

12 Executive summary Purpose The purpose of this report is to analyse the participation of Civil Society Organisations (CSOs) in the European research framework programmes (FPs), in particular FP6 and FP7, and to derive policy recommendations on how to strengthen their role. In doing so, the study pursued the following objectives: (1) developing a practical classification of CSOs that can help differentiate and understand the roles CSOs can play in European research framework programmes in the context of Responsible Research and Innovation (RRI), as well as their specific knowledge, properties and latent preferences determining their participation; (2) analysing the extent of CSO involvement in the FPs and determining their specific contribution to research and network performance through Social Network Analysis; and (3) simulating different policy scenarios and their effects on CSO participation, by complementing knowledge-based agent behaviour (as used in existing modelling approaches) with an orientation towards solving societal challenges based on the UN Sustainable Development Goals (SDGs). Methodological approach Several methodological approaches were combined for this study. This included literature reviews on Social Network Analysis (SNA) and Civil Society Organisations in research and innovation (R&I), desk research and a survey for developing a suitable definition and typology of CSOs in the FPs, and, on this basis, identifying and categorising CSOs in the FP6 and FP7 project datasets. The main methods applied in the empirical parts of the study were case studies involving in-depth interviews with participating CSOs, social network analysis of FP6 and FP7 research networks with a specific focus on CSOs, an analysis of publication output and media coverage of research projects with/without CSOs, as well as agent-based modelling to simulate the effect of policy interventions on the FP networks and the role and position of CSOs. The developed simulation approach expands previous models by focusing not only on knowledge flows in research networks, but also by additionally taking into account research impacts on societal challenges derived from the UN SDGs. Key findings and recommendations CSO participation in the FPs CSOs constitute a very heterogeneous group of different kinds of organisations (regarding size, strategic orientation, business model, funding streams, target groups, etc.), which makes it challenging to clearly distinguish, characterise and identify them among FP participants. Consequently, the study developed an approach that distinguishes between four types of CSOs: citizen-oriented ( core ) CSOs (CSO1), 8

13 society-oriented/public-funded CSOs (CSO2), society-oriented/business-funded CSOs (CSO3), and business-oriented CSOs (CSO4). The study s results show that this is a feasible way for assessing CSO participation patterns in the FPs. We strongly suggest better categorising CSOs in the CORDA dataset in the future, following the typology applied in this study. CSOs are marginal players in the FP network. Although on average one out of thirteen projects involves a CSO, they only account for about 6% of all participating organisations in FP7, and receive only about 1% of the funding provided by the European Commission. Only about 10% of the CSOs in FP7 coordinated a project, and only about 1% of all FP7 projects were coordinated by a CSO. Furthermore, they usually do not occupy a central role when participating in a project, which may be related to several issues they are facing: low status in consortium hierarchies, lack of clarity in task division, and concerns among academic researchers that CSO participation may weaken scientific legitimacy. Having in mind their marginal role in the FP networks, CSOs cannot be expected to have a significant impact when participating in research projects. CSOs do not show a systematic pattern of participation, although there are themes in which they appear more frequently and prominently than in others: Science in Society (SiS) is the only theme in which citizen-oriented (CSO1) and society-oriented (CSO2 and CSO3) CSOs play a more than marginal role. Regions of Knowledge (RoK) and Research for the benefits of SMEs (SME) show above-average shares of CSOs, which are mostly gained by business-oriented CSOs (CSO4). Activities of International Cooperation (INCO), Research policies (COH), and Socio-economic sciences and Humanities (SSH) show small but above-average budget shares with a substantial involvement of citizenand society-oriented CSOs (CSO1-3). In absolute terms, Health and Information and Communication Technologies (ICT) are important funding sources for all types of CSOs, because of the size of the allocated EU funding budgets. Important parts of the FP, in which CSOs virtually do not participate at all, are the European Research Council (ERC), the Marie Skłodowska-Curie actions, as well as Nanotechnologies (NMP) and Transport. Increasing the budget for themes that already have higher levels of CSO participation would help strengthen the role of civil society in the FPs and bring science closer to citizens. This budget shift needs to be accompanied by additional interventions such as smaller projects and adjusting evaluation weights. Most of the CSOs are not able to build up long-standing relationships with the FPs. Their drop-out rate of 85% from FP6 to FP7 is the highest of all types of participating organisations. About 70% of the CSOs in FP7 participated only once; for FP6, this figure is even higher. This lack of consistency does not allow cross-fertilization through CSO involvement (which would be possible if e.g. the same CSOs were to participate in both SiS and in other themes). High turnover of staff and a lack of organisational memory in CSOs constitute further hampering factors. 9

14 Institutional capacity-building and continuous development of skills is required to make CSOs fit for FP participation; this could involve staff exchange, launching specific initiatives and institutional capacity-building. CSOs find it difficult to link up with the highly competitive and excellence-driven logic of research and innovation. Mission-driven CSOs focus much more on policy impacts and the needs of their stakeholders than on scientific publications and building up academic track records. Their mission is to influence current national or regional policy-making; consequently, they can rarely gain benefits from being partners in projects lasting several years. On the other hand, many of the service oriented CSOs do not see much added value in research and innovation for their core business, which often is not highly innovative. As a consequence, both types of CSOs participate slightly more in coordination and support actions rather than in research and/or innovation projects. Apart from participating in coordination and support actions, other options for involving CSOs could be in agenda-setting, proposal evaluation and dissemination of results. CSOs often find it difficult to gain a European added-value from participating as partners in EU-wide projects, as they often target national or regional policies. This is particularly valid for social issues, where no common EU policies exist. Compared to the FPs, funding opportunities in the European Structural and Investment Funds seem to better address the regional/local geographical scope of many CSOs. Better aligning the FPs with the European Structural and Investment Funds, as suggested by the FP7 ex-post evaluation, could indirectly strengthen CSO involvement in transnational research and innovation activities. Benefits related to CSO participation in the FPs CSOs neither follow the logic of academia (characterised by excellence in scientific disciplines) nor the logic of business (shaped by competitiveness and profit). Instead, their logic focuses on solving societal problems either as mission-driven CSOs, by influencing policy making, or as service-oriented CSOs, by improving the situation of their primary target group. As a consequence of this different logic, CSO contribution to research performance is most likely to materialise where research promotes the CSO aims. Due to their marginal role in the FPs, CSOs also do not contribute to network performance. As shown through social network analysis, CSO participation hardly depends on network morphology; in FP7 there are virtually no correlations between network measures and CSO participation. In the FP networks, CSOs are hangers on, they do not build sub-clusters or bridge gaps, and they do not have an important brokerage function (and consequently do not plug structural holes). Notably, removing CSOs would have virtually no effect on network parameters and morphology. 10

15 The broad diversity of benefits related to CSO participation in the FPs cannot be adequately assessed by applying success criteria such as network performance or research performance. This is due to the fundamentally different logic (or currency ) of CSOs: while the FPs are shaped by research, which focuses on scientific excellence, and the business sector, which focuses on profits and competitiveness, CSOs follow the logic of societal impacts. Assessing the benefits of involving mission-driven CSOs needs to take into account policy impacts; the benefits of involving service-oriented CSOs can be measured through positive impacts on the stakeholders they represent. We therefore advocate a new view on the benefits related to CSO participation in the FPs, by assessing the (potential) societal benefits of the FPs as a whole, and by identifying the roles in which CSOs can best contribute to maximising these societal benefits. How to increase CSO involvement in the FPs A series of case studies and the preliminary scenarios of the agent-based model have shown that micro-management of the programme results in only marginal effects on the network as well as on the involvement of CSOs and their impacts. Micro-management of individual programme features results in only marginal effects. We consequently suggest considering a broader range of roles that CSOs could play in future FPs: (1) in proposal evaluation boards (at least for the impact section), (2) in agenda setting and work programme development (beyond online-based consultation processes), and (3) in ethics boards and steering committees (on project and programme level). Many of these roles would better fit to the mission-driven core logic of CSOs and could, therefore, be more attractive than being involved as marginalised project partners. Four scenarios were developed, tested by an agent-based model and analysed regarding network morphology, budget shares of different types of organizations, scientific excellence and positive impacts on the Sustainable Development Goals. These results were compared to a well-calibrated baseline scenario that mirrored budget shares and project sizes of FP7. Scenario Competitiveness & high-tech aimed at strengthening competitiveness through promoting technology-oriented research fields (such as ICT, Nanotechnologies (NMP) or Transport) and funding fewer, but bigger projects than in the past. The results showed a higher share of the business sector, comparable high concentration effects, but also higher shares of business-oriented CSOs (CSO4) than in the baseline scenario, while the share of citizen- and society-oriented CSOs (CSO1-3) remained marginal. This scenario also showed the strongest learning effects in terms of scientific disciplines and positive effects on capacity building. 11

16 Scenario Social cohesion & well-being focused on societal challenges and process-oriented types of research (such as Health, SSH, Science in Society (SiS) and Regions of Knowledge) and funding more, but smaller projects than in the past. It showed the strongest increase of CSOs involvement to 5% of organisations and 1.3% of the budget, allowing in particular citizen- and society-oriented CSOs (CSO1-3) to double their numbers and budget. This scenario also showed a more stable network (higher average degree, higher clustering) with collaborations being more equally distributed among organisations (less centralisation). Furthermore, it showed positive effects on capacity building and the highest learning effects regarding societal challenges, but without diluting scientific excellence. Scenario Fortress Europe focused research and innovation primarily on securityand health related topics in view of keeping Europe safe and making it more selfsufficient. Out of the four scenarios, it shows the least differences compared to the baseline scenario in terms of promoting specific organisations or creating strong network or learning effects. When looking at CSOs, however, this scenario stands out with respect to showing the smallest budget share for CSOs (about 0.9% only). Scenario FP rollback simulates a dramatic budget cut of the collaborative, missiondriven parts of the FP either reflecting an overall cut in the FPs budget, or reflecting budget shifts towards the ERC. This budget cut results in a dramatic drop-out of smaller organisations (including CSOs), while big players (such as universities) manage to remain core players. This leads to a densely connected network that is dominated by a smaller number of actors. Furthermore, the capacity of organisations in terms of both knowledge and impact are significantly smaller than in the other three scenarios. As the results of the agent-based modelling have shown, the four scenarios are quite distinct in terms of their effects on the overall research and innovation network, the involvement of CSOs and the learning effects of all types of organisations involved. If addressing societal challenges, bringing science closer to citizens and increasing the diversity and resilience of the European research and innovation network are perceived as important policy goals, the scenario Social cohesion & well-being is the most beneficial: It significantly increases the capacities of all actors to contribute to solving societal problems without diluting scientific excellence and without disrupting the existing network or completely altering the nature of the programme, and, thus, still being capable to adequately address the logic of the main players (universities, research organisations and companies). As a side effect, it doubles the involvement of CSOs in FPs. 12

17 Recommendations for the upcoming FP9 As concerns the upcoming FP9, decisions on the strategic orientation of the programme and its thematic foci are required. They will have to consider the different logics of the key actors involved, as well as the currencies 1 and the governance principles they apply: The first pillar of Horizon 2020, Excellent Science, focuses on frontier research, which aims at new discoveries, targets individual scientists, addresses their currency excellence as measured by publications and academic careers, and awards grants in a highly selective process based on evaluation carried out by other researchers. Through its high ambition and focus on excellence, this process is highly selective, but free from any influence beyond scientific communities. Governance is, therefore, self-referential, and considering societal impacts is often perceived as a potential dilution of excellence. The second pillar of Horizon 2020, Industrial Leadership, focusses on innovation, targets highly innovative business sectors, and aims at supporting their competitiveness, growth and jobs. The currency of this pillar is, therefore, profit, and patents are used to safeguard competitive advantages. This leads to an exclusion (of potential competitors) and joint agenda setting (in an inner circle of participants) as governance principles. The third pillar of Horizon 2020, Societal Challenges, focuses on impacts of societal practices, targets citizens and civil society, and follows a key logic of (political) responsibility. Increasing positive (and reducing negative) impacts are, therefore, success criteria for projects. While the other two pillars run the risk of a lock-in (through selection on the one hand, and exclusion on the other), this pillar can balance these tendencies and serve as an integrative element. Moreover, while the other two pillars follow insider-oriented governance principles, the third pillar has the potential to address major societal challenges that are set on a global scale. If European research and innovation policy wants to safeguard its legitimacy vis-à-vis European citizens, an independent third pillar, focusing on the grand societal challenges, is vital (no merging of the second and the third pillar under the headline of mission oriented research ). The globally agreed Sustainable Development Goals could serve as an orientation and avoid that agenda setting is driven by interest groups. In order to bring people closer to research and research closer to people, arenas for knowledge co-creation and innovation need to be designed into which a broad variety of stakeholders should be involved. Funding instruments should reflect 1 The notion of currencies stems from the understanding that in different social fields different units exist for measuring status within this field (e.g. publication track record for people working at universities). The metaphor of currency highlights that while each currency values something in a specific field, there should also be a way to convert a currency into another one (similar like the Euro that can be converted to US Dollar), thus allowing exchange across different fields. 13

18 that societal challenges are wicked problems, and that innovations are not final deliverables but continuous processes. Therefore, interventions should focus on governing landscapes (and not only projects), develop institutional capacities and structures, and allow for more continuous processes (overcoming the limited perspective of individual projects). Responsible research and innovation (RRI) should become a guiding principle in all three pillars and themes of the FPs. This will ensure that future FPs do not only promote frontier research at universities and research organisations or support competitiveness of businesses, but that they also consider the needs of citizens and address relevant societal challenges. 14

19 Résumé Cette étude examine l implication des organisations de la société civile (OSC) dans les programmes-cadres de recherche de l Union Européenne (PC) sur base de revues documentaires, d une série d études de cas, d analyses des réseaux sociaux et de la modélisation basée sur des agents. Elle révèle que les OSCs ont joué un rôle marginal dans le PC6 et le PC7 et n ont eu que des effets limités sur la performance des réseaux et sur les résultats de la recherche. Ceci peut être expliqué suivant différentes logiques : les PCs sont façonnés par les chercheurs, mettant l accent sur l excellence scientifique et le secteur des entreprises mettant l accent sur les profits et la compétitivité, alors que les OSCs mettent l accent sur les impacts sociétaux. Par conséquent, en sus d essayer de faire intervenir les OSCs en tant que partenaires, les futurs PCs pourraient optimiser le potentiel des OSCs plus efficacement en (1) les impliquant dans d autres rôles (p.ex. définition des programmes, évaluation des propositions), (2) mettant plus particulièrement l accent sur les impacts sociétaux de l ensemble du programme, et (3) finançant davantage de plus petits projets afin de contrer les effets de concentration et de réduire les barrières à l entrée. Notre simulation a montré qu un tel scénario augmenterait considérablement les contributions des PCs aux grands objectifs politiques (tels que les ODD des Nations Unies), valoriserait les compétences respectives de différents types d organisations (universités, entreprises et OSCs) et rapprocherait la recherche et les citoyens sans affecter l excellence scientifique. 15

20 Résumé Analytique Objectif L objectif du rapport consiste à analyser la participation des organisations de la société civile (OSC) aux programmes-cadres de recherche européens (PC), notamment aux PC6 et PC7, et à élaborer des recommandations politiques sur la façon de renforcer leur rôle. Ce faisant, l étude cherche à atteindre les objectifs suivants: (1) développer une classification pratique des OSCs qui pourra aider à distinguer et à comprendre les rôles que les OSCs peuvent jouer dans les programmes-cadres de recherche européens dans le cadre de Recherche et Innovation Responsable (RIR), ainsi que leurs connaissances spécifiques, leurs propriétés et leurs préférences latentes déterminant leur participation; (2) analyser l ampleur de la participation des OSCs aux PCs et déterminer leur contribution spécifique à la recherche et à la performance des réseaux par l analyse des réseaux sociaux ; et (3) simuler différents scénarios politiques et leurs effets sur la participation des OSCs en complétant le comportement des agents de la connaissance (tels que ceux utilisés dans les approches de modélisation existantes) par une orientation vers la résolution d enjeux sociétaux fondés sur les Objectifs de développement durable des Nations Unies (ODD). Approche Méthodologique Pour cette étude, différentes approches méthodologiques ont été combinées. Celles-ci incluent l examen documentaire sur l analyse des réseaux sociaux (ARS) et sur les organisations de la société civile (OSC) dans la recherche et innovation (R&I), une recherche documentaire ainsi qu une enquête visant à l élaboration d une définition et d une typologie appropriée des OSCs dans les PCs, et, sur cette base, l identification et la catégorisation des OSCs dans l'ensemble des données se rapportant aux projets des PC6 et PC7. Les principales méthodes qui ont été appliquées dans les parties analytiques de l étude sont des études de cas impliquant des entrevues en profondeur avec les OSCs participantes, une analyse des réseaux sociaux des réseaux de recherches du PC6 et PC7 axée spécifiquement sur les OSCs, une analyse des publications et de la couverture médiatique des projets de recherche avec ou sans OSCs ainsi qu une modélisation basée sur des agents destinée à simuler les effets des interventions politiques sur les réseaux du PC et le rôle et la position des OSCs. Le procédé de simulation mis au point a permis d élargir l horizon des modèles précédents en mettant non seulement l accent sur le flux de connaissances au sein des réseaux de recherche, mais aussi en prenant en considération les impacts de la recherche sur les défis sociétaux provenant des UN ODD. 16

21 Résultats clés et Recommandations Participation des OSC aux PCs Les OSCs constituent un groupe très hétérogène de différents types d organisations (taille, orientation stratégique, modèle commercial, flux de financement, groupe cible, etc.), rendant relativement ardues leur nette distinction, leur caractérisation et leur identification parmi les participants du PC. Par conséquent, l étude a débouché sur une approche qui distingue quatre types d OSCs: les OSCs orientés vers les citoyens («le noyau») (OSC1), les OSCs orientés vers la société/financés publiquement (OSC2), les OSCs orientés vers la société/financés par les entreprises (OSC3) et enfin, les OSCs orientés vers les entreprises (OSC4). Les résultats de l étude montrent que cette approche rend possible l évaluation des tendances en matière de participation des OSCs aux PCs. À l avenir, nous recommandons fortement une meilleure catégorisation des OSCs dans les données CORDA suivant la typologie appliquée dans cette étude. Les OSCs sont des acteurs tout à fait marginaux dans le réseau PC. Bien qu une OSC soit, en moyenne, impliqué dans un projet sur treize, elles représentent seulement 6 % de l ensemble des organisations participantes au PC7 et ne reçoivent que 1% des fonds fournis par la Commission Européenne. Environ seulement 10% des OSCs du PC7 ont coordonné un projet et seulement environ 1% des projets PC7 ont été coordonnés par un OSC. De plus, ils ne prennent généralement pas un rôle central lorsqu ils participent à un projet pouvant être lié aux différents enjeux auxquels ils font face : le statut inférieur dans les hiérarchies de consortium, le manque de clarté dans le partage des tâches et l inquiétude parmi les chercheurs universitaires par rapport à la participation des OSCs peuvent affaiblir la légitimité scientifique. Connaissant leur rôle marginal joué par les OSCs dans les réseaux de PC, ceux-ci n auront vraisemblablement pas un impact considérable lors de leur participation aux projets de recherche. Les OSCs ne montrent pas un modèle de participation systématique, bien qu il y ait des sujets où ils apparaissent plus fréquemment et éminemment que dans d autres : Science dans la Société (SdS) est le seul sujet dans lequel des OSCs orientés vers les citoyens (OSC1) et les OSCs orientés vers la société (OSC2 et OSC3) jouent un rôle plus que marginal. Régions de la Connaissance (RdC) et Recherche visant l'intérêt de SMEs (SME) affichent une part supérieure à la moyenne des OSCs généralement obtenus par des OSCs orientés vers les entreprises (OSC4). Activités des Coopérations internationales (INCO), Politiques de la recherche (COH), et Sciences socioéconomiques et humaines (SSH) font apparaître des parts budgétaires qui sont certes petites, mais supérieure à la moyenne, avec une implication importante des OSCs orientés vers les citoyens et vers la société (OSC1-3). En termes absolus, la santé ainsi que les technologies de l'information et des communications (ICT) sont des sources financières importantes pour tous les types d OSCs confondus et ce, dû à la taille du budget de financement délivré par l UE. Le 17

22 Conseil européen de la recherche, les actions de Marie Skłodowska-Curie ainsi que les nanotechnologies et le transport forment des parts importantes du PC auxquelles les OSCs ne participent pratiquement pas. Une augmentation du budget pour les sujets qui ont déjà une plus haute participation des OSCs sera utile pour le renforcement du rôle de la société civile dans les PCs et du rapprochement de la science et des citoyens. Ce revirement du budget devra être accompagné par des interventions supplémentaires. La plupart des OSCs ne sont pas capables d entretenir une relation durable avec les PCs. Leur 85% de taux de décrochage de FP6 à P7 représente le taux le plus élevé de tous les types d organisations participantes. Environ 70% des OSCs du PC7 n ont participé qu une seule fois: pour le PC6, ce chiffre est encore plus élevé. Ce manque de cohérence ne permet pas la fertilisation croisée au travers l implication d OSCs (cela aurait été possible si, par exemple, ces OSCs avaient participé aux deux SiS ainsi qu à d autres sujets). La rotation fréquente du personnel et un manque de mémoire organisationnelle des OSCs représentent d autres types d obstacles. Le renforcement des capacités institutionnelles et le développement permanent des compétences est requis pour que les OSCs puissent être aptes à participer au PC ; cela pourrait impliquer un échange de personnel, le lancement d initiatives spécifiques et le renforcement des capacités institutionnelles. Les OSCs peinent à rejoindre la logique de recherche et d innovation hautement compétitive et axée sur l excellence. Les OSCs axés sur leur mission se concentrent bien plus sur les impacts politiques et les besoins des parties prenantes que sur les publications scientifiques et sur la construction d'un palmarès académique. Leur mission consiste à influencer la politique actuelle, qu elle soit nationale ou régionale. Ainsi, ils ne retirent aucun avantage à être partenaires dans un projet d une durée supérieure à trois ans. Cela dit, beaucoup d OSCs orientés vers le service ne bénéficient pas vraiment de valeur ajoutée à la recherche et à l innovation pour leur activité de base étant, par ailleurs, souvent peu innovatrice. Par conséquent les divers types d OSCs participent un peu plus aux actions de coordination et de soutien qu aux projets de recherche et/ou aux projets d innovation. Hormis la participation aux actions de coordination et de soutien, d autres options impliquant les OSCs pourraient être l élaboration de programmes, l évaluation de propositions et la diffusion des résultats. Pour les OSCs, il est souvent fort peu aisé d acquérir une valeur ajoutée européenne suite à la participation à des projets menés à l échelle européenne car ceux-ci visent des politiques nationales ou régionales. Ceci vaut en particulier pour les sujets sociétaux pour lesquels il n existe aucune politique commune de l Union Européenne. Comparés aux PCs, les possibilités de financement des Fonds structurels européen et du Fonds européen d'investissement semblent mieux cibler la portée régionale/locale de nombreux OSCs. Un meilleur alignement des PCs entre le Fonds structurel européen et le Fonds européen d'investissement, comme proposé par l'évaluation ex-post du PC7, pourrait indirectement renforcer l implication des OSCs dans des activités d innovation et de recherches transnationales. 18

23 Avantages liés à la participation des OSCs aux PCs Les OSCs ne suivent ni la logique du monde universitaire (caractérisée par l excellence dans des disciplines scientifiques) ni la logique des entreprises (façonnée par la compétitivité et le profit). En revanche, leur logique met en exergue la solution des problèmes sociétaux soit comme OSCs motivés par leur mission, par l influence des actions politiques, soit comme OSCs orientés vers le service par l amélioration de la situation de leur principal groupe cible. Par conséquent, suivant cette logique différente, nous n avons pas trouvé de preuve mettant en évidence le fait que des OSCs contribuent à la performance des recherches. En raison de leur rôle marginal dans les PCs, les OSCs ne contribuent pas à la performance des réseaux. Comme indiqué dans les analyses des réseaux sociaux, la participation d une OSC n est pas dépendante de la morphologie du réseau ; il n y a pratiquement pas de corrélation entre les mesures des réseaux et la participation d une OSC dans le PC7. Dans les réseaux des PCs, les OSCs sont des «suiveurs», ils ne constituent pas de sous-groupe ni ne remplissent des vides et ils n assurent pas de fonction d'intermédiaire significative (par conséquent, ceux-ci ne bouchent pas de trous structurels). En outre, la suppression des OSCs n aurait pas eu de véritable impact sur les paramètres des réseaux et de leur morphologie. La grande diversité des avantages liés à la participation des OSCs aux PCs ne peut pas être évalué de façon adéquate par l application des critères de succès tels que «performance des réseau» ou «performance de la recherche». Ceci est dû à la logique foncièrement différente (ou monnaie ) des OSCs: les PCs sont façonnés par les chercheurs, mettant l accent sur l excellence scientifique, et par le secteur des entreprises, mettant l accent sur le profit et la compétitivité, alors que les OSCs suivent la logique des impacts sociétaux. L évaluation des avantages de l inclusion des OSCs motivées par leur mission doit considérer les impacts politiques ; Les avantages de l implication des OSCs orientées vers le service peuvent être mesurés à l aide de l impact positif pour les parties prenantes qu ils représentent. Ainsi, nous favorisons une nouvelle façon d appréhender les avantages liés à la participation des OSCs aux PCs par l évaluation des avantages sociétaux (potentiels) des PCs dans leur ensemble et par l identification des rôles dans lesquels les OSCs peuvent au mieux contribuer à maximiser ces avantages. Comment augmenter l implication des OSCs dans les PCs Une série d études de cas et de scénarios préliminaires du modèle basé sur des agents ont montré que la microgestion du programme n entraînent seulement que des effets marginaux sur le réseau ainsi que sur l implication des OSCs et leurs impacts. La microgestion de caractéristiques individuelles de programmes n a que des effets marginaux. Par conséquent, nous proposons de considérer une gamme plus ample de rôles que les OSCs pourraient jouer dans de futurs PCs : (1) dans les groupes d évaluation des propositions (au moins pour la section «impact»), (2) dans la création d ordres du jour et dans le développement de programmes de travail (audelà des processus de consultation en ligne) et (3) dans les conseils d éthique et les comités directeur (au niveau des projets et des programmes). Nombre de ces rôles 19

24 rentreraient mieux dans la logique fondamentale des OSC (axés sur les missions) et pourraient, par conséquent, être plus intéressants que le rôle de participant comme partenaires marginalisés dans les projets. Quatre scénarios ont été développés, examinés par un modèle basé sur les agents, et analysés par rapport à la morphologie des réseaux, les parts de budget des différents types d organismes, l excellence scientifique et les impacts positifs sur les objectifs de développement durable. Ces résultats ont été comparés avec un scénario de référence bien calibré qui reflétait les parts de budget et les dimensions des projets du PC7. Le scénario «Compétitivité & haute technologie» avait pour objectif d accroître la compétitivité par la promotion de secteurs de recherche axés sur la technologie (tels que les TIC, les nanotechnologies (NMP) ou le transport) et de financer des projets moins nombreux, mais plus amples que dans le passé. Les résultats ont montré une plus grande part du secteur des affaires, des effets de haute concentration comparables, mais aussi de plus grandes parts d OSC axées sur les affaires (OSC4) que dans le scénario de référence, alors que la part des OSC axées sur les citoyens et les sociétés (OSC1-3) est resté marginale. Ce scénario a aussi démontré de forts effets d apprentissage en matière de disciplines scientifiques et des effets positifs sur le renforcement des capacités. Le scénario «Cohésion sociale & bien-être» se concentrait sur les défis sociétaux et sur les types de recherche axés sur les processus (tels que la santé, les SHS, la science dans la société (SiS) et les régions de la connaissance) en finançant des projets plus nombreux, mais de moindre ampleur que dans le passé. Il a montré la plus forte augmentation de l implication d OSC à 5% des organismes et à 1,3% du budget, permettant ainsi, particulièrement aux OSCs visant les citoyens et les sociétés (OSC1-3), de doubler leur nombre et leur budget. Ce scénario a aussi montré un réseau plus stable (plus haut degrès moyen, plus de regroupement) avec des collaborations qui étaient distribuées plus équitablement entre les organismes (moins de centralisation). En outre, il a montré des effets positifs sur le renforcement de capacités et le plus grand effet d apprentissage en ce qui concerne les défis sociétaux, mains sans affaiblir pour autant l excellence scientifique. Le scénario «Europe-forteresse» a surtout concentré sa recherche et son innovation sur des sujets tournant autour de la sécurité et de la santé en vue de maintenir la sécurité en Europe et de la rendre plus autonome. Parmi les quatre scénarios, c est celui-ci qui montre le moins de différences par rapport au scénario de référence en matière de promotion d organismes particuliers ou de création de forts réseaux ou d effets d apprentissage. Cependant, si l on regarde les OSCs, ce scénario se démarque en raison de la plus petite part de budget pour les OSCs (environ 0,9% uniquement). Le scénario «PC démantèlement» simule une coupe budgétaire dramatique des parties collaboratives et axées sur les missions des PCs, soit par le reflet d une coupe générale dans le budget des PCs, soit par le reflet de redistribution de budget vers le CER. Cette coupe budgétaire a pour résultat un abandon dramatique d organismes plus petits (y compris des OSCs), alors que les grands acteurs (tels que les universités) parviennent à rester des acteurs fondamentaux. Cela mène à un réseau étroitement connecté qui est dominé par un nombre diminué de parties prenantes. En 20

25 outre, la capacité des organismes, autant en termes de connaissances que d impacts, est considérablement plus petite que dans les trois autres scénarios. À l instar des résultats du modelage axé sur les agents, les quatre scénarios se distinguent les uns des autres en fonction de leurs effets sur le réseau de recherche et d innovation en général, de l implication des OSCs et des effets d apprentissage de tout type d organisme impliqué. Lorsqu il s agit de relever les défis sociétaux, le rapprochement de la science et des citoyens et l augmentation de la diversité et de la résilience des réseaux européen de recherche et d innovation sont considérés comme des objectifs stratégiques importants, ce qui a pour conséquence que le scénario «cohésion sociale & bien-être» est le plus propice : il accroît de manière considérable les capacités de toutes les parties prenantes pour contribuer à résoudre des problèmes sociétaux sans affaiblir pour autant l excellence scientifique et sans perturber les réseaux existants ou changer complètement la nature du programme, et est donc encore en mesure de s occuper de manière adéquate de la logique des acteurs principaux (universités, organismes de recherche et entreprises). Cela a pour effet de, indirectement, doubler l implication des OSCs dans les PCs. Recommandations pour le PC9 à venir En ce qui concerne le PC9 à venir, des décisions sur l orientation stratégique du programme et ses axes thématiques doivent être prises. Elles devront prendre en considération les différentes logiques des parties prenantes les plus importantes impliquées ainsi que les «monnaies» 2 et les principes de gouvernance qui sont utilisés par les parties : Le premier pilier d Horizon 2020, la «Science excellente», met l accent sur la recherche exploratoire qui a pour objectif de nouvelles découvertes, vise des chercheurs individuels, s occupe de leur excellence en termes de «monnaie» telle qu elle est mesurée par les publications et les parcours universitaires, et remet des bourses dans un processus hautement sélectif qui se base sur l évaluation effectuée par d autres chercheurs. À travers ses hautes ambitions et son accent sur l excellence, ce processus est hautement sélectif, mais libre de toute influence audelà de la communauté scientifique. Par conséquent, la gouvernance ne fait référence qu à elle-même et considérer les impacts sociétaux est souvent considérée comme un affaiblissement potentiel de l excellence. Le deuxième pilier d Horizon 2020, le «primauté industrielle», se concentre sur l innovation, vise des secteurs d affaires hautement innovateurs et a pour objectif de stimuler leur compétitivité, leur croissance et leur création d emplois. La «monnaie» de ce pilier est donc le profit et les brevets sont utilisés pour assurer des avantages compétitifs. Cela mène à l exclusion (de possibles concurrents) et un établissement 2 L idée des «monnaies» provient du point de vue que différentes unités existent dans différents champs sociaux pour mesurer le statut à l intérieur de ce champ (par ex. le bilan des publications pour des personnes travaillant dans une université). La métaphore de «monnaie» souligne le fait que, tandis que chaque monnaie a une valeur particulière dans un champ particulier, la conversion d'une monnaie en une autre devrait également être possible (à l instar de l euro qui peut être converti en dollars américains) afin de permettre ainsi l échange à travers différents champs. 21

26 conjoint de l ordre du jour (dans un cercle interne de participants) comme principes de gouvernance. Le troisième pilier d Horizon 2020, les «Défis sociétaux», met l accent sur les impacts de pratiques sociétales, vise les citoyens et la société civile et suit une logique fondamentale de responsabilité (politique). L augmentation d impacts positifs (et la réduction des négatifs) constitue, ainsi, le critère de succès pour les projets. Alors que les deux autres piliers courent le risque d un «verrouillage» (par la sélection d un côté et l exclusion de l autre), ce pilier peut mettre en équilibre ces tendances et servir d élément intégrant. En outre, alors que les deux autres piliers suivent des principes de gouvernance axés sur les initiés, le troisième pilier a le potentiel de relever des défis sociétaux majeurs d une pertinence globale. Si la politique européenne de recherche et d innovation souhaite assurer sa légitimité face aux citoyens européens, un troisième pilier qui se concentre sur de grands défis sociétaux est d une importance vitale (et non la combinaison du deuxième et du troisième pilier sous le titre de «recherche axée sur des missions»). Les objectifs de développement durable fixés au niveau global pourraient fournir une orientation et empêcher que l établissement de l ordre du jour soit motivé par des groupes d intérêts. Afin d ouvrir davantage le monde de la recherche à la société et inversement, d ouvrir la société au monde de la recherche, des espaces de cocréation de connaissances et d innovation devraient être fondés en impliquant une grande variété de parties prenantes. Les mécanismes de financement devraient refléter le fait que les défis sociétaux soulèvent des problèmes «pernicieux» et que les innovations ne sont pas des produits finaux, mais des processus continus. Par conséquent, les interventions devraient se concentrer sur la gouvernance d'écosystèmes (et pas seulement de projets), développer des capacités et des structures institutionnelles et permettre davantage de processus continus (en surmontant la perspective limitée de projets individuels). La recherche et l innovation responsable (RIR) devraient alors devenir un principe de guidance dans les trois piliers et thèmes des PCs. Cela constituera une garantie pour que de futurs PCs mettent en avant non seulement la recherche exploratoire au sein d universités et d organismes de recherche ou soutiennent la compétitivité des entreprises, mais aussi, qu ils prennent en considération les besoins des citoyens et qu ils relèvent des défis sociétaux. 22

27 Kurzfassung Ziel Ziel dieses Berichts ist es, die Teilnahme von zivilgesellschaftlichen Organisationen (Civil Society Organisations CSOs) in den Europäischen Forschungsrahmenprogrammen (Framework Programmes FPs), insbesondere in FP6 und FP7, zu untersuchen und daraus Politikempfehlungen zur Stärkung der Rolle von CSOs abzuleiten. Die Studie verfolgte dabei die folgenden Ziele: (1) die Entwicklung einer praktikablen Klassifizierung von CSOs, welche helfen soll, die Rollen der CSOs innerhalb der Europäischen Forschungsrahmenprogramme im Rahmen von Responsible Research and Innovation (RRI) zu unterscheiden, sowie deren spezifisches Wissen, Verhalten und latente Präferenzen die ihre Teilnahme bestimmen zu verstehen; (2) die Untersuchung des Ausmaßes der CSO Beteiligung in den FPs und die Bestimmung ihres spezifischen Beitrag zum Forschungsnetzwerk mittels sozialer Netzwerkanalyse; und (3) die Simulation verschiedener Politikszenarien und deren Effekte auf die Teilnahme von CSOs mittels agentenbasierter Modellierung, wobei das Agentenverhalten nicht wie in bestehenden Modellierungsansätzen üblich rein durch wissenschaftliche Exzellenz determiniert ist, sondern mit einer Orientierung zur Lösung gesellschaftlicher Herausforderungen, die auf den Nachhaltigkeitszielen der UN (Sustainable Development Goals SDGs) basieren, ergänzt wird. Methodischer Ansatz In dieser Studie wurden mehrere methodische Ansätze verknüpft. Dies umfasste Literaturrecherchen über Soziale Netzwerkanalyse (SNA) sowie zivilgesellschaftliche Organisationen in Forschung und Innovation (F&I), Recherchen und eine Umfrage zur Entwicklung einer geeigneten Definition und Typologie von CSOs in den Rahmenprogrammen, und basierend darauf die Identifizierung und Kategorisierung von CSOs in FP6 und FP7 Projektdatensätzen. Die zentralen Methoden für den empirischen Teil dieser Studie waren Fallstudien mit Tiefeninterviews mit teilnehmenden CSOs, soziale Netzwerkanalysen der Forschungsnetzwerke in FP6 und FP7, mit spezifischem Fokus auf CSOs, Publikations- und Medienanalysen von Forschungsprojekten mit CSO Beteiligung, sowie agentenbasierte Modellierung zur Simulation der Effekte politischer Interventionen auf die FP-Netzwerke und insbesondere auf CSO-Beteiligungen. Der für diesen Zweck entwickelte Simulationsansatz erweitert existierende Modelle, indem nicht nur Wissensflüsse in Forschungsnetzwerken sondern auch die Problemlösungskompetenzen der teilnehmenden Akteure in Bezug auf soziale Herausforderungen wie z.b. den UN SDGs berücksichtigt werden. 23

28 Zentrale Erkenntnisse und Empfehlungen CSO-Teilnahme in den FPs CSOs stellen eine sehr heterogene Gruppe verschiedener Organisationstypen (hinsichtlich Größe, strategischer Orientierung, Geschäftsmodell, Finanzierungsquellen, Zielgruppen, usw.) dar, wodurch eine klare Trennung, Charakterisierung und Identifizierung von CSOs in den Rahmenprogrammen erschwert wird. Vor diesem Hintergrund entwickelte die Studie einen Zugang, der zwischen vier Typen von CSOs unterscheidet: bürgerorientierte ( kern ) CSOs (CSO1), gesellschaftsorientierte/öffentlich-finanzierte CSOs (CSO2), gesellschaftsorientierte/unternehmensfinanzierte CSOs (CSO3), und unternehmensorientierte CSOs (CSO4). Die Resultate der Studie zeigen, dass die Herangehensweise geeignet ist, die Teilnahmegewohnheiten von CSOs in FPs zu beurteilen. Wir raten dringend zu einer zukünftig besseren Kategorisierung von CSOs im CORDA- Datensatz, entsprechend der in dieser Studie angewandten Typologie. CSOs sind vergleichsweise unbedeutende Akteure im FP-Netzwerk. CSOs sind im Durchschnitt in einem von dreizehn Projekten eingebunden; sie machen ungefähr 6% aller teilnehmenden Organisationen in FP7 aus, erhalten aber nur etwa 1% der von der Europäischen Kommission zur Verfügung gestellten Förderungen. Nur etwa 10% aller in FP7 teilnehmenden CSOs koordinierten ein Projekt, und nur etwa 1% aller FP7 Projekte wurden von CSOs koordiniert. CSOs nehmen in der Regel keine zentrale Rolle in den Projekten ein. Dies liegt unter anderem an einer niedrigeren Stellung innerhalb des Projektkonsortiums, Unklarheiten bei der Aufgabenteilung, und an der Befürchtung von akademischen Forschungspartnern, die Teilnahme von CSOs könnte sich negativ auf die wissenschaftliche Reputation auswirken. Aufgrund ihrer unbedeutenden Rolle in FP-Netzwerken können von CSOs keine signifikanten Wirkungen bei einer Teilnahme an Forschungsprojekten erwartet werden. CSOs weisen kein systematisches Verhaltensmuster in Bezug auf FP-Beteiligung auf, in machen Themen treten sie aber öfter und stärker in Erscheinung als in anderen: Science in Society (SiS) ist das einzige Thema in dem bürger- und gesellschaftsorientierte CSOs (CSO1-3) eine bedeutsame Rolle spielen. Auch Regions of Knowledge (RoK) und Research of the benefits of SMEs (SME) zeigen überdurchschnittlich hohe Anteile von CSOs, was aber vor allem an unternehmensorientierten CSOs (CSO4) liegt. Activities of International Cooperation (INCO), Research policies (COH), und Socio-economic Sciences and Humanities (SSH) zeigen kleine aber dennoch über dem Durchschnitt liegende Budgetanteile für bürger- und gesellschaftsorientierte CSOs (CSO1-3). Absolut gesehen stellen auch die Themen Health und Information and Communication Technologies (ICT) aufgrund der Höhe des zur Verfügung gestellten EU Förderungsbudgets eine wichtige Finanzierungsquelle für CSOs dar. In anderen wichtigen Teilen des FPs, wie dem European Research Council (ERC), den 24

29 Marie Skłodowska-Curie Actions sowie in den Themen Nanotechnologies (NMP) und Transport nehmen CSOs praktisch nicht teil. Eine Erhöhung des Budgets für Themen die eine höhere CSO-Beteiligung aufweisen würde zu einer Stärkung der Rolle der Zivilgesellschaft in den FPs führen und Forschung den BürgerInnen näher bringen. Derartige Budgetverschiebungen bedürfen aber noch weiterer flankierender Maßnahmen, wie kleinere Projekte und eine Veränderung der Gewichtungen bei der Antragsevaluierung. Die meisten CSOs schaffen es nicht, mehrmalig bzw. längerfristig an den FPs teilzunehmen. 85% der CSOs aus FP6 nahmen nicht mehr in FP7 teil, was der höchsten Ausstiegsrate aller teilnehmenden Organisationstypen entspricht. Rund 70% der CSOs in FP7 nahmen an nur einem einzigen Projekt teil, für FP6 ist dieser Anteil sogar noch geringfügig höher. Diese fehlende Kontinuität verhindert die Verbreitung positiver Effekte der CSO-Teilnahme (dies wäre möglich wenn dieselben CSOs in mehreren Projekten und in unterschiedlichen Themen teilnehmen würden). Gründe dafür sind eine hohe Personalfluktuation in CSOs und, daraus resultierend, ein mangelndes organisationales Gedächtnis. Der Aufbau institutioneller Kapazitäten und die kontinuierliche Kompetenzentwicklung in CSOs sind Voraussetzungen zur Teilnahme an den FPs. Dies könnte durch Mitarbeiteraustausch und mittels gezielter Maßnahmen zum Aufbau institutioneller Kapazitäten erreicht werden. Die Logik von CSOs steht diametral zur kompetitiven und exzellenzorientierten Forschungs- und Innovationslogik. Missionsgetriebene CSOs orientieren sich an politischen Herausforderungen und den Bedürfnissen der von ihnen vertretenen Akteure und legen wenig Wert auf wissenschaftliche Publikationen oder akademische Laufbahnen. Ihr Ziel ist die Beeinflussung aktueller politischer nationaler und regionaler Entscheidungen, daher nutzt ihnen die Teilnahme an mehrjährigen Projekten kaum. Andererseits ziehen viele dienstleistungsorientierte CSOs, deren Kerngeschäft meist wenig innovativ ist, nur sehr geringen Zusatznutzen aus Forschung und Innovation. CSOs nehmen in den FPs daher eher in Koordinierungs- und Unterstützungsmaßnahmen (CSAs) als in echten Forschungs- und/oder Innovationsprojekten teil. Die Einbindung von CSOs in Agenda-Setting, in die Antragsevaluierung und in die Verbreitung von Ergebnissen würde eine substanziellere Involvierung von CSOs in die FPs ermöglich, welche über die Teilnahme an Koordinierungs- und Unterstützungsmaßnahmen hinausgeht. CSOs ziehen oft keinen Europäischen Zusatznutzen durch die Teilnahme an EUweiten Projekten, da sie sich zumeist an nationalen und regionalen Politiken orientieren. Das betrifft insbesondere sozialpolitische Fragestellungen für welche keine gemeinsame EU-Politik existiert. Im Vergleich zu den FPs wird der Europäische Strukturund Investitionsfonds der regionalen/lokalen geografischen Ausrichtung vieler CSOs besser gerecht. 25

30 Eine bessere Abstimmung der FPs und des Europäischen Struktur- und Investitionsfonds, wie in der FP7 ex-post Evaluierung vorgeschlagen, könnte indirekt zu einer stärkeren Einbindung von CSOs in transnationale Forschungs- und Innovationsaktivitäten führen. Der Nutzen aus der Teilnahme von CSOs in den FPs CSOs verfolgen weder eine akademische (auf Exzellenz in wissenschaftlichen Disziplinen) noch eine wirtschaftliche (geprägt von Wettbewerbs- und Gewinnstreben) Logik. Stattdessen liegt ihr Fokus auf der Lösung gesellschaftlicher Probleme entweder als missionsgetriebene CSOs durch die Beeinflussung politischer Entscheidungen, oder als dienstleistungsorientierte CSOs, durch die Verbesserung der Situation ihrer primären Zielgruppe. Aufgrund dieser Ausrichtung ist ein wissenschaftlicher Beitrag von CSOs vor allem dort zu erwarten wo Forschung die Ziele von CSOs fördert. Wegen ihrer unbedeutenden Rolle in den FPs tragen CSOs zudem nicht zur Netzwerk- Performance bei. Eine soziale Netzwerkanalyse zeigte, dass die Teilnahme von CSOs praktisch nicht von der Netzwerkmorphologie abhängt: in FP7 existiert nahezu keine Korrelation zwischen Netzwerkparametern und der Teilnahme von CSOs. CSOs sind hangers-on in FP-Netzwerken, das heißt sie spielen keine bedeutende vernetzende Rolle, sind keine Broker, konstituieren nicht oder nur in geringem Maße Communities, und verbinden nicht unterschiedliche Netzwerkbereiche (d.h. sie überbrücken auch keine strukturellen Löcher). Sie sind daher auch nicht entscheidend für den Netzwerkaufbau auf Makroebene: das Entfernen von CSOs hätte praktisch keinen Effekt auf Netzwerkmaße und -morphologie. Der Nutzen der Teilnahme von CSOs in den FPs kann nicht hinreichend durch die Verwendung gängiger Erfolgskriterien wie Netzwerk-Performance oder Forschungs- Performance bestimmt werden. Dies liegt an der grundsätzlich unterschiedlichen Logik (oder Währung ) von CSOs: während die FPs der Logik von Forschung ausgerichtet auf wissenschaftliche Exzellenz und Unternehmen ausgelegt auf Gewinn und Wettbewerb folgen, denken CSOs in der Logik gesellschaftlicher Auswirkungen. Zur Beurteilung des Nutzens der Teilnahme missionsgetriebener CSOs müssen politische Auswirkungen berücksichtigt werden, während die Vorteile aus der Beteiligung dienstleistungsorientierter CSOs durch positive Auswirkungen auf die von ihnen vertretenen Akteure gemessen werden können. Wir empfehlen daher, den Nutzen der Teilnahme von CSOs in den FPs aus einem neuen Blickwinkel zu betrachten, nämlich durch die Beurteilung des gesellschaftlichen Gesamtnutzens der FPs, und durch die Identifizierung von (neuen) Rollen mittels derer CSOs am besten zur Erhöhung dieses gesellschaftlichen Nutzens beitragen können. 26

31 Wie die CSO-Beteiligung in den FPs erhöht werden kann Eine Reihe von Fallbeispielen und die Szenarien der agentenbasierten Simulation haben gezeigt, dass ein Mikro-Management einzelner Teile des FPs nur geringe Auswirkungen auf das Netzwerk sowie die Beteiligung von CSOs hat. Mikromanagement individueller Programmteile hat nur marginale Auswirkungen auf die Beteiligung von CSOs als Projektpartner. Wir empfehlen daher, eine größere Bandbreite an Rollen für CSOs in zukünftigen FPs anzudenken: (1) in Evaluierungsgremien für FP-Anträge (zumindest für den Impact Teil), (2) beim Agenda-Setting und der Entwicklung von Arbeitsprogrammen (in substantiellerer Weise als online-basierte Konsultationen), und (3) in Ethik-Gremien und Lenkungsausschüssen (auf Projekt- und Programmebene). Viele dieser Rollen würden besser zur missionsorientierten Logik von CSOs passen und könnten daher für diese attraktiver sein, als eine untergeordnete Rolle als Projektpartner zu spielen. Im Rahmen der Studie wurden vier Szenarien mittels eines agentenbasierten Modells simuliert. Diese wurden dann hinsichtlich ihrer Effekte auf Netzwerkmorphologie, Budgetanteilen verschiedener Organisationstypen, Wissenschaftsexzellenz und deren positiven Auswirkungen auf die UN Nachhaltigkeitsziele (SDGs) analysiert. Die Ergebnisse wurden mit einem Basisszenario, das Budgetanteile und Projektgrößen von FP7 widerspiegelt, verglichen. Das Szenario Wettbewerbsfähigkeit & High-Tech strebt die Stärkung der Wettbewerbsfähigkeit durch die Förderung von technologieorientierten Forschungsbereichen (wie z.b. ICT, Nanotechnologies (NMP) oder Transport) und die Finanzierung von wenigen, aber größeren Projekten als früher an. Gemäß den Simulationsergebnissen führt dieses Szenario zu einem höheren Anteil des Unternehmenssektors, vergleichsweise hohen Konzentrationseffekten, aber auch einem im Vergleich zum Basisszenario höheren Anteil von unternehmensorientierten CSOs (CSO4). Der Anteil von bürger- und gesellschaftsorientierten CSOs (CSO1-3) bleibt hingegen unbedeutend. Dieses Szenario weist die größten Lerneffekte in Bezug auf wissenschaftliche Disziplinen auf. Das Szenario Soziale Kohäsion & Wohlbefinden legt den Fokus auf die Adressierung gesellschaftlicher Herausforderungen durch mehr prozessorientierte Forschung (wie z.b. Health, Socio-economic Sciences and Humanities (SSH), Science in Society (SiS) und Regions of Knowledge (RoK)) und die Förderung von mehr, aber kleineren Projekten. Gemäß den Simulationsergebnissen weist dieses Szenario mit 5% der Organisationen und 1,3% des Budgets die höchsten Anteile an CSO-Beteiligung auf. Insbesondere bürger- und gesellschaftsorientierte CSOs (CSO1-3) profitieren überdurchschnittlich stark. Dieses Szenario weist auch ein stabileres Netzwerk auf (höherer average degree, stärkeres clustering) mit gleichmäßiger verteilten Kooperationen zwischen Organisationen (weniger Zentralisierung). Außerdem zeigen sich die größten Lerneffekte hinsichtlich gesellschaftlicher Herausforderungen, wobei die wissenschaftliche Exzellenz nicht beeinträchtigt wird. 27

32 Das Szenario Festung Europa simuliert eine Abschottung Europas und fokussiert Ausgaben für Forschung und Innovation daher in erster Linie auf sicherheits- und gesundheitsrelevante Themen bzw. Versorgungssicherheit mit Energie. Unter den vier Szenarien weist dieses die geringsten Unterschiede zum Basisszenario hinsichtlich der Förderung spezifischer Organisationen, der Bildung starker Netzwerke, oder bezüglich Lerneffekten auf. Es sticht jedoch heraus in Bezug auf CSOs, welche hier mit ca. 0,9% den geringsten Budgetanteil aller Szenarien erhalten. Das Szenario FP-Rollback simuliert eine dramatische Budgetkürzung der kollaborativen bzw. missionsorientierten Teile des FP. Dies spiegelt entweder eine grundsätzliche Kürzung des EU-Budgets für Forschung und Innovation oder eine signifikante Budgetverschiebung in Richtung des ERC wider. Gemäß den Simulationsergebnissen resultiert diese Budgetkürzung in einem dramatischen Rückgang kleinerer Organisationen (inklusive CSOs), während große Akteure (wie Universitäten) Schlüsselakteure bleiben. Das führt zu einem deutlich dichteren Netzwerk, welches von wenigen zentralen Akteuren dominiert wird. Außerdem zeigen sich deutlich niedrigere Lerneffekte der teilnehmenden Organisationen als in den anderen drei Szenarien. Die Ergebnisse der agentenbasierten Modellierung zeigen deutliche Unterschiede der vier Szenarien hinsichtlich ihrer Effekte auf das gesamte Forschungs- und Innovationsnetzwerk, der Beteiligung von CSOs, und den Lerneffekten der beteiligten Organisationstypen. Wenn die Lösung gesellschaftlicher Herausforderungen, eine bessere Verankerung von Wissenschaft in der Gesellschaft, sowie die Erhöhung von Diversität und Resilienz des Europäischen Forschungs- und Innovationsnetzwerks als wichtige politische Ziele angesehen werden, zeigt das Szenario Soziale Kohäsion & Wohlbefinden den größten Nutzen: Es steigert die Kapazitäten aller Akteure zur Lösung gesellschaftlicher Probleme deutlich, ohne dabei die wissenschaftliche Exzellenz und bestehende Netzwerke zu schwächen oder das Programm komplett zu verändern. Noch dazu wird der Logik der wichtigsten Akteure (Universitäten, Forschungsorganisationen und Unternehmen) genügend Rechnung getragen. Als Nebeneffekt wird die Beteiligung von CSOs in den FPs verdoppelt. Empfehlungen für das 9. Forschungsrahmenprogramm (FP9) Im Hinblick auf das nächste Forschungsrahmenprogramm dem FP9 stehen strategische Entscheidungen betreffend der Ausrichtung und der thematischen Schwerpunkte an. Dabei müssen die verschiedenen Logiken der involvierten Akteure sowie die Währungen 3 und deren Governance-Grundsätze berücksichtigt werden: 3 Der Begriff Währungen basiert auf dem Verständnis, dass in verschiedenen sozialen Kontexten unterschiedliche Einheiten zur Messung des sozialen Status existieren (z.b. Publikationsleistungen von ForscherInnen an Universitäten). Die Metapher Währung soll darauf aufmerksam machen, 28

33 Die erste Säule von Horizon 2020, Wissenschaftsexzellenz, konzentriert sich auf Pionierforschung zur Gewinnung neuer Erkenntnisse, durchgeführt von einzelnen (exzellenten) ForscherInnen. Exzellenz als Währung dieser Säule bemisst sich in Form von Publikationen und akademischen Laufbahnen. Förderungen werden in einem stark selektiven Prozess, basierend auf Begutachtungen von anderen WissenschaftlerInnen vergeben. Die hohen Ambitionen und der Fokus auf Exzellenz basieren auf einem Prozess, der stark selektiv, jedoch frei von Einflüssen außerhalb der Wissenschaft ist. Die Governance dieser Säule ist somit selbstreferenziell, und die Berücksichtigung gesellschaftlicher Auswirkungen wird oft als potentielle Verwässerung von Exzellenz erachtet. Die zweite Säule von Horizon 2020, Führende Rolle der Industrie, richtet ihren Fokus auf Innovation, zielt auf hoch innovative Wirtschaftssektoren ab, und strebt die Förderung von Wettbewerbsfähigkeit, Wirtschaftswachstum und die Schaffung von Arbeitsplätzen an. Als Währung dieser Säule gelten Profite sowie Patente zur Sicherung von Wettbewerbsvorteilen. Dies führt zu Governance-Grundsätzen, die von Exklusion (potentieller Mitbewerber) und gemeinsamen Agenda-Setting (innerhalb eines kleinen, geschlossenen Kreises) geprägt sind. Die dritte Säule von Horizon 2020, Gesellschaftliche Herausforderungen, richtet ihren Fokus auf die Lösung gesellschaftlicher Herausforderungen, zielt auf BürgerInnen und die Zivilgesellschaft ab, und folgt der Logik von (politischer) Verantwortung. Zu den Erfolgskriterien dieser Projekte zählen deshalb die Erhöhung positiver (und die Verminderung negativer) Auswirkungen. Während für die anderen zwei Säulen die Gefahr eines Lock-ins (einerseits durch Selektion, und andererseits durch Exklusion) besteht, kann diese Säule entsprechende Tendenzen ausgleichen und als integratives Element dienen. Zudem verfolgen die beiden die anderen Säulen Insider-orientierten Governance-Grundsätzen, während die dritte Säule das Potential hat, die wichtigsten gesellschaftlichen Herausforderungen auf globaler Ebene aufzugreifen. Wenn Europäische Forschungs- und Innovationspolitik ihre Legitimität gegenüber Europäischen BürgerInnen bewahren will, ist eine unabhängige dritte Säule mit einem Fokus auf die großen gesellschaftlichen Herausforderungen entscheidend (d.h. kein Zusammenlegen der zweiten und dritten Säule unter dem Titel missionsgetriebene Forschung ). Die globalen UN Nachhaltigkeitsziele (SDGs) könnten als Orientierungshilfe zur Schwerpunktsetzung dienen und vermeiden, dass Agenda- Setting von Interessensgruppen gelenkt wird. Um Forschung näher an Menschen und Menschen näher an Forschung zu bringen braucht es Plätze zur gemeinsamen Wissensfindung und Innovation, in die eine Vielfalt von Stakeholdern eingebunden werden. Die Förderinstrumente sollten die Tatsache widerspiegeln, dass es sich bei dass jede einzelne Währung eines bestimmten Bereiches auch in eine andere Währung konvertiert werden können sollte (ähnlich dem Umwechseln von Euro Banknoten in US Dollar), um einen Austausch zwischen verschiedenen Kontexten zu ermöglichen. 29

34 gesellschaftlichen Herausforderungen um komplexe Probleme ( wicked problems ) handelt, und dass Innovationen nicht endgültige Ergebnisse sondern vielmehr stetige Prozesse sind. Daher sollte der Fokus von Interventionen auf der Gestaltung von offenen Prozessen (und nicht nur Projekten) und dem Aufbaue institutioneller Kapazitäten liegen und stetige Prozesse erlauben welche die limitierte Perspektive individueller Projekte aufheben. Verantwortungsvolle Forschung und Innovation (Responsible Research and Innovation - RRI) sollte ein Leitprinzip aller drei Säulen und Themen der FPs werden. Dadurch kann gewährleistet werden, dass zukünftige FPs nicht nur Pionierforschung an Universitäten und Forschungsorganisationen fördern oder die Wettbewerbsfähigkeit von Unternehmen unterstützen, sondern auch die Bedürfnisse der BürgerInnen berücksichtigen und sich den relevanten gesellschaftlichen Herausforderungen annehmen. 30

35 1 Introduction Responsible Research and Innovation (RRI) relies on the assumption that, in order to find sustainable research and innovation (R&I) solutions for societal challenges, all societal actors (researchers, citizens, policy makers, business, civil society organisations) should work together during the whole R&I process, aligning priorities of research and innovation to the values, needs and expectations of society. Horizon 2020, the EU s current research framework programme, goes conform with this understanding of involving civil society into the overall research landscape: With the aim of deepening the relationship between science and society and reinforcing public confidence in science, Horizon 2020 should favour an informed engagement of citizens and civil society on research and innovation matters by promoting science education, by making scientific knowledge more accessible, by developing responsible research and innovation agendas that meet citizens' and civil society's concerns and expectations and by facilitating their participation in Horizon 2020 activities. Previous monitoring and evaluation activities related to the FPs have mainly investigated participation patterns of the classical organisation types defined in the framework programmes: universities (HES), private research organisations (REC), companies (PRC), public bodies (PUB), and other organisations not fitting to any of the other four categories (OTH). These analyses continue to show that European R&I activities are dominated by universities and companies. The currency of universities is excellence, measured in the form of scientific publications. The currency of companies is profit, in terms of filed patents. FP7 specifically addressed these two currencies by creating two new vehicles in 2007: On the one hand the European Research Council (ERC) was set up, encouraging the highest quality research in Europe through competitive funding and supporting investigator-driven frontier research across all fields, on the basis of scientific excellence. On the other hand, Joint Technology Initiatives (JTIs) were created to support large-scale multinational research activities in areas of major interest to European industrial competitiveness. The concept of RRI shifts the focus on finding sustainable R&I solutions for societal challenges; it seeks to achieve this by better engaging society in R&I activities. RRI thus introduces a third currency to the R&I landscape - the currency of responsibility. A number of recent research activities have consequently focused on the question of how to better involve civil society organisations (CSOs) which have so far been a marginal player in the EU s framework programmes in R&I activities. These include FP-funded research projects such as CONSIDER ( Civil Society Organisations in Designing Research Governance ; SecurePART ( Increasing the Engagement of Civil Society in Security Research ; and Responsible-Industry ( Against this background, the DG Research & Innovation has commissioned a study aimed at identifying CSOs in the FPs, analysing their participation patterns in FP6 and FP7, and simulating policy interventions for strengthening CSOs role in FP-funded research. 31

36 This report summarises the findings of this study, which pursued the following objectives: 1. Developing a practical typology of CSOs that can help differentiate and understand the roles CSOs can play in European research Framework programmes in the context of RRI, as well as their specific knowledge, properties and latent preferences determining their participation. 2. Analysing the extent of CSO involvement in European research framework programmes and determining their specific contribution to research and collaborative research network performance through Social Network Analysis. 3. Simulating different policy scenarios and their effects on CSO participation, through expanding already existing modelling approaches by complementing knowledge-based agent behaviour with an orientation towards solving societal challenges based on the Sustainable Development Goals (SDGs). This report is structured according to the main tasks carried out in the project. Chapters 2 and 3 summarise the findings from initial literature reviews on social network analysis (SNA) approaches and on CSO definitions and logics. Chapter 4 presents the work on identifying and categorising CSOs participating in the FPs. Chapter 5 summarises the findings from case studies with different kinds of CSOs. Chapter 6 describes the results of the social network analysis (SNA), focusing particularly on the role of CSOs in the European research networks. More detailed SNA results are included in the Annex to this report. Chapter 7 is dedicated to the agent-based simulation model that was developed for the purpose of this project. It briefly explains the model and looks in more detail into the results of four main scenarios that were simulated in view of deriving policy recommendations on CSO involvement in the FPs. A discussion with stakeholders at a workshop held at the EuroScience Open Forum (ESOF) 2016 in Manchester in July 2016 helped to refine the model and to develop the scenarios. The summary of the workshop as well as the detailed description of the model itself is included in the Annex to this report. Chapter 8 finally presents conclusions and policy recommendations regarding the involvement of CSOs in the European research framework programmes as well as an outlook on future research topics. 32

37 2 Literature review of network analysis approaches The main objective of this task was to review SNA (Social Network Analysis) literature in terms of R&I (research and innovation) performance and cooperation structures in general and in EU Framework Programs and to review of R&I related SNA literature in the context of agents based modelling. The task also aimed at developing a Multi-Level Multi- Theory approach composed of existing theories combined in a new way that provides a deeper analysis and understanding of social network structures in the context of R&I performance and the integration of CSOs in the cooperative processes. Finally, a terminological guideline was made to define a common understanding of the used terms. The use of SNA for evaluating European cooperation structures is not a radically new approach; however, the discipline itself is constantly developing and unfolds into more and more specialized subfields ranging from social sciences, computer science, physics to life sciences. Since increasing IT capacities led to vast possibilities of data mining and collection, SNA methods and tools were forced to constantly improve in order to handle big data. Due to the complexity and dynamics of the discipline it is crucial to establish an overview of the state of the art. The literature research was structured as follows: Stocktaking and systematically reviewing existing literature on Social Network Analysis (SNA), Building a conceptual and analytical SNA framework in the context of the tendered project, Determining strengths and weaknesses of different academic fields of SNA, Detail the most accepted terminology describing the terms to be used in the study, Review of Commission SNA studies and projects in this area. All in all around 120 relevant journals articles, book chapters and studies were reviewed for Task 1. The output comprises a report which describes the crucial results in detail. Additionally an annex was created containing an overview of all reviewed sources in form of a table following the categories: Author and Title; Analyzed Dataset; Main Variables; (Impact) Indicators; Methods and Types of Calculation; Main Findings; Differences, Strengths and Weaknesses; Findings relevant for the project. The following pages give a brief overview of the main results. 2.1 Critical analysis of previous SNA approaches and studies in R&I evaluation and measurement of impact in general This sub-task focused on a critical analysis of SNA approaches in studies related to R&I evaluation, impact measurement and scientific team performance in general apart from the EU Research Program. SNA based evaluation of scientific networks and R&I related networks since the 1960s and 1970s focused on the study of scientific collaboration networks such as citation, bibliometric and patent networks (e.g. Ko et al 2014, Newman 33

38 2002/2004, Barabasi et al. 2002/2008, Hicks and Katz 1996, Price 1965) 4, the evaluation of collaboration on the basis of CVs and home universities (e.g. Bozeman et al. 2009, Gaughan et al 2008, Tuire et al 2001), the exploration of the invisible college 5 (Crane 1965/1972, Tuire 2001) and the examination of knowledge and communication flows (e.g. Allen/Cohen 1969; cf. Mote et al, 2007: 191/Rogers et al 2001: 162). The data for these studies are mostly gained from scientific databases and journals of different disciplines 6, patent databases (e.g. Ko et al., 2014) and CVs (e.g. Gaughan et al., 2008). Some common relational indicators in this field are for example the Erdös number, degree of clustering, clustering coefficient, degree distribution, network interconnectedness and average separation, average degree and centrality measures (cf. e.g. Newman 2001/2004, Barabasi 2001/2008, Chinchilla-Rodriguez 2012). Another important topic is the evaluation and comparison of formal and informal networks and the impact of the team composition and team structure on the performance (e.g. Allen et al. 2007, Borgatti et al. 2002; Mehra et al. 2006, Mote et al. 2007). Data were collected using questionnaires in different organizations, R&I teams and projects. Important variables to measure the team performance are for example productivity, key attributes of organizations and co-membership in projects. Besides the most common statistical approach, regression analysis, different SNA centrality measures (degree, closeness, betweenness, eigenvector) were applied in the mentioned studies. The following overview briefly describes the main results from this sub-task: Co-authorship networks have features of small world networks like a short node-tonode-distance and a large clustering coefficient (e.g. Newman 2000&2004, Barabasi 2001&2008). Referring to the scientific world, the participation and integration of CSOs in R&I collaborations might be difficult, because the prestige-factor of being quoted often or working together with central scientists seems to be very dominant in this field. The so called Matthew-Effect (people who are quoted often are going to be quoted more often than people who are quoted less often 7 ) leads to a strong center and the exclusion of other players and peripheral actors who have less prestige (e.g. Tuire et al, 2001). Following this hypothesis, the participation of CSOs in the scientific field/academic research is rather unlikely, as long as success is only measured by publications and other values (e.g. mainstreaming research results to the benefit of a non-academic public) are not systematically considered All sources are listed in the Annex of Deliverable D2, arranged according to the Sub-tasks in alphabetical order. Community of intercommunicating scientists working within a specific paradigm/field organized in an unofficial networks and contributing to the growth of knowledge due to the knowledge transfer, the connection to each other and the ability to link scientists (Social Research Glossary: [ ]) e.g. Medline, (biomedical research), Los Angeles e-print Archive (theoretical physics), Spires (high energy physics) and Networked Computer Science Technical Reference Library (computer science) (cf. Newman, 2000). R.K. Merton: [ ] 34

39 Informal networks of R&I teams should not be underestimated for example when it is about problem-solving strategies within teams. In these cases informal networks have often more potential besides the formal structures. SNA enables the identification of important persons and institutions within informal networks and can help to further investigate the role of CSOs within informal and formal networks and the mechanisms to enhance the capacities of a better team performance (e.g. Borgatti et al, 2002; Mehra et al., 2006; Mote et al. 2007; Allen et al., 2007). CSOs often work through activating informal networks. In this sense CSOs might have the professional competence to solve problems informally and thus increase the R&I team s resilience and productivity. The heterogeneity of a team can increase the team performance, if the team is coordinated properly and the members are well connected to each other (e.g. Reagan and Zuckerman, 2001). The Input of CSOs, which are close to (certain groups of) the civil society could make an enriching contribution to often more homogenous R&I teams. National boundaries seem to restrict the collaboration and flow of information and a shared physical location and organizational identity support the effective collaboration and knowledge exchange in R&I teams. Related to international R&I teams, finding strategies to overcome the restriction of national borders seems to be important. According to Sell et al. (2004) NGOs are often structured as transnational advocacy networks (p. 149), which could lead to a high team performance when CSOs are involved because they are used to collaborate across borders. In this sense transnationality could be an enabling factor for CSOs to participate in R&I teams, because they are well connected on an international level. CSOs are often well integrated and involved in political processes, which leads other countries to do the same. There is something like a domino-effect and the impact of good examples. It seems that one driving factor for the participation of actors on the fringes such as CSOs are legitimacy gains for political actions. Programs, addressing and taking up topics from society are more likely to integrate CSOs as justification in terms of the common goods. Best practice examples of other countries including CSOs in terms of a soft social pressure might be utilized to motivate other groups (such as scientists or politicians) to follow the bottom-up approach. But the integration of CSOs just for the sake of legitimacy should be regarded carefully to avoid CSOs being misused as token-partners to justify political actions in the name of the common good (e.g. Böhmelt 2014). All in all the results of the analysed SNA studies are rather descriptive. The network structures are described in a superficial way. It seems that there are few far-reaching analyses done of the network structures, which generate a deeper understanding of the network morphology. Further, the studies rest on a limited theoretical basis. The theoretical approach is in most cases narrowed down to the small world theory (Watts/Strogatz 1998) and the strength of weak ties (Granovetter, 1973). 35

40 2.2 Critical analysis of previous SNA studies in FP1-7 in regard of effects on R&I, CSO participation and impact on European society This sub-task critically reviewed papers and studies about SNA and R&I collaboration in the context of the European Framework Programmes. The European Framework Programmes for research and innovation are unique and distinguish themselves from other R&D instruments by some common structural key elements. First, only projects of limited duration that mobilize private and public funds at the national level are funded. Second, the focus of funding is on multinational and multi-actor collaborations that add value by operating at the European level. Third, project proposals are to be submitted by self-organized consortia and the selection for funding is based on specific scientific excellence and socio-economic relevance criteria (Barber et al, 2013). The methods and approaches used for FP's evaluation show a trend over time away from panels and towards (larger-scale) professionally conducted evaluations and studies. They are strongly focused on participant surveys, interviews and analysis of EC-internal databases (notably of FP participants) and documents. These are sometimes supplemented by case studies. Seldom there are FP databases matched to external databases and surveys, such as the Community Innovation Survey or data about regional production or value added. There has been some broadening of techniques from about 2005, when Social Network Analysis (SNA) and bibliometrics began to be used (EPEC 2011). Evaluations focused on four types of outcomes: Knowledge generation & learning, knowledge networks, knowledge spillovers and outcomes enabling commercial exploitation. The ultimate aim was to search for impacts of FPs on knowledge exploitation, collaboration in R&D, technology exploitation, innovation in industry, innovation in market structures, knowledge spill-overs to the education system and other R&D policies, improved policy development & regulations and finally innovation acceptance among end users (EPEC 2011). Some common SNA indicators used in reviewed studies are for example: evaluation of strong/weak ties, number of vertices/edges, fragmentation, clustering coefficient, mean degree and degree/eigenvector/closeness/betweenness centrality. The following overview briefly describes the main results from this sub-task: Participants are highly networked and partnerships are based on previous collaborations. Large organizations dominate the FP's networks, the key organizations are taking leading roles within research teams and projects (Wagner et al 2005, Malerba et al 2006, Wagner et al 2005, Malerba et al 2006, Avedas et al 2009, Malerba et al 2013). Therefore, the connectivity crucially depends on few organizations. Cooperation with CSOs might be of benefit for others in buffering and catalysing the overall capacity of networks along the stages of technology cycles (Ulanowicz 2000, Vedres & Stark 2010, Katzmair 2012, Walker & Salt, 2006 and 2012). Partners may directly gain from the diverse expertise and unconventional approach of civil society groups in carrying out a certain research project or in meeting legal commitments (Raustiala 1997; see also Betsill & Corell 2001; 2008; Biermann & Pattberg 2008). The research networks within the FP's 1-7 are dense and hierarchical: A highly connected core of frequent participants, taking leading roles within consortia, linked to a large number of peripheral actors, forming a giant component that exhibits the 36

41 characteristics of a small world. Participation by type of organization is skewed towards HES and REC (EC, 2013, FP7 Monitoring Report). The FP network is built around five core Western European countries, i.e. Germany, the United Kingdom, France, Italy and the Netherlands (EC 2014 EPIRIA, Ortega et al 2010). SNA studies attest the centrality of HES and REC as hubs and their role as gatekeepers. Hence, their preferences towards CSO involvement play an important role in facilitating their participation. Networks of Excellence are centred around academic institutions, while Specific Targeted Research Projects combine academic and central corporate members. SMEs and OTHERS, including CSOs participate in FP6 research networks, but they are likely to be on the fringes rather than in the centre. Consultancy and organisations of type OTHER had no or only negligible representation among core organisations. After FP3, 100 per cent of core organisations have been classified as old boys : the pool of core nodes appears to exhibit remarkable stability and has not been renewed in almost two decades (Heller-Schuh et al, 2011). The FP is a scale-free network with an accelerated growth (Heller-Schuh et al, 2011). The most important consequence of the presence of such network hubs is that the overall connectivity of the network as well as its topological properties crucially depends on few organizations. Scale-free networks are vulnerable to the targeted removal of the most important nodes thereby decreasing the ability of the remaining nodes to interact with each other (Vonortas 2013). CSOs could act as brokers between the excellence-oriented research, profit-oriented companies and societal challenges. Collaborations between partners are established by means of preferential attachment. In other words, the participants with more connections establish new ones at higher rate than participants with few connections. As a consequence, the so-called richget-richer phenomenon arises, in which the most connected participants increase their collaborations at the expense of the latecomers (Almendral et al 2007, Avedas et al 2009). Network analysis found that the main variables driving project efficiency and research success in terms of measurable outputs (such as publications or patents) are: national culture, working together over time, project and funding duration, project size, common goals, project management and stability of project members. They also found that performance decreases as project size increases (Avedas 2009, Arnold 2012). In summary, it can be suggested that research program design ought to acknowledge the value-adding function of CSOs in representing the general public by: a) opening up agenda setting to bottom up initiatives by CSOs in line with principles of responsible research and innovation RRI; b) mandating a more formal inclusion, also in coordination roles, of CSOs, recognizing their role in brokering between excellence oriented research, profit-oriented companies and societal challenges; c) broadening the indicators for performance assessments to capture disruptive innovations and more long-term effects, recognizing the function of CSOs in buffering and catalysing the overall capacity of networks along the stages of technology cycles. 37

42 2.3 Critical analysis of R&I related SNA Literature in the context agent based simulation modelling, p* modelling and systems dynamics This sub-task focused on critical analysis of R&I related SNA Literature in the context agent based simulation modelling, p* modelling and systems dynamics. The literature on simulation and networks has grown extensively over the past 10 years. Some of the primary methods are agent-based modelling, multi-agent-based simulation, mathematical equation-based models, system dynamics (Maier, Frank H. 1998; Sterman, 2000), statistical models, cellular automata worlds[1] (Epstein, 2008), exponential random graph models (ERGMs) (Borgatti et al., 2013), p*modelling (SIENA) (Snijders et al., 2006) and hybrid models. Hybrid models are a combination of any of the abovementioned techniques and have become popular recently (Axelrod, 2006). There has been also advances in computation power in this field, which have made it more tractable to compute models with large numbers of variables. Over the past 15 years, agent-based modelling has become widely used in various research fields. There is proliferation of agent-based modelling in different fields. The following overview briefly describes the main results from this sub-task: The main focus of current studies has shifted towards studying the interactions of agents. For example, in various studies (Moss et al. 1998; Korber, et al. 2011; Kiesling, Elmar et al. 2012), agents have more complex characteristics like being able to adapt to different changes in the environment or being influenced or not by another agent. Also agents can take into consideration the communities opinions (Sarker, and Tapabrata,2010; Li, Yongli et al 2013). Our model is similar to Axelrod s approach, which combines the discrete and highly interactive manner of actor decisions and the continuous process of the technology life-cycle. The interaction between these two processes will create a more realistic scenario in terms of applicability of the simulation results. Our model draws on previous research by Gilbert et al. (2001 and 2007), in particular the SKIN model (Simulating knowledge dynamics in innovation networks) developed by Gilbert et al. (2007) that focuses on market interaction and knowledge exchange among firms. The SKIN model outperforms previous theoretical attempts in economics to analyse the innovation process. The novelty of the model is in its use of empirical evidence from case studies to model the decision procedures instead of integrating strategic alliances and cooperative R&D in a standard equilibrium model of competition. Korber, et al. (2014) is also of interest due to the way they assign agents attributes. They categorised their agents based on their research fields (e.g. dermatology, oncology, proteomics, etc.) or on their scientific, technological field or business domain. Then, they assign core competencies like R&D, sales, service, etc. based on particular competencies within the specific research field. The main simulation approaches and software environments used currently are NetLogo based MASON, Repast, and Swarm. 38

43 The main variables used in R&I studies were the kenes, agent's capability to interact with the market, an agent's ability to adapt or innovate as well as innovation itself, and the ability of an agent to be conservative or progressive when entering the market. The studies that focused on agents examined the environments under which they operate i.e. their social standing and tried to mimic the real world conditions in the simulations. The weaknesses of agent-based simulation experiments are related to the approach of agent based simulation in contrast to system dynamics ( continuous modelling ). Agent based models provide a setup to learn and formulate hypothesis (later tested with real data) rather than to predict. Our simulation model will interact with the network structure, which is also created by a dynamic process. The network model is defined as a recursive system, where each actor within a consortium is an input and an output at the same time, allowing for a mutual exchange of information and resources. We integrate models from other disciplines such as Game Theory (Axelrod, 1984, 1997; Von Neumann et al., 2007), Systems Ecology (Odum, 2000, 2007), Resilience Theory (Gunderson and Holling, 2002) and Cultural Theory (Thompson, 2003) to extend standard SNA and standard agent based modelling. 2.4 Development of Multi Theory Multi Level SNA methodology (MTML) A large number of SNA studies reviewed in the former Sub-tasks have been made and look at R&I collaboration from different angles, but often limited to a descriptive analysis and one-dimensional interpretation. They describe networks based on common SNA indicators such as weak and strong links, density, centrality and clusters of actors, but fail to provide answers to more fundamental questions pertaining to the normative level, i.e. how should networks be designed to deliver better R&I outcomes and what is the impact of particular collaboration structures on inclusiveness, European value add, resilience of the ERA and the European innovation system. The results are often restricted to a specific case under study without providing an epistemological frame to interpret the results in its wider environment or cultural context and in relation to rules governing the functioning of the scientific environment. For that account, we use a more sophisticated multi-level multi-theory idea rather than the typical one-dimensional approaches. A multi-level multi-theory approach could serve as a tool to study network development, morphology, constellations, structures and actions from different perspectives and hence, offer opportunities to provide substantive insights for policy intervention and the design of future research programmes. The present research design undertakes to study European Research Framework Programmes project collaboration with state-of-the art SNA and ABM methodology and constructs a MTML model for hypothesis building and interpretation of results that rests on four pillars: System Ecology (e.g. Lietar 2010; Fath, 2914), Resilience Theory (e.g. Gunderson and Holling, 2002, Fath/Dean/Katzmair 2014; Vedres/Stark 2010) Agent Based Modelling/Game Theory (e.g. Axelrod/Dion 1988; Axelrod 2003), and Cultural Theory (e.g. Thompson, 2003; Douglas 2007). 39

44 The variables being used in this study are inspired by Michael Thompson s Cultural Theory. These variables describe the value system and norms and the types of action and behaviour of the different actors. We assimilate in this case the Egalitarians with CSOs and other agents on the fringes. By analysing the collaboration-structure of the framework programs, using SNA, the variables extracted from case studies and from the Cultural Theory are tested. The meaning and importance of different currencies in different fields (e.g. publications, patents, common good, profit, etc.) and the culture of collaboration arising from the logic of the currencies are going to be incorporated in the model. Against the background of the particular mindset and rationality of each group (according to the groups logic), the relational Figure 1: MLMT approach influence of the players (referring to the concept of Game Theory) and the system referring to decision-making-processes and actions in general, is going to be simulated by defining rules in ABM. ABM allows finding the combination of variables, including the players who stabilize the system between efficiency and diversity or who respectively destabilize it (according to System Ecology and Resilience Theory). Further, the different phases of the collaboration should be considered in more detail in the frame of the conceptualized model to find out in which stage of the adaptive cycle the R&I teams are resilient and which SNA factors and network constellations lead to more (or less) resilience. In this sense, SNA will not only be used to analyze emerging network morphologies but also casual understanding of what are the underlying drivers of change. This MTML model can provide an explanation for the factors underlying current participation patterns of CSOs in the European Framework programs by including different theoretical perspectives. 40

45 3 Literature review on CSOs in EU and in R&I Task 2 explored academic, policy and research project literature in order to come to a view on the nature and perception of civil society organisations (CSOs) in research. This is because despite widespread discussion on this issue, there is little established agreement upon the meaning and scope of CSOs. 3.1 Background There is a general trend in research policy and practice toward broader stakeholder engagement in technical and scientific projects. 8 Civil Society Organisations (CSOs) are apparently thought of as important actors who can realise the promise of participative research, responsive to the real world. 9 As CSOs often represent specific interest groups that they have specific knowledge of given areas their input into research may lead to a broader knowledge base and thus more robust knowledge. It can increase the legitimacy of findings, offer transparency and heighten public awareness and discourse. As a consequence there are numerous attempts to stimulate participation in research and embed participative processes in research governance. 10 There is, however, no particularly fixed consensus on what counts as participation, on the role and definition of CSOs, on the ways in which research can or ought to use participative methods, and so on. The theoretical benefits and disadvantages of participation are disputed. A standard definition for the term Civil Society Organisation is absent from present literature. This raises a number of theoretical and empirical problems. If the definition is too narrow, then the set of candidate examples will be too small. On the other hand, very broad definitions comprise a too large a variety of CSOs which are unlikely to be highly compatible with one another. Given its prominence in European policy debates, the popularity and history of the subject implies a broad reaching application of the term CSOs as seen in the EU s own broad definition of a CSO as being Any legal entity that is non-governmental, non-profit, not representing commercial interests and pursuing a common purpose in the public interest. 11 Considering the various interests of organisations is crucial for defining CSOs. Some definitions exclude organisations that represent mainly commercial interests such as industry and business associations. Hence, the need to determine this project s working definition. In order to allow a useful analysis and simulation, the definition of the term CSO used in the project allows for varying degrees of correspondence between the organisation and the definition. Therefore a simple binary definition of the term x is/is not a CSO is here rejected and instead a typology of CSOs, accounting for the diversity of organisations participating in European framework programmes is suggested. This comes from an extensive literature review. 8 9 Smismans, S, Reviewing Normative Theories on Civil Society Participation, see, for instance, the influential Latour, B, Science in Action: How to Follow Scientists and Engineers through Society 10 see EU (2014) Horizon 2020 'Science with and for Society Providing advice on potential priorities for research and innovation in the work programme consultation paper

46 For the purposes of this research, it is vital to address the lack of evidence of the effect of CSO participation within research. This will require at least two steps, one descriptive and one explanatory. The descriptive step will begin with a literature review from research projects that have participated with CSOs or that have otherwise explored this aspect of research. This serves to give a picture of how CSO participation has been carried out or how it has been approached in research projects. The explanatory step comes from an analysis of what we discovered in order to explain why discussion of CSO participation has been realised in the way it has. Together, these two steps provide a basis for suggesting revisions to enable more optimal participatory practices, bearing in mind the perspectives of academic researchers, project participants and policy discourses. 3.2 Methodology The review aimed to clarify the nature of CSOs by reviewing several streams of literature that are relevant to CSOs in research. To achieve maximum complete coverage of the literature, the project therefore reviewed academic, policy and project (e.g. deliverables, reports) documents. Searches for evidence were precisely targeted for information tightly linked to the study aims. Inclusion and exclusion criteria were determined with the aim of generating search results with the maximum relevance and impact in understanding perspectives on CSOs in research. The review and analysis made use of NVivo10, a textmining resource, in order to permit qualitative analysis of large bodies of literature. The present study sought to put forward a working definition of CSOs, effects of participation and the empirical support for the main hypotheses. Therefore, data was searched focusing on the key term, Civil Society Organisation. Given that the application of the term CSO may vary, alternative terms of CSOs such as nongovernmental organisations (NGOs), third sector organisations, and non-profit organisations were used. These terms were used as the initial search terms since they represented the topic name and its alternative names. More search terms were developed as the researchers increased their familiarity with terminology used within this field through reading articles. To get more specific results which produce useful evidence within this framework of study, some terms were added to augment the key search terms. These were effectively contextualising terms. As shown in the table below, terms and alternative terms were used to ensure that as much data as possible within each context could be located. Table 1: Key search strings used Civil Society Organisation Civil Society Organisation AND European Union AND Responsible Research and Innovation Civil Society Organisation AND European Union Civil Society AND European policy Civil Society Organisation AND European Union AND Research and Innovation Non-governmental Organisation Civil Society Organisation AND European Union AND Responsible Innovation Non-governmental AND European Union AND Responsible Research Innovation 42

47 Non-governmental Organisation AND European Union Third sector organisation AND European Union AND Research and Innovation Non-profit Organisation AND European Union AND Research and Innovation Non-governmental Organisation AND European Union AND Research and Innovation Third sector organisation AND European Union AND Responsible Research and Innovation Non-profit Organisation AND European Union AND Responsible Research and Innovation Third sector Organisation Third sector organisation AND Research and Innovation Non-profit Organisation AND Responsible innovation Third sector organisation AND European Union Non-profit organisation AND European Union Non-profit organisation AND Research and Innovation Since the project was interested in critically reviewing definition and terminology used in relation to CSO participation in R&I, academic literature was one domain in which the search-terms were deployed. Given that a critical review on relevant EU rules and initiatives for engaging CSOs in research and innovation activities across different DGs and for involving them in policy development, in particular research policy, was also intended, policy discourse was examined. Finally, a critical analysis of the role of CSOs in R&I and a review of CSOs relations with other R&I stakeholders and partners was required, and so project outputs formed a final domain of interest. Thus the literature review covered the following bodies of literature: Academic Literature European policy-relevant literature Projects outputs of public engagement projects In order to ensure as complete coverage of the literature as possible, articles were searched for in different databases and sources based on the inclusion criteria listed above. 3.3 Summary of findings CSO definition in different bodies of literature What is immediately striking in the academic literature is the diversity of the discussion of CSOs, which, at the very least, is a verification of the dynamic interest in this field of study. In the academic literature, non-profit, non-governmental, selfconstituting voluntary associations are centrally discussed characteristics of CSOs, and therefore constitute a core definition of CSOs in academic circles. However, benefits in terms of participation, empowerment and capacity-building, feature much more widely in the academic literature than CSO definitions do. Since such terms appear so prominently in academic discussion of CSOs, the benefit-bringing nature of CSOs could be considered as indicative of a defining characteristic of CSOs in the academic literature. 43

48 The discussions of CSOs in policy literature are cognate with standard EC definitions, but what is of interest is the limited scope and emphasis suggested in the policy discourse. Common cause NGOs, non-governmental and non-profit definitions dominate the discussion. The discourse of CSOs in policy literature is also centred on matters of current CSO involvement in research and the benefits of CSO involvement in research. Within the discussions of CSO involvement it can be seen that reasons for including CSOs are widely discussed, alongside CSO roles, constraints and enablers, and impact on research. The most prominently discussed benefits are the enabling of citizen participation in research and the quality of research itself. In contrast to the academic and policy discourse, project outputs discuss CSOs current involvement in research about as much as they discuss definitions of CSOs, benefits of CSO inclusion, downsides of inclusion and empirical support combined. Although discussions on current involvement seem to be very vibrant and unsettled, a few focal points could be identified, namely constraints on CSO participation, activities and impacts of CSOs in research, and the motivations of CSOs in research. Beyond this, the discourse is highly fragmented, suggesting very detailed discussion and, perhaps, open questions. This doesn t mean, however, that the discourse is incoherent the many references outside of the apparent focal points could certainly be related back to broader themes but it does suggest the need for close scrutiny. The following table compares the EC/EU view, the academic view, the policy view and the emerging project view, based on the most frequently discussed themes from project outputs. Table 2: Working CSO definitions EC/EU View Working Academic View Working Policy View Working Project View non-state non-governmental non-governmental non-governmental not-for-profit non-profit non-profit non-profit / no commercial interest non-partisan nonviolent through which people organise to pursue shared objectives and ideals political, cultural, social or economic promotes the public good self-constituting, voluntary associations Common cause NGOs faith-based association / alternative legal status --- no single definition (?) / think tanks --- public interest / public well-being Consensus, it seems, emerges on the non-state, not-for-profit basis of CSOs, but beyond that, more is up for grabs Action Logic of CSOs It is possible to reconstruct from the analyses above a kind of action logic for CSOs, from the perspectives of the three literature reviews undertaken. Firstly, it was useful to compile the general preoccupations indicated by the three literature reviews: 44

49 Table 3: Preoccupations in literature reviews Academic Literature Policy Literature Project Literature Benefits Current involvement in research Downsides Capacity building Empower Increase transparency current CSO involvement in research benefits of CSO involvement in research reasons for including CSOs CSO roles constraints and enablers impact on research enabling citizen participation quality of research constraints on CSO participation activities and impacts of CSOs in research motivations of CSOs These themes show various focal points of project literature. This can be instrumental in generating criteria for case-study selection and in setting up key interests for an agent model, from a project literature point of view. For case-study selection this is interesting because it throws up hot topics in project literature and so provides a basis for either pursuing themes that preoccupy current projects, or grounding a gap analysis and finding interestingly understudied cases. The same approach can be applied to the literature reviewed from the academic and policy perspectives. Essentially, we can develop a basis for an action logic for CSOs from the perspectives of the literature reviewed. 3.4 Refinement of proposed CSO typology Drawing upon all of the research and analysis above we can propose a streamlined table of CSO typology, including examples and hints at preoccupations. This provided insight into both the nature and the action-logic of the different CSOs. With the views developed here, it became increasingly fruitful to look at the various levels of CSO-ness in order to come to a detailed view of CSO identity and action across a range of dimensions. Using the methods from above, these various levels were modelled by screening out various nodes in order to constitute the different levels, narrow, medium and broad definition. This showed us, in the various domains of academic, policy and project discourses, how each level of definition is discussed. At the very least, from each domain of discourse it was possible to visualise the extent to which each level of definition was discussed. This was possible by viewing the distribution of nodes across the domains. An immediate result from this was a clear view of the ways in which different domains of 45

50 discourse typically discuss CSOs. The preliminary typology emerging from this exercise is shown in the following table. Figure 2: Levels of "CSO-ness" Table 4: Preliminary CSO typology from the literature review Name Definition Common cause Their focus is on contributing to the public good. Prosperitie s / characteri stics Source EU-based / policy literature Example AGE Platform Europe Terminolo CSO, not for gy profit, NGO Area of Environment activity Sustainable development Research and innovation Nongovernmental Not-for profit Non-partisan Non-violent Shared objectives Shared voice Researchoriented Industry oriented Academic literature Participate in Promote concentrates on how research. interest of CSOs are companies but constituted to are not profitmaking express or support a particular position organisations Academic literature Project outputs Not in the literature Greenpeace IARS ACEA uk/ CSR Europe NGOs, Voluntary associations Environment Sustainable development Economic development Non-government Non-profit Self-constituting Voluntary associations Faith-based / values Research Ethics Oversight Non-government Non-profit / no commercial interests Faith-based associations Alternative legal Promote the interests of a organisations with a profit interest Nongovernment Not for profit Have commercial interests Sectoral 46

51 Name How they can be identified by their mission Legal status Areas of interest Action logic in research Common cause Political, social, cultural, economic groups Promote the public good Public good orientation Shared voice Specific cause Researchoriented status No single definition Public interest / public wellbeing Adding value to research practices Being responsible in research Industry oriented wellbeing Lobbying Promote a sector Legal status is likely to be relevant but difficult for us to determine Promotion of public engagement, deliberative democracy Interested in how CSOs research can influence policy. Will engage in research that clearly promotes the public good. Promote transparenc y in research Promote public engagement as expression of common interest Benefits that CSOs bring: Capacity building Enable citizen participation Increased transparency Mitigate effects of business interests Strengthen representation Research needs to benefit from the inclusion of this voice. Or: The voice needs to benefit from the research Are interested in specific research topics relevant to their mission Raise visibility of their mission Steer research towards specific questions and outcomes. Interested in constraints, activities, impact and motivations of CSOs in research Interested in specific topics and / or CSO implications in research itself Interested in open research opportunities Interested in and aware of research landscape Raise visibility of research and CSO involvement Promote engagement as means of improving research, increasing legitimacy of findings Interested in research that further the interest of the sector Demonstrate the social responsibility of the sector Seeking to extend specific networks Raise visibility of the sector Research to support or achieve specific (policy) goals This table, based in the synthesis of the above literature reviews outcomes, provides a grounding for a further step. Essentially, this material can be operationalised in such a way that it provides a methodology for classifying CSOs. Rather than a list of criteria, as the table is presented, this operational version of the typology gives a means of actively 47

52 using the literature review material to select and to sort CSOs. This decision tree is grounded in the insights from the literature review and so it proceeds in a principled rather than an ad hoc way. Figure 3: Operationalised typology ("decision tree") for categorising CSOs Rather than a direct mapping of the typology to the decision tree, this represents an iteration. This is owing to the fact that an additional step of analysis takes place in producing the decision tree from the typology. Mainly, this involves non-duplication of headings or themes, and general streamlining. For instance, in the typology headings for common cause, shared voice, research-oriented, profit oriented CSOs. In the operationalised decision-tree image this shrinks to a tripartite functional split between service, advocacy and research. This is because service can be seen to cover common cause and some advocacy CSOs, as well as profit-oriented CSOs (e.g. the business advocacy CSO, ACEA mentioned in the typology). In short, in creating the CSO classificatory tool in the above image streamlining and conceptual clarity was facilitated further, without sacrificing the strong foundation provided by the literature review. Moreover, owing to the conceptual coherence between the typology and the decision-tree tool, each can be used in conjunction with the other in a useful, productive tension in order to provide robust insights regarding CSO identification and evaluation. 48

53 4 Compilation of project dataset The purpose of this task was to prepare the project s database of organisations participating in FP6, FP7 and H2020 projects, and identifying the different types of CSOs therein. After carrying out the required cleaning and harmonisation across the three different sources, the project dataset was comprised of 220,454 entries, consisting of 45,031 organisations and 37,662 projects (from FP6, FP7 and Horizon 2020). This dataset consequently served as a basis for identifying and categorising the CSOs participating in the FPs. This was done though an online survey with organisations categorised as OTH in the project dataset and a subsequent manual screening of organisations. 4.1 Survey with organisations categorised as OTH A survey questionnaire was developed on the basis of the decision tree, which included some additional questions that may help categorise the CSO. About 8,000 contact details for OTH organisations were retrieved (for a number of organisations, more than one person was approached in view of ensuring a high return rate). The full survey was launched in October 2015 and remained open for about two months. From the around 8,000 contacts that were approached, about 3,200 turned out to be non-existent (i.e. the respective addresses included in E-CORDA were not active anymore). During the two months in which the survey was open, about 1,500 replies were received. 4.2 Refined CSO typology and additional screening of OTH organisations The analysis of the 1,500 replies from the online survey allowed reflecting on the proposed CSO typology from the literature review (see above), also in view of coming up with a manageable procedure for manually screening more than 5,700 organisations concerning their CSO status. The analysis revealed that the distinction of CSOs according to the two dimensions income and target group was feasible, while the results regarding the potential third dimension ( function ) were not homogenous. For instance, the results showed that the function carrying out research does not describe a specific type of research-oriented CSO (as was suggested in our proposal), but rather researchoriented CSOs can be of very different types (e.g. big, industry-oriented CSOs as well as smaller core CSOs). The CSO typology used for this study was consequently refined in view of focusing on the two dimensions income and target group, distinguishing between the following four types of CSOs: 1) CSO 1: core CSOs (oriented towards individuals/citizens): this category includes organisations funded by individuals or other non-profit organisations (NPOs), whereby this funding source is clearly specified in financial reports or other official documents of the organisation, and with individuals as the main beneficiaries. Examples of CSO 1 organisations include: European Patients Forum, Red Cross, WWF, European Brain Council. 2) CSO 2: oriented towards society/public, financed by public: this category includes organisations funded by the state (government) and having individuals or public authorities as the main beneficiaries. This category also includes organisations 49

54 potentially being CSO 1, but where the funding source is not clearly specified in official documents. Examples of CSO 2 organisations include: Transparency International, European Platform of Women Scientists, Alzheimer Europe. 3) CSO 3: oriented towards society, financed by businesses: this category includes organisations funded by businesses but having a clear social/environmental focus concerning target groups. Examples of CSO 3 organisations include: European Food Information Council, European Aquaculture Society. 4) CSO 4: oriented towards businesses (e.g. business associations): this category includes organisations financed by profit-oriented companies or government/state/public funds with the aim of supporting/enhancing business, industries or professional groups and their interests. Examples of CSO 4 organisations include: Association of European Railway Industries, European Business and Innovation Centre Network, European Business Register. The refined CSO typology described above was used as the basis for the manual screening of the remaining 5,700 organisations categorised as OTH. Additionally, a check of other organisations (e.g. categorised as REC etc.) was carried out by looking at CSO-umbrella organisations and their members and cross-checking this with the organisations in the project dataset (i.e. organisations participating in the framework programmes). For a number of European CSO umbrella organisations (such as European Citizen Action Service (ECAS), ECRE European Council on Refugees and Exile, European Union Fundamental Rights Platform, Europe+, EU Civil Society Contact Groups), the lists of member organisations were retrieved and cross-checked with the organisations included in the project dataset to identify additional CSOs listed under other categories. 4.3 Finalisation of project dataset The CSO categorisation described above was eventually merged with the project dataset. Organisations that could not be allocated to any of the four CSO types remained in the project dataset under the category OTH. By doing so, the category OTH was split into five categories (CSO1, CSO2, CSO3, CSO4, OTH). Moreover, organisations that were obviously wrongly categorised as OTH, but were in fact public bodies or profit-oriented, were re-categorised into the correct category ( PRC or PUB ) 12. The outcome of this sub-task was a final project dataset (by the end of January 2016) with the following organisation types to be applied in the remainder of the study: HES: Higher or secondary education REC: Research organisations PRC: Private for profit (excluding education) PUB: Public body (excluding research and education) CSO1: core (citizen-oriented) CSOs CSO2: society-oriented/public-funded CSOs CSO3: society-oriented/business-funded CSOs CSO4: business-oriented CSOs OTH: Other (non-csos) 12 Due to this re-categorisation of OTH organisations, the numbers for PRC and PUB in the present study will therefore differ from other analyses on FP6 and FP7 participations. 50

55 The main novelty of this study (and its dataset) is the identification of CSOs according to the four different types among all the organisations participating in FP6, FP7 and H2020. The following table provides an overview of the numbers of CSOs included in the project dataset, including the remaining OTH organisations. Table 5: Numbers of CSOs included in the final project dataset Type FP6 FP7 H2020* Total FP7 & H2020* (no duplicates) Total FP6, FP7, H2020* (no duplicates) CSO CSO CSO CSO OTH Total CSO Total CSO Total "OTH" * Data from June

56 5 Case studies of networks involving CSOs CSOs and other potential research partners operate on different logics, such that Framework Programme funding does not appear in the same way to each. Tweaks within the existing system can be used to incentivise CSOs to participate more; however, where the goal is to bring science closer to society, a different kind of change is probably necessary. The work carried out in this task aimed at identifying and implementing further case studies based on the case studies carried out in the course of the CONSIDER project ( Civil Society Organisations in Designing Research Governance ; The idea was to go beyond existing cases and detect gaps in the knowledge created by CONSIDER and to fill these gaps using additional cases. 5.1 Guiding Questions The following questions structure the overall approach beyond CONSIDER case studies. They were informed by the results of the network analysis and the discussions related to the agent-based modelling. They were designed to capture interesting cases that can inform simulation tasks. This is based on insights about, for example, the resilience of CSOs that appear to fail, yet keep trying. The overall areas structuring the approach are as follows: CSOs that tried participating several times and did not succeed, but keep trying Important "pure" CSOs (type 1) that never tried participating CSOs that (tried to) coordinate(d) projects in FP6/FP7 Highly-active CSOs (e.g. coordinators and/or active in many projects) in FP6 that got lost in FP7 5.2 Methodology CONSIDER re-analysis The data collection and analysis of the CONSIDER project have been described in detail in the project deliverables (see: For the purposes of this task, the data were revisited with the aim of finding additional insights concerning the research questions. This was done via the coded data of the CONSIDER project and a re-analysis of the case study summaries. It is important to underline, however, that the CONSIDER project, by nature of its chosen cases, could not provide responses to some of the questions. All CONSIDER cases were of active research projects that included CSOs. This included some cases where CSOs were coordinators. What the project data does not provide, however, are insights into any sort of research in which CSOs are not part of consortia, e.g. answers to the question why successful CSOs have dropped out of participation. In total, there were slightly over 30 CONSIDER case studies that were analysed. These case studies were all successfully involved in research. However, the level of involvement differed. As such, looking at the 4 CSO type categories that have been elected for use in our study, we can discount CONSIDER case studies as falling under the first two categories of: 52

57 i. CSOs that tried participating several times and did not succeed, but keep trying. ii. Important "pure" CSOs (type 1) that never tried participating. We can however expect CONSIDER case study CSOs to fall under the last two categories: iii. iv. CSOs that (tried to) coordinate or coordinated projects in FP6/FP7. Highly-active CSOs (e.g. coordinators and/or active in many projects) in FP6 that got lost in FP7. To gather responses to these questions, additional data had to be collected Additional case studies The use of blanket, fully-fledged case studies was revised, in view of providing insights through the targeting of specific types of CSOs, in particular circumstances, with specific experiences. Although the CONSIDER cases were fully-fledged case studies, the cases for the CSO tender were more explanatory/inspirational interviews. The interviews were semi-structured and open in order to ensure that participants were not restricted, but had the opportunity to reflect and expand on the questions if need be. This allowed for the collection of detailed data. Given the difference in focus, it was not possible to follow the CONSIDER methodology, which used as its unit of analysis the case of projects including CSOs. This logic could not be transferred to unsuccessful CSOs and, therefore, needed to be revised. For the sake of consistency with the proposal, we nevertheless continue to use the term case study when talking about the empirical work. The cases in this case are the collection of data used to answer the individual guiding questions listed above Selection of cases In order to make a justified selection of interviewees, the guiding questions were used to frame areas of enquiry. The CORDIS database was then used to find groups of CSOs relevant to each framing question. The typology developed in the project was used to further refine the relevance of targets to the aims of the study. Primarily, those CSOs of type 1 and 2 were given priority, owing to their being more like the sorts of CSOs that governance structures tend to seek, in order to include in research (i.e. without commercial interests etc.). 5.3 Findings Summary of findings additional cases The table below summarises the outcomes of the work beyond CONSIDER based on interviews with key players in organisations identified as relevant. The organisations were identified using information from prior tasks. 53

58 Table 6: Main findings from additional cases Question CSOs that tried participating several times and did not succeed, but keep trying Why do they keep trying? Important "pure" CSOs (type 1) that never tried participating Why are they not trying? Under what conditions would they be interested in participating? CSOs that (tried to) coordinate(d) projects in FP6/FP7 why did they (seek to) coordinate? Highly active CSOs (e.g. coordinators and/or active in many projects) in FP6 that got lost in FP7 Why did they drop out? Response Funding Commitment to Mission Steering research toward valuable ends In a sense, some are sometimes by proxy Capturing participation requires scrutiny (e.g. a researcher applying for funds from a lab funded by another agency is an arguably participation) See FP as peer (or rival?) funding scheme agenda clash or complementarity Constitutional and legal factors may make it impossible (e.g. rules for endowment disbursement) Where possible, co-determining research agendas at high levels would be incentive New challenges Capacity-building internally Elevate role beyond minor partner Create impact in an area of interest Steer research, non-trivially Lack of capacity to keep up with calls Administrative burdens/hurdles (e.g. visas for non EU members) Bad luck (entropy dropping out was not a decision) Participation lacked the expected impact on research (e.g. through low esteem) Summary of findings CONSIDER cases The table below summarises the re-analysis of the CONSIDER cases. Table 7: Main findings from re-analysis of CONSIDER cases Question What was the motivation for CSO participation? Response Desire to contribute to research Improve lives of CSO interest groups as they have easy access to research outcomes Keep abreast of scientific developments Grow reputation in research field Act as intermediaries between researchers, industry and CSO interest groups 54

59 Why were some CSOs involved several times? What were CSOs experiences? How FP participation changed the organisation? WP leadership and coordinator CSO power Existing networks and collaboration Prior research experience Funding purposes Prior research experience Experience of working with vulnerable groups Raising company profile Improving CSO standing in research CSO power 5.4 Conclusions As can be concluded from the findings above, CSOs are clearly motivated to participate in research due to a number of reasons. Such motivations may potentially be extended to other CSOs when research is more conducive for CSOs to have the opportunity to contribute, grow their reputation and keep abreast of scientific developments. This may be realised if CSOs are given roles that are deemed as important in the consortium structure. These roles should enable the CSOs to have the ability to influence the research agenda and decision-making in the research process. Such roles may include WP leadership or coordination roles. However, there is need for caution when it comes to the possible assigning of coordination roles to CSOs, especially within the current EU research frameworks, as CSOs consider EU research regulations as bureaucratic, particularly for CSOs who often do not have research capacity. The analysis has also shown that some CSOs are likely to be involved several times in research. This has usually happened when there had been long-term networks and collaborations, prior research experience and for funding purposes. It can, therefore, be concluded that it is essential to nurture and ensure that CSOs continue to be involved and collaborate in research. This can be done through the creation of networking platforms by funders and policy makers, so that they are the norm and not the exception. Moreover, the inclusion of CSOs in agenda-setting at the highest levels of FP formulating is likely to be the most efficient way to include their values, norms, insights, knowledge, and influence. This kind of change could alter the character of FPs, such that collaborative research, centred on socially relevant areas, would be incentivised. Including CSOs in project evaluations would likely serve to alter the character of a) proposals and b) consortium makeup by incentivising other researchers to take CSO involvement in research seriously. 55

60 6 Role and position of CSOs in FP networks 6.1 Network Analysis Data Base In order to determine the role and functional impact of CSOs in the EU research framework programmes and their networks, we use CORDIS data of FP6, FP7, and Horizon projects. Based on the common definition that a network is a set of nodes connected through a set of links, we treat organisations as actors that are connected to each other through EU funded projects. These networks are called two-mode, because they consist of two classes of nodes (organisations and projects) and can be transformed into a one-mode network of organisations (two organisations are connected if they cooperate in at least one project) or into a one-mode network of projects (two projects are linked if at least one common organisation participates in them). Our analysis mainly refers to the one-mode organisation networks Data Management and Network Generation Relationships between FP participants are derived from project data by linking organisations that cooperate in one or more EU funded projects. Imagine a project with four participants; it leads to a fully connected graph, i.e. a network of four organisations in which each organisation is linked to every other organisation. Each of the six relationships would be treated equally because it is derived from cooperation in exactly one project. In cases where organisations cooperate in more than one project, the link between them would receive a strength that correlates with the number of common projects. This, however, tends to generate artificial relational data: We have to assume that the intensities of relationships are not equal, and that strength differs between different pairs of actors within one project depending on certain criteria. For example, we would expect that there is more communication between the coordinator of the project and the participants than among the participants themselves. Furthermore, interaction happens more frequently between participants within short geographical distances and that share the same language. Additionally, if we compare different projects with each other, we will assume that there is more communication and more transaction of resources in longterm projects. For these reasons, we developed a model for the estimation of the variance of interaction and computed encounter probabilities, which estimate the amount of communication and resource transaction among participants according to the following criteria: 1. Number of common projects: encounter probability rises with the number of projects in which two organisations cooperate. 2. Position within project: encounter probability is higher between coordinators and participants than among participants. 3. Duration: encounter probability is higher in long-term projects (duration in months). 4. Funding: encounter probability is higher in highly funded research projects (EU contribution). 13 End of period for H2020 data: April See the annex for the numbers of projects, institutions, and funding in detail. 56

61 5. Geographical distance: encounter probability is higher among organisations of the same country (same language). 6. Project size: encounter probability decreasing with the number of participating organisations (the higher the number of participants, the lower the interaction intensity). According to these influence factors, we computed the strength of relationships of all pairs of participants within each thematic area of FP6 and FP7. In doing so, we transformed the CORDIS project data into network data of organisations that cooperate with each other in EU funded projects, in which the amount of estimated interaction (i.e. strength of relationships) corresponds to encounter probability (Figure 4). The computation of encounter probabilities is our approach to come to a better understanding of the meaning of relationships and networks. Figure 4: Transforming CORDIS project data into network data. Line strength corresponds to encounter probability Graphic: FASresearch. Figure 5 represents the core of the FP7 cooperation network. In order to represent the overall structure of the network, it is necessary to reduce the number of relations and to select the most important organisations. An arc from A to B indicates that A was the coordinator of B in at least one FP7 project. For each organisation, the five most important collaboration partners in terms of the number of common projects are depicted. The strength of the arcs refers to the robustness, which is measured by the number of common neighbours of each collaboration relation. The size and the colours of the organisations indicate aggregated encounter probability. The thematic core areas are clearly visible: On the left side we find organisations in the fields of engineering, ICT, and energy (also most of the companies are located in this sector). On the right side, there are the institutions doing research in health and life sciences. This duality is typical for the EU funded research and science. Around these two centres, additional clusters can be found notably agriculture/food and aerospace. Of course these are not all thematic areas and this is not the complete FP7 network it is the absolute core, the key drivers of the network that dominate it, and around which the structure of the collaboration network emerges. 57

62 Figure 5: Core of FP7 cooperation network (694 out of organisations) Data source: CORDIS, WU. Analysis and visualisation: FASresearch. 58

63 6.1.3 CSO Participation General Statistics As already described above within the participant data among the OTHERS category different types of CSOs have been identified. Table 8 lists the organisation types by numbers and percentages. Table 8: FP6, FP7, and H2020* types of institutions FP6 FP7 H2020 Type of Institution Number Percent Number Percent Number Percent HES , , ,2 REC , , ,4 PRC , , ,5 PUB , , ,7 OTH 624 2, ,7 58 1,1 CSO , ,7 48 0,9 CSO , ,3 93 1,8 CSO , ,7 58 1,1 CSO , , ,2 Total , , ,0 Data source: CORDIS, WU. Table: FASresearch. *April 2015 Altogether, (FP6) and (FP7) CSOs have been found, that is 6,6% and 5,8% of all institutions respectively. Without the business oriented type 4, only 3,2% (FP6) and 2,7% (FP7) of all organisations are CSOs. We can see that the amount of CSOs decreases from FP6 to FP7, but increases when we include H2020 data until April 2015 (8,1% CSOs, 3,9% CSOS without business oriented types 4). Finding 1: All in all, we found fewer CSOs than expected. There are only (FP6) and (FP7) CSOs in the database, that is 6,6% and 5,8% of all institutions respectively. Without the business oriented type 4, only 3,2% (FP6) and 2,7% (FP7) of all organisations are CSOs. Table 9: Funding distribution among organisation types (in 1,000 Euro and %) FP6 FP7 H2020* Types Thousand Percent Thousand Percent Thousand Percent HES ,8 38, ,1 43, ,6 35,8 REC ,6 29, ,1 27, ,3 31,9 PRC ,6 25, ,2 24, ,8 25,3 PUB ,5 3, ,0 2, ,2 4,5 OTH ,2 1, ,3 0, ,5 0,2 CSO ,7 0, ,3 0, ,0 0,2 CSO ,2 0, ,8 0, ,0 0,5 CSO ,4 0, ,3 0, ,1 0,3 CSO ,1 0, ,7 0, ,5 1,3 Total ,0 100, ,7 100, ,1 100,0 Data source: CORDIS, WU. Table: FASresearch. *April 2015 When we look at the funding distributions among organisation types, proportions become even clearer. All in all, in FP7, CSOs received 1,4% of funding (FP6: 1,3%); disregarding business oriented CSOs type 4 it is only 0,6 % (FP6) and 0,4% (FP7). H2020 numbers 59

64 until April 2015 exhibit increases on a low level (2,3%; 1,1% without business oriented CSOs). Funding percentages indicate that the FPs are more customized for universities and research organisations (plus specific programmes for enterprises), and this fact has a strong impact on the currencies that are regarded as valuable within these systems: scientific excellence and applicability. Finding 2: The share in funding of CSOs is even lower than the share in institution numbers and in project participations. In FP7, CSOs received 1,4% of funding (FP6: 1,3%); disregarding business oriented CSOs type 4 it is only 0,6 % (FP6) and 0,4% (FP7). Another indicator that there are obstacles for CSOs to take part in Framework Programmes is their low survival rate (Figure 6). Only 15.3% (19 out of 124) of the CSOs type 1 ( core CSOs) in FP6 can be found in FP7 again. The rates for business oriented CSOs are a little bit higher (20,6% or 154 out of 748 for CSOs type 4). Figure 6: Survival rate of institution types from FP6 to FP7 in % of organisations Data source: CORDIS, WU. Chart: FASresearch. Finding 3: CSOs exhibit a high dropout rate from FP6 to FP7. Only 15.3% (19 out of 124) of the CSOs type 1 ( core CSOs) in FP6 can be found again in FP7. For CSOs type 2, the rate is 16.3%, and for CSOs 3, 13,8%. The rates for business oriented CSOs are a little bit higher (20,6% or 154 out of 748 for CSOs type 4). There are specific playgrounds of CSOs thematic areas in which CSOs participate more than in others. It is obvious that universities and non-university research institutions dominate the arena. There are thematic areas specific to companies (Information and Communication Technologies, Research for the benefit of SMEs). Here we also find a number of business oriented CSOs (type 4). However, when it comes to more pure CSOs (types 1 and 2), numbers are small. Regarding FP7, there are 48 CSO1s in Health (which is 0.6% of all organisations in Health), 36 in Science and Society, and 33 in ICT. CSOs type 2 were mainly found in Science in Society (76), Environment (46), and Health 46). These patterns become even clearer when we look at the funding distribution among types of CSOs across thematic areas. The share in funding of CSOs within the thematic areas (Table 10) is very small, even if thematic areas that are typical for CSOs are considered. The core CSOs 1 received 3.7% of the funds of Science in Society (FP7), for CSOs of type 2 the percentage is 6.1. All in all, CSOs types 1, 2, and 3 received 12.8% of 60

65 SiS funding in FP7. Business oriented CSOs received at least 11.8% in Regions of Knowledge. Table 10: Percentages of funding by CSO type, programme, and thematic area (row sum = 100) CSO1 CSO2 CSO3 CSO4 CSO1-CSO3 Thematic Area FP6 FP7 FP6 FP7 FP6 FP7 FP6 FP7 FP6 FP7 Activities of International Cooperation 0,9 0,5 1,7 1,2 0,2 0,8 0,6 2,3 2,8 2,6 Regions of Knowledge 0,0 0,2 0,6 0,9 1,3 0,8 4,8 11,8 1,9 1,9 Research for the benefit of SMEs 0,1 0,1 0,2 0,3 0,6 0,6 8,2 6,4 0,9 0,9 Research Infrastructures 0,0 0,1 0,5 0,8 0,7 0,7 1,4 1,8 1,2 1,6 Research Potential 0,0 0,0 0,0 0,0 0,0 0,0 0,0 1,4 0,0 0,0 Science in Society 3,1 3,7 8,1 6,1 3,3 3,0 5,8 3,8 14,4 12,8 Energy 0,0 0,1 0,0 0,1 0,1 0,3 0,5 1,8 0,1 0,5 Environment 0,2 0,2 0,9 0,6 0,4 0,2 1,0 0,5 1,4 1,0 Food 0,2 0,2 0,4 0,3 0,6 0,7 1,2 1,2 1,2 1,2 Health 0,3 0,4 0,4 0,3 0,0 0,3 0,4 0,3 0,7 1,1 ICT 0,1 0,2 0,4 0,2 0,2 0,2 0,7 0,9 0,7 0,6 Nanosciences 0,0 0,0 0,2 0,2 0,2 0,2 1,0 1,1 0,4 0,4 Security 0,0 0,7 0,0 0,3 0,0 0,5 0,0 1,5 0,0 1,5 Socio-economic sciences and Humanities 0,2 0,8 0,9 1,0 0,4 0,2 2,7 0,5 1,4 1,9 Space 0,3 0,2 0,2 0,1 0,4 0,0 0,7 0,3 0,9 0,3 Transport (including Aeronautics) 0,0 0,0 0,1 0,2 0,1 0,1 1,0 1,6 0,2 0,3 European Research Council 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 Nuclear Research 0,0 0,1 0,1 0,0 0,0 0,0 0,5 0,5 0,1 0,1 General Activities 0,0 0,0 0,8 0,3 0,0 0,0 0,7 1,0 0,8 0,3 Joint Technology Initiatives 0,0 0,1 0,0 0,2 0,0 0,4 0,0 0,5 0,0 0,7 Research policies 0,2 0,2 0,8 1,7 0,6 1,7 1,3 2,4 1,5 3,5 Marie-Curie Actions 0,1 0,0 0,1 0,1 0,1 0,1 0,2 0,2 0,3 0,2 Total 0,1 0,2 0,4 0,3 0,2 0,3 1,0 1,1 0,8 0,9 Read: Within FP6 in Activities of International Cooperation CSOs type 1 get 0,9% of funding. Data source: CORDIS. Table: FASresearch. By looking at the column percentages (Table 10) we can determine how important each thematic area is for each type of institution in terms of funding. In FP7, CSOs of type 1 receive 22.4% of their funding through Health, 16.1% through ICT, 13% through Security, and 12.7% through Science in Society. 61

66 Table 11: Percentages of funding by CSO type, programme, and thematic area (column sum = 100) CSO1 CSO2 CSO3 CSO4 CSO1-CSO3 Thematic Area FP6 FP7 FP6 FP7 FP6 FP7 FP6 FP7 FP6 FP7 Activities of International Cooperation 5,9 1,1 3,6 1,6 0,8 1,1 0,5 0,9 3,1 1,3 Regions of Knowledge 0,0 0,4 0,0 0,8 0,1 0,8 0,1 3,3 0,1 0,7 Research for the benefit of SMEs 0,7 1,0 1,0 1,8 4,7 3,8 15,6 12,5 2,1 2,3 Research Infrastructures 0,0 2,7 7,0 16,1 16,2 13,8 8,0 10,5 8,6 12,0 Research Potential 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,1 0,0 0,0 Science in Society 4,7 12,7 4,0 13,3 2,8 6,9 1,2 2,5 3,8 10,8 Energy 0,0 1,6 0,2 1,8 1,3 3,9 1,4 7,0 0,5 2,6 Environment 16,3 5,3 24,7 9,3 17,2 3,8 11,5 2,6 21,0 6,3 Food 9,5 6,6 7,3 5,8 16,9 12,4 8,8 6,3 10,6 8,4 Health 33,0 22,4 12,0 12,0 1,5 12,5 5,1 3,0 12,4 14,7 ICT 15,7 16,1 24,3 12,9 17,9 11,6 15,7 16,4 20,9 13,2 Nanosciences 1,3 1,2 4,3 4,2 6,8 6,8 9,7 9,0 4,5 4,4 Security 0,0 13,0 0,0 4,1 0,0 6,0 0,0 5,5 0,0 7,0 Socio-economic sciences and Humanities 2,9 5,1 4,0 4,3 3,1 0,7 5,5 0,6 3,6 3,2 Space 2,6 2,3 0,7 0,4 2,1 0,3 1,0 0,7 1,4 0,8 Transport (including Aeronautics) 4,2 2,0 1,5 4,4 3,4 4,1 12,0 13,3 2,5 3,7 European Research Council 0,0 0,3 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,1 Nuclear Research 0,0 0,3 0,2 0,1 0,1 0,1 0,8 0,5 0,1 0,1 General Activities 0,0 0,0 1,9 0,1 0,0 0,0 0,7 0,1 1,0 0,0 Joint Technology Initiatives 0,0 5,6 0,0 4,9 0,0 10,2 0,0 4,2 0,0 7,1 Research policies 1,9 0,0 2,7 0,1 3,5 0,1 2,0 0,0 2,8 0,1 Marie-Curie Actions 1,2 0,1 0,5 2,0 1,3 1,2 0,4 0,9 0,9 1,3 Total Read: Within FP6 CSOs type 1 get 5.9% of their funding through Activities of International Cooperation. Data source: CORDIS. Table: FASresearch. Participation of CSOs in areas like Health, ICT or Environment is significantly below average. However, these are big thematic areas (much funding, many projects). Therefore, in absolute terms, these thematic areas are important funding sources for CSOs. Finding 4: There are specific playgrounds for CSOs thematic areas in which CSOs participate more than in others. In relative terms (CSO share in each thematic area), Science in Society, Activities of International Cooperation, Research policies, and Socioeconomic sciences and Humanities are most significant. In absolute numbers, Health and ICT are also important funding sources for CSOs because of their sheer size. Furthermore, there is a clear correlation between the participation of different institution types and funding schemes, as Table 12 shows. The last of the three tables, which includes the column percentages, indicates that the amount of participation in Coordination and support actions (CSA) is higher among CSOs than among universities, research institutes and enterprises. For example, 43.5% of CSO type 1 participation takes place in CSA projects; this number for universities is only 10.7%, for research institutes 16.8%, and for enterprises 9.3%. This means that CSOs (like public bodies) are rather engaged in coordination, networking and dissemination activities than the research-oriented participants in framework programmes. Participation of CSOs only takes place in CP or in CSA projects; we hardly find them in other funding schemes (ERC, JTI, MC, or NOE). 62

67 Table 12: FP7 participation of institution types in funding schemes: numbers (first table), row % (second table), column % (third table) Scheme HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total CP CP-CSA CSA ERC JTI MC NOE Total Scheme HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total CP 31,6 23,5 39,6 3,2 0,21 0,29 0,25 1,15 0,2 100,0 CP-CSA 37,5 43,3 8,5 7,8 0,10 0,69 0,36 1,45 0,3 100,0 CSA 27,1 28,2 19,0 17,7 0,72 1,64 0,89 3,70 1,0 100,0 ERC 73,2 25,2 1,2 0,3 0,04 0,00 0,00 0,02 0,0 100,0 JTI 23,3 19,3 54,8 1,3 0,16 0,14 0,26 0,61 0,1 100,0 MC 65,7 22,6 10,5 0,8 0,01 0,07 0,03 0,08 0,1 100,0 NOE 53,0 31,7 11,1 2,7 0,27 0,00 0,27 0,91 0,1 100,0 Total 37,3 24,8 30,4 5,0 0,2 0,5 0,3 1,3 0,3 100,0 Scheme HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total CP 49,9 55,7 76,7 37,4 50,31 37,50 47,70 51,14 34,5 58,8 CP-CSA 3,8 6,5 1,1 5,9 1,54 5,67 4,36 4,10 3,6 3,7 CSA 10,7 16,8 9,3 52,6 43,52 53,50 42,13 41,29 56,4 14,8 ERC 8,0 4,2 0,2 0,2 0,62 0,00 0,00 0,06 0,0 4,1 JTI 2,7 3,4 7,9 1,1 2,78 1,33 3,63 1,99 1,9 4,4 MC 23,7 12,3 4,7 2,3 0,31 2,00 1,45 0,85 3,3 13,5 NOE 1,2 1,1 0,3 0,4 0,93 0,00 0,73 0,57 0,3 0,8 Total 100,0 100,0 100,0 100,0 100,0 100,0 100,0 100,0 100,0 100,0 Data source: CORDIS. Table: FASresearch. Finding 5: Funding schemes have a significant impact on CSO participation as well. CSOs are engaged in coordination, networking and dissemination activities (CSAs) rather than in research activities (CPs). Our conclusion is that we found less real CSOs in the FP project data than we expected. With regard to project participation and funding, the importance of CSOs within the European Framework Programmes is rather small. Nevertheless, there are specific thematic areas in which we identified a bigger number of CSOs ( playgrounds of CSOs). In relative terms, Science in Society (a small thematic area relative to the others) is most important. In absolute terms, the big areas Health (for core CSOs) and ICT (for the business oriented CSOs) contain a notable number of projects with CSO participation. When it comes to funding schemes, CSO participation is much higher in coordination and support actions than in genuine research oriented projects. This indicates that CSO participation in EU funded projects does not, first and foremost, target advancing scientific excellence, research, and the production of knowledge. 63

68 6.1.4 Network Parameters Key Network Analysis We use the following parameters to describe the overall network structures. Funding: Percentage of funding of each thematic area within the whole Framework Programme; it is an indicator for the foci of the Framework Programmes. Organisations: Number of institutions within the network (universities, nonuniversity research, companies, public institutions, four types of CSOs, and others). The number of organisations is an indicator for the volume of a thematic area and the size of a scientific and technological field. Lines: This number shows the amount of cooperation relations within the networks. Density: Percentage of existing relations compared to possible relations. Because of the limited number of contacts each social actor is possible to maintain (limited time and energy), density usually depends on network size: The bigger a network, the lower its density. Density indicates the amount of connectedness of a thematic area. Average degree: Average number of project partners each organisation has within the thematic area. It shows whether an area is dominated by big research projects with many participants. Fragmentation: Amount of actors within the network that cannot be reached by direct and/or indirect contacts. If a network consists of different components that are not connected to each other, the scientific or technological field is still small or emerging. We also find high fragmentation in thematic areas in which cooperation does not belong to the goal functions of the calls. Centralisation: Network analysis calls degree the number of partners an organisation is connected to. Degree centralization of a network is the variation in the degrees of vertices divided by the maximum degree which is possible in the network of the same size. The higher the centralisation, the more dominated the network is by a single organisation or by a few actors (monopoly-like distribution of linkages). Centralisation is also an indicator for the maturity of a technological field (established organisations dominate the arena). Modularity: Indicator for the existence of subnetworks (modules). Networks with high modularity have dense connections within the sub-networks, but sparse connections between them. Networks with high/low modularity are called multi peaked/single peaked networks. Average Clustering Coefficient: First measure for the amount of clustering. In general, clustering refers to the existence of triadic relationships. Triadic means if A cooperates with B and A cooperates with C, then A and C are also cooperating (transitive relationships). The clustering coefficient of an organisation indicates to which degree the cooperation partners of each actor are connected to each other as well. The table lists the average values for the area networks. 3-Rings: Second measure for the amount of clustering. It also refers to the existence of triadic relationships but takes the number of common partners into account as well. A value of 20, for example, indicates that, on average, each connected pair of organisations has 20 cooperation partners in common. CSO percentages: Percentages of CSOs types 1 4 within the network of each thematic area. The results for all thematic area networks in detail can be found in the Annex. 64

69 6.1.5 The Correlation of Network Parameters and CSO Participation The objective is to find out if there are any statistical correlations between the network parameters described in chapter and the participation of CSOs in EU funded projects. Do networks with a higher amount of CSOs have specific characteristics? The analysis takes the following parameters into account: (1) Thematic area: How does the content of a thematic area influence the participation of CSOs? Are there specific (research) questions and topics that foster the participation of CSOs? And is there a correlation between the currency of an area (scientific excellence, benefit, mission) and CSO participation? (2) Share of funding: Is there a relation between the size of an area and CSO participation? Do we find CSOs in big or in small programmes? (3) Organisations: Is there a connection between network size and CSO participation? Do we find CSOs in small (or in big) networks? (4) Lines and density: Is there a correlation between network density and CSO participation? Is it easier for CSOs to enter less connected networks? (5) Average degree: Does the mean number of project partners have an impact on CSO participation? Do we find CSOs in big or in small consortia? (6) Fragmentation: Does the degree of fragmentation influence CSO s attendance? Is it easier for CSOs to join fragmented and less hierarchical networks? (7) Centralisation: Is there a correlation between the degree of link monopolisation and CSO frequency? Is there a higher probability to find CSOs in less centralised networks? (8) Modularity: Do we find CSOs in networks with high modularity, i.e. multi peaked networks that consist of densely connected sub-networks that are only loosely connected to each other? (9) Average clustering coefficient: Is there a correlation between the amount of triadic relationships around each organisation and CSO participation? Do we find CSOs in less clustered networks? (10) Clustering: Is clustering measured by 3-rings, the absolute amount of triadic relationships within each area network, an obstacle for the participation of CSOs? We correlate each of these parameters with the relative amount of CSOs for FP6, as well as for FP7 (see Annex for details and tables). To get a better overview, we transform this matrix into a network graph (Figure 7). The nodes represent the network parameters, lines indicate significant positive (solid) and negative (dotted) Pearson correlation coefficients. The node for the percentages of CSOs 1 (and for the number of organisations as well) is isolated. This means that there is no significant correlation between the participation of CSOs type 1 and specific network parameters in FP6. The network structure itself has no impact on the amount of CSOs type 1. There are two groups of parameters with positive correlations within the groups and negative correlations between the groups. The group in the centre of the network consists of parameters that refer to network size (funding, number of lines) and 65

70 clustering (average degree, 3-rings, centralization) respectively. In particular, there is a strong correlation between funding and number of links, between funding and clustering (measured by the amount of triadic relationships within the network: the higher the funding, the higher the network clustering), and between funding and the average degree (the average number of project partners each organisation has within the area network). More money means more cooperation, more cliques and a higher number of project partners for each organisation within the network. Figure 7: Correlations between network parameters (FP6) Continuous lines indicate positive correlations, dotted lines refer to negative correlations. Chart: FASresearch The parameters on the right indicate if the networks are single peaked or multi peaked. Single peaked means there is only one core or one centre within the network; multi peaked means that there is more than one core and more than one centre (the network of Figure 7 is multi peaked, as it contains two clusters of parameters). High modularity and high fragmentation characterise multi peaked networks. A high clustering coefficient (the degree to which project partners cooperate with each other) also indicates modularity because the modules emerge through triadic relationships. We can see that there is a tension between the clustering parameters, on the one hand, and the parameters of multi peaked networks, on the other. If the networks of the framework programmes grow (higher funding, more organisations, more linkages), clustering also increases (triadic relationships, cliques); new links connect different sub-networks and close the gaps between them; a single peaked network emerges. The analysis identifies the following correlations between network parameters and CSO participation for the FP6 thematic area networks: Centralisation: There is a negative correlation between the participation of CSOs (types 2, 3, and 4) and network centralisation. We rather find these CSOs in less centralised networks that are less dominated by a few organisations. Modularity: There is a positive correlation between modularity and the participation of CSOs type 3 and CSOs type 4. We rather find these CSOs in multi peaked networks. Fragmentation: There is a positive correlation between fragmentation and the participation of CSOs type 3 and 4. We rather find these CSOs in networks with a 66

71 higher degree of fragmentation (networks with sub-networks that are not connected to each other: Fragmentation leads to modularity). Average clustering coefficient: There is a positive correlation between the average clustering coefficient of the thematic area networks and the percentages of CSOs type 4. The average clustering coefficient also indicates modularity, the existence of several sub-networks with a high amount of internal transitive relationships. All in all, there are just a few significant correlations between the network parameters and CSO participation in FP6. For CSOs type 1, we do not find any correlations. For CSOs types 2, 3, and 4 we can say that they rather appear in less centralised and multi peaked networks. Diagrams, which show the correlations described above, can be found in the Annex. For the 7 th framework programme, it is even more difficult to detect significant correlations between network parameters and CSO participation. There is only a relation between the relative amount of CSOs type 4, on the one hand, and modularity and fragmentation, on the other. The two thematic areas, Regions of Knowledge and Research for the benefit of SMEs, are leading in terms of CSO type 4 participation (type 4 represents business oriented CSOs), and they consist of networks with a high amount of modularity. In the case of FP7, we additionally ran a regression analysis (for FP6 this procedure brought no significant results), which indicated that there is a significant co-variance of network centralisation (domination of a few actors in terms of the number of project partners) and CSO participation. In the case of FP7, we find the highest amount of CSOs type 1 ( core CSOs) within Science in Society and in SSH, and both of them consist of less centralised networks. These correlations are significant, but weak. Our conclusion is that there are just a few correlations of network parameters and CSO participation. The absolute CSO participation is low even in networks which fulfil the criteria of these parameters. Changes of these parameters would change CSO participation to a minor degree. We take it for granted that the network parameters that describe network size and structure, as well as CSO participation itself, are not causes, but effects of other factors, which influence both network structure and CSO participation. Finding 6: CSO participation hardly depends on network morphology. If anything, CSO participation is like network morphology an effect of other influencing factors, especially of funding distribution and cyclicity. We find CSOs involved more in small thematic areas with less centralised networks that have a higher degree of modularity and specific thematic foci Key Actor Analysis Network Position of Institution Types The first task of the key actor analysis is to define the criteria for determining the key actors. In order to describe the position of the institution types (universities, research institutions, companies, CSOs etc.) within the overall network, we use the social capital concept of brokerage and closure. 67

72 In terms of network analysis, actors or key players are organisations that are somehow well embedded in the network of organisations connected to each other through EU funded projects. Network embeddedness, in particular, refers to centrality measures. There are different ways to be central in a network 15. Degree centrality, for example, indicates the number of cooperation partners of each organisation (an indicator for activity). Closeness centrality refers to the distance each actor has to cover to reach every other actor within the network. The smaller this distance, the higher the closeness centrality and the better the accessibility of the actor. There are a lot of further centrality measures, but in the end two dimensions of network centrality, or in other words, of social capital, are decisive: brokerage and closure 16. Brokerage means that an organisation is connected to many other organisations that are not connected to one another. Thus, broker organisations bridge structural holes between other actors. This is an advantageous position within a network. The hypothesis is that network actors that are not connected to each other differ in the resources they own. If an organisation is connected to disconnected organisations, it has access to a more diverse resource portfolio. Closure, on the contrary, indicates that an organisation is connected to many organisations that are also connected to each other (triadic relationships). Closure capital means that an actor belongs to one or more groups. The assumption is that the members of a group are similar regarding the resources they own. Therefore, closure capital rather enables to accumulate and to save resources than to get access to new ones. Being well embedded means to have both closure and brokerage capital stabilisation of existing resources and efficient access to new resources. Organisations with a high amount of brokerage and closure belong to the cores of the FP cooperation networks. These cores consist of structural folds, i.e. of many cohesive groups whose relationships overlap with one another 17. Structural folds reduce tension and close the gap between robust relationships to similar organisations, on the one hand, and efficient but vulnerable contacts to different project partners, on the other. Social capital is unequally distributed among the different types of institutions within the FP networks. We computed brokerage and closure for every organisation 18 and added up these two dimensions in a general index for network embeddedness. If we sort the organisations by this index in descending order and divide them into three groups, we get the organisations of the core (top 30%), the semi-periphery (40% in the middle), and the periphery of the networks (30% at the end of the list). Table 13 shows the distribution of organisation types over the network core, the semi-periphery, and the periphery separately for FP6 and FP7. The numbers are column percentages and row percentages respectively. 15 Wasserman, S./Faust, K. (1994), pp. 167ff. 16 Burt, R.S. (2007). 17 Vedres, B./Stark, D. (2010). 18 Brokerage is measured by betweenness centrality (Freeman, L.S. [1979]) and closure by the two-step Eigenvector Centrality (Bonacich, P. [1972]) within the two-mode networks of organisations and projects. 68

73 Table 13: Percentages of organisation types within the core, semi-periphery, and periphery of the FP networks (FP6 and FP7) FP6 column percentages Network area HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Core 54,8 45,7 22,2 29,7 13,7 18,1 19,7 25,9 16,3 30,0 Semi-Periphery 22,7 30,5 44,6 43,4 33,1 45,3 45,7 48,9 44,9 40,0 Periphery 22,5 23,8 33,2 26,9 53,2 36,5 34,6 25,1 38,8 30,0 Total FP6 row percentages Decile HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Core 19,2 25,0 40,7 8,7 0,3 1,0 0,6 3,0 1,6 100,0 Semi-Periphery 6,0 12,5 61,2 9,6 0,5 1,9 1,0 4,2 3,2 100,0 Periphery 7,9 13,0 60,6 7,9 1,0 2,0 1,0 2,9 3,7 100,0 Total 10,5 16,4 54,8 8,8 0,6 1,7 0,9 3,4 2,9 100,0 FP7 column percentages Network area HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Core 60,3 49,2 23,0 35,0 15,9 15,8 24,5 26,1 18,8 30,0 Semi-Periphery 23,1 29,7 42,7 43,5 52,8 51,2 50,5 47,6 51,0 40,0 Periphery 16,6 21,2 34,4 21,5 31,3 33,1 25,0 26,3 30,2 30,0 Total FP7 row percentages Decile HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Core 16,0 20,5 50,5 8,2 0,4 0,7 0,6 2,7 0,4 100,0 Semi-Periphery 4,6 9,3 70,4 7,6 1,0 1,7 0,9 3,7 0,9 100,0 Periphery 4,4 8,8 75,5 5,0 0,8 1,5 0,6 2,7 0,7 100,0 Total 8,0 12,5 66,0 7,0 0,7 1,3 0,7 3,1 0,7 100,0 Deciles based on brokerage and closure capital of organisations in FP6 and FP7. Data source: CORDIS, WU. Analysis and table: FASresearch. An example from the first table: In FP6, 54.8% of the universities are located in the network core, 22.7% belong to the semi-periphery, and 22.5% to the periphery. The second table tells us that in FP6, 10.5% percent of all institutions are universities, but within the network core, they account for 19.2%. In FP7, we find almost half (49.2%) of the non-university research institutions in the network core, 29.7% in the semiperiphery, and 21.2% in the periphery. 12.5% of all organisations in FP7 refer to nonuniversity research, but within the core they add up to 20.5%. This shows once more that universities and non-university research institutions are the driving forces behind the FP networks. The concentration increased from FP6 to FP7 (from 54.6% to 60.3% of the universities in the core, from 45.7% to 49.2% of the nonuniversity research in the core). In FP6, more than 53.2% of all CSOs of type 1 (66 out of 124) are located in the periphery of the network; only 13.7% (17 out of 124) are located in the centre, and 33.1% (41 out of 124) in the semi-periphery of the FP6 network. All in all, CSOs seem to improve their network position slightly from FP6 to FP7: We find a higher amount of them in the semi-periphery (52.8%), the percentage in the centre is stable (15.9%), and there is a smaller amount in the periphery (31.3%). Furthermore, the numbers indicate that the more business oriented CSOs (types 3 and 4) are slightly better embedded: Within FP7 more than 26% of CSOs type 4 are located in 69

74 the centre (for CSOs type 1 this number is 15.9%); we find only 25% of CSOs 4 in the periphery (compared to 53.2% in case of CSOs 3). Finding 7: Universities and non-university research institutions are the key drivers of the FP networks. We mainly found CSOs in the periphery and in the semi-periphery of the networks. Finding 8: The more business oriented CSOs (lobbying groups, type 4) exhibit a better network embeddedness than the core CSOs (type 1) Networks with and without CSOs To illustrate the position of CSOs, we compare the values of different network parameters how will they change if we remove CSOs from the FP7 network? And how will they change if we remove the universities (Table 14). Table 14: Selected network parameters: FP7 without universities and without CSOs Data source: CORDIS, WU. Analysis and table: FASresearch. Removing the CSOs would mean removing 2.1% of the organisations and 1.8% of the cooperation relations. Removing universities means 7.8% fewer actors, but 46.8% less linkages, which further underlines the central position of universities. Network density (which is the proportion of existing and possible links) would increase by removing CSOs. Without universities, we would have 37.3% less density. The average distance between all actors in the network would not change if there were no CSOs, but would be 9.2% larger without universities. The average number of cooperation partners of each organisation within the total network is 43.3, without CSOs it would be 43.4, without universities only 25. Without CSOs, the network would be a little bit more centralised, meaning they are at the periphery of the network. Without universities, it is less centralised, as they are in the core of the network. Finally, removing CSOs would not change the average number of 3-rings, which indicates the amount of clustering (the existence of cliques). Without universities, there would be 52% less clustering. These numbers indicate that CSOs do not play an important role for the emergence, for the connectedness, nor for the stability of the FP7 collaboration network. Finding 9: Removing CSOs would not change network parameters and morphology (assuming all other things remaining the same). CSOs were hangers on, they did not build sub-clusters or bridge gaps, and they did not have an important brokerage function. 70

75 6.1.7 Key Link Analysis Connections of CSOs Duration of Cooperation Which kinds of relations are typical for CSOs? At first, we assume that CSOs, in general, are more involved in short-term projects. They participate in coordination and support actions (CSAs) more often than universities, research institutes, or enterprises; and the average duration of CSAs is shorter than of collaborative projects (28.2 months vs. 39 months). They do not participate in ERC projects (with three exceptions) whose average duration is 58.2 months. Figure 8: FP7 average duration of projects and relative amount of CSO participations (all types) X-axis: percentages of CSO participations within FP7 thematic areas (all CSO types). y-axis: average duration of projects in months. Data source: CORDIS. Chart: FASresearch. Figure 8 represents average project duration and CSO participation in FP7 thematic areas. We know that CSO participation is highest within Science in Society and Regions of Knowledge. Regions of Knowledge is about coordination and support actions, and only 25% of all projects within Science in Society are collaborative research projects. Both of them rather consist of short-term projects (average duration about 32 months). Also within Research policies (25 months), International cooperation (37,6), and Research for the benefit of SMEs (27 months), project duration is shorter and CSO participation is higher than within the research-oriented thematic areas. The chart indicates that CSOs appear more frequently in thematic areas with lower project duration. This could be one reason that it is more difficult for CSOs to build up long-term relationships to other organisations through EU funded projects (high drop-out rate). Finding 10: CSO participation was higher in thematic areas and funding schemes with projects with shorter average duration. Thus, it was more difficult for CSOs to build up more stable and more continuous relationships. 71

76 Coordination and Participation What are the roles and positions of institution types and of CSOs, in particular, within EU funded projects? Who is coordinator and who is participant? Who has the lead if CSOs take part in FP projects? In general, CSOs rarely function as coordinators. Only 6.8% of all CSO type 1 participations (22 out of 302) are in a coordination position. This value for universities is 28.8% ( out of ) and 21.5% (7.081 of ) for non-university research institutes. At least 11.2% of all CSO type 4 project participations (196 out of 1.756) are in a coordination position. CSOs 2 and CSOs 3 are coordinators only in 9.7% and 8.9% of their project participations. Enterprises, in general, exhibit a high number of project participations, but a low proportion of coordination positions (7.6%). Even within their specific playgrounds, CSOs rarely act as coordinators. Within Science in Society, only 2 out of 183 projects are coordinated by CSOs of type 1 (1,1%), 4.9% (= 9 projects) by CSOs type 2, 7 projects by CSOs type 3, and 6 projects by business oriented CSOs. This type of CSO coordinates 16.7% of all projects within Regions of Knowledge (14 out of 84) and 5.5% (57 out of 1.028) within Research for the benefit of SMEs. Within Socio-economic sciences and Humanities, CSOs do not act as coordinators at all. 19 We can assume that there is a correlation of the coordination and the participation of certain institution types. Depending on the types of participants, certain types of coordinators appear. Table 15 represents the relationship of coordination and participation for the overall FP7. Note that the numbers indicate relationships, rows represent the institution types as coordinators, and columns represent institution types as participants. For example, (first table), universities are embedded in relationships, in which they act as coordinators, and in which the institutions they are connected to act as participants. There are relations in which universities are coordinators and participants. Within the FP7 network, there are 132 relations in which universities are coordinators and CSOs type 1 are participants. To compare the numbers for different types, we have to look at the column percentages (second table). Table 15: FP7 coordination and participation of institution types (absolute numbers) Coordination Participation Type HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total HES REC PRC PUB CSO CSO CSO CSO OTH Total See the Annex for a detailed listing of roles and positions of institution types across the thematic areas. 72

77 Coordination Participation Type HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total HES 48,1 33,9 26,6 21,2 43,7 35,4 30,0 18,1 20,0 35,1 REC 30,6 38,7 30,5 38,0 24,2 26,4 27,6 29,0 29,7 32,8 PRC 16,9 20,9 37,8 17,9 15,2 17,2 26,3 32,2 29,7 25,5 PUB 2,7 4,4 2,1 18,3 7,6 9,0 5,8 7,1 10,0 3,9 CSO1 0,1 0,1 0,0 0,1 2,6 2,0 1,9 0,2 0,3 0,1 CSO2 0,2 0,2 0,2 0,5 3,0 4,2 1,6 0,3 1,2 0,3 CSO3 0,2 0,1 0,1 0,2 0,7 1,8 0,5 0,4 1,8 0,2 CSO4 1,0 1,5 2,6 3,3 1,0 3,1 6,1 12,4 5,5 2,0 OTH 0,1 0,2 0,1 0,5 2,0 0,7 0,3 0,4 1,8 0,2 Total Data source: CORDIS. Table: FASresearch. If universities are participants, in 48.1% of the relations universities are coordinators as well. If CSOs type 1 are participants, then in 43.7% of the cases universities are coordinators. Thus, CSOs type 1 are the institution type for which universities as coordinators are most important (with the exception of universities themselves). In other words, CSOs of type 1 are rather invited to participate by universities. This is because we find them in thematic areas in which mainly universities are coordinators. The pattern for CSOs type 2 is similar (research institutes, enterprises, public bodies and CSOs type 4 are slightly more important as coordinators for them than universities). For the more business oriented CSOs and lobbying groups (CSOs types 3 and especially type 4), which we rather find in thematic areas specific to companies, it is the companies that are important as coordinators. If a CSO type 4 is a participant, the probability that an enterprise is coordinator is 32.2% (research institutes: 29%, universities: 18.1%, and CSO type 4 themselves 12.4%). Finding 11: CSOs rarely acted as coordinators. It hardly depended on the type of coordinator if a CSO was invited to participate in a project or not. If a thematic area addressed excellent research, the number of universities and research institutes was above average, and these types of institutions then coordinated the projects. If a thematic area was designed for the private sector, companies more often acted as coordinators. In the first case, CSOs were rather invited by universities or research institutes. In the second case, it was the enterprises that asked for CSOs to take part in EU funded projects. 6.2 Publication Analysis The first part of the network analysis of our study described the network indicators of the European FP networks and their correlation with CSO participation. The following part refers to the analysis of the research indicators, i.e. of the scientific output of EU funded projects and the involvement of CSOs in this output activities. For this purpose, we use OpenAIRE publication data Data Base The OpenAIRE team provided publication data in October All in all, they included a list of unique publications of them are connected to unique IDs of EU funded FP7 projects. Our project network data set includes FP7 projects. Thus, we have publication data for 46,7% of the projects in our network data set. Certain funding schemes produce more publications than others. Above all, the amount of ERCprojects and of Collaborative projects (CPs) is higher within the publication data than 73

78 within the overall project data (28% vs. 18% and 34.5% vs. 27.5%). On the other hand, just 5% of projects with publications are CSAs (Coordination and support actions), whereas their total amount in CORDIS is 10%. This is important because we know that CSO participation is higher among CSAs than among CPs or ERC projects. The European Research Council projects, Information and Communication Technologies, and Health are the thematic areas with the highest number of publications, followed by the Marie-Curie Actions, Environment, Research Infrastructures, and Environment. Our first impression is that there are fewer publications in areas with a (relatively) high amount of CSO participation. Figure 9 illustrates this correlation: Within Science in Society, CSOs (types 1, 2, and 3) are involved in 42.1% of the projects, but only 0,1% of the publications in our OpenAIRE data set originate from Science in Society. Within the European Research Council, on the other hand, only 2 of projects (= 0,04%) happen with CSO participation, but 36.4% of the OpenAIRE publications derive from ERC. Figure 9: FP7 thematic areas by CSO participation and percentages of publications within OpenAIRE data. Data sources: OpenAIRE, CORDIS, WU. Chart: FASresearch. Looking at the diagram we can propose that it is the Health area that exhibits a number of projects with CSO participation and a publication output above average. Finding 12: With regard to publication output, Health and ICT are by far the most important thematic areas (they dominate the ERC funded projects as well). Finding 13: The publication output is dominated by thematic areas, in which the amount of projects with CSO participation is rather low. 74

79 The main questions of the analysis of the publication data are: How can we describe the contribution of CSOs to the publication output of EU funded projects? In which areas do we find the highest amount of scientific output? And how important is the participation of CSOs for these areas? We use the All Science Journal Classification (ASJC) to describe the scientific fields in which the publication output takes place. ASJC is provided by the SCImago Journal & Country Rank portal 20. There is an important limitation regarding the OpenAIRE data: They do not include the information about the organisations of the authors (universities, institutes, companies etc.). Therefore, we cannot estimate the real contribution of CSOs to publishing results of EU funded projects. However, we know which FP7 projects led to publications, and we know whether CSOs participated in these projects. All in all, the ASJC Journal Classification contains 307 different classes of scientific disciplines in 24 different fields (for example, Agricultural and Biological Sciences, Arts and Humanities, Biochemistry, Genetics and Molecular Biology, Business, Management and Accounting, etc.). These classes are added to the Journals to describe the main content of the publications that are published within them. This is the second limitation of the OpenAIRE data: The ASJC classes do not refer to the content of the specific publication, but rather to the Journal in which the article was published. Thus, we have to combine three data sets to determine the research performance of CSOs in EU funded projects: CORDIS data with FP7 projects and participants, OpenAIRE data with publications (including the CORDIS project-ids of the FP7 projects, to which the articles belong, and the ISSN International Standard Serial Number of the journal in which the article was published), and the SCImago data which add the ASJC scientific disciplines to the journals (Figure 10). Figure 10: Combining different data sources to generate FP7 publication networks Chart: FASresearch. The CORDIS data include the information about projects (thematic area, funding, duration etc.) and about the participating organisations (universities, research institutes, public bodies, CSOs ). The OpenAIRE data contain the articles (tiltle, publication date, name of the author, journal, ISSN number), including the CORDIS project IDs through which we can connect them to the CORDIS database. Additionally, we have the list of journals and the ASJC classes, which we add to the other data sets via the ISSN number of the journals. Journal data (SCImago) were not available for all of the publications. In the end, we have a comprehensive data set containing unique articles with 20 SCImago. (2007). SJR SCImago Journal & Country Rank. Retrieved July, 2016, from 75

80 information about the FP7 projects and about the scientific disciplines of the journals in which they were published Core Areas of Scientific Output Figure 11 shows in which scientific disciplines the articles appear and through which thematic area they were funded. Figure 11: Network of FP7 thematic areas and ASJC disciplines Data source: CORDIS, OpenAIRE. Analysis and visualisation: FASresearch. Arcs lead from thematic areas to scientific disciplines and indicate that both are connected through at least 500 publications which derived from EU funded projects within the particular thematic area. For example, in the field of Genetics articles were published that were funded through projects in the FP7 Health area. Only links which represent at least 500 articles are depicted. We see that ERC, Health, Marie-Curie Actions, and ICT produce the highest number of articles. It is also visible (at this highly aggregated level) that, on the one hand, there are area-specific disciplines that are only connected to one thematic area (e.g., articles in the field of Cellular and Molecular Neuroscience only arise in the Health theme), and, on the other hand, we have disciplines that connect different areas. However, it is remarkable that the areas are highly disconnected. Articles that were funded by Health projects are linked to different disciplines than articles that were funded within the ICT theme. We can also see the disciplines that dominate the publications of the ERC and the Marie Curie-Action projects Scientific Output and CSO Participation Now we combine the three data sets (publications, scientific disciplines, and projects) once more and determine the amount of CSO participation in the different scientific fields in which the articles were published. Note that the numbers in Table 16 indicate publications (for example, all in all we identified 21 different publications in the discipline Physics and Astronomy with the participation of at least one CSO1), but multiple 76

81 mentions are possible (one article can be assigned to more than one discipline). And there is another limitation: We do not know if the CSOs themselves participated in writing the article, we only know that the article was funded by an FP7 project in which at least one CSO 1 participated. Table 16: Scientific disciplines and types of institutions by number of publications (absolute numbers). ASJC scientific area HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Physics and Astronomy Biochemistry, Genetics and Molecular Biology Medicine Materials Science Engineering Decision Sciences Computer Science Agricultural and Biological Sciences Chemistry Environmental Science Immunology and Microbiology Chemical Engineering Pharmacology, Toxicology and Pharmaceutics Neuroscience Social Sciences Energy Psychology Nursing Health Professions Economics, Econometrics and Finance Arts and Humanities Business, Management and Accounting Veterinary Dentistry Total Multiple mentions possible. Data source: OpenAIRE, CORDIS, WU. Computation and table: FASresearch. We find the highest numbers of publications with (at least indirect) CSO1 participation in disciplines which belong to the Health area: Medicine (461); Biochemistry, Genetics, and Molecular Biology (177), and Neuroscience (119). Delving deeper into the CORDIS data, we see that in many cases these are charities, patient organisations, and self-help groups in the fields of human medicine and diseases. The organisations with the highest number of projects that lead to publications are: Age Platform Europe, Foundation for Innovative New Diagnostics, Autism Europe, The European Foundation for Clinical Nanomedicine, AbilityNet, and Skalbes (a Crisis Intervention Centre in Latvia). For CSOs type 2 (besides Medicine) disciplines show up which refer to Decision Sciences (238) and Environmental Sciences (235). The most important CSOs here are the Stichting Nederlands Instituut Voor Beeld En Geluid, ICLEI European Secretariat GmbH, the Smithsonian Institution, Centro de Referencia em Informacao Ambiental, Species 2000, Fondazione Per Adroterapia Oncologica Tera, Fundacion Phantoms, and the Stichting Wetlands International. Publication numbers are again higher for business oriented CSOs (types 3 and 4). They take part in projects in the thematic areas that refer to economy and business and address companies and enterprises (Research for the benefit of SMEs, ICT, etc.), and the articles are published in the fields of Physics and Astronomy; Engineering; Medicine; Biochemistry, Genetics and Molecular Biology; and Agricultural and Biological Sciences. In determining the amount of CSO participation within each scientific field (Table 17), we see that in the Medicine category only 1.2% of the articles were funded by FP7 projects with CSO type 1 participation. The relative number is highest within the Nursing category 77

82 (1.6%), but this concerns only 26 articles. Nevertheless, the relative numbers show us that if there are any publications in which CSOs participate, then this happens in the fields of human medicine and diseases in many cases. For the business oriented CSOs and lobbying organisations, Business, Management and Accounting is the most important category. Table 17: Scientific disciplines and types of institutions by number of publications (row percentages). ASJC scientific area HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Physics and Astronomy 45,9 34,9 14,1 2,7 0,0 0,3 0,4 1,2 0,5 100,0 Biochemistry, Genetics and Molecular Biology 37,5 31,0 22,3 6,7 0,4 0,4 0,4 1,0 0,2 100,0 Medicine 33,9 28,6 22,0 11,0 1,2 0,9 0,7 1,2 0,3 100,0 Materials Science 44,3 31,1 19,5 3,1 0,0 0,3 0,2 1,3 0,3 100,0 Engineering 39,3 29,5 24,3 4,1 0,1 0,2 0,3 1,8 0,4 100,0 Decision Sciences 42,3 31,9 12,4 10,7 0,2 1,1 0,3 0,8 0,2 100,0 Computer Science 39,7 29,2 24,0 4,3 0,2 0,4 0,6 1,2 0,4 100,0 Agricultural and Biological Sciences 35,6 29,8 19,1 10,7 0,4 0,9 1,0 2,1 0,5 100,0 Chemistry 42,4 29,7 20,2 5,2 0,1 0,2 0,4 1,4 0,4 100,0 Environmental Science 33,5 29,1 20,2 12,2 0,6 1,6 0,6 1,8 0,3 100,0 Immunology and Microbiology 33,3 30,3 22,1 11,0 0,7 0,3 0,5 1,6 0,2 100,0 Chemical Engineering 42,5 29,5 22,5 3,2 0,1 0,3 0,1 1,5 0,4 100,0 Pharmacology, Toxicology and Pharmaceutics 32,8 27,3 24,0 11,3 0,9 0,3 0,7 2,6 0,2 100,0 Neuroscience 40,3 30,4 19,7 5,8 1,5 0,4 0,6 1,3 0,1 100,0 Social Sciences 45,9 25,5 16,3 7,3 1,2 1,4 0,7 1,3 0,4 100,0 Energy 34,7 30,3 26,8 4,9 0,1 0,5 0,5 1,7 0,5 100,0 Psychology 51,4 28,3 14,1 4,5 0,4 0,4 0,4 0,5 0,1 100,0 Nursing 31,0 27,3 24,4 10,0 1,6 0,5 2,4 2,5 0,4 100,0 Health Professions 36,1 29,4 23,4 6,8 0,0 2,8 0,2 1,1 0,3 100,0 Economics, Econometrics and Finance 52,3 24,5 12,0 6,5 0,5 0,8 1,1 1,8 0,4 100,0 Arts and Humanities 67,3 21,7 6,3 3,0 1,1 0,4 0,0 0,1 0,0 100,0 Business, Management and Accounting 35,6 24,9 20,7 10,2 0,9 0,9 0,4 5,6 0,9 100,0 Veterinary 31,6 30,4 27,5 9,4 0,0 0,0 0,0 1,2 0,0 100,0 Dentistry 35,7 28,6 33,3 2,4 0,0 0,0 0,0 0,0 0,0 100,0 Total 39,7 30,6 19,9 6,7 0,4 0,5 0,5 1,4 0,3 100,0 Multiple mentions possible. Data source: OpenAIRE, CORDIS, WU. Computation and table: FASresearch. Table 18: Scientific disciplines and types of institutions by number of publications (column percentages). ASJC scientific area HES REC PRC PUB CSO1 CSO2 CSO3 CSO4 OTH Total Physics and Astronomy 17,0 16,7 10,4 5,8 1,5 7,1 12,4 13,1 23,1 14,7 Biochemistry, Genetics and Molecular Biology 12,7 13,6 15,0 13,5 12,8 9,7 12,1 9,8 8,8 13,4 Medicine 9,8 10,7 12,6 18,7 33,4 20,0 17,3 10,1 10,8 11,4 Materials Science 10,6 9,7 9,3 4,3 1,2 4,5 3,7 8,9 8,8 9,5 Engineering 8,2 8,0 10,2 5,1 2,2 3,6 4,7 11,0 8,8 8,3 Decision Sciences 7,1 7,0 4,2 10,7 3,8 13,0 4,7 3,9 4,2 6,7 Computer Science 6,0 5,7 7,3 3,9 3,0 4,7 7,1 5,4 7,1 6,0 Agricultural and Biological Sciences 4,6 5,0 4,9 8,1 4,5 8,3 11,2 7,7 6,8 5,1 Chemistry 5,0 4,5 4,7 3,6 1,3 1,6 3,7 4,7 5,5 4,7 Environmental Science 3,8 4,2 4,5 8,1 7,0 12,9 5,7 5,8 4,5 4,4 Immunology and Microbiology 3,3 3,9 4,3 6,4 7,0 2,5 3,9 4,6 2,2 3,9 Chemical Engineering 2,9 2,6 3,1 1,3 0,4 1,4 0,6 3,0 3,1 2,7 Pharmacology, Toxicology and Pharmaceutics 2,0 2,1 2,9 4,0 5,0 1,1 3,3 4,6 1,3 2,4 Neuroscience 2,4 2,3 2,3 2,0 8,6 1,8 2,9 2,2 0,8 2,4 Social Sciences 1,5 1,1 1,1 1,5 4,1 3,5 2,0 1,3 1,4 1,3 Energy 0,9 1,0 1,3 0,7 0,2 0,9 1,0 1,3 1,4 1,0 Psychology 0,7 0,5 0,4 0,3 0,5 0,3 0,4 0,2 0,1 0,5 Nursing 0,4 0,4 0,6 0,7 1,9 0,4 2,4 0,9 0,5 0,5 Health Professions 0,3 0,3 0,4 0,3 0,0 1,8 0,1 0,3 0,3 0,3 Economics, Econometrics and Finance 0,4 0,2 0,2 0,3 0,4 0,4 0,6 0,4 0,3 0,3 Arts and Humanities 0,4 0,2 0,1 0,1 0,7 0,2 0,0 0,0 0,0 0,2 Business, Management and Accounting 0,2 0,2 0,2 0,3 0,4 0,3 0,2 0,8 0,5 0,2 Veterinary 0,0 0,1 0,1 0,1 0,0 0,0 0,0 0,0 0,0 0,1 Dentistry 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 0,0 Total 100,0 100,0 100,0 100,0 100,0 100,0 100,0 100,0 100,0 100,0 Multiple mentions possible. Data source: OpenAIRE, CORDIS, WU. Computation and table: FASresearch. 78

83 Table 18 indicates which scientific fields are most important for each category of CSO. One third of all publications with (at least indirect) CSO 1 participation are assigned to the Medicine category. For CSOs type 2, this number is 20%, followed by Decision Sciences (13%), and Environmental Science (12.9%). Articles of projects with the participation of CSOs type 3 are related to Medicine (17.3%), Physics and Astronomy (12.4%), Biochemistry, Genetics, and Molecular Biology (12.1%). For CSOs of type 4 Physics and Astronomy (13.1%), Engineering (11%), and again Medicine (10.1%) are most important. Although these are interesting cases, organisations and fields, the numbers of publications that were funded by FP7 projects with the participation of CSOs are marginal when we compare it to the publication numbers of universities, research institutes, and companies. Furthermore, the scientific fields in our publication data with a higher relative amount of CSO participation refer to smaller thematic areas (less funding, less projects, less organisations). Figure 12 shows this correlation. It represents the network of scientific disciplines (ASJC classes at the 4-digit level) and their connections through publications that are derived from EU funded projects. Disciplines can be connected because articles were published in journals with more than one class or in different journals. We can treat disciplines that are linked to each other as similar regarding the knowledge they produce. The size of the classes refers to the number of articles in the years that were assigned to them. The colours indicate the amount of project participations of CSOs types 1, 2, and 3 (i.e. without the genuine business oriented CSOs) within the projects that produced publications in the particular scientific field. The maximum value for CSO participation is 4.3% in the category Tourism, Leisure and Hospitality Management with 11 articles. We can easily identify two big clusters: technical and engineering sciences on the left and life sciences on the right. The classes with the biggest numbers of articles are Condensed Matter Physics; Astronomy and Astrophysics; and Electronic, Optical and Magnetic Materials on the right and Molecular Biology, Biochemistry, and Genetics on the left. It is typical that the scientific discipline, which mainly connects technical and life sciences, is Computer science applications. Both technical and life sciences need the mathematical and computer sciences for methods and techniques of analysis. All in all, the percentages of CSO participations are low the average value for the disciplines within the network is 0.46%. But we can see that CSO participation is a little bit higher among the classes within the life science sector than among the classes within the technical sciences sector. This corresponds to the result that the Health thematic area is the only theme with publication output and CSO participation above average. Life science categories with a relatively high CSO participation are Health Policy, Developmental Neuroscience, Complementary and alternative medicine, and Anaesthesiology and Pain Medicine. Note that these are small categories with respect to the number of articles. 79

84 Figure 12: Network of ASJC-disciplines (All Journal Science Classification) which are connected through EU funded publications Data source: OpenAIRE. Analysis and graphic: FASresearch. 80

85 Beyond the two clusters of technical and life sciences, there are disciplines at the periphery of the network which exhibit the highest amount of CSO participation. They belong to the economical sciences, to the social sciences, and to the humanities. The ones with the highest amount of CSO participations are Tourism, Leisure and Hospitality Management (11 articles/4.3% CSO participation); Business, Management and Accounting (7/3.8%); History (215/1.9%), and Cultural Sciences (95/1.8%). Finding 14: The network of scientific disciplines derived from publications is driven by two thematic cores: engineering and computer sciences, on the one hand, and human medicine and life sciences, on the other. In both of them, CSO participation is rather low. The results so far seem to demonstrate that CSOs do not participate in EU funded projects with a high publication output. Research and publishing research results does not seem to be the main goal of CSOs when they take part in EU funded projects. Our next step is to show that this correlation more projects with higher publication output and lower CSO participation is significant. Table 19: Numbers of projects with/without CSO participation (all types) and with/without publications Publications No Yes Total Observed CSO No values participation Yes Total Publications No Yes Total Expected CSO No values participation Yes Total Chi square 0,00 Data sources: CORDIS, WU, OpenAIRE. Table: FASresearch. Table 19 represents a cross tabulation with the number of projects that were conducted with or without the participation of CSOs (all types) and which led to publications or not. The first table includes the observed numbers of projects. All in all, we have projects in our FP7 data set. There are projects (48,3%) without any CSO participation and without publication output. We have identified projects (44.1%) with publications but without CSOs. In projects (5%) CSOs took part but no publications were written. And finally, there are 661 projects (that is 2.6% of all FP7 projects) with CSOs and with publications. The second table contains the values we would expect if there was a random distribution of projects. Based on the given distribution of projects with and without publications and CSOs respectively, we would expect 896 projects with CSO participation and with publication output. But we only have observed 661, that is 35.6% less than expected. And the chi-square test shows that this divergence is significant. There is a significant negative correlation of CSO participation and publication output. One important reason might be that CSOs are more engaged in coordination activities than in research projects (see Table 12), and coordination activities have a lower publication output than research projects. The average number of publications per project among CSAs is 8.2; among CPs the number is

86 Finding 15: There is a significant (statistical, not causal) negative correlation of CSO participation and publication output across EU funded projects. We rather found CSOs in thematic areas with a smaller number of publications. Scientific excellence and thus a high publication output is not the objective of CSOs when they take part in EU funded projects. Finding 16: Our conclusion is that CSOs do not have an impact on research performance as measured by publication output. The participation of CSOs in EU funded projects does not increase the number of scientific articles. On the contrary, we rather find them in projects that do not generate, or generate fewer, publications. One reason is that CSOs take part in more coordination projects than in research projects. Research and scientific excellence are not the main goal functions of CSOs when they take part in EU funded projects. They play a different role in the innovation process. 6.3 Media Analysis Background and Data Base We consider publicity and media exposure as important currencies of civil society organisations. If it is true that the main goal of CSOs in FPs is not scientific excellence (as it is for universities and research institutions) and not applicability, sales, and profit (as it is for companies), but finding solutions for societal problems, they need to be wellknown and raise awareness for the problems they deal with. For this reason, we tried to determine the presence of CSOs in media reports of newspapers and magazines (print as well as online) about EU funded projects. We used international media data bases (e.g. Factiva) as data sources. Within the content they provide, we identified articles about EU funded projects with specific search terms. 21 The period of investigation was from 2007/01/01 until 2015/12/31. For this period, we detected a total number of articles that contained the search terms mentioned above. From these, we took a random sample of articles for an in-depth qualitative analysis. We went through each article and identified the EU funded project it pertained to, recorded the institutions that were mentioned and classified them (universities, research institutes, companies, public institutions, CSOs, and others), and, finally, we determined the topic of the EU funded project that was reported in the article. Regarding these topics, we took the target classification of the EU Sustainable Development Strategy (EU SDS) and added additional categories when necessary. A certain number of articles in our data set deal with EU funded projects outside of the Framework Programmes; nevertheless, they are important for our analysis, and the SDS target classification, plus additional categories, is more suitable to describe the content of the articles and projects that is why we did not use the thematic areas of the CORDIS database. 21 "eu project" or "eu funded project" or "project funded by the eu" or "european union funded project" or "eu research project" or "european union research project" or "h2020 project" or "fp7 project" or "horizon2020 project" or "horizon 2020 project" or "seventh framework programme project" or "7. framework programme project". 82

87 6.3.2 Types of institutions mentioned in media reports After data cleaning, we could refer to a data set with 864 articles about FP projects and articles about EU funded projects outside the FPs (INTERREG, European Neighbourhood and Partnership Instrument, European Development Fund, European Regional Development Fund, etc.). Within these articles we found unique institutions mentioned by name. The type with the highest number is public institutions (41,3%), followed by companies (25,8%), universities (14%), CSOs (12,9%), and research institutes (4,7%); the rest is others (1,2%). Table 20: Numbers of institutions mentioned in media reports about EU funded projects (FP related or not) Table: FASresearch FP related Not FP related Type Number % Number % Total Universities , ,9 823 Non-university research 172 8,7 85 2,6 257 Companies , , Public institutions , , CSOs 121 6, ,3 651 Others 11 0,6 51 1,6 62 Total Within the articles reporting on FP projects, companies dominate the arena (42,4%), followed by universities (26,9%), public institutions (15,3%), research institutes (8,7%), and CSOs (6,1%). We can compare these numbers with the percentages of institutions within our FP7 CORDIS data set (not listed in the table). Within the media reports universities and public institutions are overrepresented (universities: 26,9% vs. 8%; public institutions: 15,3% vs. 7%). There is a similar amount of CSOs in both of the data sets (media reports: 6,1%; FP7: 5,8%). There are fewer companies (42,4% vs. 65,9%) and fewer non-university research institutes (8,7% vs. 12,5%) in the media reports than in the CORDIS project data. Therefore, the articles about FP projects like the projects themselves are dominated by universities and companies (by the currencies of scientific excellence and profit). When it comes to media reports about EU funded projects outside the Framework Programmes, media exposure of CSOs (and of public institutions as well) is much higher. These projects rather belong to the spheres of regional cooperation or development aid than to R&I. More than 16% of all institutions mentioned in the articles about projects outside the FPs are CSOs, compared to 6% in FP related articles. The projects outside of FPs seem to address the interests, needs, and skills of CSOs rather than projects that are essentially scientific. Finding 17: Within the articles about FP projects the presence of CSOs is as marginal as in the FP projects themselves. Finding 18: Media exposure of CSOs is much higher in articles about EU funded projects outside of the FPs. If we take media exposure as an indicator for CSO participation, programmes and funds like the European Development Fund, the European Regional Development Fund or INTERREG seem to address the needs and capabilities of CSOs more than the research or business oriented Framework Programmes. 83

88 6.3.3 Topics of Media Reports The topics of projects are different when we differentiate between FP projects and projects outside of FPs. 28.7% of articles about FP projects refer to Health. Health is the most important topic of articles about FP funded projects in our data set, followed by Science, innovation, and technology in general (21.3%), and Climate change and clean energy (17.7%). Finding 19: The most important topics of media reports about EU funded projects are Health, Innovation and Science in general, Climate Change, Information and Communication Technologies, and Natural Resources. Topics of articles about EU funded projects outside of FPs are different. The most important topic is Promoting sustainable regional development (26.4%), Social inclusion, demography, and migration (20.8%), and Promoting and strengthening SD governance (18.5%). Within these fields we will find a higher amount of CSOs mentioned in the articles. Finding 20: Media reports about EU funded projects outside of the FPs mainly refer to Promoting sustainable regional development; Social inclusion, demography, and migration; Promoting and strengthening SD governance; Climate Change, and Natural Resources Types of Institutions and Topics Our next step is to combine these dimensions of information kinds of projects, types of institutions, and topics. We have seen that universities and research institutes appear more often in articles about FP projects, whereas CSOs and public institutions dominate the media reports about EU funded projects outside of the Framework Programmes. We can assume that this correlates with the specific topics of FP projects and non-fp projects. We know that regarding media reports Health is the most important topic of FP projects for universities. 26.5% of all cases, in which universities show up, refer to Health projects. Health is followed by Science/Innovation/Technology (14.2%), ICT (12.4%), Conservation and management of natural resources (11.8%), and Sustainable consumption and production (9.3%). When it comes to articles about non- FP projects with university participation, the most important topic is Promoting sustainable regional development (13%), Health (12.3%), Social inclusion, demography, and migration (11.3%), Conservation and management of natural resources (9.9%), and Climate change and clean energy (8.2%). Articles about FP projects with company participation mostly refer to Science, Innovation & Technology (16.5%), Climate change and clean energy (15%), Health (14.2%), ICT (13.9%), and Sustainable Transport (11.7%). Media reports about EU funded projects outside of the FPs, in which companies are namely mentioned, deal with Climate change and clean energy (17.3%), Promoting sustainable regional development (12.9%), Conservation and management of natural resources (8.5%), Science/Innovation/Technology (7.8%), and Sustainable Transport (7.2%). Let us see in which media reports the CSOs appear. The most important topic of articles about EU funded projects, in which CSOs are mentioned, is Social inclusion, demography, and migration. 28.2% of all articles with CSOs belong to this thematic 84

89 area. But the importance of this topic only results from its importance within the non- FP projects (only 5.2% of the FP related articles refer to Social inclusion). The pattern is similar for the next two topics: Promoting sustainable regional development (26.6% of all articles in which CSOs are mentioned) and Promoting and strengthening SD governance (17.9%) get their importance from non-fp projects. The fourth topic Conservation and management of natural resources, however, is equally important in both FP and non-fp projects with CSO participation (16,5% and 16% respectively). And the fifth topic is Health (15%), which is the most important topic within the articles about FP projects with CSO participation. If there are CSOs mentioned in media reports about FP projects, the topics are similar to the topics of FP projects in general Health, Science/Innovation/Technology in general, ICT, Conservation and management of natural resources, and Climate change and clean energy. If we compute the differences of the percentages for articles about projects inside and outside the FPs, we can identify the topics that are overrepresented within the articles about FP projects and the ones that are overrepresented within the articles about non-fp projects respectively with CSO participation in both cases (Figure 13). Figure 13: Topics of projects with CSO participation in media reports comparison inside/outside FPs Data base: FP and non FP related articles in which CSOs are mentioned (N = 523). Diagram: FASresearch In general, media exposure of CSOs is higher in articles about EU funded projects outside of FPs. In these articles, the following topics are important: Sustainability, Regional development, Social inclusion, Global poverty, and Preserving of historic environment and national heritage. In articles about FP funded projects with CSO participation, health, science and innovation, in general, and ICT are topics with an above average frequency. 85

90 Figure 14 shows the five most important topics of each institution type as a network graph. Grey nodes indicate institution types; arcs lead to the topics (grey nodes), and the strength and the darkness of the arcs depend on the percentages of articles that refer to the institution type and the topic. We can easily identify two clusters, meaning that there are institution types with typical topics. On the left there is the cluster with companies, universities, and research institutes. It represents the currencies of scientific excellence and profit. Typical topics of this cluster are Health, ICT, and Science and innovation in general. On the right we find a cluster that connects politics and civil society through their topics Sustainable regional development, Sustainable development governance, and Social inclusion. There are at least two topics that connect these two clusters: Health and Conservation and Management of natural resources. The structure of this network indicates that the currencies of the scientific field (excellence), and of the economic field (profit), differ from the currency of the civil society (problem solving, awareness). Figure 14: Top 5 topics of each institution type in media reports about EU funded projects (FP and non FP related) Data base: FP and non FP related articles (N = 2.339). Visualisation: FASresearch. Finding 21: Within the media reports about EU funded projects, different types of organisations typically appear together with specific topics. Both organisation types and topics represent the three currencies of innovation: Science (universities and research institutions/basic research in general and health); economy (companies/ict, energy and transport), and civil society (CSOs/social inclusion, sustainable regional development, and SD governance). Finding 22: There are typical topics of articles about EU funded projects in which CSOs participate. Topics which support the participation of CSOs are most notably sustainability, regional development, international development and cooperation, social inclusion, demography, migration, and poverty. 86

91 7 Simulation of policy interventions This section describes the agent-based simulation model that was developed in the course of the project and presents the results from several scenarios that were simulated. A detailed technical description of the simulation model is included in the Annex to this report. 7.1 Overview of the simulation model An agent-based model (ABM) was developed for the purpose of this study, simulating the Framework Programmes (FPs) and their participating organisations in terms of call design, consortium formation, proposal evaluation, and project implementation. The core of the simulation model lies in consortium formation, i.e. the process by which a coordinator searches for project partners with respect to a call for proposals that requires specific (combinations of) skills. As a product of this formation phase, the consortium produces a proposal that is evaluated based on its expertise levels in the required skills, its management aspects, as well as other eligibility conditions (e.g. requested budget, number of partners and country distribution). The following sub-sections briefly summarise the main elements of the model; a more comprehensive (technical) description of the model explaining each element in detail is included in the Annex to this report Scope of the model The ultimate aim of the model was to simulate the participation of Civil Society Organisations (CSOs) in the FPs. Since CSO is not a pre-defined category for organisations participating in the FPs, it was necessary to first define, characterise and identify them in the previous FPs (FP6 and FP7) and then to analyse their participation patterns; the results of these tasks are documented in the previous chapters of this report. Based on these analyses, the scope of the model was defined so as to simulate the top-down defined, collaborative parts ( themes ) of the FPs. With respect to FP7, this includes the Cooperation programme and some parts of the Capacities programme, but excludes the European Research Council (ERC; referred to as Ideas programme in FP7) and the Marie Skłodowska-Curie actions (MCSA; referred to as People programme in FP7). With respect to Horizon 2020 (H2020), this includes the societal challenges pillar, a major part of the industrial leadership pillar, as well as a small part of the excellent science pillar Organisations All organisations are modelled as nodes of a social network graph. They are categorised according to the organisation typology developed for the purpose of this study, distinguishing between universities (HES), research organisations (REC), companies (PRC), public bodies (PUB), other (OTH), as well as the four CSO-categories (see section 4.3 above for an overview of the organisation types used in this study). Each organisation also has a network position attribute big, medium or small that is calculated based on this organisation s encounter probability (a measure derived from the social network analysis; see section above for an explanation). 87

92 7.1.3 Skill vectors A central element of the simulation model is the different skills of the participating organisations, upon which they form consortia and upon which their proposals are then evaluated. Skills have names combined with respective levels of expertise. In order to simulate consortium formation and proposal evaluation, skills need to be defined for both organisations (so that they can contribute their expertise to a consortium) and topics (so that proposals can be evaluated according to the skills required by the topic). Two different domains of skills are used in the model: knowledge and impact. Knowledge refers to scientific disciplines according to the Scopus classification of subject categories (also called minor subject areas in Scopus 22 ). Impact refers to the 169 targets of the Sustainable Development Goals adopted by the United Nations in There are several reasons why a distinction between knowledge and impact makes sense for simulating CSO participation in the FPs. Most importantly, knowledge in the sense of scientific excellence is not a currency according to which CSOs act (as has been shown in the previous chapters); instead, their motivation stems from creating societal impact, i.e. solving societal problems. Moreover, excellence and impact are two separate evaluation categories for FP proposals; an analysis of evaluation scores in FP7 proposals has revealed that there exists no correlation between the scores received for the two criteria (r = 0.12). We can expect CSOs to mainly contribute to a consortium s impact score, whereas their contribution to excellence would be negligible. Skills also represent resources that an organisation can contribute, thus representing staff capacity. Consequently, in the model, each skill has a certain capacity, which can be reserved, resulting in a utilization of that skill. In other words, an organisation can only work on so many proposals and projects at the same time as given by its staff capacity. Big organisations as defined by their size can consequently participate in several proposals and projects at the same time, whereas small organisations are only able to participate in one or few activities. Skill vectors for topics and organisations were generated using two sources. The knowledge-vector was generated based on actual publication output of FP7 projects, as documented in the OpenAIRE database (see section above). The impact-vector was generated building on previous work of the contractor for DG Research & Innovation in the course of the FP7-4-SD.eu project 24, by linking FP7 topics with the 169 targets of the UN Sustainable Development Goals (SDGs). This linking was done based on the same cross-referencing methodology developed for, and applied in, the FP7-4-SD.eu project 25 (see also see Dimitrova et al., 2015). 22 Scopus, Content Coverage Guide, January 2016, available at: data/assets/pdf_file/0007/69451/scopus_content_coverage_guide. pdf, section 4.4, p.21, accessed 7 July See also see United Nations (2015), Transforming our world: the 2030 Agenda for Sustainable Development, available at: accessed 7 July Project Monitoring system for FP7 contribution to sustainable development, contract no. 30- CE / See 88

93 While the two data sources give a realistic picture in terms of knowledge and impact skills, it needs to be acknowledged that due to data gaps not all organisations actually participating in FP7 can be covered by this approach: knowledge-vectors can only be assigned to organisations participating in topics with publication output (this is the case for slightly more than three quarters of organisations); impact-vectors can only be assigned to organisations participating in topics with SDG-relevance (this is the case for about two-thirds of organisations). Organisations participating more frequently in FP7 are more likely to participate in a topic related to publication output and/or SDG relevance; this tends to be the case for bigger, more central actors, and, vice versa, organisations participating only once (or few times) are more likely to not be equipped with skill vectors that would allow them to participate in the simulation game. This tends to be the case for smaller, more peripheral actors, such as public bodies (PUC), CSOs and other organisations (OTH), which consequently do not participate in the simulation. Looking at total numbers, FP7 data contain about 23,000 unique organisations participating in the themes simulated in the model (see section above). In comparison, the dataset used for the simulation contains about 20,000 organisations having knowledge- and/or impact-vectors, i.e. about 3,000 mainly small and peripheral organisations less. However, as shown in section below, this has no major consequences for the simulation results. The skill vectors used in the present model are quite similar to what others call kene vectors (see e.g. Ahrweiler, Pyka and Gilbert, 2004), albeit with the addition of an explicit resource aspect (i.e. capacity, utilization). Another difference lies in the fact that we do not only model knowledge, but also impact, hence, the terminus skill vector to differentiate Call design The topics fed into the model in order to stimulate consortium formation are based on actual FP7 topics published between 2007 and These have been transformed into skill vectors for each topic (as described in section above), representing knowledge- and impact-requirements of this topic. To steer the model in view of simulating different scenarios, the steering parameters listed in Table 21 below can be used to operate the list of topics fed into the simulation. This can be done on the theme-level (for influencing the theme s budget, topic size, and number of projects per topic) and on the overall programme level (for evaluation weights). 89

94 Table 21: Parameters for scenarios Parameter Level Characteristics No. of topics Theme Multiplier, influencing the number of topics called for per theme (and thus also the budget allocated to the theme, as each topic already has its budget attached to it) Topic size Theme Multiplier, influencing the size of the topics in terms of budget (increasing topic size without correspondingly reducing the number of topics leads to an increase in the theme s budget; increasing topic size without adjusting the number of projects per topic leads to larger projects) No. of projects per topic Evaluation weights Theme Overall Multiplier, influencing the number of projects funded per topic (increasing the no. of projects without adjusting topic size will lead to smaller projects) Factor specifying the weights of the evaluation criteria excellence, impact and management; the weighting can be set separately for each criterion Participation Participation in a proposal is represented by an edge (i.e. a link) between an organisation and a consortium. In that, a participant is either a participant or coordinator, receiving a certain budget. Furthermore, organisations contribute a certain skill (knowledge and/or impact). All these contributions are aggregated in the consortium s knowledge and impact vectors, as is the requested budget. Consortia are linked to their respective topic by an edge of type evaluation. For the latter, the score of the proposal (as outcome of the evaluation) and the funding decision (accept or reject) of the evaluation phase is stored. A successful consortium has the linked evaluation status accepted and an additional attribute duration that turns the consortium into a project. After the project ends (duration), organisations of the project are linked by a relationship edge, which carries an attribute weight, reflecting the total number of joint project participations Simulation The simulation uses the social network graph as input and simulates its evolution over a specified time interval in discrete steps of one month. For the purpose of this study, the timing observed for FP7 was followed, resulting in topics being generated over a simulated period of 7.5 years (i.e. 90 months). The simulation stops after all projects have been terminated. The input of the simulation model consists of the basic graph, a list of topics to be generated, and evaluation weights (see section above). In case that no basic graph exists, the data of project participations from FP6 and FP7 is imported and the calibration is initiated. Thereafter, the base graph and the list of topics to be generated are loaded and the supplied evaluation weights are read. In every simulation step (i.e. month), consortia are formed for all new topics, establishing proposals. Each proposal is evaluated and the consortium is either disbanded or a project is funded. Finally, projects that have achieved their specified duration are terminated, knowledge is transferred ( learning ) and project partners are linked. 90

95 7.1.7 Output The output of the simulation comprises a set of key performance indicators, including: Number of consortia (created, eligible, rejected, shortlisted, funded) Numbers and shares of organisations (unique) and budget (EC contribution), per OrgType: HES, REC, PRC, PUB, OTH, CSOs (separately and grouped into CSO1-3 (i.e. excluding business-oriented CSOs) and CSO1-4 ) Network measures (see section above for an explanation): Per theme: lines, density, average degree, centrality, 3-rings Per organisation type (HES, REC, PRC, etc.): brokerage & closure Skill vectors per organisation (knowledge and impact), allowing aggregation per organisation type and for the whole network Budget per organisation (aggregated over all projects), allowing analysis of regional budget distribution Additionally, the complete graph at the end of a simulation run is exported, to allow for further, in-depth analyses on the level of the individual organisations. The results of the various simulation runs, based on above-mentioned performance indicators and on further in-depth analyses, are presented in section 7.3 below Calibration and validation of the model An important step in developing the agent-based simulation model was its calibration, in view of producing realistic results. This comprised two main aspects: calibrating the proposal evaluation approach in the model based on the generated skill vectors, and calibrating the consortium formation process. Calibration of the model with respect to the proposal evaluation process was based on indepth analyses of the generated skill vectors for individual topics and organisations. Additionally, a simulation of the evaluation phase was performed using the software package R, by aggregating the generated skill vectors for actually formed FP7 proposals (for which data were available) and then calculating the proposal score and the respective decision of whether a proposal would be approved or rejected. A comparison with actual funding decisions for submitted FP7 proposals (as documented in CORDA) showed that the proposed approach based on the generated skill vectors and the evaluation method included in the model was able to reproduce 72% of the actual funding decisions in FP7. Calibration of the model with respect to consortium formation was based on an in-depth analysis of the structural characteristics of consortia at the social network level. This referred to the decision of coordinators to coordinate a project, the choice of partners (which and how many partners to involve), the choice of budget distribution among the partners, the project duration, etc. Based on these analyses, respective probabilities for each component were derived and implemented in the model. Validation of the model was based on comparing a baseline scenario, referring to a business-as-usual continuation of FP7 (with the same parameter settings), against actual FP7 statistics. Table 22 below shows an overall good fit of the simulation results with actual FP7 figures, in particular concerning projects, budget, and budget shares of 91

96 the different organisation types. Slight discrepancies are visible with regards to the (unique) numbers of organisations, which are related to the fact that the simulation overemphasizes bigger organisations (as explained in section above). These differences do not influence the reliability of the model as concerns the other indicators; however, they need to be kept in mind when interpreting the shares of organisation types (based on unique numbers) from the simulation runs in relation to actual FP7 figures. Table 22: Comparison of baseline scenario with FP7 data Parameter FP7 data* Simulation (baseline scenario) Projects 7,680 7,613 Budget 28.2 billion 28.1 billion Shares of OrgTypes Budget shares of OrgTypes HES: 9% REC: 14% PRC: 63% PUB: 8% CSOs: 6% OTH: 1% HES: 36% REC: 29% PRC: 30% PUB: 4% CSOs: 1% OTH: 0% HES: 11% REC: 17% PRC: 62% PUB: 6% CSOs: 3% OTH: 0% HES: 35% REC: 31% PRC: 29% PUB: 4% CSOs: 1% OTH: 0% Note: * FP7 data refers to the themes simulated in the agent-based model only (to allow comparison with the simulation results). Another validity check of the model involved sensitivity tests by simulating a number of preliminary test scenarios featuring different parameter settings regarding project size, evaluation weights and budget distribution between themes. For the latter, both more moderate and more radical budget shifts were assumed. About 20 preliminary test scenarios were simulated, and their results were compared with the business as usual (baseline) scenario. An analysis of these 20 simulation test runs is included in the Annex to this report. These simulation runs did not only lead to refinements of the model, but also helped in developing four main scenarios (described below) that were simulated in view of deriving policy recommendations for the study. 92

97 7.2 Simulated scenarios Four main scenarios were developed referring to four different strategic orientations of the FP. They reflect significant but realistic changes in the FP s structure and budget and feature distinct combinations of the model s steering parameters in terms of budget distribution to themes, project size and evaluation weights. The four main scenarios are: M.1: Competitiveness and high-tech, strongly promoting the H2020 pillar of industrial leadership and the more technological themes of the societal challenges pillar; M.2: Social cohesion and well-being, strongly promoting the socioenvironmental parts of the societal challenges pillar and the cohesion aspects of H2020 by enhancing the parts Science with and for Society and Regions of Knowledge; M.3: Fortress Europe, assuming an encapsulation of Europe from other parts of the world and thus focusing on security-related aspects, such as the themes Health, Energy and Security; and M.4: FP rollback, simulating a dramatic cut of the FP budget for collaborative research by about two thirds in favour of other spending (such as significantly increasing the European Research Council s (ERC) budget) or due to an overall reduction in the EU budget. Figure 15 below shows the budget distribution across themes for the four main scenarios. The FP7 baseline scenario is included to allow comparison of the simulated budget shifts in relation to FP7. In addition to the budget shifts, the scenarios also apply different settings regarding the project size (larger projects for Competitiveness and high-tech and Fortress Europe vs. smaller projects for Social cohesion and well-being ) and evaluation weights (stronger weight of criterion excellence for Competitiveness and high-tech vs. stronger weight of criterion impact for Social cohesion and well-being ). The exception is FP rollback, where project size and evaluation weights were not changed (see bottom part of Figure 15 below). Additionally, both Competitiveness and high-tech and Social cohesion and well-being feature an increase in their total budget by about 20%. In this sense they can be seen as reflecting opposite strategic orientations of the FP in the one case putting the focus on technological advancement with the aim of increasing the competitiveness of European industry, and in the other case strongly promoting processes through which European (civil) society would have a much stronger role in the FPs, thus contributing to social cohesion aspects. 93

98 Figure 15: Budget distribution (EUR billion) and evaluation weights for main scenarios evaluation weights evaluation weights evaluation weights evaluation weights evaluation weights - excellence 1 - excellence 3 - excellence 1 - excellence 2 - excellence 1 - impact 1 - impact 1 - impact 3 - impact 2 - impact 1 - management 1 - management 2 - management 2 - management 1 - management 1 project size project size project size project size project size - same as FP7 - larger projects - smaller projects - larger projects - same as FP7 94

99 7.3 Analysis of scenarios The simulation runs of the scenarios described above were analysed in terms of the key performance indicators listed in section above Overview across different scenarios The figures below provide an overview of the results from simulating the four main scenarios in terms of budget shares of the different organisation types (Figure 16), the total budget received by those organisations (Figure 17), the shares of organisation types based on unique numbers (Figure 18), as well as the actual (unique) numbers of organisations per type (Figure 19). In terms of budget shares of the different organisation types, the scenarios M.1 Competitiveness & high-tech and M.2 Social cohesion & well-being stand out, showing considerably different budget distributions than the other scenarios. Scenario M.2 shows the highest budget share for CSOs, with 1.3%. Scenario M.1 also shows a higher budget share for CSOs, with 1.2%. However, the two scenarios differ in terms of budget shares for business-oriented CSOs (CSO4) and the society-oriented CSOs (CSO1-3), with M.1 showing a higher share for the business-oriented CSO4 group. Figure 16: Budget shares of different organisation types across the main scenarios The budget figures shown in Figure 17 reflect the total budgets set for the scenarios (see Figure 15) and the budget distribution by organisation type (see Figure 16). 95

100 Figure 17: Budget to the different organisation types across the main scenarios (EUR billion) In terms of shares of the different organisation types based on counting unique organisations (see Figure 18), scenario M.4 FP rollback shows a higher share of universities (HES) than all other scenarios; it also shows the smallest share of companies (PRC) and the highest share of private research organisations (REC). Scenario M.2 Social cohesion & well-being shows a considerably higher share of CSOs (almost 5%) than the other scenarios, being more than 50% above the share of CSOs in the baseline scenario. Finding 23: Scenario Social cohesion & well-being shows the strongest effects on promoting CSO participation in the FPs. Figure 18: Shares of different organisation types (unique numbers) across the main scenarios In terms of numbers of organisations participating in projects, Figure 19 shows the expected results based on the settings applied in the scenarios (see Figure 15 above), 96

101 i.e. in total higher numbers of organisations in scenarios M.1 and M.2, and lower numbers in scenario M.4. The differences between M.1 Competitiveness & high-tech and M.2 Social cohesion & well-being are due to differences in project size; while in M.1 larger projects were funded, M.2 funded considerably smaller consortia and therefore also shows a much higher number of projects (almost 20,000 projects this is almost three times as much projects as in the baseline scenario). Scenario M.2 also shows the by far highest numbers of CSOs; in particular, numbers of society-oriented CSOs (CSO1-3) are twice as high as in the baseline scenario. Figure 19: Numbers (unique) of different organisation types across the main scenarios Comparing Figure 19 and Figure 17, it is interesting to note that while in scenario M.2 Social cohesion & well-being there are only about 20% more universities (HES) than in the baseline scenario, their budget increases by almost two thirds. Moreover, while society-oriented CSOs (CSO1-3) double their numbers in scenario M.2 compared with the baseline scenario, the increase in their budget is even stronger (+125%). As concerns scenario M.1 Competitiveness & high-tech, a similar effect can be observed for companies (PRC) and the industry-oriented CSOs (CSO4): while their numbers only increase by about 10%, their budget grows by more than 50% Summary of findings per scenario In the following the key findings from the four main scenarios (in relation to the baseline scenario) are summarised based on the figures shown above and based on further simulation results such as numbers of proposals and projects. Scenario M.1 Competitiveness & high-tech M.1 can be interpreted as the industry-oriented scenario, with companies (PRC) having the highest budget share (34%; slightly more than universities (HES) with 33%). Despite the much higher overall budget, the number of funded projects is about 10% lower than in the baseline scenario, which can be explained by larger project sizes. The number of submitted proposals does not differ significantly compared with the baseline scenario. 97

102 The slight increase in the number of organisations by about 7% is mainly due to more companies (PRC) participating. In relative terms, also industry-oriented CSOs (CSO4) and other organisations (OTH) show increases of more than 10%. Scenario M.2 Social cohesion & well-being M.2 can be interpreted as the scenario promoting universities (HES) and society-oriented CSOs, in particular CSO1 and CSO2. It is also characterised by a dramatically higher number of projects (almost three times more) and proposals (almost two times more) than in the baseline scenario. It consequently also shows a much higher number of organisations (+20%), with the highest increases (in relative terms) for CSO1 and CSO2 of more than +100%. Looking at all CSOs (CSO1-4), scenario M.2 shows the by far highest shares of CSOs in terms of numbers (almost 5%) and budget (1.3%). Scenario M.2 is also showing the strongest effects regarding concentration effects and capacity building of the participating organisations (see analyses on concentration effects and learning further below). Scenario M.3 Fortress Europe Despite the considerable budget shifts in favour of the themes HEALTH, ENERGY and SECURITY (see Figure 15), scenario M.3 shows the least differences to the baseline scenario across all simulated scenarios in terms of budget and organisations. It features about 15% fewer projects (and a similarly lower amount of proposals), mainly due to larger project sizes as compared with the baseline scenario. Although the total number of organisations is virtually the same, public bodies (PUB) and citizen-oriented CSOs (CSO1) show increases of about 7%, which are likely related to the strong promotion of the themes HEALTH and SECURITY. At the same time, however, scenario M.3 shows the smallest budget share for CSOs, with CSO1-4 having a budget share of less than 0.9%. Scenario M.4 FP rollback The budget cut applied for scenario M.4 (see Figure 15) by about two thirds results in a correspondingly lower number of projects and proposals. This scenario consequently also features a much lower number of participating organisations (-37%), mainly due to significantly fewer companies (PRC), but also CSOs (CSO1-4) and other organisations (OTH), all of which show reductions of more than 40% compared with the baseline scenario. This can be explained by small organisations (SMEs, but also CSOs and OTH) dropping out of the programme, while only the larger ones remain as project participants (see the analysis on concentration effects below). In this sense, universities (HES) are least affected by the budget cut simulated in scenario M.4, with their numbers only falling by about 20% (and their share of organisation types consequently increasing to 15%, as shown in Figure 18 above). Under the assumption that the budget cut simulated in scenario M.4 is due to a correspondingly major increase of the budget for the European Research Council (ERC) a programme clearly dominated by universities this would lead to a strong (double) promotion of universities (HES) in the whole Framework Programme over other organisation types. 98

103 7.3.3 Effects of the main scenarios on network parameters The four main scenarios also differ in terms of their effects on key network parameters (which stem from the Social Network Analysis; see chapter above). Scenario M.1 Competitiveness & high-tech shows a much lower network density, as well as lower avg. degree, centralisation and clustering compared with the baseline scenario. These effects are due to the moderately larger number of organisations combined with a considerably smaller number of (larger) projects, resulting in less collaboration opportunities and consequently a lower average number of cooperation partners, as well as less triadic relationships. Scenario M.2 Social cohesion & well-being also shows a much lower network density and lower centralisation, due to the much larger number of organisations participating in this scenario. In contrast to M.1, however, M.2 features a higher avg. degree and higher clustering, indicating stronger collaboration relationships (more cooperation partners per organization, more triadic linkages) between participating organisations (in particular in the theme Regions of Knowledge ). This can be explained by the significantly higher number of consortia, with almost three times as many projects as in the baseline scenario. Scenario M.3 Fortress Europe shows a similar pattern as M.1, with all network parameters being slightly lower than in the baseline scenario. Similar to M.1, this is a result of fewer (but larger) projects being funded in this scenario, while the total number of participating organisations does not change. Scenario M.4 FP rollback shows the effects as expected from its settings: a much higher density and also higher centralisation. As shown in Figure 19 above and Figure 20 below, this scenario features a significantly lower number of participating organisations, with especially small, peripheral actors dropping out of the programme. With only large organisations and disproportionally more universities (HES) remaining in the network, the network is much more densely connected and more dominated by a smaller number of actors. Table 23 below gives an overview of the network effects of the four main scenarios. It shows that in particular scenario M.2 Social cohesion & well-being leads to beneficial network effects, with collaborations being more equally distributed among the participating organisations (less centralisation), but at the same time leading to more and stronger collaboration patterns (higher average degree, more 3-rings), which increase the overall stability of the network. Table 23: Effects of main scenarios on the overall network Competitivene ss & high-tech Social cohesion & well-being Fortress Europe FP rollback Density Avg. Degree Centralisation Rings Note: a single + or - means a change of between 5% and 20%; a ++ or -- means a change of more than 20%; empty cells mean no change or a change of less than 5%. 99

104 7.3.4 Differences between the main scenarios in terms of concentration effects Concentration effects can be analysed in terms of (a) shares of organisations by size (big, medium, small), (b) budget shares of the central players in the network (top 10 organisations), and (c) the regional distribution of budget, in particular between old and new Member States. Figure 20: Differences in shares of organisations by size (big, medium, small) across the main scenarios relative to the baseline scenario (percentage points difference) Figure 20 shows the differences in the shares of organisations (based on unique numbers) by size (big, medium, small) across the four main scenarios in relation to the baseline scenario. Organisation size is determined by the encounter probability, a network measure used in the social network analysis (see section above). Scenarios M.1, M.2 and M.3 all feature lower shares of big organisations compared with the baseline scenario, in favour of higher shares for medium-sized and small organisations. The differences are most distinct in scenario M.2 Social cohesion & wellbeing, with big organisations having a 6 percentage points lower share than in the baseline scenario, and in particular medium-sized organisations gaining shares. This indicates less concentration on the big players in this scenario in favour of more diversity of participating organisations. In contrast, scenario M.4 FP rollback shows strong concentration effects on the big organisations, whose share increases by more than 10 percentage points. As already indicated above, this is due to medium-sized and small organisations dropping out of the programme due to the dramatic budget reduction, making the big organisations even more dominant in the network. Looking at the very centre of the network, the top-10 organisations are very stable across all simulation runs. Five organisations rank among the top-10 across all simulation runs; the remaining five places in the top-10 are shared by a total of eight further organisations across the scenarios. This indicates that the composition of the very core of the network is only slightly affected by the different scenarios. The situation is however different when looking at the budget share of the top-10 organisations (see Figure 21 below). 100

105 Figure 21: Budget share of the top-10 organisations across the main scenarios The differences in terms of shares of big organisations are to some extent also reflected in the budget shares of the top-10 organisations in the network (see Figure 21). Similar to the real situation in FP7, the top 10 organisations (representing less than 0.1% of all participating organisations) receive almost 10% of the total budget in the baseline scenario. In line with the results above, scenarios M.1, M.2 and M.3 show a lower concentration of the budget on the top 10 organisations. Again, scenario M.2 Social cohesion & well-being shows the most distinct difference, with the top-10 organisations receiving only 7.7% of the total budget. In contrast, scenario M.4 FP rollback shows a slightly higher budget allocation to the top-10 organisations in the programme. Figure 22: Differences in regional budget distribution across the main scenarios relative to the baseline scenario (percentage points difference) Note: the big-5 countries refer to Germany, France, United Kingdom, Italy and Spain; the EU-12 + HR refer to the new Member States that have joined the EU since 2004, including Croatia (which has joined the EU only at the end of FP7). Figure 22 shows the differences in regional budget distribution across the four main scenarios relative to the baseline scenario. In relation to concentration effects, we look at the budget received by organisations from the big-5 countries (Germany, France, United Kingdom, Italy and Spain) and the budget received by organisations from the 101