D6 Final evaluation report

Size: px
Start display at page:

Download "D6 Final evaluation report"

Transcription

1 Impact assessment of the SME-specific measures of the Fifth and Sixth Framework Programmes for Research on their SME target groups outsourcing research D6 Final evaluation report Prepared for: European Commission Research Directorate-General Directorate T Implementation of Activities to outsource - SME IDEA Consult in collaboration with: - Ecorys Nederland BV - CSIL - Ecotec Research and Consulting Brussels, January 8 th, 2010 IDEA Consult nv Phone: (+32) Kunstlaan 1-2, box 16 Fax: (+32) B 1210 Brussels

2 Table of Contents p. 1 Executive summary Introduction Evaluation methodology and tools Participation statistics Reach of the target group Project involvement and implementation Project outputs and results Project s effects and impacts Success factors Collective versus Cooperative projects Monitoring and Review Process Relevance and value added Relevance of the SME specific measures Value added of the SME specific measures An outlook to FP Policy recommendations Strategic level Programme level Project / Implementation level Introduction The objectives of the study Content of this evaluation report The SME-specific measures Introduction CRAFT, Cooperative research and Research for SMEs Collective research and Research for SME Associations The uniqueness of SME-specific measures Main trends and changes in the SME-specific measures New instruments in EU Research Policy Increased emphasis on Collective research Modified participation criteria Adapting the financial support to encourage outsourcing of research Increasing importance of the business relevance Budget discretion Reviewing process of projects: towards more systematic follow-up Intellectual Property Rights: increased flexibility Increased support to SMEs to facilitate participation Evaluation methodology and tools Introduction Evaluation framework Hierarchy of objectives and effects Levels of analysis Evaluation tools Strengths and weaknesses of our approach Evaluation results Introduction Participation statistics Project and proposal statistics Unique and recurring participants Projects, proposals and success rate per country Reach of the target group Degree of R&D-formalisation R&D cooperation experience Project involvement and implementation Proposal preparation Project implementation

3 5.4.3 Prior collaboration Project additionality Project outputs and results Reach of project objectives Dissemination and training activities by IAGs Effectiveness of project implementation Intellectual property rights Project s effects and impacts Introduction Change in R&D-formalisation and capabilities (Dis)continued collaboration Application to FP Impact on R&D and economic performance Other effects and impacts of the project Benefit-cost ratio Success factors Collective versus Cooperative projects Monitoring and Reviewing Process Relevance and value added of the SME specific measures Relevance of the SME specific measures Value added of the SME specific measures An outlook to FP Policy recommendations Introduction Strategic level Programme level Project / Implementation level

4 1 EXECUTIVE SUMMARY 1.1 Introduction The rationale of the SME-specific measures is to involve in Cooperative or Collective research projects SMEs that do not have the capacity to carry out the required research on their own, either because they have little or no in-house R&D capacity or because their R&D capacity lies in another domain or sector. The underlying philosophy is to provide funding to SMEs and Industrial Associations or Groupings (IAGs) to allow them to outsource research to RTD organisations, which would in turn provide R&D services for the SMEs. The SME-specific measures differ significantly from the thematic programmes of the Framework Programmes in various perspectives: Their target group is different. While thematic programmes address mainly technology pioneers, SME-specific measures aim at reaching SMEs with little or no in-house R&D capacity. The nature of the R&D activities supported under SME-specific measures is different : the focus is on finding solutions to technical problems that SMEs identify (applied research). SME-specific measures concern more the innovation process of SMEs than the scientific excellence of the conducted research. The final objective is above all the socio-economic gains for SMEs. The concept of outsourcing research (or buying-in research services) is central. Also, more emphasis is put on networking. SME-specific measures respond to a pure bottom-up logic based on the needs of the SMEs in any domain. Ownership of project results in Cooperative Research projects is retained by the SME participants. In Collective Research projects, IPR are owned exclusively by the IAGs. The Cooperative Research Measures target SMEs with no or little in-house R&D capacity, but with innovation needs that can be tackled via a specific research project whereby research activities are outsourced to a knowledge provider (RTD performer). Cooperative research projects under FP6, on average, lasted a little over 2 years, included 9 participants and had a budget of 1,4 Million Euro. The Collective Research Measures aim at expanding the knowledge base and reinforcing the technological base of large communities of SMEs through enabling RTD performers to undertake scientific and technological research activities on behalf of Industrial Associations or Groupings. One of the major components of the Collective Research scheme is the dissemination of results to large communities of SMEs through Industrial Associations and Groupings, and the training of SME staff. Collective Research projects last longer, have bigger consortia and receive larger budgets than Cooperative Research projects: on average, under FP6, the duration of a Collective Research project was nearly 3 years, the consortia included 20 participants and the budget amounted to 2,7 Million Euro. There have been some important changes in research policy and in the way SME-specific measures were (are) designed and implemented under the 5th, 6th 4

5 and 7th Framework Programmes. In this context, we can distinguish several trends: The focus on funding for coordination of existing national / regional research programmes or so-called joint initiatives (introduced under FP6), besides the more classical grants given at project-level in a centralised manner. These recent initiatives represent a new way of organising support to SMEs. There is an increased emphasis on Collective research, which reflects a clear policy rationale: Collective research is considered to induce larger impacts than the Cooperative research scheme thanks to its assumed stronger multiplier effect, that is, through the dissemination of research results to all other SME members of an industrial grouping. The participation criteria have been modified over the course of the FPs with regard to the minimum number of RTD performers in consortia, the criteria for the country of origin of the participants and the assignment of the coordination tasks. The organisation of the financial support has been adapted to encourage the outsourcing of research by SMEs. The increasing importance given to the business relevance of a proposal during evaluations. The project review process after completion of projects has been organised more and more systematically. With regard to Intellectual Property Rights, the regimes have become more flexible, especially under FP7 in which the consortia are free to reach a tailormade agreement, as long as it is in the best interest of all partners. There has been an increased emphasis on the support (e.g. IPR helpdesk) to SMEs in order to facilitate their participation. The objectives of this evaluation of the SME specific measures were the following: to obtain evidence on the impact of the SME-specific measures of FP5 and FP6 on their SME target groups; to verify the relevance, effectiveness, efficiency, utility and added value of the SME-specific measures implemented in FP5 and FP6 as well as the relevance of those of FP7 to evaluate whether the expected impacts are realistic; to identify ways of improving and strengthening the SME-specific measures; to develop indicators for the monitoring of SME performance/activity and the impacts of SME-specific measures under the Framework Programmes for future assessment purposes. The measures for which the impact on SMEs were evaluated are the CRAFT programme under FP5 and the Cooperative and Collective Research programmes in FP6. 5

6 1.2 Evaluation methodology and tools The main objective of this project is to assess the impact of the SME-specific measures of the Fifth and Sixth Framework Programmes for Research on their SME target groups outsourcing research. In order to assess this impact, different and complementary methodologies were applied: Online surveys were set up that addressed to both participants (the experimental group) and rejected applicants (the control group). The experimental group was composed of participants that participated in the SME-specific measures either under FP5 or FP6 and for which we had an address available. The net sample contained 4814 potential respondents and we received 443 responses which implies a response rate of 9,2%. The control group included organisations (SMEs, SME Associations, RTD-performers, ) that were involved in a proposal submitted for the Cooperative and Collective Research programmes under FP6, whose proposal passed the evaluation threshold but was not selected for EC-funding because the available funding was exhausted. Out of the net sample of 3752 potential respondents, 248 responses were received, which implies a response rate of 6,6%. 45 case studies were conducted with 3 to 4 interviews for each case study. We established a group of 8 countries from which these cases were selected: Belgium, Finland, France, Hungary, Poland, Slovenia, Spain and the UK. Case studies provide case-based, empirical evidence on the impacts of participation on SMEs and the impact of SME participation on projects. Special attention was given to the question of how and why certain impacts materialized. In addition to the interviews in the context of the case studies, a series of interviews and a workshop with (other) stakeholders (such as innovation agencies, National Contact Points and SME representative organisations) took place. The additional interviews mainly served to complement the different case studies in the selected countries with relevant context information for these countries. The workshop aimed at policy reflection on the SME-specific measures from a more global point of view on the basis of the survey and case study results. The AMADEUS database was used with economic and financial data on millions of EU companies, to evaluate the impact of the SME specific measures on the participating SMEs. This was a pioneering exercise since this kind of data has not been used before in the context of the evaluation of the impact of participation in the FPs. Each of the above mentioned methodologies has its strengths and weaknesses. Overall, identifying the impact of R&D-support on the competitiveness of those supported, remains a very difficult exercise (among other things because of the so called attribution problem). However, the approach we followed by combining the evidence of the four evaluation tools described above provided an evidence base that is much stronger when compared to an approach based on one single methodology. 1.3 Participation statistics In terms of all proposals submitted, FP6 outnumbers FP5, but the success rate was lower under FP6 (12%) than under FP5 (24,5%). Under FP6, a high amount of proposals did not live up to standards, as only 36,5% of submitted proposals 6

7 ended up being above threshold. On the other hand, almost one-quarter of the proposals submitted under FP6 were evaluated above threshold but did not get EC-funding, leaving aside many valid projects (most of them under the Cooperative measure). Based on the survey results for the control group, 2 out of 3 projects have been abandoned completely. Another 30 % were continued but with significant changes to the project (mainly lower budget / scale and much less international partnership). When we look more closely at the distribution of the projects across the different countries, we find that the United Kingdom, Germany, Spain and Italy account for almost 60% of the total number of projects. France is relatively less involved and its share decreased between FP5 and FP6. The New Member States only account for a very small share of the total number of projects, but their share is increasing (from 0,9% in FP5 to 2,3% in FP6). We also studied the difference in success rates (i.e. the number of contracts or funded proposals divided by the total number of all proposals submitted) across the participating countries. Looking at the total of FP5 and FP6, the Netherlands have the highest success rate: they have a success rate of over 20%, closely followed by the United Kingdom and Denmark. New Member States recorded a progress in their success rate. Conversely, France, Belgium and Ireland noticed a drastic drop in their success rates between FP5 and FP6 (respectively from 26% to 8%, 22% to 9% and 23% to 4%) As regards the frequency of participation in the SME-specific measures, we observed that most participants in the SME-specific measures participated only once (90% of the participants). However, 15 (under FP5) / 8 (under FP6) RTD performers were involved in at least 10 projects, with the most actively involved RTD performer participating in 18,3% of all the projects under the SME-specific measures of FP Reach of the target group The SMEs participating in the SME specific measures have less formalised R&Dactivities compared to the SME participants in the other FP measures: 24% of the SME respondents participating in our survey could be characterized as having a high degree of R&D-formalisation 1 compared to 37% of the SME respondents participating in the other measures (mainly the thematic programmes). This difference is statistically significant. Participation in nationally or regionally funded R&D programmes seems to be a stepping stone to participation in the SME specific measures: 3 out of 4 SME respondents participated before in a nationally or regionally funded R&D- 1 A construct degree of R&D formalisation was defined, based on the responses on two survey questions addressed to companies, i.e. whether the company had a separate R&D department and whether there was a yearly budgeting of the R&D expenses. Three types were defined: low degree of R&D-formalisation: no yearly budget for R&D expenses and no separate R&D department medium degree of R&D-formalisation: a yearly budget for R&D expenses but no separate R&D department high degree of R&D-formalisation: a yearly budget for R&D expenses and a separate R&D department 7

8 programme. But 1 out of 2 had also already been involved in an R&D project funded within the EU Framework Programmes. Overall, there is a variety in the level of R&D capabilities among the SMEparticipants. The SME specific measures reach their target group of SMEs with little or no in-house R&D as well as SMEs with in-house R&D but not in the domain considered. However, the measures also attract SMEs with high in-house R&D capacity in the field considered that could very likely also participate under the thematic programmes (more detailed information can be found in section 5.3 of the Evaluation Report). 1.5 Project involvement and implementation The origin of the project ideas and the involvement of the different types of project partners in the preparation of the proposal were examined. Nearly 70% of all respondents indicated that the original idea came from the RTD-performers whereas about 44% mentioned SMEs are the initiators 2. The RTD-performers are also the most intensively involved in the preparation of the proposals, while the SMEs and IAGs, on average, only give limited input. As regards the involvement of SMEs in the project implementation, the impact of SMEs on the project mainly occurs through their involvement in the project meetings where they can keep the project on track by making sure that the project s results serve their technical as well as their business needs. Near the end of the project, the SMEs become more actively involved by performing tests and trials. There is a difference in project involvement between the SMEs in the Cooperative and Collective Measures: the SMEs in the Cooperative projects are more involved during the execution of the project compared to the SMEs in the core group 3 of the Collective projects. In order to obtain more insight into the extent to which the projects result in new partnerships, the existence of prior collaboration with the other partners involved in the projects were studied. The SME-specific measures lead to more new partnerships compared to the other instruments/programmes in FP6, in particular with foreign partners and with firms, less so with domestic universities or research institutes. 1.6 Project outputs and results Since SMEs are not always closely involved in the development of the proposals, we examined whether the project objectives matched the SME s own objectives. The answer is affirmative: overall, the project s objectives were to more than a reasonable extent in line with the SMEs own objectives. The most important project objective for SMEs, is the development of a new or improved product (for 70% of the SME-respondents). Surprisingly, the adaptation of a product, service or process to a new regulation, norm or standard is a much less important project 2 3 It was possible to tick multiple answers. Under the Collective Research Measures, a minimum of two SME participants had to be represented in the consortium to contribute to the project from the definition phase to the dissemination of results. These SME participants constitute the SME core group. 8

9 objective (for only 10% of SME-respondents) even for the SMEs participating in a Collective project. When looking more closely at the reach of the project objectives, around 60 % of the SME and IAG respondents felt that, overall, the project objectives were partially reached, a little more than 20 % indicated they were totally reached while just under 20 % indicated they were not reached at all. For the SMEs, there is a positive relationship between the intensity of project involvement and reaching the project objectives: SMEs that are more intensively involved in the project consider that the project objectives were better reached than those SMEs with a less intensive project involvement. RTD-performers are more positive about the reach of the project objectives compared to the SMEs because of their different initial objectives. RTD-performers are research oriented and happy if the project reaches its goals from a technical and research point of view, even if a project shows that a specific technology cannot be applied in a commercial context. SMEs however are typically more focused on commercial output when assessing the success of a project. Following from the above, when the project resulted in new knowledge but not in any commercial output, the RTD-performers are likely to be satisfied with the results, while the SMEs seek the fulfillment of their business objectives also. In 6 out of 40 cases for which we have the information available, the project already resulted in a commercial output. In 13 other cases there are expectations for commercial output in the future. Projects for which the original project idea was initiated by the SMEs have more chances of obtaining commercial outputs. The effectiveness of the projects implementation was also analyzed. According to the respondents, the effectiveness of the overall project implementation was good. The exploitation of results of the project on the market is rated the least effective factor. Nevertheless, 30,5% of the respondents indicated the project resulted in the creation of (or access to) IPR new to their organisation. The exploitation of the IPR is, however, not always evident: the ownership for IAGs and SMEs is not always ideal, the collective sharing of IPR may make exploitation afterwards more difficult and patent applications are expensive to be financed by SMEs. The Collective measure aims to reach a wider community of SMEs or whole industrial sector through the dissemination of project results. The type of dissemination and training activities organized by the IAGs were examined and the most effective activities in reaching SMEs were identified: seminars and workshops were the most popular dissemination activities while conferences and fairs reached more SMEs. 1.7 Project s effects and impacts The project outputs and results may lead to more sustainable effects and impacts of project participation on the whole organisation. In order to evaluate the impact of project participation, the situation before project participation was systematically compared to the situation at the time of the evaluation, and the results for the experimental and control group were related to each other. Change in R&D formalisation and capabilities 9

10 There has been an increase in the degree of R&D-formalisation among the participating SMEs, in particular among the SMEs under the Collective measure. One aspect of R&D-formalisation is the availability of a yearly budget for R&D: we find that the increase in the number of SME-participants who have a yearly R&D budget (following participation in a project funded under the SME-specific measures) differed significantly from the control group results. Following project participation, also the self-declared R&D-capabilities of the SME-participants increased and this increase is also significantly larger than for the SMEs in the control group. So overall, it seems that there is a positive impact on the R&D and innovation capabilities of the SME-participants which can be attributed to project participation. However, project participation has not, or only to a limited extent, led to more outsourcing of R&D by the SMEs. The examination of the effects of project participation on the IAGs R&D and innovation related activities brought to light that in particular the IAGs intensity of involvement in projects funded within the context of the EU R&D Framework Programmes has increased significantly when compared to IAGs in the control group. (Dis)Continued Collaboration The SMEs participating in the SME-specific measures tend to continue cooperation more with the domestic universities and research institutes (for nearly 90 % of the SME-respondents this was the case); international partnerships are much less likely to be continued in the future under the SME-specific measures (57% of the SME-respondents) compared to the partnerships that are set up under other FP6 funding measures (around 81 %). The lower the R&D capability of the SMEs, the more likely that they do not continue any form of partnership or collaboration with any of their project partners after the completion of the research project. Application to FP7 As regards the application of SMEs to FP7, participants to the SME-specific measures are more reluctant to apply to FP7 than the SME-participants in other FP6 measures. However, the SMEs application to FP7 is positively linked with the reach of the project objectives: SMEs in successful projects are much more likely to have applied for FP7 or consider applying in the future compared to SMEs participating in projects where the project objectives were not reached at all. Impact on R&D and economic performance A second series of evaluation results pertains to the impact of project participation on a number of R&D and economic performance variables of the SMEs involved. Using survey data, we find that the trend for half of the 16 R&D and economic performance variables that we examined, is significantly more positive for the SMEs in the experimental group than in the control group. The most significant differences are for the trends in R&D intensity, the number of transnational market oriented partnerships, the competence level of employees, national market shares and the overall competitiveness. Moreover, completed projects lead to a significantly more positive trend in a number of variables in comparison to ongoing projects (73% of the projects in the survey were completed, 27% of the projects were still ongoing) The above results suggest a possibly important economic impact of participation in the SME-specific measures, however, these positive changes are attributed only to a limited extent to project participation by the SMEs, making it difficult to draw strong conclusions (attribution problem). 10

11 Changes in the overall economic standing of the SME participants following their project participation were also analyzed: 35 % of the SMEs experienced a positive change in their overall economic standing following their project participation; 60% of the SMEs experienced no change. The overall change in the SMEs economic standing is more positive for SMEs who felt that their project s objectives were totally reached, i.e. there is a positive correlation between the change in economic standing of the SMEs and the reach of their project s objectives. Using the AMADEUS database, however, no evidence for an impact of project participation on the economic performance of the SME-participants can be detected, neither on the basis of a univariate analysis, nor by applying multivariate regressions. The cases show that in many projects new technologies have been developed or are at least close to development at the end of the project but this has not been translated yet into potential commercial products or process improvements. Therefore many projects would benefit from a follow-up project to translate the research/technological results into commercially viable products or process improvements. Other effects and impacts SME participants acknowledge that project participation results in increased scientific and technological knowledge and improved network abilities. These impacts are larger for the SMEs under the Cooperative measures compared to the SMEs in the core group under the Collective measures. The case studies underline these positive impacts but also indicate an improved market insight, increased visibility and project management skills as positive consequences of project participation. For the IAGs, there is also an important positive impact on their networking abilities as well as on their ability to disseminate technological information to the SME members of their organization. Benefit-Cost Ratio The benefit-cost ratios related to project participation are as follows: on average, project participants evaluate the benefits of project participation to be slightly higher than the costs. However, there are large differences by type of respondent: for the RTD-performers, the benefit-cost ratio is the highest, while for the SME-participants, it is the lowest. This can be explained by the fact that RTD performers get their research services to the SMEs reimbursed up to 100% whatever the outcome of the project. 1.8 Success factors Based on the case studies, we identified 8 factors that contribute positively to the success of a project: The commitment from partners is what makes a project. All project partners need to deliver what was agreed upon, even if this contribution to the project is overall quite limited as is the case for most SMEs under the measures considered. In order to obtain commitment from, in particular, the SME partners in a project, it is important that what will be developed is sufficiently near to the 11

12 market and serves the business needs of SMEs as well as their technological needs. A too high level of EC-funding allocated to outsourced R&D services to the RTD performers limits the involvement of the SMEs. Therefore a critical minimum amount of funding should be allocated to the non-research partners. Contributing to commitment also, regular and face-to-face meetings between the partners are to be conducted. R&D-projects are by definition risky business since the outcomes will not be clear from the beginning. Therefore, when a project develops and intermediate findings show that the intended results will not be achieved, it is important that the project is adopted in a flexible way and in line with the modified objectives. With regard to IPR, a good, detailed definition of IPR rules ex-ante (e.g. consortium agreement during negotiations) is important so that everybody plays a fair game : all IPR aspects should be carefully and exhaustively considered, discussed and defined. Another success factor is strong leadership by the coordinator : the role of the coordinator is very important due, in particular, to the international cooperation which complicates the projects. The lack of strong leadership and management by the coordinator is often quoted as the cause for the delay or low effectiveness in a number of cases. SMEs with the capacity to absorb the research results, RTD-performers that are (private) client oriented and an early focus on exploitation are factors that have a positive impact on the chances of obtaining a commercially viable product/service/process as project output. 1.9 Collective versus Cooperative projects Based on the results from the survey and from the case-studies, key relative characteristics or differences between the Collective and the Cooperative Measure were identified. Cooperative and Collective projects attract different, distinctive profiles of SMEs: the SMEs in the Cooperative projects are somewhat larger, have higher R&D-capabilities and are more experienced with R&D cooperation compared to the SME participants in the core-group of the Collective projects. Collective projects are considered by SME participants as not as close to the market when compared to the Cooperative projects. Linked to the larger scale of the Collective projects, participants also report higher coordination costs and a relatively lower level of involvement by SMEs in both the conception and development phase and the actual project implementation compared to Cooperative projects. However, one of the advantages of the Collective research scheme is that it fosters the pan-european inter-connectedness of national/ regional/local pockets of collaboration (even more than the Cooperative scheme). Another factor that differs somewhat between the Cooperative and Collective measures is the match between the project and the SME s own objectives. SMEs under the Collective measure are somewhat more satisfied with their project s own objectives than the SMEs under the Cooperative measures. This is not necessarily contradicting the observation that SMEs are relatively less involved in (both the conception and the execution of) the Collective projects, since operating 12

13 intermediaries (IAGs) under the Collective measure is expected to convey and defend - the needs of their SME members towards the RTD-performers. Collective projects mainly support SMEs with low R&D capabilities to reach a first level of formal organization of R&D activities, while more R&D-experienced SMEs participating under the Cooperative scheme appear to be more able to translate benefits of participation into further improving their R&D capacity and their economic performance (based on self-declaration) Monitoring and Review Process The case studies in particular generated insights in the way project participants consider the monitoring and review process of their project by the EC services. There are many complaints about the limited involvement of the EC and regular changes in scientific officers. Monitoring by the EC is generally found to be too limited to purely administrative or legal matters. The administrative follow-up was in general appreciated and good, but participants would appreciate a more active involvement from the scientific content point of view. The presence of the EC scientific officers at some of the project team meetings in the context of FP7 projects was very much appreciated by the project partners. The overall assessment by independent reviewers tends to be in line with our own assessment of the success of the projects. Review reports however fail to detect projects that appear to be, afterwards, a failure since they are to assess the implementation of the project based on the Description of Work (annex I to the grant agreement). No project is considered unsatisfactory in the review reports examined under this evaluation Relevance and value added Relevance of the SME specific measures The SME-specific measures are particularly relevant if they respond to the needs of the SMEs. The SME survey respondents indicated that financing research and innovation activities is their most important R&D and innovation need, followed by access to new scientific and technological knowledge and solving technical problems. The SME-specific measures address very well the two most important needs while Solving technical problems appears to be much less well addressed. Participating SMEs report that the Cooperative measure addresses their innovation needs to a larger extent than the Collective measure. This may be due to the fact that Cooperative projects are more specific and Collective projects are usually of a more generic nature. Case study findings reveal that the SME-specific measures are very relevant for two types of SMEs: SMEs without high in-house R&D capabilities that want or need to develop a project in an international context; SMEs with higher R&D capabilities that want to complement their R&D activities by buying in additional knowledge assets from an RTD-performer. 13

14 These SMEs all have a high level of business maturity, e.g. they have a good knowledge of their own strengths and weaknesses (not only from a scientific or technological point of view). This allows them moving up to a European project or seeking additional expertise Value added of the SME specific measures The SME-specific measures have a clear added valuebwhen compared to other relevant policy measures or programmes at European, national or regional level that serve similar objectives: complementarity : SME-specific measures occupy a unique niche within the whole supply of policy measures and therefore complement well other measures at regional, national and EU level. These measures have at EU level a clear value added because they address (1) a technological need which is common to a community of SMEs that is broader than a single country or region, or where the critical mass needed to address it is not present at the regional or national level only, or (2) a specific technological problem which cannot be solved through the available research capacities in the home country or region, but only by having recourse to research activities abroad. allows for access to international cooperation and the widening of the own local market and strategy of the companies beyond regional or national borders; purely bottom-up driven, addresses the technical needs of SMEs while the majority of other initiatives at European, national or regional level have thematic priorities. The common benefits of national programmes compared to the European SMEspecific measures are identified in: usually shorter time span of projects and thus quicker reaction to changes in the market; easier and shorter application procedures with less administrative requirements (but national or regional project induce per definition a lower degree of complexity in terms of proposal development and consortium management); higher success rate (but this may be biased by the lower competition at regional / national level) An outlook to FP7 Even though FP7 remains outside the scope of the evaluation, we noticed some positive and negative differences in the SME-specific measures between FP7 and FP6, as they are perceived by SME participants or members of other organisations. Regarding the FP7 funding rules, there is a large confusion among SMEs, RTDperformers, IAGs and even National Contact Points about the characteristics and advantages of the FP7 funding rules. There are mixed feelings about the administrative burden under FP7: paperwork is more streamlined but some problems still remain. For example, according to some RTD-performers interviewed, there are now inconsistencies in the budgeting and accounting 14

15 system. This assessment from some RTD-performers contrasts sharply with our observation that rules are now clearer for SMEs under FP7. There were relatively much less SMEs complaining about bureaucracy (compared to RTD-performers), but this might be biased by the fact that RTD-performers play on average a more active role in project initiation, proposal preparation and coordination. Finally, SMEs from the new Member States perceive FP7 (including the SME-specific measures) as too high-tech, which leads to the exclusion of lower-tech SMEs as revealed in various interviews from new Member States Policy recommendations The following policy recommendations are proposed, organized according to three levels: Strategic level (What is the rationale of the policy tool? Is it still pertinent to keep it in the future, and if yes under which form?); Programme level (Which type of programme is needed?); and Implementation or project level (How should such a programme be actually designed?) Strategic level 1. There is a need to keep an SME-dedicated R&D support measure. Beside the well-known market failure arguments, one of the underlying reasons behind this recommendation is that SMEs represent the very large majority of the industrial texture. Supporting and enhancing their innovative activities can therefore contribute significantly to the realization of the post-lisbon objectives. 2. An SME-dedicated measure for SMEs with limited R&D capabilities is appropriate at EU-level if it addresses (1) a technological need which is common to a community of SMEs that is broader than a single country or region, or where the critical mass needed to address it is not present at the regional or national level, or (2) a specific technological problem which cannot be solved through the available research capacities in the home country or region but only by having recourse to research services abroad. 3. Given the needs it has to fulfil and the options to organize it, this SMEdedicated measure is at the moment best organized as a Community initiative and not through existing national/regional initiatives nor offered via existing inter-governmental initiatives or ART 169-like measures. As far as Community funds are a valid option to organize the SME-dedicated measure at EU-level, different existing policy tools can be used such as the Framework Programme for Research and Technological Development (i.e. the current situation), the Structural Funds, or the Competitiveness and Innovation Framework Programme (CIP). The most realistic choice is the Framework Programme for Research and Technological Development, however better integrated with CIP to be better aligned with the innovation and business needs of SMEs. 15

16 4. Therefore, an in-depth debate is still needed on the respective role of the relevant Community initiatives in this respect and on the potential synergies between them, in particular to better exploit the complementarity between the Framework Programme for Research and Technological Development and the Competition and Innovation Framework Programme. 5. If the SME-dedicated measure is kept organized under the Framework Programmes, it would definitely need some adjustments at programme- and implementation-level (see below) Programme level 6. The bottom up character of the measure should be preserved as a programme characteristic as it serves SMEs technical needs better. 7. The SME-dedicated measure would benefit from a comprehensive approach in which research activities and business development get better integrated within the same project, including e.g. funding for follow up demonstration activities. 8. The budget allocations for the Research For SMEs scheme and the Research for SME Associations scheme should be decided independent from one another because they serve different target groups and have different effects. It seems that Collective projects mainly help SMEs with low R&D capabilities to reach a first level of formal organization of R&D activities, while more R&D-experienced SMEs participating under the Cooperative scheme appear to be more able to translate benefits of participation into further improving their R&D capacity and their economic performance. 9. It would be better to support more projects with smaller budgets since there are large oversubscription rates in the SME-specific measures, which result in a double loss: (1) the investment made in the proposal and (2) the potential output of the project which is abandoned in 2 out of the 3 cases Project / Implementation level The first recommendations under this operational level (recommendations nr 10 to 13) are related to the success factors that we described above. Their objective is to increase the chances to have projects that serve the needs of the SMEs that are committed partners and are actively involved within an outsourcing context. 10. Project proposals should first pass the SME needs test. For the Cooperative projects, we would suggest to introduce a two-stage selection and evaluation procedure similar to the Collective projects. In the first stage, applicants should develop the business case for the project. This limited proposal should be evaluated from a business perspective and therefore by business representatives mainly. Those proposals that pass this SME needs test, could then be further developed into a research proposal where the evaluation is mainly done by scientific and technical experts. 11. SME-participants and IAGs must sign a letter of commitment. 12. With regard to IPR configuration, there is a need to both keep flexibility and foster transparency within project consortia. It is recommended to reduce information asymmetries within the consortia with regard to the IPR set-up 16

17 through a close interaction between all projects members and the European Commission. 13. Funding rules should be reconsidered in such a way that RTD-performers run some risk as well. 14. Make sure SMEs can participate in the project meetings and let them validate the interim project results. 15. There is a need for more balanced requirements for international cooperation. This recommendation springs from the finding that distance, cultural and language barriers make it sometimes difficult to compose a partnership that satisfies the requirements in terms of international composition. 16. Dissemination and training activities in the Collective projects towards the SME-members of the IAGs need pro-active monitoring. The reasoning behind this recommendation is that these activities are necessary to realise the intended multiplier effect for the Collective projects while we found from the case studies that some SME members of the core group were against wider dissemination of the results to the other SME members in the sector. 17. In order to be able to evaluate the effects and impacts of the SME specific measures in the future, there is a need for additional information collection, at the right time to balance the need for detailed information with the requirement to keep the administrative burden low. 17

18 2 INTRODUCTION 2.1 The objectives of the study The overall strategic objective of the study was: to provide a well founded evidence-based social and economic impact assessment, both qualitatively and quantitatively, of the SME-specific measures of the Fifth (FP5) and Sixth Framework Programme (FP6) for Research, Technological development and Demonstration activities on the SMEs and SME Associations (both the SME core group and other SME members) outsourcing research. The measures for which the impact on SMEs were evaluated are the CRAFT projects under FP5 and the Cooperative and Collective Research programmes in FP6. Based on the outcomes of this assessment, it should be possible to link the identified economic and social impacts directly to programme participation and the added value of participation in EC Framework Programmes for SMEs and SME Associations specifically. The more operational objectives of this evaluation can be described as follows: To obtain evidence on the impact of the SME-specific measures of FP5 and FP6 on their SME target groups; To verify the relevance, effectiveness, efficiency, utility and added value of the SME-specific measures implemented in FP5 and FP6 as well as the relevance of those of FP7 to evaluate whether the expected impacts are realistic; To identify ways of improving and strengthening the SME-specific measures; To develop indicators for the monitoring of SME performance/activity and the impacts of SME-specific measures under the Framework Programmes for future assessment purposes. A description of the wider policy context of the study is included in the background report to this study. 2.2 Content of this evaluation report After the executive summary and the introduction, this report further consists of 4 main Chapters. In Chapter 3 we introduce the SME-specific measures that are within the scope of the evaluation. In Chapter 4 we summarise the evaluation methodology and the tools we applied. We introduce the evaluation framework, the hierarchy of objectives and effects that we (re)constructed as well as the levels of analysis we applied. We also describe briefly the evaluation tools we applied, and present the main strengths and weaknesses of our evaluation methodology. In Chapter 5 we discuss the evaluation results. Chapter 6 presents our policy recommendations. 18

19 Please note that there is a Background Report available which includes all detailed analyses and further information that supports the conclusions presented in this evaluation report. Throughout this Evaluation Report, the sections of the background report where further details can be found are indicated. 19

20 3 THE SME-SPECIFIC MEASURES Introduction SMEs are major contributors to economic and social development, new economic activities, job creation and competitiveness. There are, however, good reasons to believe that SMEs meet specific constraints when it comes to carry out research and innovation activities or using and exploiting their results. Among these specific constraints, one usually mentions the limited access to finance for SMEs to engage in research or innovation activities, the lack of critical mass and absorptive capacity to participate in cooperative networks, or the larger difficulty to appropriate the benefits from research and innovation. To help alleviate these constraints, innovation-driven SME-specific policies were developed over the past 15 years in Member States and at the EU level. SMEspecific measures in the Research Framework Programmes, for instance, were conceived in the nineties in order to facilitate SME participation in -and their benefit from- EC research programmes. These SME-specific measures were continued and reinforced after the launch of the Lisbon Strategy; their budget was almost doubled between FP6 and FP7 5. They address the technological needs of SMEs, helping them to innovate and to acquire new knowledge from RTD providers to improve their competitiveness. The rationale of these SME-specific measures is to involve in collective or cooperative research projects SMEs that do not have the capacity to carry out the required research on their own (either because they have little or no in-house R&D capacity at all or because their R&D capacity lies in another domain or sector). The underlying philosophy of the SME-specific measures is to provide funding to SMEs to allow them outsourcing research to RTD organisations, which would in turn provide R&D services for the SMEs. In this chapter we provide a summary of the key characteristics and evolution of the SME-specific measures that are within the scope of this evaluation, i.e. the CRAFT Measure under FP5 ( ), and the Cooperative Measure and the Collective measure under FP6 ( ), as well as their equivalent measures under FP7 6. We first provide the key characteristics of the CRAFT / Cooperative scheme and the Collective scheme, respectively. In the fourth section we summarize what makes these two measures a unique policy tool, as compared to other instruments of the Framework Programmes. In the final section, we present the main changes that these measures underwent over the past ten years from FP5 until FP7, as well as other relevant trends identified This chapter is based on chapters 2 and 3 (in particular) of the Background Report. About 66 mio a year under FP6 ( ) and about 127 mio a year (budget figures) under FP7 ( ). While this evaluation focuses on the SME-specific measures under the 5 th and the 6 th Framework Programmes, we have also analysed the key characteristics and evolution of the equivalent measures under the 7 th Framework Programme, as the analysis of measures under FP5 and FP6 may result into relevant adjustments of the current measures under FP7. 20

21 3.2 CRAFT, Cooperative research and Research for SMEs Under FP5, the CRAFT (Cooperative Research Action For Technology) action supported SMEs that could innovate but which had no research capacities of their own. It brought together SMEs from different countries with a specific research objective or need which would assign a large part of the work required to RTD performers (e.g. universities or research centres) with a European Community financial participation of up to 50% of total eligible costs. The ownership and IPR of the research remained exclusively with the SMEs. The CRAFT measure evolved under FP6 into the Cooperative Research measure, and into the Research for SMEs measure under FP7. These measures under FP6 and FP7 shared the general characteristics of the CRAFT projects under FP5. They have in common that they target SMEs with no or little in-house R&D capacity, but with innovation needs that can be tackled via a specific research project whereby research activities are outsourced to ( bought from) a knowledge provider (RTD performer). These measures finance research projects where minimum 3 SMEs from different countries have set up a consortium with one or more RTD performers, with a budget of maximum 2.0 mio (1.5 mio under FP7) and a maximum duration of 2 years. The choice of the project is left to the SMEs and the associated RTD performers: the measures under FP5, FP6 and FP7 remain fully bottom-up driven, in order to reflect as much as possible the specific needs and problems of the SMEs interested. However, some adjustments were made to the measures in the course of FP6 and FP7. Table 1 below compares the CRAFT scheme under FP5 with the Cooperative Research (under FP6) and the Research for SMEs (under FP7) on a number of criteria. Because most of these adjustments implied the Collective measure as well, we will present and discuss them for both measures, under the final section of this chapter. 21

22 Table 1: CRAFT (FP5) vs. Cooperative Research (FP6) vs. Research for SMEs (FP7): a comparison CRAFT FP5 Participants - SME proposers: min 3 from 2 different countries - RTD performers: min 1 - Other enterprises may not participate Cooperative Research FP6 - SME proposers: min 3 from 2 different countries - RTD performers: min 2 from 2 different countries - Other enterprises may participate Research for the benefit of SMEs FP7 - SME proposers: min 3 from 3 different countries - RTD performers: min 2 from any country - Other enterprises may participate Duration 1-2 years 1-2 years 1-2 years Cost M M 2.0 M M 2.0 M M 1.5 Thematic areas EC-Budget spent Financial support Application Other conditions Any thematic priority as long as it is in line with the general objectives of at least one thematic programme under FP5 7 Any research topic across the whole field of science and technology 8 Any research topic across the entire field of science and technology mio mio Maximum 891 mio (budget figures) - 50% of total eligible cost (RTDs are 100% funded) Two possibilities: - Via Exploratory Award - Direct submission of final proposal - RTD performers are subcontractors, cannot be coordinators - 5 main evaluation criteria Source: IDEA Consult, based on CORDIS - 50% for research and innovation (RTDs are 100% funded) - 100% for management (up to 7% of EC contribution) - Single-stage submission of final proposal - SMEs and RTD performers can be coordinators - 6 main evaluation criteria - 110% rule! - 75% for research and technological development - 50% for demonstration activities - 100% for management (no ceiling anymore) Single-stage submission of final proposal - SMEs, RTD performers and other consortium partners can be coordinators - 3 main evaluation criteria Call description CRAFT 2000 (January edition) Work Programme 2002 (January, July and August editions) Work Programme 2007 (February edition) and 2008 (January edition) 22

23 3.3 Collective research and Research for SME Associations The Collective Research scheme evolved from the Collective Research pilot action used in FP5. The pilot action on Collective Research was introduced with the purpose of expanding the knowledge base of large communities of SMEs through the direct involvement of industrial associations and groupings representing their SME members. The projects enabled associations to define research topics in the interest of their SME members and to outsource the required research to RTD performers, expanding their knowledge base and, thus, improving their general standard of competitiveness. There were 5 pilot projects funded with 40 partners involved under FP5. The projects were of substantial size and of medium-term duration. The pilot action was funded under the Innovation SMEs Horizontal Programme of FP5. Table 2 compares the cooperative versus collective research actions on a number of criteria. Table 2: Cooperative vs. Collective Research (FP6): a comparison Cooperative Research Participants - SME proposers: min 3 from 2 different countries - RTD performers: min 2 from 2 different countries - Other enterprises may participate Duration 1-2 years 2-3 years EC-Budget 330 mio 146 mio spent Cost M M 2.0 M M 5.0 Thematic areas Financial support Application Other conditions Any research topic across the whole field of science and technology 10-50% for research and innovation (RTDs are 100% funded) - 100% for management (up to 7% of EC contribution) Single-stage submission of final proposal SMEs and RTD performers can be coordinators Source: IDEA Consult, based on CORDIS Collective Research - Proposers: min 2 National Associations from different countries or 1 European Industrial Association - RTD performers: min 2 from 2 different countries - SME Core Group: min 2 from 2 different countries Any subject across the whole field of science and technology 11-50% for research and innovation (RTDs are 100% funded) - 100% for management (up to 7% of EC contribution) - 100% for training Two-stage submission of outline proposal and consequently, final proposal SME Associations and RTD performers can be coordinators Cooperative research for SME s under the 6 th Framework Programme, Work Programme 2002 (January, July and August editions). Collective research for SME s under the 6 th Framework Programme, Work Programme 2002 (January, July and August editions). 23

24 Collective Research aimed at expanding the knowledge base and reinforcing the technological base of large communities of SMEs through enabling RTD performers to undertake scientific and technological research activities on behalf of Industrial Associations or Groupings. One of the major components of the Collective Research scheme (and one of its distinctive characteristic as compared to the cooperative scheme) was the dissemination of results to large communities of SMEs through Industrial Associations and Groupings, and the training of SME staff. The projects were larger and ran longer (two to three years) than Cooperative Research projects. The total EC-budget for this programme was EUR 150 million. The FP6 Collective measure evolved under FP7 into the Research for SME Associations. Compared to FP6, relatively more budget is earmarked for the Research for SME Associations scheme under FP7 than for the Collective measure. In acknowledgement of the role played by SME Associations with respect to the dissemination of project results, a larger number of SMEs can be reached through this scheme, and therefore it is considered that Collective research schemes were having a higher multiplicator-effect than Cooperative research schemes. Hence, a greater emphasis is put on the support to SME Associations. This resulted in a share of at least 1/3 rd of the total budget (EUR billion) available for this programme under FP7 compared to maximum 1/3 rd under FP6. Besides, the current measure kept the general characteristics than its predecessor. The main changes between FP6 and FP7 are discussed below under the last section of this chapter. 3.4 The uniqueness of SME-specific measures Considering this detailed description and analysis of the SME-specific measures, and in spite of the fact that these measures evolved to some extent between FP5, FP6 and FP7 (the main trends in this evolution are delineated under the subsequent section), SME-specific measures in general differ significantly from the thematic programmes of the Framework Programmes from different viewpoints. First of all, their target group is different. While thematic programmes address mainly technology pioneers (i.e. high-tech, research-intensive companies), SMEspecific measures aim at reaching SMEs with little or no in-house R&D capacity (but with specific R&D needs). These SMEs often haven an innovative idea but lack the research capacity to bring the concept to fruition. SME-specific measures also address a larger segment of enterprises: low- and medium-tech SMEs from any industrial sector. It follows that the nature of the R&D activities supported under SME-specific measures will be different as well: less frontier technology development than under the thematic programmes, but more applied, problem-solving technical research, and more importance given to the dissemination of best available technologies. Consequently, SME-specific measures concern more the innovation process of SMEs than the scientific excellence of the conducted research and the scientific breakthroughs achieved. These measures are developed in relation to the rationale of the market, thus following a different logic and approach than the trans-national collaborative research partnerships under other schemes in the Framework Programmes. Thematic programmes of course also consider the innovative impacts of the research conducted, but the 24

25 priority balance is clearly different. For SME-specific measures, the final objective is above all the socio-economic gains for SMEs. Third, the concept of outsourcing research (or buying-in research services) is central. While under the thematic programmes the companies (including SMEs) in general carry out themselves (a part of) the research activity 12, the SMEs targeted by SME-specific measures rely much more on outsourcing the research activities to RTD performers in order to satisfy their innovation needs since they have no or little in-house R&D capacity. In this context, SME-specific measures give SMEs money to buy-in the research services needed. Also, much more emphasis is put on networking : i.e. to support SMEs to get embedded in a (cross-border) network where they can find more easily answers to their innovation needs (Cooperative scheme) or to expand the knowledge base of large groups of SMEs via the involvement of e.g. industrial associations (Collective scheme). By supporting the development of (more or less sustainable) crossborder networks in which SMEs (customers) and RTD performers (suppliers) codevelop innovative solutions, SMEs can absorb new and external knowledge for faster innovation. In that context, the choice of partners or RTD performers is also left open: the key point is that the SME gets the best value for money, in its business interest. Finally, SME-specific measures respond to a pure bottom-up logic based on the needs of the SMEs: there is no thematic nor technological focus put by the EC. Under these schemes, SMEs have the freedom to choose the focus of the conducted research. While under the thematic programmes proposals have to fit into a specific technological focus, proposals for support under SME-specific measures can be in every industrial sector or technological domain, without any restriction. 3.5 Main trends and changes in the SME-specific measures In the previous sections of this chapter, we have already touched upon some important changes in the way SME-specific measures were (are) designed and implemented under the 5 th, 6 th and 7 th Framework Programmes. In the subsequent paragraphs, we further detail and comment on the most important trends detected. For more detailed information on the evolution of the SMEspecific Measures under the 5 th, 6 th and 7 th Framework Programmes, we refer to chapter 3 of the Background Report New instruments in EU Research Policy The SME-specific measures of the EC Framework Programmes do not evolve in isolation. On the contrary, changes in their objectives, design or actual implementation are strongly influenced by the broader evolution of research policies (SME-policies) and their respective instruments. European Research policy has undergone many important changes over the past ten years. One of them is the increased focus by the European Commission on funding for coordination of existing national / regional research programmes, besides the more classical grants given at project-level in a centralised manner (as it is the case for the SME-specific measures which form the object of this study). 12 Provided the logical division of labour between partners: more upstream research carried out by e.g. a university research lab and more downstream-research (more market-oriented) left to the company. 25

26 Good examples of these new instruments are the ERA-Net Scheme or the Article 169. Article 169 enables the Community to participate in research programmes jointly undertaken by several Member States and can therefore contribute to integrating national programmes at European level. The ERA-Net scheme was launched in 2002 as part of FP6. It aims at stimulating the cooperation and coordination between national (regional) research programmes, including their mutual opening and the development of joint calls. It typically targets research programmes owners or managers (ministries, government agencies or research councils) and invites them to submit proposals in self-nominated topic areas (bottom-up principle). The ERA-Net scheme is one of the flagship instruments of the European Commission for the further development of an integrated European Research Area (ERA). Two ERA-Net projects were recently launched as extensions of the SME-specific measures as implemented at EU level by the EC, ie. CORNET and ERA-SME. CORNet (ERA-Net Collective Research) was launched in It intends to network and coordinate national and regional programmes on Collective research for SMEs from 15 countries/regions. ERA-SME intends to coordinate and network national and regional programmes promoting research co-operation between SMEs and research organisations from 16 countries (with the aim of expanding the partnership to other countries). Beyond these two ERA-Nets, one should also mention the recent joint EU-Eureka Eurostars initiative, launched in October 2007 and based on article 169 of the EC Treaty. Eurostars aims at providing financial support, from both the EU and the participating countries' funds, to research-performing SMEs. Through this initiative the EC is providing Community funding to support the integration of national schemes into one joint programme (the EC funding amounts up to 100 million euros on top of 300 millions euros provided by the 32 participating Countries). Although these new instruments are beyond the scope of this study, it is in our opinion crucial to consider this evolution in the EU research policy landscape as important background element when analysing the relevance and impact of the FP SME-specific measures. This trend towards networked or joint and mutually open calls has been taken into account when we further examine the relevance of SME-specific measures organised under the Framework Programmes Increased emphasis on Collective research The CRAFT / cooperative scheme and the Collective research scheme have been created with different rationale and objectives. The Cooperative research scheme brings together SMEs from various EU countries with a specific research objective or need which would assign a large part of the work to RTD performers. The Collective research scheme aims at expanding the knowledge base of large communities of SMEs or sectors through the involvement of industry associations or groupings. Associations define research topics in the interest of their SME members and outsource the research to RTD performers. Collective research projects are typically much larger projects (ca. 20 partners per project, against 10 partners per project in the case of cooperative projects) conducting another type of research (more often normative or pre-normative research, research on new standards) and usually with more focus on dissemination activities. In the course of FP5, FP6 and FP7, the Collective research Scheme has gained importance in comparison with the Cooperative scheme, as attested by the shift within the total budget for the SME-specific measures in favour of the Collective 26

27 scheme. Starting from a pilot action with minor budget under FP5, Collective Research represented 31% of the total budget spent on SME-specific measures under FP6 (i.e. 146 mio out of 476 mio spent). Under FP7, it is assigned at least 1/3 of the total budget reserved for SME-specific measures (i.e. at least 445 mio out of 1,336 bn ) 13. This increased importance reflects a clear policy rationale: the EC considers that Collective research induces larger impacts than the cooperative research scheme (thanks to its assumed- stronger multiplicatoreffect) Modified participation criteria As the SME-Specific measures of the Framework Programmes typically target SMEs with a need to outsource research, their participation in joint research projects and their cooperation with RTD performers is a key element of the design of these measures. Throughout the three Framework Programmes, FP5, FP6 and FP7, the emphasis on SMEs in CRAFT/Cooperative Research projects is apparent: at least three independent SME participants should be part of consortia. The minimum number of RTD performers in consortia has changed slightly over the course of the FPs. Whereas FP5 required a minimum of one RTD performer, in FP6 and FP7 this number was increased to at least two RTD performers. The main reason underlying this change is that the EC wanted to avoid any situation where new knowledge generation would be too concentrated in one region or country, which would have hampered efficient dissemination throughout the consortium (or would have biased it towards some more closely located- SMEs). Modifying the number of RTD performers involved is an important parameter: often consortia consist of different pairs with one SME and one RTD performer from the same country. The criteria for the country of origin of the participants have evolved throughout the FPs in question. In FP5 and FP6, SME participants were required to be from at least two different countries, while FP7 entailed a more internationally mixed composition of consortia regarding the SME participants (at least three different countries involved). This reflects not only the need to have more countries involved in the context of the enlarged European Union, but also the concern to stimulate wider participation (network creation) and learning effects. A similar increasingly flexible approach over the course of the FPs is visible for the coordination tasks of research projects carried out under the SME-Specific measures. While FP5 strictly reserved the role of the coordinator for one of the SME participants in the consortium, FP6 allowed RTD performers to take up this role as well. In FP7 coordination can be assigned to any participant in the consortium (except subcontractors). The initial idea (under FP5) was that SMEs would take the lead of the project. However, in reality it became obvious under FP5 that the RTD performers were often the true leaders of the project. By introducing more flexibility in this regard, the EC wanted to better align its rules with reality. Instead, the EC tried to give a more leading role to the SMEs by adapting the financial rules (see infra) Adapting the financial support to encourage outsourcing of research Under FP6 and FP7, the financial rules were progressively adapted to further encourage the outsourcing of research activities. While under FP5 and FP6, RTD- 13 Figures on FP7 refer to committed budgets and not actual support spent, which is not yet available. 27

28 performers had to carry out at least 40% of the research activity (in budget, i.e. the eligible costs) 14, under FP7 RTD-performers have to carry out the bulk of the research activity, which is expected to be in the range of 70-80% (in budget). The main change in this regard was the introduction of the so-called 110%-rule under FP7. This rule limits the total support to the project to 110% maximum of the total amount of subcontracting to the RTD performer (for RTD activities), in order to stimulate a greater allocation of resources within the project to outsourced activities (for RTD activities). Moreover, since the price of these outsourced research activities has to be invoiced by the RTD performer to the SMEs, SMEs keep a tight control of the budget spending (they pay the invoices, and the EC considers these as SME cost ) Increasing importance of the business relevance The evaluation of project proposals is based on a number of evaluation criteria, which have evolved and changed throughout the FPs. Over time, it is observed that both the number of criteria and their emphasis have undergone changes. The main trend here consists of the increasing importance given to the potential impact of the proposal, which implies that the intention has been to promote research projects which are not only scientifically sound and innovative but also of business interest for the SMEs. Potential impact was introduced as new evaluation criterion under FP6. This trend is continued under FP7: the impact -criterion has received (much) more weight (it has become 1 out of 3 instead of 1 out of 6 criteria and compared to the other 2 evaluation criteria, the threshold value is higher: 4/5 instead of 3/5). Proposals must therefore include estimates of the expected benefits (increase in income, reach of other markets, comply with future legislation, ). The increased importance of the impact -criterion is also reflected by the fact that under FP7 relatively more evaluators come from the business community. This may be linked to the introduction of a remote evaluation phase before having a central consensus meeting in Brussels. Under FP6, the evaluations were mostly centrally organised in Brussels. But since industry people are normally less keen to come to Brussels for central evaluation, relatively more academic participants participated in these evaluations. By introducing the remote phase, the EC minimizes the time spent away from their business. This way, the European Commission tries to ensure that at least 50% of external evaluators come from the business/industrial community, which is a specific requirement under the SME specific measures Budget discretion From FP3 to FP5, the SME-specific measures were managed by the thematic programmes and funded from their core budget. This allowed the thematic programmes to adjust call budgets in line with demand. From FP6 onwards however, the SME-specific measures were centrally managed by the SME unit with a dedicated budget. This gave more visibility to the SME measures but less flexibility on the budgetary side. Due to the ever increasing interest for the SMEspecific measures, which has not been matched by the budget, the success rate between FP5 and FP6 has consequently decreased dramatically (see Section 5.2). 14 Under FP5, in terms of man-days, the RTD-performer(s) had to carry out at least 50% of the research activity. 28

29 3.5.7 Reviewing process of projects: towards more systematic follow-up Even though the actual implementation of projects is largely beyond the control of the European Commission, the way the follow-up process of projects is organised at the side of the European Commission may have an impact on their outcomes (monitoring, follow-up, reporting, evaluation). Between FP5, FP6 and FP7, the reviewing process did not change fundamentally: principles remained the same. The review produces two reports: 1) content-wise progress of the project and the planning of the next steps for the subsequent phases, and 2) management report. The review is based on the reports delivered by the research partners, which allows the Commission Services to evaluate whether the work that was announced (in the technical annex of the contract), has been carried out correctly, including the adequate allocation of resources. This review is an essential input for the Commission to justify the payments. The evaluation is thus only considering the good implementation of the workplan and the correct allocation of resources; there is no evaluation of impacts or effects whatsoever. From FP6 on, some incremental improvements were brought into the system, in two directions. On the one hand, the reviewing process took place more and more systematically, i.e. in a more homogeneous and transparant manner across all DG RTD services. Under FP5, there was no systematically implemented external feedback on the follow-up of projects, leaving the actual reviewing process to a large extent to the own judgement of the scientific officer. There were guidelines, but the extent at which these were actually and systematically implemented differed between the services. Approval of a series of reports on the basis of which invoices were paid was an interactive process between the project officer and the coordinator. It was often an iterative process with various revisions, but it depended much on the diligence and persistence of the project officer and was thus not standardized. A more systematic and coherent procedure, however, was implemented in the course of FP6. Guidance notes were issued in October 2004 to list and describe in addition to the specifications in the contract and its annexes, the reports and deliverables which had to be submitted to the EC services by the project coordinator on behalf of the whole consortium (guidance notes were directly addressed to the consortium coordinators) 15. These notes set the guidelines for the timing of delivery (usually within 45 days after the end of the reporting period), or the nature and provisional content of each type of report (e.g. interim report versus final report, management report, planning for next 18 months, plan for using and dissemination of knowledge, final report on the distribution of the Community s contribution, etc). The procedure foresaw also periodic reviews but only for part of the cooperative projects, and with the possibility (but not the obligation) of involving external reviewers. These reviews (that could occur during the lifetime of a project or after it had been finalised) principally assessed things such as the degree of fulfilment of a project, the necessity of the resources that a project had consumed, the management aspects of the project etc. If, as a result of the review, there was a reason to believe that the project had deviated from its declared aims or was in some other way unable to achieve its goals, the outcome of the review may have involved a re-focusing of the project on modified goals, or recommended the suspension of part or the entire project. 15 EC, Project reporting in FP6. Guidance notes for Integrated Projects, Networks of Excellence, Specific Targeted Research and Innovation Projects, Coordination Actions, Specific Support Actions, Cooperative Research Projects and Collective Research Projects, October 2004, Brussels. 29

30 On the other hand, DG RTD decided from FP6 on to have recourse in a systematic way to external reviewers. These reviewers are technical experts in the various technological fields covered by the projects funded. They evaluate the progress of the project from a technical point of view and in this quality assist the EC scientific officer. It is not mandatory to follow the judgement of the external reviewer: the scientific officer keeps the end responsibility of the review process. Once the external reviewer has given his/her assessment, the scientific officer integrates the insight into a final review reports, which is sent to the project s coordinator, wich is a specific characteristic of the SME-specific measures (for instance in the thematic programmes, the project coordinator receives only a letter that says that the reports have been approved and that payments are on their way). Under FP7, guidelines recommend to all services sending the review reports to the coordinators as well; in that context one can say that the SMEspecific measures under FP6 were avant-garde. The EC has to inform the project coordinator on beforehand on the identity of the external reviewer in order to avoid any conflict of interest Intellectual Property Rights: increased flexibility Ownership of project results in CRAFT and Cooperative Research projects is retained by the SME participants for the course of FP5 and FP6. Exploitation of project results as well as the commercialisation of them are expected from the SMEs after the completion of the project. Even though RTD performers can participate as coordinators in Cooperative Research projects of FP6, they do not have any ownership of the IPR resulting from the project. These belong to the SME participants exclusively. Nevertheless, RTD performers may benefit from early access to and preferential use of the project outcomes. With Collective Research projects in FP6, IPR are owned exclusively by the Industrial Association or Groupings. The RTD performers and the SME participants cannot derive property rights from the obtained project results. However, similar to Cooperative Research projects, they can take advantage of early access to and the preferential use of the project outcomes. A major change occurred in FP7, which provides much more flexibility with regard to IPR. Under FP7, consortia or associations may decide to follow the default regime as set out for FP5 and FP6. However, they are free to reach a different, tailor-made agreement, as long as it is in the best interest of all partners. Nonetheless, SMEs should be facilitated and provided with all the rights to use and to exploit project results as intended by them. If the SMEs or SME associations do not retain the IPR that results from the projects they are involved in, this should be reflected in a lower cost of the RTD-services. The rationale behind this increased flexibility is that the EC wants to better reflect business reality: RTD performers may want to further develop results, sometimes on demand of SMEs, or they are better capable of taking a patent (cfr TechTransfer services at universities). One should avoid a situation where research results are insufficiently commercially exploited because the official owners of IPR (ie SMEs) do not have enough capacity to register a patent Increased support to SMEs to facilitate participation Several supporting measures were provided for applicants in order to guide applicants and facilitate easier proposal submissions. Differences exist in these supporting measures for FP5, FP6 and FP7. Some of the measures are more common throughout Framework Programmes (like the European Information 30

31 Centres) and others are quite unique to a particular Framework Programme (like the ProTool Help Pages under FP5). Also, some were specifically designed to support SMEs (like the National Contact Points), others to support all types of enterprises (like the Cordis Partner Search Facility). An important evolution is the increasing use of digital tools such as the Electronic Proposal Submission Service (EPSS) to submit proposals. While under FP5, proposals could only be submitted in hard copy, most of them were submitted in electronic format under FP6 while under FP7, electronic submission is obligatory. 31

32 4 EVALUATION METHODOLOGY AND TOOLS 4.1 Introduction In this chapter we present the pillars of our evaluation methodology and the evaluation tools that we applied in the course of the study (see Figure 1). For the detailed description of the evaluation tools and methodologies that we used, we refer to Chapters 4 and 7 of the Background Report. Figure 1: Evaluation methodology and tools 4.2 Evaluation framework We distinguished the following evaluation topics (in line with the tender specifications): - Relevance: How well chosen are the objectives and interventions of the SMEspecific measures given the needs of the EU SME Community and industry? - Effectiveness: What outputs and results were achieved by the measures? How do they relate to the objectives of the programmes? - Efficiency: Which resources where used to achieve the results? Were they allocated optimally? - Utility: To what extent did the results correspond to the identified needs of the SME community? 32

33 - Sustainability: How sustainable is the impact of the SME-specific measures? - Impact and added value: What has been the overall impact of the SMEspecific measures and what s the added value of the SME-specific measures compared to other instruments or funds? These evaluation topics require a well-defined framework to analyse all aspects in a consistent manner. In Figure 2, we present the framework that we used for this evaluation. It refers to the typical intervention logic when policy makers want to address a specific problem, like in this case the existence of SMEs that could benefit from innovating but that lack the research capabilities of their own. Figure 2 General framework of the evaluation 4.3 Hierarchy of objectives and effects In order to facilitate the link between what we evaluate and the objectives of the SME specific measures, we identified the objectives of the SME specific measures and structured them into a hierarchy of objectives (for a definition, see Box 1). Box 1: Definition of hierarchy of objectives The hierarchy of objectives is a tool that helps to analyze and communicate the objectives of a programme or other policy intervention. It organizes these objectives into different levels (strategic objectives, sub-objectives, activity-related objectives and horizontal objectives) in the form of a hierarchy or a tree, thus showing the logical links between the various levels. 33

34 In ex-post assessments like this study, problem analysis and intervention logics are just as critical as ex-ante or mid term evaluations as they provide a way of drafting questions to test the 'fit' of the identifiable outputs against the way in which the programme was intended to work. The lack of a formal ex-ante or mid term evaluation of the FP5 programme as a whole or the SME-specific parts of it means that it has been necessary to construct a hierarchy of objectives. So it is important to stress that this exercise is kind of a reconstruction. As such the basis for our work was the information contained within the text of the programme decisions and the corresponding work programmes, but not only this. We have also consulted the ex-ante evaluation for the seventh framework programme and ex-post evaluations of fifth and sixth framework programmes which have attempted a similar reconstruction. These written sources were complemented by a series of exploratory individual interviews with EC officials as well as an interactive intervention logic seminar held on November the 19 th, 2008 with the EC. During this seminar, we discussed extensively the initial and specific rationale and objectives of both SME-specific measures under FP5 and FP6. This seminar formed the most valuable base for reconstructing the initial objectives of the measures. Box 2 defines the four levels of objectives. Figure 3 presents the reconstructed objectives at the various levels; the arrows show the relationships between them, or in other words to which higher-level objective each lower-level objective contributes. When reading this hierarchy of objectives, it is important to keep in mind that: We have been selective in structuring the strategic objectives in the sense that we have focused on objectives that are directly connected to the SME specific measures. The intermediate objectives also link closely to the SME specific programmes. While the specific objectives are common to both measures (they both aim at e.g. addressing the specific research needs of SME), at the lowest, operational level we have made a distinction between those that are in particular valid for the CRAFT/Cooperative research programme versus those that are in particular valid for the Collective research programmes. When linking the different levels of objectives, we have been selective by only indicating the links that seem to be most evident, although typically on the basis of implicit assumptions without hard evidence. Lower level objectives can contribute to more than one higher-level objective, but we have only indicated the presumably most relevant one in order not to complicate the picture. The hierarchy of objectives is relevant to the SME-specific programmes and not to the whole range of initiatives and programmes aiming at boosting innovation and R&D. This means that in particular the intermediate objectives and activity-related objectives listed are related to the activities that take place in the SME-specific programmes and as such are not comprehensive. However, other EU support programmes and their activities contribute to the same higher level strategic objectives. On the basis of this hierarchy of objectives we also constructed a hierarchy of effects (see Figure 4). The purpose of this exercise is to illustrate what measurable outputs and effects the programme is designed to have. These effects are derived from our attempt to operationalise the objectives. In Figure 4, each 34

35 effect has also been complemented by specific examples. These are of course not exhaustive; their aim was to better define and understand the expected effect in the light of this impact assessment. Box 2: Four levels of Objectives / Effects Global Strategic objectives represent the highest-level objectives and are defined very generally. These have, for example, a direct link with the Lisbon Strategy. SME-measures contribute only to a small extent to their realisation. Global Strategic objectives provide a good basis for assessing an intervention to longer term and more diffuse effects (or global impacts). Intermediate Strategic objectives refer to the medium-term objectives of the SMEspecific measures taken together contributing to the global strategic objective. Intermediate strategic objectives provide a basis for assessing an intervention in relation to its medium-term effects (or intermediate impacts) on both direct and indirect beneficiaries/recipients of assistance. Specific objectives indicate through which channels the intermediate strategic objectives can be reached. They have a (more) direct link with the core activities that take place under the SME-specific programmes. They provide a basis for assessing an intervention in relation to the medium-term results that occur at the level of direct beneficiaries/recipients of assistance. Operational (activity-related) objectives are the lowest-level, most specific, shortterm objectives and have a close link with the activities undertaken by the partners in the projects funded. They provide a basis for assessing an intervention in relation to its outputs. The latter can be defined as what is directly produced/supplied through the implementation process. 35

36 Figure 3: Hierarchy of objectives Global strategic objective (long-term, wider impact) OBJ 1: To improve the international competitiveness of European SMEs in the Knowledge-Based Economy Intermediate strategic objective (mid-term, impact on direct and indirect recipients) OBJ2: To increase the level of European SME/industry investment in (international) R&D activities OBJ3: To facilitate the internationalization of European SMEs OBJ4: To facilitate the access to S&T knowledge for SMEs with limited or no in-house R&D capacity and for SME Associations Specific objectives (mid-term, impact on direct recipients, results) OBJ 5: To address the technological innovation needs of participating SMEs and SME Associations OBJ 6: To promote the continued involvement of SMEs and SME associations in trans-national research OBJ 7: To promote the take-up of research results and technology transfer from RTD performers to SMEs Operational objectives (short-term, projectrelated, impact on direct recipients, outputs) OBJ 8: To assist SMEs in outsourcing research activities in any domain OBJ 9: To facilitate transnational co-operation in research and networking between participating SMEs, research performers / providers and larger companies OBJ 10: To promote the quality of research and benefits to participating SMEs (monitoring and review process, IPR access) OBJ 8: To enable large communities of SMEs to outsource, via SME Associations, research activities in any domain OBJ 9: To facilitate the involvement of participating SME Associations in trans-national research activities OBJ 10: To promote the quality of research and benefits to participating SME Associations / industrial groupings (monitoring and review process, IPR access) OBJ 11: To enhance the dissemi-nation of research results OBJ 12: To support the training of staff from SME members of SME Associations / Industrial groupings on the take-up of research results COOPERATIVE SME Specific Collective 36

37 Figure 4: Hierarchy of effects Global strategic impacts (long-term, wider effects) Effect 1: Improved international competitiveness of European SMEs in the Knowledge-Based Economy E.g.: World export market share of European SMEs increases Intermediate impacts (mid-term effects on direct and indirect recipients) Effect 2: European SMEs/industry increase their investment in (international) R&D activities E.g. SME-specific BERD intensity is growing in the EU Effect 3: Pace of internationalisation of European SMEs increases E.g. Share of exports in total turnover of SMEs increases Effect 4: SMEs with limited or no in-house R&D capacity and SME Associations get an easier access to S&T knowledge E.g. European SMEs work more closely and more frequently with RTD performers Results (mid-term effects on direct recipients) Effect 5: The technological innovation needs of participating SMEs and SME Associations are addressed E.g. Participating SMEs were able to substantially improve a specific production process Effect 6: Involvement of participating SMEs and SME associations in trans-national research continues (beyond project duration) E.g. Participating SMEs launched other research projects with partners abroad after the project had come to an end Effect 7: Effective technology transfer and absorption of research results by participating SMEs E.g. Research results were successfully implemented in the company, with a sustainable impact on the production process Outputs (short-term, projectrelated, effects on direct recipients) Effect 8: Effective outsourcing of research activities by participating SMEs E.g. Participating SMEs outsource or buy in more research services during the project than before Effect 9: Effective involvement of participating SMEs in trans-national research cooperation with research performers and larger companies E.g. Number of partners (ie both SMEs and RTD performers / large companies) from other countries participating in the project, as compared to the situation before any participation in the scheme Effect 10: Research conducted leads to benefits for participating SMEs and SMEs have access to IPR E.g. Level of creation and exploitation of IPR (as owner or via user rights) Effect 8: Effective outsourcing of research activities by participating SME Associations E.g. Participating SME Associations outsource more research services during the project than before Effect 9: Effective involvement of participating SME Associations in transnational research activities E.g. Number of partners (ie SME Associations, SMEs and RTD performers / large companies) from other countries participating in the project, as compared to the situation before any participation in the scheme Effect 10: Research conducted leads to benefits for participating SME Associations / industrial groupings, and SMEs have access to IPR E.g. Level of creation and exploitation of IPR (as owner or via user rights) Effect 11: Effective dissemi-nation of research results E.g. Number of SMEs (outside the core-group) having participated in dissemination activities, compared to the total number of participating SMEs in the sector/group Effect 12: Effective training of staff from SME members of SME Associations / Industrial groupings on the take-up of research results E.g. Proportion of SMEs (outside of the coregroup) of which staff has participated in a training activity SME-specific: COOPERATIVE scheme SME-specific: COLLECTIVE scheme 37

38 4.4 Levels of analysis Taking into account the requirements expressed in the terms of references (ToR) as well as during the kick off meeting and in the course of the exploratory interviews, we distinguished among the following levels of analysis ( program and project levels as mentioned in the ToR are intertwined herein): 1. The level of the (design of the) support scheme 2. The level of the delivery structure 3. The level of the SMEs/SME Associations 4. The level of the broader economic, scientific and technological environment The interrelation among these levels is depicted in the Figure below. Figure 5: Interrelation among the various levels of analysis Source: IDEA Consult We prepared the list of evaluation questions, inspired by the hierarchy of objectives and the hierarchy of effects and structured by the four levels of analysis and the different evaluation topics. This list is included in the background report (section 4.3). 4.5 Evaluation tools The main objective of this project is to assess the impact of the SME-specific measures of the Fifth and Sixth Framework Programmes for Research on their SME target groups outsourcing research. In order to assess this impact, we applied different but complementary methodologies: We have set up online surveys addressed to both participants (the experimental group) and rejected applicants (the control group). The 38

39 experimental group was composed of participants that participated in the SME-specific measures either under FP5 of FP6 and for which we had an address available. The net sample contained 4814 potential respondents. We received 443 responses which implies a response rate of 9,2%. The control group included organisations (SMEs, SME Associations, RTD-performers, ) that were involved in a proposal submitted for the Cooperative and Collective Research programmes under FP6, whose proposal passed the threshold but was not selected for EC-funding because the available funding had been exhausted. We had a net sample of 3752 potential respondents and received 248 responses which implies a response rate of 6,6%. The main challenge when analysing the survey results was to prove the impact of project participation because of the attribution problem: if we observe a certain effect among the participants, can we attribute this to their participation in the project? In the context of the survey and the analysis we have made, we have dealt with this problem in the following different ways: - We asked the participants a series of questions about for example the R&D behaviour and activities before the start of the project and today in order to be able to compare. Assume that there is an increase in the percentage of participating SMEs that today have a separate budget for R&D. The question then remains whether this effect can be attributed to the project participation. We consider these chances are higher when among the control group of SMEs there is a statistically lower increase in the number of SMEs that today have a separate budget for R&D. - The respondents, both in the experimental and the control group, were also asked about the trend in a number of economic performance variables like their turnover or their export share. Again, by comparing the trend among the participants vis-à-vis the control group, we can attribute differences in economic performance to project participation. When analysing the survey results, we compared some of the results of our survey with results from two other surveys that we conducted in the context of other FP6 evaluations. More specifically, we refer to the Behavioral Additionality Survey (BAS) and the Participation Survey (PS). This comparison is a unique exercise because it is the first time in the evaluation of the Framework Programmes that effects on SMEs of different FP funding measures can be compared. In this case, we compare the effects of the SMEspecific measures and of the Thematic Priorities on participating SMEs. The BAS was launched within a study to assess the behaviour additionality of FP6. The survey sample for the experimental group covered participants under all FP6 instruments and Programmes except for the SME-specific Measures, the Marie Curie Mobility Actions and the Programmes implemented jointly by Several Member States. We linked the results from our survey with that of the BAS: we made a comparison between the answers of the SMEs within our survey and the answers of the SMEs in the BAS to check whether e.g. SME respondents in our survey have less/more R&D experience than the SME respondents in the BAS, which could give insight into the extent the SMEspecific measures reached their target group. The PS was launched within the context of an evaluation study that analyzed the implementation and impacts of the research activities completed under the 6 th Framework Programme. The survey sample for the experimental and 39

40 control groups was set up in a similar way as in the BAS, thus also excluding the SME-specific measures. The BAS and PS each reached a different set of (SME-) respondents: the whole sample was split in two 16 (to avoid oversurveying), so that each possible (SME-)respondent received only one survey. The PS included the same question as in our survey on the R&D capabilities of the SMEs before the start of their project, thus again allowing a comparison between the two surveys. We have set up 45 case studies covering interviews with 85 SMEs, 17 IAGs and 54 RTD-performers (i.e. on average 3,4 interviews per case with 2 interviews with SMEs). We established a group of 8 countries from which these cases were selected: Belgium, Finland, France, Hungary, Poland, Slovenia, Spain and the UK. Information obtained is case-based, empirical evidence on the impacts of participation on SMEs and the impact of SME participation on projects. Special attention was given to the question of how and why certain impacts materialised. They complemented the empirical evidence gathered through the online surveys and at the same time broadened a qualitative dimension of the impact analysis. They also formed a basis for a detailed critical assessment from the users perspective. The results from the case studies have been analysed in two complementary ways: by a quantitative horizontal cross case analysis as well as by a qualitative vertical reading of all case reports. In addition to the interviews in the context of the case studies, we also had a series of interviews and a workshop with (other) stakeholders. The additional interviews mainly served to complement the case studies in the selected countries with relevant context information for these countries. The workshop had the objective to reflect on the SME-specific measures from a more global point of view on the basis of the survey and case study results. We also used the AMADEUS database with economic and financial data on millions of EU companies, to evaluate the impact of the SME specific measures on the participating SMEs. This has been a pioneering exercise since this kind of data has not been used before in the context of the evaluation of the impact of participation in the FPs. We applied a univariate analysis to assess whether there is a significant difference between the performance of participants in the SME specific measures and the performance of rejected applicants, as well as a multivariate regression analysis to get further insight in the relation between FP participation and the economic performance of the participants 4.6 Strengths and weaknesses of our approach Each of the above mentioned methodologies has its strengths and weaknesses, on its own as well as compared to the other methodologies. The surveys for instance allow collecting opinions from a larger group of respondents but can go less in depth compared to the case studies. The response rate to the surveys was reasonable although below expectations. However the response analysis indicated no biases for the dimensions that could be compared with the sample characteristics. The set-up of a survey among a control group furthermore allowed comparing in a systematic way the changes in R&D and innovation 16 The sample split was done in such a way that the two samples (PS and BAS samples) both showed the same characteristics as the whole sample: both samples for instance included 11,9% SMEs. 40

41 behaviour as well as in the economic performance of participants and nonparticipants. This way we could deal with the attribution problem, a problem most traditional impact studies suffer from. The set-up, execution and analysis of the case studies have been very challenging and labour intensive. The cases allowed collecting qualitative illustrations of the results from the survey outcomes. But in addition, because of their large number, we were also able to draw statistics and compare them with the survey results. For most of the topics, the survey and case study results reinforced each other which raises their reliability. The case study interviews further allowed posing the why -questions which further broadened our insight into the processes going on. We must keep in mind however that both the respondents to the survey as well as the interviewees for the case studies can be biased. While surveys and case studies are largely based on opinions of those surveyed or interviewed, the AMADEUS database provides factual information about the performance of the companies. This is a major strength. But the analysis that could be developed was, due to the limitations of the data and the fact that companies can go bankrupt or are taken over, based on a subset of SMEs. Proving the impact of R&D-support on the competitiveness of those supported, remains a very difficult exercise. The very nature of Framework Programme research, based typically on pre-competitive collaborative research projects means that there are likely to be many steps, and complementary inputs downstream of the research, before commercial or other types of return can be realised. Further still, the academic literature shows very clearly that there are severe measurement challenges when seeking to assess the totality of research impacts. These relate to both the problems of attribution, connecting impacts with the underpinning research, as well as with the many 'spillovers' which flow from a research action. Data availability, especially tracking research over the longer term is also a severely limiting factor. By using the complementary approaches we described, we tried to deal with these challenges to the best possible extent. In any case, the approaches we followed provided us with an evidence base that is much stronger when compared to an approach based on one single methodology. 41

42 5 EVALUATION RESULTS 5.1 Introduction This chapter builds to a large extent on the information collected by means of the surveys and the case studies. The survey results have been analysed in full detail, not only question by question but also by combining questions and taking into account participant and project characteristics like for example the degree of R&D-formalisation within the SMEs or whether the project was already finished or still ongoing. We also compared the survey results with those of other surveys, set up in the context of the evaluation of FP6. These surveys serve as interesting benchmarks to evaluate for instance the reach of the target group by the SME specific measures. This chapter also includes the evaluation results from the case studies. These have been analysed in two complementary ways: by a quantitative horizontal cross case analysis as well as by a qualitative vertical reading of all case reports. In addition, we also integrated the results from a series of additional interviews that were carried out with national contact points, SME representative organisations and R&D and innovation support agencies in different countries as well as the further insights collected during a stakeholder workshop. The order in which we present the evaluation results corresponds in first instance with the structure of the questionnaires used for the survey: R&D-behaviour and activities before the start of the project, project involvement and implementation, project outputs and results, project effects and impacts on the organisation and relevance and value added of the SME specific measures. But other insights derived from the cases as well as from the additional interviews, have been entered into additional sections: Success and failure factors, Collective versus Cooperative projects, Monitoring and review process and an outlook on FP7. But we start off with some evaluation results based on an analysis of the participation statistics. Please note that the more detailed evaluation results are included in the Background Report. 42

43 Some abbreviations and conventions: EXP refers to the experimental group, CON to the answers from the control group. COLL refers to the Collective measure, COOP to the Cooperative measure SSM-participants are the SME participants in the SME specific measures. Other participants are the SME participants in the other framework programmes, mainly the thematic programmes. n = the number of observations used to calculate the mean value or the %- share. In most cases I don t know or not applicable have not been taken into account to calculate these mean values or %-shares. T-0 refers to the situation before the submission of the project proposal, T-1 refers to the situation today, i.e. at the time the respondent replied to the survey 5.2 Participation statistics 17 In this paragraph we summarize the descriptive statistics on participation in the SME-specific measures under FP5 and FP6. We analysed participation statistics along three dimensions, i.e.: - According to the number of projects, proposals, participants, average budget and duration for the various measures and FPs; - According to the number of unique participants, i.e. we analysed multiple participations and the distribution of recurring participants ; - According to the share of each country in the total numbers of projects and proposals, as well as its success rate (e.g. number of successful proposals compared to total number of proposals) over time. 17 The results presented in this section are a summary of the material provided in Chapter 5 of the Background Report. 43

44 5.2.1 Project and proposal statistics In terms of all proposals submitted, FP6 outnumbers FP5, but the success rate (i.e. the number of contracts or funded proposals divided by the total number of all proposals submitted) was lower under FP6 (12%) than under FP5 (24,5%). Table 3: General overview participation statistics FP5 CRAFT FP6 COOP COLL Total Total number Contracts Proposals -proposals above threshold na all proposals Succes rates proposals above threshold/all proposals na 40,30% 18,20%* 36,50% Contracts/proposals above threshold na 29,60% 68,30%* 32,90% Contracts/all proposals 24,50% 11,90% 12,40% * 12,00% Oversubscription (Proposals above threshold - contracts)/all proposals na 28,37% 5,78% * 24,51% Note: * stage 1 and 2 combined: above threshold gives an indication of the last evaluation a proposal has gone through. For the 2-stage procedure of the collective measure, this means the outcome of the 2 nd stage. Source: IDEA Consult on the basis of FP5 and FP6 dbase The success rate of the Cooperative measure under FP6 (11,9%) was slightly lower than the Collective measure (12,4%). Under FP6 a smaller number of projects were funded in comparison to FP5. These FP6 projects, however, received on average a clearly higher budget and EC contribution, especially for the projects under the Collective measure. A high amount of proposals did not live up to standards, i.e. only 36,5% of submitted proposals ended up being above threshold. In this regard, proposals submitted under the cooperative scheme were more successful than their counterparts under the collective scheme: 40,3% were successfully evaluated against 18,2% under the collective scheme. However, slightly more than 70% of these successful proposals did not get any funding under the cooperative scheme, as the budget had been exhausted by the top ranked proposals. Under the collective measure, the available budget allowed relatively more successful proposals to be funded: only 32% (against more than 70% under the cooperative scheme) of the proposals above threshold did not get funded. Almost one-quarter of all proposals submitted were evaluated above threshold but did not get any funding, leaving aside many valid projects. Table 3 (last row) also shows the number of projects that were above threshold but did not turn into a funded project, divided by the total number of proposals. This is an indicator of the oversubscription rate. Overall, almost one-quarter of all proposals submitted were evaluated above threshold but did not get funding. For 44

45 the cooperative measure, the number is relatively high with 28% of all proposals being positively evaluated but not funded. For the collective measure, the percentage is lower with about 6%. This can be caused by either the application of the 2-stages procedure under the Collective scheme, which may have allowed filtering out proposals of lower quality, or by the relatively larger available budget (i.e. compared to the number of submissions), or by a higher quality of proposals under the collective scheme (as compared to cooperative ones) Unique and recurring participants 90% of the participants participated only once in the SME-specific measures Under FP5, 89,5% of the participants participated in only one project. 6,8% participated in two projects and 1,8% participated in three projects. Under FP6, 92,1% participated in one project, 5,5% in two projects and 1,3% in three projects. A few RTD-performers are involved in a large number of the projects, and this concentration has increased between FP5 and FP6 Under FP5, there were 15 organisations with each more than 10 participations. This list contains only RTD-performers from the old EU Member States. The list was topped by Pera Innovation Ld (UK), TNO (NL) and Fraunhoffer (DE). Under FP6 only 8 organisations had at least 10 participations. Under FP6 the top-three consisted of Pera Innovation Ld (UK), Fraunhoffer (DE) and TWI Ld (UK) (TNO 6 th ). The most recurring participant is the RTD-performer Pera Innovation Limited which has participated 87 times or in 18% of all projects under FP6, and in 7% of the projects in FP5. All together the company has participated in 132 projects in FP5 and FP6. Pera was the participant which increased the most his market share between FP5 and FP6 (from 7% to 18%), but some others followed the same trend: Fraunhoffer, for instance, increased from 5,1% to 7,2%. Moreover, the list of these few organisations does not modify significantly between FP5 and FP Projects, proposals and success rate per country 18 The United Kingdom, Germany, Spain and Italy account for almost 60% of the total number of projects (FP5 and FP6 together). France has a share of almost 11% of all projects in FP5 which has further decreased to around 7% in FP6. The New Member States only account for a very small share of the total number of projects, but their share is increasing (from 0,9% in FP5 to 2,32% in FP6). Moreover, their share in projects has increased more from FP5 to FP6 than their share in proposals, implying a progress in terms of success rate A project was assigned to a country on the basis of the coordinator of that project being located in that country. Defined here as the ratio between the number of funded projects and the total number of proposals. 45

46 Looking at the total of FP5 and FP6, the Netherlands have the highest success rate (23,58%). United Kingdom and Denmark also have a success rate of over 20%. For EU15 countries the success rate amounts up to 16,21%, which is a lot higher than for the EU10 countries (5,8%). In terms of success rate of a proposals turning into a proposal above threshold, the percentages are on average higher in the Old Member States compared to the new Member States. The quality of the proposals of Old Member States seems therefore higher which can possibly be explained by the higher level of experience of most of the co-ordinators from those countries. Success rates dropped significantly between FP5 and FP6, especially for France (26% to 8%), Belgium (22% to 9%) and Ireland (23% to 4%). 5.3 Reach of the target group 20 In this section, we give an overview of the profile characteristics that are related to the R&D-behaviour and activities of the SME-respondents before the submission of the proposal. Their analysis allows evaluating the reach of the SME specific measures in terms of target groups. We have used some of these profile characteristics later on in the analysis to check whether responses were significantly different according to these characteristics Degree of R&D-formalisation We defined a construct degree of R&D formalisation based on the responses on two questions addressed to companies only, i.e. whether the company had a separate R&D department and whether there was a yearly budgeting of the R&D expenses. We defined 3 types : low degree of R&D-formalisation: no yearly budget for R&D expenses and no separate R&D department medium degree of R&D-formalisation: a yearly budget for R&D expenses but no separate R&D department high degree of R&D-formalisation: a yearly budget for R&D expenses and a separate R&D department Our analysis allows drawing the following conclusions. The SMEs participating in the core group of the Collective Measures have a lower degree of R&D formalisation than the SMEs participating in the Cooperative Measures In a little less than 45 % of the SMEs in the experimental group (the Cooperative and Collective projects taken together), there was a low degree of R&Dformalisation before the start of the project, 31% had a yearly budgeting of R&D expenses, which we consider as the weaker form of R&D-formalisation and 25% of the SMEs in the experimental group even had a separate R&D department (see Figure 6). Smaller SMEs have a lower degree of R&D formalisation. Further 20 More detailed information can be found in section 6.2 in the Background Report. 46

47 analysis also showed that the SMEs participating in the core group 21 of the Collective Measures have a lower degree of R&D formalisation than the SMEs participating in the Cooperative Measures: nearly 55% of the core group SMEs in the Collective Measures can be considered as having a low degree of R&D formalisation at the onset of the project compared to 42% in the Cooperative Measures. When compared to the experimental group, we observed that the SMEs within the control group, on average, have a higher degree of R&D formalisation. Overall, we would have expected a lower degree of R&D-formalisation, as the SME-specific measures target low- and medium-tech SMEs. But without a real benchmark, it is difficult to draw strong conclusions. Therefore the comparison with the SME respondents in the other FP funding measures 22 is very interesting (see Figure 6). Figure 6: Degree of R&D formalisation SMEs before start of the project (a) (a) Before the start of the project, was there a yearly budget for R&D/a separate R&D department in your organisation? SSM-participants: n = 216, other participants: n = 324 Source: IDEA Consult on the basis of survey results The SMEs participating in the SME specific measures have less formalised R&D-activities compared to the SME participants in the thematic programmes. It turns out that at the start of the project, the SME respondents in the other (non-sme specific) measures of FP6 had a higher degree of R&D-formalisation than the participants in the SME-specific measures. This difference is statistically significant. 37% of the SME respondents participating in these other measures (mainly the thematic programmes) could be characterized as having a high degree of R&D-formalisation compared to 24% in our survey The SMEs in the core group are actual participants in the Collective projects which is not the case for the SME-members of the IAGs that do not participate themselves but are represented by the participating IAGs. When analysing the survey results, we compare some of the results with those from two other surveys that IDEA Consult conducted in the context of other FP6 evaluations: the Behavioral Additionality Survey and the Participation Survey. The samples for these surveys covered participants under all FP6 instruments and Programmes except for the SME-specific Measures. 47

48 Besides the objective, factual characterisation of the degree of R&D formalisation of the SMEs (yearly budget, R&D department), the SMEs were also asked to evaluate their own R&D capabilities 23. The results on this question are very similar. Table 4: Self declared R&D capabilities SMEs before start of the project (a) SSM participants Other participants Very low 5,5% 1% Low 21,2% 3% Medium 41,1% 18% High 24,2% 42% Very High 8,1% 35% Mean 3,08 4,1 n (a) Overall, how do you rate the R&D capabilities of your organisation before the start of the project... Mean values with 1 = very low, 2=low, 3=medium, 4=high, 5=very high Source: IDEA Consult on the basis of survey results Before the start of the project, the average SME in the experimental group characterized itself as having medium R&D-capabilities (see Table 4). But one out of three SMEs responded to have (very) high R&D capabilities. We also found that smaller SMEs do not seem to have lower R&D capabilities than their larger counterparts. This seems to contradict the positive relationship we found before between the degree of R&D-formalisation and the size of the SMEs. A possible explanation for this contradiction could be that within a small SME there is indeed no separate R&D budget or department, but for example the SME-owner is a scientist who conducts R&D projects in his day-to-day business. A very interesting finding results again from the comparison with the SMEs participating in the other measures under the framework programme: the R&D capabilities of the SMEs in these other measures were on average high with a much larger percentage of SMEs in the category high R&D capabilities (42% of the SMEs versus 24% in the SSM). This is another indication that the SMEspecific measures reach a different group of low/medium tech SMEs. 3 out of 4 SME respondents participated before in a nationally or regionally funded R&D-programme. 1 out of 2 had been involved in an R&D project funded within the EU Framework Programmes. 49,8 % of the SME respondents in the experimental group never conducted or participated in a project funded within the context of any EU R&D framework programme (55,6 % within the control group). Much more SMEs participated before in R&D projects publicly funded at the regional or national level. Only 26,8 % of the SMEs in the experimental group and 33 % in the control group never conducted or participated in a project funded by public R&D support programmes at national/regional level. Participation in nationally or regionally funded R&D programmes therefore appears to be a stepping stone to participation in the SME specific measures. This was confirmed by the interviewees among the National Contact Points and national and regional R&D and innovation support agencies. 23 Note that the results for the degree of R&D formalisation show a high positive correlation with the results on a question where the SMEs had to indicate their R&D-capabilities (from very low to very high ). 48

49 5.3.2 R&D cooperation experience The following conclusions are about the R&D cooperation behaviour of the SMEs and the IAGs, again before the submission of their proposal for the SME specific measures. Figure 7: R&D cooperation experience SMEs before start of the project (a) (a) Before the start of the project, was your organisation involved in? Mean values with 1 = never, 2 = occasionally, 3 = regularly. SSM-participants: n = 236, other participants: n = 373 Source: IDEA Consult on the basis of survey results Before project participation, the R&D cooperation experience of the SMEs was occasional with domestic partners but much less with foreign partners and overall also smaller compared to the SME participants in the thematic programmes. The R&D cooperation experience of the IAGs can be characterised in the same way as for the SMEs. With regard to the R&D cooperation experience of the SMEs before the start of the project, we note from Figure 7 that, on average, SMEs had already cooperated occasionally with other companies and higher education institutes/research organisations located in their own country, in the context of R&D projects. As we would expect, the R&D cooperation with companies and research organisations located abroad occurred on a less frequent basis. Further analysis showed that, of course, those SMEs that participated already before in a Framework Programme show a higher level of R&D cooperation experience. Overall, we did not notice any important differences between the experimental and the control group. The results from the cross case analysis are quite in line with those of the survey: the interviewed SMEs had, on average, also more R&D cooperation experience with higher education or research institutes compared to companies and also more with local research institutes compared to foreign research institutes. When comparing to the results of the SME participants in the other measures under the Framework Programme (in particular the thematic programmes), we find that before the start of the project, SME-participants in SME specific 49

50 measures had less R&D cooperation experience. The differences are most clear with respect to international R&D cooperation experience (see Figure 7). Similar to the SMEs, the average IAG in the experimental group had occasionally been involved in R&D cooperation with higher education institutes/research organisations located in its own country before the start of the project. There were less IAGs with R&D cooperation with research institutions abroad but within the EU. SME participants in the Cooperative projects had more R&D cooperation experience compared to the SME participants in the core group of the Collective projects. Within the experimental group, the SMEs that participated in the Cooperative measures had been engaged some more in R&D cooperation with companies from abroad and with research organisations from their own country and from abroad than the core group SMEs that were funded under the Collective measures. This is in line with the, on average, higher degree of R&D formalisation of the SME participants in the Cooperative measure compared to the Collective measure. The cases make clear that in fact there is a variety in the level of R&D capabilities among the SME-participants. So overall, the SME specific measures reach their target group, but also other SMEs that do not belong to the target group. As one of the conclusive questions for the cross case analysis, we asked the authors of the cases to give their opinion on the reach of the target group for their case. In 21 out of the 45 cases, the majority of the SME project partners were low to medium tech SMEs. In 16 cases (or a little more than 1 out of 3 cases), there was a mixed presence of high tech and low/medium tech SMEs. In the remaining 8 cases, there were more high tech SMEs than low/medium tech SMEs. This nicely illustrates the variety in the level of R&D capabilities among the SME participants. Some of the SMEs indeed have low R&D capabilities of their own. But on the other hand, there are also examples of SMEs that do have R&D capabilities of their own and could very likely also participate under the thematic programmes. So the SME specific measures reach their target group, but also SMEs that do not belong to the target group. The EPEC study 24 also addressed the issue of the reach of the target group and came to similar conclusions. However, as the high-tech SMEs having participated to the SME-specific measures fall out of the targeted group of SMEs, it is interesting to assess the reasons that may have stimulated their participation in these schemes instead of under the thematic programmes of the FP. A detailed look at the 6 cases (out of 45 casestudies) where high-tech SMEs were involved, allows for the following observations. The high-tech companies involved in these 6 projects do not only have high-tech in-house R&D capabilities on their own, their core R&D activities seem to largely correspond to the work carried out in the project. In other words, these SMEs did not participate in the SME-specific measures because of a lack of R&D capability in a particular domain or to transfer complementary knowledge to their core expertise. Based on their R&D capabilities, one would have expected them 24 EPEC, 2006, Impact Assessment for improving SME specific research schemes and measures to promote SME participation in the Framework Programme. 50

51 participating under the thematic programmes. Therefore, the reasons why they opted for the SME-specific measures instead of the thematic programmes should be sought elsewhere. Based on the assessment of the case-studies, three main reasons appear. First, the SME may have considered the outsourcing of research and collaboration with a RTD-performer as a more cost-effective option (with less economic risk) than carrying out R&D on its own (for instance because the RTDperformer had a clear cost advantage in terms of research equipment). Second, the R&D carried out under SME-specific measures is of a very specific nature and relatively close to the market (at least as compared to the nature of research activities carried out under thematic programmes); it implies also more incremental work instead of radically innovative research. Third, the bottom-up character of the measure allowed the SME designing the research agenda according the best to its needs. 5.4 Project involvement and implementation 25 The topics we deal with under this section are about the involvement in the setup and execution of the project, with special attention for the role played by the SMEs. The partnerships also get special attention, in particular the extent to which these partnerships are new Proposal preparation For a majority of the projects, the original idea comes from the RTDperformers. In a smaller number of cases, SMEs are the initiators. Figure 8: Original idea for the project (a) 80% 70% 60% 50% 40% 30% 20% 10% 0% RTDperformer SME IAG Private consultant Intermediary Other (a) Who put forward the original idea for the project? Multiple answers were possible so column totals sum up to more than 100 %. n = 384. Note: other = all other types of partners involved. Source: IDEA Consult on the basis of survey results 69,3 % of the SMEs, IAGs and RTD performers taken together indicated the RTDperformers put forward the original idea for the project, whereas 43,8% of these 25 More detailed information can be found in section 6.3 in the Background Report. 51

52 respondents stated that the SMEs came up with the initial project idea (see Figure 8). SMEs as a separate group of respondents attribute less of the initial project ideas to the RTD performers and more to the SMEs: for about 1 out of 2 SME respondents in the experimental group, either an RTD performer (56,8 %) or an SME (their own company or another company 50,7 %) launched the project idea. On the other hand, a very large proportion of RTD performers, nearly 90%, indicated they put forward the original project idea. The figures from the cross case analysis are comparable: in 59,6% of the cases it was said that the RTD performer put forward the original idea and in 38,3 % it was one or more of the SMEs. The cases further illustrated that when a project idea is originally formulated by an SME, in most cases it is subsequently developed by an RTD-performer. This explains why sometimes both the SME and the RTD-performer will claim having put forward the original for the project: the SME because he signaled the problem to be solved by means of a technology to be developed, the RTD performer because he came up with the suggestion for the technological solution. The RTD-performers are also the most involved in the preparation of the proposals. Figure 9: Involvement in proposal preparation (a) (a) To what extent was your organisation involved in the preparation of the proposal to be submitted to the EC? n = 392. Source: IDEA Consult on the basis of survey results Figure 9 shows the self-declared involvement of the different partner-types (SMEs, IAGs and RTD performers) in the preparation of the proposal. Looking at the global results for the experimental group, the RTD performers were the most involved in the preparation of the proposal: they proclaimed to have had an (almost) substantial contribution to the proposal (on average). The SMEs and IAGs stated they only gave limited input (on average) in the proposal preparation. This indicates that mostly the RTD performers write the proposals for application in the SME-specific measures. 52

53 Looking at the Collective and Cooperative measure separately, we see that SMEs indicated they were less involved in the preparation of projects under the Collective measure. Further analysis of the survey results showed that the SMEs with a medium degree of R&D formalisation (a yearly R&D budget but no separate R&D department) are most strongly involved in the proposal preparation while the SMEs with the lowest degree of R&D-formalisation are the least involved in the proposal preparation. The results of the cross case analysis confirm the leading role of the RTDperformers in the development of the proposals, even more explicit than the survey results (but based on a more limited number of observations): using the same answering possibilities, the involvement of the RTD-performers gets an average score of 3,4 (48 responses), while the SMEs (49 responses) and the IAGs (20 responses) get 2,16 and 2,2 respectively (with 1 = not really involved, up to 4 = took the lead) Project implementation SMEs in the Cooperative projects are more intensively involved during the execution of the project compared to the SMEs in the core group of the Collective projects. Table 5: Involvement of SMEs in project execution (a) COLL COOP Total FP5 FP6 TOT Mean n Mean n Mean n Mean n Mean N SME 2, , , , , (a) How was your organisation involved in the execution of the project? Mean values with 1=as a project partner involved in the exploitation and dissemination phase only, 2=as a project partner involved in a particular phase in the project, 3=as a project partner involved from the beginning to the end of the project, 4=as a coordinator. Source: IDEA Consult on the basis of survey results With a mean value just below 3, SMEs have in their own opinion on average been involved as project partners during almost the entire duration of the project (see Table 5). The distinction between SMEs financed under the Collective and Cooperative measures shows that generally SMEs (in the core group) under the Collective measures have been less involved than the SMEs active in the Cooperative measures. The SMEs felt a little more involved in FP5 compared to FP6 but the difference is too small to draw strong conclusions. Further analysis showed there are no big differences in terms of involvement in the project execution dependent on the degree of R&D formalisation of the SMEs. While the survey results are a self-assessment by the SMEs about their role in the execution of the project, we also questioned the RTD-performers on the role played by the other partners. According to the RTD performers, other research organisations contribute most to the execution of the projects, outweighing the contribution by the SMEs. RTD performers also state that SMEs provide a more sizeable input into the projects under the Cooperative measure compared to the Collective measure and that the involvement of the SMEs and the IAGs in Collective projects is of about the same intensity. 53

54 The results of the cross case analysis slightly differ and indicate a lower involvement of the SMEs with a mean value 2,52 based on 84 observations. SMEs have an impact on the project mainly through their involvement in the project meetings where they can keep the project on track, i.e. providing results that serve their technical as well as their business needs. From our case studies it is clear that throughout the project, the involvement of SMEs lies mostly in the participation to the project meetings. Near the end of the project, the SMEs become more actively involved by performing tests and trials. A UK SME active in the rotational moulding sector specified the role of the SME core group in Collective projects as follows: (1) to guide the project in the direction of results with commercial potential and to check the realism of the work plans, (2) to conduct technical guidance on what can/can t work and (3) to do in-house testing and trials. The testing occurs generally in an iterative manner with feedback provided to the RTD-performers on how to adjust the method/product/. In such a case, proximity in geographical, linguistic and cultural terms matters a lot: field-trials are more effective if you have pairs of actors, i.e. RTD-performers and SMEs in each country. In many projects, there seems to be a core of active partners, besides a number of peripheral partners. The lesser involvement of some SMEs can be due to the fact that they only had interests in minor aspects of the project or that they can only participate actively near the end of the project in e.g. the testing stage of the project. However, the passive role of some SMEs can also be explained by their lack of motivation which seems to be more often the case in Collective projects Prior collaboration The next tables are about the existence of prior collaboration with the other partners involved in the projects and provide us more insight into what extent the projects result in new partnerships. The SME-specific measures lead to more new partnerships compared to the other instruments/programmes in FP6. 54

55 Figure 10: New partnerships by SMEs (a) New partnerships with 70% 60% 50% 40% 30% 20% 10% 0% Domestic firms Firms from abroad Domestic universities/research institutes SSM participants Other participants Universities/research institutes from abroad (a) Did your organisation work collaboratively with these partners before the project? Question was only asked to respondents that had indicated before to cooperate with this type of partner in their proposal. The figure presents the percentage share for the answer with none of them, indicating new partnerships. SSM participants: n up to 227, other participants: n up to 327. Source: IDEA Consult on the basis of survey result For nearly one out of two SME respondents in the experimental group, the cooperation with the domestic firms and IAGs in their project was new, i.e. they had never previously collaborated with these partners before (see Figure 10). More than 60% of the SMEs in the experimental group had not previously worked with the foreign firms, IAGs and universities/research institutes that were active in their consortium. In the control group, the percentage of new collaborations was smaller, with only one out of three SMEs entering a new cooperating relationship with the domestic partners from their consortium. When comparing the results from our survey with those on the other instruments/programmes under FP6, we see that in general the proportion of new partnerships is lower under the other (non-sme-specific measures) in FP6 than under the SME-specific measures 26 (see Figure 10). The SME-specific measures thus lead to more new partnerships compared to the other instruments/programmes in FP6. The participating IAGs also entered into new partnerships, in particular with firms and universities or research institutes from abroad. 26 The prior collaboration with domestic universities is an exception: here there were less new partnerships under the SME specific measures than under other FP6 funding measures 55

56 Figure 11: Prior collaboration by IAGs (a) (a) Did your organisation work collaboratively with these partners before the project? Question was only asked to respondents that had indicated before to cooperate with this type of partner in their proposal. The figure presents the percentage share for the answer with none of them, indicating new partnerships. n = up to 28. Source: IDEA Consult on the basis of survey results The participating IAGs also entered into new partnerships, in particular with firms (in particular from abroad but also domestic) as well as universities or research institutes (from abroad). Just like the SMEs, they indicated that they had already cooperated with several of the domestic university/research institute partners before the start of the project. They also had cooperated previously with most of the other IAGs in their consortium. On average, partnerships are more new for the SMEs involved compared to the IAGs. The cases illustrate the fact that setting up a partnership is a challenging effort. Although many partnerships originate out of pre-existing contacts, the partnership set up is a challenging effort because of the need to find companies which fit the criteria in terms of size and also span the required number of Member States. Geographical and language barriers also make the partnership set up more difficult. Another issue in setting up the partnership which was raised in several cases is the size of the consortium. A large consortium with especially many SMEs makes it harder for each partner to benefit from the project, especially in terms of profits of any future product coming out of the project. Some projects in the case studies also suffered from early drop outs by SMEs. Most cited reasons are a misunderstanding about the actual content of the project or dissatisfaction about the funding arrangement. Another important issue taken into account in the partnership set up is the selection of partners that are preferably active in different stages of the value chain, resulting in a consortium with almost no competitors in particular within cooperative projects. A final observation is that project coordinators seem to have made limited use of partner search facilities on CORDIS, national sources that have their own search tools, as does the Enterprise Europe Network for example. The suggestion was 56

57 made to unify these search tools to allow for one well-operating tool to be used by all potential participants Project additionality Project additionalities are in place if the research project is cancelled, unless it is supported by public funds ( full project additionality). When the project is continued but changed in one or more of its dimensions, we use the term of partial project additionality. When the project is continued without any changes, there is no project additionality. Figure 12: Project additionality (a) (a) What has happened to the project after it was denied EU-funding? no project additionality = the project has gone ahead as originally planned, partial project additionality = the project has gone ahead but with significant changes to the project, full project additionality = the project has been cancelled completely (n=118) Source: IDEA Consult on the basis of survey results 2 out of the 3 projects above threshold that could not be funded have not taken place at all. Only 3,4% of the respondents in the control group replied that the project has taken place as originally planned which implies that there is no project additionality at all (see Figure 12). On the other hand, 66% of these respondents indicate that the project has not taken place at all, which implies full project additionality. But the decision is not a zero-one decision. About 30% of the projects has taken place but with changes to the project implying partial project additionality. This subgroup received a series of questions on the changes they have made. The most important changes are the cut in the number of international partners, the smaller financial scale and the more narrow research objectives. These results show that a lot of good proposals (i.e. above threshold) never get realised or only in a reduced form because of the limited EC-funding available. 57

58 5.5 Project outputs and results 27 In this section we report on the reach of the project objectives and on whether the projects outputs and results fulfilled the SMEs own objectives. Secondly we verify for the IAGs which training and dissemination activities they set up in de the context of their projects. Thirdly, we review the effectiveness of the project implementation. In a fourth section, we discuss some issues related to Intellectual Property Rights (IPR) Reach of project objectives Because the SMEs are not always closely involved in the development of the proposals, we included a question about the match between the project objectives and the SME s own objectives. The project s objectives were to more than a reasonable extent in line with the SMEs own objectives, and this match logically increases together with the level of involvement of the SME in the project s preparation and implementation. Figure 13: Match between project and SME s own objectives (a) To a very large extent To a large extent To a reasonable extent To a limited extent Not at all 0% 10% 20% 30% 40% 50% Total COOP COLL (a) To what extent were the project objectives in line with your organisation s own objectives? n = 232. Source: IDEA Consult on the basis of survey results The SMEs indicated that overall, the project s objectives were to more than a reasonable extent in line with their own objectives. In total, 37,5% of the SME respondents felt that the project matched their own objectives to a large or very large extent (see Figure 13). The SMEs under the Collective measures seemed to have been somewhat more satisfied with their project s objectives than the SMEs under the Cooperative measures but the difference is small. Further analysis indicated that the match between the project s and the SME s objectives is larger for the SME respondents 28 that were coordinator of their 27 More detailed information can be found in section 6.4 in the Background Report. 58

59 project. This observation supports the idea that when SMEs are more strongly involved (as coordinator), this has positive consequences for the match between the project objectives and the SME objectives. The development of a new or improved product is for the SMEs by far the most important objective of their project. The adaptation of a product, service or process to a new regulation, norm or standard is a much less important project objective, also in the Collective projects. Figure 14: Importance of project objectives for the SMEs (a) To develop a new or improved product To develop a new or improved process To solve a technical problem To determine the feasibility of a product, process or service To develop a new or improved service To adapt your product, service or production process to a new regulation, norm or standard 0% 10% 20% 30% 40% 50% 60% 70% 80% (a) What were the objectives of the project? Percentage in the table reflects the % of respondents that ticked the objective specified in the table. Respondents could tick multiple objectives. n = 236. Source: IDEA Consult on the basis of survey result About 70% of the SMEs indicated that the development of a new or improved product was the objective of their project. The development of a new or improved process was the project objective for 40% of the SMEs. The adaptation of a product, service or process to a new standard was the least important objective, also for the IAG respondents. The latter result is surprising when it comes to IAGs in particular as one of the programme objectives of the collective measure is to support R&D that benefits the whole sector like in the case of the adaptation of a product, service or process to a new standard. But it was confirmed by the case studies because in none of the cases it was the major project objective. 16 of the 45 cases aimed for the development of a new or improved product and in another 16 cases the focus was on development of a new or improved process. The SME and IAG respondents stated that, overall, the project objectives have been partially reached. For the SMEs, there is a positive relationship between more intensive project involvement and the extent of reaching the project objectives. 28 It is however also possible that in the projects for which the SMEs responded to be partner, another SME was the coordinator. 59

60 Overall, the SME as well as the IAG respondents felt that the project objectives have been partially reached 29 (see Figure 15). Figure 15: Overall reach of project objectives according to SMEs and IAGs (a) (a) Overall, have the objectives of the project for your organisation been reached? SMEs: n = 236, IAG: n = 30. Source: IDEA Consult on the basis of survey result Around 1 out of 5 SMEs as well as IAGs indicated project objectives were totally reached. On the other hand, nearly 1 out of 5 SMEs and 1 out of 10 IAGs responded the project did not at all reach its objectives. IAGs are on average a little bit more affirmative about the reach of the project objectives. Further analysis showed that project involvement has an influence on the SMEs opinion about the extent to which the project objectives have been fulfilled. For on average nearly 30% of the SMEs with a less important project involvement 30, the project did not at all reach its stated objectives. For the SMEs that were more intensively involved, as a partner throughout the whole project or as a coordinator, this percentage is much lower (between 4% and 15,8%). Also, more than 60% of the more involved SMEs answered that the project objectives had at least partially been fulfilled, compared to 50% of the less involved SMEs. In 6 out of 40 cases the project already resulted in a commercial output, in 13 other cases the expectation is that in the future there will be a commercial output. Projects for which the original project idea was initiated by the SMEs had more chances of obtaining commercial outputs. In the context of the cases we explicitly checked whether the project resulted in any commercial outputs. In 9 of the 40 cases for which we have this information available, it is still too early to make an assessment. Only in 6 cases the project already resulted in a commercial output. In 12 out of the remaining 25 cases, there are no commercial outputs yet and the expectations for the future are Note that a considerable percentage of SMEs and in particular IAGs indicated that it was too early to assess whether their project objectives had been accomplished. Meaning that the SME was only involved in the end phases of a project (exploitation & dissemination phases) or in one particular phase of the project. 60

61 negative, while in the other 13 cases the expectation is that in the future there will be a commercial output. It is difficult to evaluate these numbers without a benchmark. But in a survey addressed to the participants in the other measures under FP6, mainly the thematic programmes, about 35 % of the SME respondents thought they will (likely or very likely) realise commercial returns from their participation in the project. If we consider the 6 cases that already realised a commercial output and the other 13 cases where the expectations are positive for the future, we arrive at 19 out of 31 cases or about 60 % of the cases with projects that resulted or are expected to result in a commercial output. This compares positively to the 35 % in the other measures under FP6, but one must take into account however that the questions raised were not exactly the same. Further analysis shows that projects for which the original project idea was initiated by the SMEs had more chances of obtaining commercial outputs: from the 10 cases initiated by SMEs, 3 projects already resulted in a commercial output and for 4 projects, commercial results were expected. Overall, in 7 of the 10 projects initiated by SMEs, there were (expected) commercial outputs. From the 16 projects initiated by the RTD performers, only 2 cases reached a commercial project and for 8 projects this output was still expected: thus in 10 out of the 16 cases whose original project idea was launched by the RTD performer, there was an (expected) commercial output. RTD-performers are more positive about the reach of the project objectives compared to the SMEs because of different initial (research and business) objectives. The survey as well as the case study results show that the RTD-performers are typically more positive about the reach of the project objectives compared to the SMEs. This has to do with the different initial objectives. RTD-performers are research oriented and happy if the project reaches its goals from a technical and research point of view, even if a project shows that a specific technology cannot be applied in a commercial context. SMEs are typically more focused on commercial output as a criterion to evaluate the success of a project. So if a project has resulted in new knowledge but not in any commercial output, the RTD-performers are likely to be satisfied with the results, while the SMEs are not. This seems to be true in particular for those SMEs that have limited R&D capabilities of their own because they not always have a realistic view on the risks of R&D and on the complexity of translating research results into exploitable products/services/processes Dissemination and training activities by IAGs Under the Collective measure, the objective is that the results obtained are disseminated to the wider community of SMEs. Since these activities typically take place at the end of the projects and many collective projects are not completely finished yet, we have only partial results. The multiplier effect for instance could not be really evaluated yet. Seminars and workshops are the most popular dissemination activities but conferences and fairs reach more SMEs. 61

62 Figure 16: Dissemination and training activities by IAGs (a) Seminar Workshop Company Visit Conference Fair In-company training (Press) articles, Reports, Manuals 0% 20% 40% 60% 80% (a) Did your organisation set up one of more of the following dissemination or training activities in the context of the project? n = 28. Source: IDEA Consult on the basis of survey results IAGs organised mostly seminars and workshops in order to disseminate the information to their SME-members: around 65% of IAGs indicated to have organised such activities. According to more than 1 out of 2 IAGs, they reached between 10 and 50 SMEs during these gatherings. IAGs who organised conferences, fairs or distributed written information (via articles, reports and manuals) generally succeeded in reaching more SMEs in comparison to the other activities Effectiveness of project implementation On average, respondents rated the effectiveness of the overall project implementation in between limited and very effective. Of all factors, the exploitation of results of the project on the market is rated the least effective factor. 62

63 Figure 17: Effectiveness of project implementation according to the SMEs (a) Coordination project by the coordinator Transnational cooperation in the project Working relations with other partners in project Knowledge transfer between RTD performer and other project partners Overall project implementation Monitoring and review process by EC Dissemination activities The training of staff on take-up of results Access and use of IPR Exploitation of results of project on the market Not ef fective Limited Very ef f ective (a) How do you rate the effectiveness of...? n = up to 227. Source: IDEA Consult on the basis of survey results On average, SME-respondents rated the effectiveness of the overall project implementation in between limited and very effective. Of all factors, the exploitation of results of the project on the market is rated the least effective factor (on average and in total), which is a noteworthy finding. On all of the dimensions of project implementation that we inquired, the SMEs rank effectiveness lowest. Similarly, on all of the dimensions the effectiveness of project implementation is ranked higher for the projects under the Collective measure compared to the Cooperative measure. Note that we checked for a difference between finished and ongoing projects for the 4 lowest ranked dimensions of project implementation. The hypothesis was that participants in finished projects would be more positive on these dimensions that relate to activities which will typically take place more towards the end of the project. However, the opposite is true Intellectual property rights 90 % of the respondents indicated that there was a specific consortium agreement on IPR. 30,5% of the respondents indicated that the project resulted in the creation of (or the access to) IPR new to their organisation. The exploitation of it is however not always evident. Table 6: Intellectual property rights SME IAG RTD Total % having a specific consortium agreement 86,7% 88,5% 95,0% 89,7% n % for which project lead to creation of new IPR 32,5% 54,5% 22,7% 30,5% n Source: IDEA Consult on the basis of survey results 63

64 90 % of the respondents indicated that there was a specific consortium agreement on IPR (see Table 6). This seems to be a high number but it is not clear however whether the default regime 31 agreement to be concluded in the context of the project contracting, has also been considered by the respondents as a specific agreement. Also in the context of the case studies, 42 out of the 45 cases mention a specific consortium agreement on IPR. But our experience indeed is that participants, in particular SMEs, do not know the details of the default regime. In total, 30,5% of the respondents indicated that the project resulted in the creation of (or the access to) IPR new to their organisation. A much larger proportion of the IAGs indicated this was the case for them. Again we checked for a difference between the ongoing and finished projects. We found out, surprisingly, that the mean total percentage for ongoing projects (38%) is higher than for finished projects (29%). In the cross case analysis, the proportion of cases in which new IPR was created as a result of the project amounted to 38 % (16 out of the 42 cases for which this information is available). The detailed results of the survey on IPR further indicate that: The IPR was in most projects assigned to more than one of the partners (joint ownership); The IPR was in most projects owned by one or more of the SMEs, but in a considerable number of projects also by the RTD performers, the latter being a surprising result under the SME-specific measures. In 3 cases, the new IPR has already been exploited. In 10 other projects, this has not been the case yet, but interviewees expected it to be the case in the future. Out of the 38 cases where the issue was explicitly addressed, there was only one case where IPR was really a problematic issue. Regarding Intellectual Property Rights, various issues were raised in the context of the case-studies. The most recurrent issues were the following: - A thorough discussion of the IPR configuration (including a discussion on the various types of IPR) at the beginning of the project, leading to a clear IPR arrangement, is an important pre-requisite for a fair implementation of the project. - Ownership for all the IAGs involved (in the case of collective projects) or all the SMEs involved (in the case of cooperative projects) is not always ideal since not all partners have the same interest in the development and exploitation of IPR; - The collective sharing of IPR may make the exploitation afterwards more difficult. For example, in one of the cases we investigated, an SME reported that at the end of the project it is difficult to carry on the participation from all the SMEs in taking a product to the market. The SME felt that attempting to include all the original SME partners would delay commercialisation of proven prototypes. 31 This refers to the situation where all the IPR developed in the context of the project belongs to the SMEs (in the cooperative projects) and the IAGs (in the collective projects). 64

65 - Patent applications take a considerable period of time, they are expensive and difficult to finance by SMEs, which may cause a delay in the commercialization of project outcomes. More detailed information our findings regarding IPR under the SME-specific measures can be found in section of the Background Report 5.6 Project s effects and impacts Introduction The evaluation results we present in this chapter deal with the effects and impacts that the project may have had on the whole organisation, in particular the SMEs, and in a sustainable manner (i.e. leading to a structural adaptation of the whole organisation). The first evaluation results are about changes in the R&D behaviour and activities of the SME-respondents, in particular changes in the degree of R&D formalisation, R&D capabilities and R&D cooperation behaviour. A second series of conclusions is about the impact of project participation on R&D and in particular economic performance variables of the SMEs involved. In order to evaluate the impact of project participation (cfr. the problem of attribution), we systematically compare the situation before project participation and today and between the experimental and the control group Change in R&D-formalisation and capabilities There has been an increase in the degree of R&D-formalisation among the participating SMEs, in particular among the SMEs under the Collective measure with the increase in the number of SME-participants with a yearly R&D budget differing significantly from the control group. Figure 18: Change in degree of R&D formalisation SMEs (a) 32 More detailed information can be found in section 6.5 in the Background Report. 65

66 (a) T0:Before the start of the project (EXP) /the submission of the project proposal (CON) was there a... in your organisation?/t1: Is there today a...in your organisation? The figure denotes % Yes to this question. Source: IDEA Consult on the basis of survey results There has been an increase in the degree of R&D-formalisation within the participating SMEs: in the experimental group, when looking at the total numbers, there was a slight increase (+5%-points) of SMEs having a separate yearly R&D budget as well as of SMEs having a separate R&D department (see Figure 18). The main difference between the SMEs under the Collective and Cooperative measures is that the percentage of SMEs with a yearly R&D budget increased more markedly in the Collective group (+11%-points) than in the Cooperative group (+4%-points). We checked by means of significance tests, whether the progress made with regard to R&D formalization in the experimental group was significantly larger than the progress made in the control group. We found that the percentage of SMEs in the experimental group who declared an improvement in its R&D budget 33 was significantly larger than the similar proportion in the control group. With regard to the percentage of SMEs with a separate R&D department, there was no significant difference between the experimental and the control group. The increase in the self-declared R&D-capabilities of the SMEparticipants is significantly larger than for the SMEs in the control group. Figure 19: Change in self-declared R&D capabilities SMEs (a,b) Very High High Medium Low Very low 0% 10% 20% 30% 40% 50% T0 T1 (a) T-0: Overall, how do you rate the R&D capabilities of your organisation before the start of the project? n= 236. (b) T-1: Overall, how do you rate the R&D capabilities of your organisation today? n = 227. Source: IDEA Consult on the basis of survey results On average, the SMEs in the experimental group rate their R&D capabilities today higher than before the start of the project (see Figure 19). On the basis of the 33 improvement = an SME who responded no yearly R&D budget at T0 and a yearly R&D budget at T1 66

67 survey results, there are no big differences in this increase by type of measure (Cooperative versus Collective), nor between coordinators or partners. The respondents in the control group also rate their R&D capabilities today higher than at the time when they submitted the project proposal. The increase is significantly smaller though than that in the experimental group. For the 85 SMEs that were interviewed in the context of the cases, the interviewers estimate is that for 37 of them (43,5%), their project participation had no impact on the SMEs R&D and innovation capabilities, for 25 SMEs (29,4%) there was a low impact, for another 17 (20%) a medium impact and for 6 SMEs (7,1%) a high impact. The impact on the R&D- and innovation capabilities is related with the involvement of the SMEs in the project. Interviewers assessed the impact to be highest for coordinating SMEs and lowest for SME-partners that were only involved in the dissemination phase. Overall, it seems that there is a positive impact on the R&D and innovation capabilities of the SME-participants which can be attributed to project participation. Project participation has not, or only to a limited extent, led to more outsourcing of R&D by the SMEs. Compared to their R&D behaviour before project participation, there are today significantly more SMEs that (a) outsource R&D, (b) conduct or participate in R&D projects funded at national or regional level and (c) conduct or participate in a project funded by the R&D framework Programme. However, in the control group, the SMEs also indicated to have, on average, experienced a positive change in these types of R&D activities. When comparing the progress made in the experimental group with that of the control group, we found no significant differences between the two groups. This means that improvements experienced by the SMEs in the experimental group cannot be attributed to project participation. In the context of the cases we focused on the question whether the SMEs today outsource R&D. 40 out of the 78 SMEs do so occasionally (51,3%) and another 19 or 24,4% even do so regularly. But a large majority of 53 interviewed SMEs (68,8%) indicated no relationship at all with project participation while only 5 SMEs said their outsourcing behaviour had changed to a large extent as a consequence of project participation (i.e. after project participation they outsource more R&D than before project participation). Project participation also changed the R&D and innovation related activities of the IAGs and in particular their involvement in projects funded within the context of the EU R&D Framework Programmes. 67

68 Figure 20: Change in R&D activities IAGs (a, b) (a) T0 = Before the start of the project, did your organisation...? n= up to 28. (b) T1 = Does your organization today...? n= up to 28. Source: IDEA Consult on the basis of survey results Compared to the change in R&D activities of the SMEs, participation in the SME specific measures seems to have had a bigger impact on the IAGs, although we must be careful because of the overall limited number of respondents. But for all the activities listed in Figure 20, except the last one, we noted a significant increase in the mean value for the participating IAGs. The largest increase relates to the participation in a project funded within the context of any EU R&D framework programme. For this activity as well as for the last one, this increase is moreover significantly different from the control group for which, in fact, the mean value for these activities decreased (Dis)continued collaboration The SMEs are most likely to continue cooperating with the domestic universities and research institutes they teamed up with. The partnerships with the other types of partners are much less likely to be continued under the SME-specific measures compared to the partnerships that are set up under the FP6 thematic programmes. 68

69 Figure 21: Discontinued and continued collaboration by SMEs (a) Domestic Firms Domestic firms Other SSM DISC DISC CONT CONT Firms from abroad Firms from abroad Other SSM DISC DISC CONT CONT Domestic Universities/ Research /research Institutes institutes Other SSM DISC DISC CONT CONT Universities/ Research Institutes stitutes from from abroad abroad Other SSM DISC DISC CONT CONT 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% (a) Did the cooperation continue /Do you expect the cooperation will continue after the project was/is completed? SSM: n = up to 199, other: n = up to 308. Source: IDEA Consult on the basis of survey results The SMEs are most likely to continue cooperating with the domestic universities and research institutes they teamed up with in the context of the project. These were also the partners they knew from earlier cooperation. More in general, the SMEs are more likely to continue cooperating with domestic partners, which can probably be related to their limited R&D-capabilities implying that international R&D cooperation is not a realistic option. Further analysis shows that: A more intensive involvement in the project leads to more partnerships being continued after the finalization of the project; The higher their degree of R&D formalisation, the more likely the SMEs continue cooperating with their project partners afterwards. When we compare our results with those of the survey with participants in the other funding measures, we notice that the partnerships under the SME-specific measures are much less likely to be continued than under the other FP6 funding measures: the percentage of discontinued partnerships (see Figure 21) is much higher for the SME-specific measures than for the other FP6 funding areas, especially for foreign partnerships (i.e. with partners from abroad). In the context of the cross case analysis we checked in particular whether the interviewed SMEs continued to cooperate with one or more of the foreign consortium partners. 39 SMEs (54,2 %) did so compared to 33 (45,8 %) who didn t, a result which is in line with the survey results. The IAGs also continued their cooperation mostly with the domestic universities and research institutes as well as with the domestic firms they teamed up with. But their cooperation with IAGs from abroad also continued in most cases. 69

70 Figure 22: Discontinued and continued collaboration by IAGs (a) (a) Did the cooperation continue /Do you expect the cooperation will continue after the project was/is completed? n = up to 27. Source: IDEA Consult on the basis of survey results Looking at the results in Figure 22, it seems that more IAGs have continued cooperating with their partners afterwards compared to the SMEs. They most regularly continued cooperating (or are likely to do so) with the domestic firms and the domestic university or research institute. Cooperation was only occasionally continued with the domestic IAG 34. Again we see that also the IAGs continued cooperating more with domestic partners compared to foreign partners (except for the partnerships with other IAGs) Application to FP7 SME application to FP7 is positively linked with the reach of the project objectives but participants to the SME-specific measures are more reluctant to re-apply to FP7 than participants in other FP6 measures. 34 But note that we had only 13 IAG-respondents, the lowest number, that cooperated with another IAG from the same country. 70

71 Table 7: Application for FP7 by SMEs (a) EXP CON SSM Other SSM Other Yes 38,2% 61,3% 43,3% 63,21% n "Research for SMEs" 53,8% 43,6% "Research for SME associations 9,0% 12,8% Thematic programmes 52,6% 64,1% Other 16,7% 10,3% I don t know 9,0% 7,7% No, but we plan to apply 16,7% 13,8% 14,4% 12,3% n No, and we don't want to apply 21,1% 5,4% 12,2% 7,6% n No, and we don't know yet whether we will apply 24,0% 19,5% 30,0% 17,0% n (a) Did your organisation apply for funding under FP7? If yes, did your organisation apply for FP7- funding under? Multiple answers possible Source: IDEA Consult on the basis of survey results 38,2% of the SMEs in the experimental group already applied for funding under FP7 and another 16,7% plans to do so (see Table 7). 21,1% is sure not to apply for FP7, while 24 % doesn t know yet. Those that applied already, did so mainly under the SME-specific measure research for SMEs and (about the same number) under the thematic programmes. The results for the control group are a little different and in a surprising way (but based on a smaller number of observations). There are more respondents in the control group that already applied for funding under FP7 (43,3 % compared to 38,2%) and there are less respondents that are sure not to apply for FP7 funding (12,2 % compared to 21,1%). When we compare our results with those of the survey with participants in the other FP6 measures, we see that the application to FP7 is much higher for the respondents that participated in the other (non-sme specific) measures of FP6 than for those that were active in the SME-specific measures (61% compared to 38% in our survey for the experimental group). This indicates that participants to the SME-specific measures are more reluctant to re-apply to FPs than participants in other FP6 measures. Further analysis shows that SMEs in (partially) successful projects are much more likely to have applied for FP7 or to apply in the future, compared to SMEs participating in projects where the project objectives were not reached at all. Or inversely, SMEs participating in projects where the project objectives were not reached at all are much more likely not to have applied for FP7 and not planning to apply either compared to SMEs in (partially) successful projects Impact on R&D and economic performance For half of the listed R&D and economic performance variables, the trend is significantly more positive for the SMEs in the experimental group than in the control group. Moreover we find that finished projects lead to a 71

72 significantly more positive trend in a number of variables in comparison to ongoing projects. These survey results suggest a possibly important economic impact of participation in the SME-specific measures but the low scores on attribution make it difficult to draw strong conclusions here. 72

73 Table 8: Trend in R&D and economic variables for SMEs (a,b,c) EXP CON COLL COOP TOTAL COLL COOP TOT trend 1 attribution 1 trend 1 attribution 1 trend 1 attribution 1 trend 1 trend 1 trend 1 R&D expenditure (as % of total turnover) +++ 2,37 1,45 2,40 1,64 2,39 1,60 2,29 2,14 2,15 n R&D personnel (as % of total staff) ++ 2,22 1,42 2,29 1,51 2,28 1,49 2,25 2,10 2,12 n Number of contracts outsourcing research activities + 2,22 1,24 2,14 1,30 2,16 1,29 2,00 2,04 2,03 n Number of transnational R&D partnerships with other companies and/or research institutes + 2,16 1,48 2,26 1,52 2,24 1,51 2,13 2,10 2,10 n Number of transnational market oriented partnerships +++ 2,29 1,47 2,29 1,44 2,29 1,45 2,13 2,03 2,03 n Time-to-market for new or improved products or services 2,26 1,30 2,27 1,45 2,27 1,42 2,33 2,25 2,26 n Level of product or service diversification 2,35 1,29 2,51 1,62 2,48 1,56 2,43 2,46 2,45 n Competence level of employees +++ 2,58 1,51 2,61 1,77 2,61 1,72 2,38 2,42 2,41 n Production costs 2,06 1,20 2,21 1,36 2,18 1,33 2,33 2,13 2,15 n Overall productivity level in your organization 2,24 1,34 2,42 1,44 2,38 1,42 2,67 2,33 2,36 n Turnover 2,28 1,24 2,48 1,41 2,44 1,38 2,50 2,39 2,40 73

74 EXP CON COLL COOP TOTAL COLL COOP TOT trend 1 attribution 1 trend 1 attribution 1 trend 1 attribution 1 trend 1 trend 1 trend 1 n Share of export in turnover 2,14 1,24 2,40 1,33 2,35 1,31 2,30 2,29 2,29 n National market share +++ 2,19 1,18 2,47 1,34 2,42 1,31 1,90 2,24 2,20 n International market share 2,12 1,21 2,40 1,35 2,35 1,32 2,30 2,35 2,34 n Total number of employees 2,19 1,18 2,31 1,33 2,28 1,31 2,20 2,38 2,36 n Overall competitiveness of your organization +++ 2,47 1,45 2,72 1,72 2,68 1,67 2,50 2,45 2,46 n (a) What has been the trend in the following variables for your organisation since the start of the project we identified in the beginning of the questionnaire? Please also indicate to what extent this trend is related to your participation in the project. With mean trend: 1= Negative, 2= No change, 3= Positive, and with mean contribution: 1= No relationship, 2= Limited extent, 3= Large extent, 4= Totally (b) The + -signs indicate a statistically significant difference for that variable between the trend for the experimental group (total) compared to the trend for the control group (total). +++ refers to a 1% significance level, ++ to a 5% significance level and + to a 10% significance level. (c) The variables in the first column that are shaded are variables for which the average trend is statistically significantly more positive for the finished projects compared to the ongoing projects. About 75 % of the projects were finished, the other 25 % ongoing. Source: IDEA Consult on the basis of survey results 74

75 The trend in the listed R&D and economic performance variables of the SMEs can be situated somewhere between no change and positive. The trend in the overall competitiveness and the competence level of employees is described as the most positive (mean trend of 2,68 and 2,61 respectively). The number of contracts outsourcing research activities on the other hand has not changed according to the SMEs in the experimental group. But the average SME respondent attributes only a marginal fraction of these trends to their participation in the project: the attribution is in all cases described by the respondents as below a limited extent. When comparing the SMEs under the Collective and Cooperative measures, we notice that nearly always, the trend is described as more positive under the Cooperative measures than under the Collective measures and the attribution is also larger according to the SMEs under the Cooperative measures. The comparison between the experimental and control group shows us that the mean trend in the experimental group is significantly larger than that in the control group for half of the economic variables described in Table 8. This result suggests a possibly important impact of participation in the SME-specific measures on the economic indicators of the participating SMEs. But the low scores on attribution make it difficult to draw strong conclusions here. Further analysis shows that finished projects lead to a significantly more positive trend in a number of variables in comparison to ongoing projects. Respondents that answered the survey for a finished project, also attributed a somewhat larger proportion of the trend in the economic and R&D variables to their project than respondents that were involved in an ongoing project. However, the difference in attribution between the ongoing and finished projects is very small and remains for all projects below to a limited extent. The trends in the R&D and economic variables for the SMEs that were interviewed in the context of the cases are very similar to the ones reported for the SMErespondents in the survey, i.e. in between no change and positive but much closer to no change. The mean values are typically a little lower. Similarly, the extent to which these trends are attributed to project participation according to the interviewed SMEs is also very low with average values closer to no relationship than to to a limited extent. And again, the mean values are a little lower for the interviewed SMEs compared to the SME-respondents in the survey. 35 % of the SMEs experienced a positive change in their overall economic standing following their project participation, 60% experienced no change. The overall change in economic standing correlates positively with the overall reach of the project objectives. 75

76 Figure 23: Overall change economic standing SMEs as a consequence of participation (a) 80% 70% 60% 50% 40% 30% 20% 10% 0% Negative change No Change Limited improvement Large improvement Very large improvement COLL: SME members of IAGs COOP COLL: SME core group members TOTAL (a) Overall, has there been a change in the economic standing of your organisation as a consequence of your participation in the project? Source: IDEA Consult on the basis of survey results 35 % of the SMEs experienced a positive change in their overall economic standing following their project participation: 30,5 % indicated a limited improvement, 4,3% a large improvement and 0,5% a very large improvement. 60,5 % of the SMEs experienced no change in their overall economic standing following their project participation (see total in Figure 23). Most other participants indicated a limited improvement. On average, the trend in the economic standing of the SMEs interviewed in the context of the cases was nearly equal compared to the SME respondents in the survey. The distribution is somewhat different with in particular more SMEs in the cases that indicated they experienced no change (75% compared to 60,5%). Further analysis showed that: SMEs as project coordinators seem to have gone through a relatively larger change in economic standing compared to the SMEs who were partners in the consortium; The overall change for the SMEs participating in the Cooperative measure is more positive than for the SMEs in the core group of the Collective measure, but the difference is small; The overall change for the SMEs participating in the completed projects is more positive than for the SMEs participating in the ongoing projects, but the difference is small. The overall change in economic standing correlates positively with the overall reach of the project objectives; for instance, only 33 % of the SMEs that participated in projects that reached their project objectives totally, indicated no change, while 62% of them mentioned an improvement. IAGs are more positive about the change in the economic standing of their SME members as a consequence of the Collective projects compared to the SMEs that participated in the core group of the Collective projects (see Figure 23. Our explanation for this counterintuitive result is that, like we observed 76

77 before, SMEs are more critical about the project outcomes and effects compared to the IAGs. Using AMADEUS data, we find no evidence for an impact of project participation on the economic performance of the SME-participants 35. We also used the AMADEUS database with economic and financial data on millions of EU companies, to evaluate the impact of the SME specific measures on the participating SMEs. We tried to identify potential links between companies economic performance and their FP-participation, a pioneering exercise in the context of the evaluation of the Framework Programmes. We find no evidence for a positive impact of FP participation on the economic performance of the companies, not on the basis of our univariate analysis, nor by applying multivariate regressions. While one should keep in mind the limits of this pilot exercise, this basic result is in line with the conclusions we derived from the survey and the case studies. From the cases we know that many projects would benefit from a follow-up project to translate the technological results into potential commercial products or process improvements. This may explain the limited economic impact or project participation up to now. What appears from the cases is that often the project does not result in commercial outputs, as we noted already before. In many projects new technologies have been developed, or are at least close to development but this has not been translated yet into potential commercial products or process improvements. In those cases where at the end of the project, the consortium is close to commercialization of the project output, the partners often express their need for follow-up of the project. A lack of resources and funding mainly hinders this follow-up from within the consortium itself. The following quote from an SMErespondent nicely illustrates this: additional work is needed to increase the commercial quality of the new developed product and to develop a large scale manufacturing process. The necessary work could be performed only with the participation of all members of the consortium. This has, however, not been done at present because the SME can t provide the financing alone. The SMEs often link the success of the project to the outcome of a commercial product. The question is of course if this is a fair criterion. There is indeed an inherent and unavoidable contradiction in se in the configuration of the SMEspecific measures between the pre-competitive nature of FP-funding on the one hand and the need to foster commercial and competitive development on the other hand Other effects and impacts of the project SME participants agree that project participation results in increased scientific and technological knowledge and improved network abilities. 35 The economic performance comparison on the basis of Amadeus data is described in full detail in Chapter 7 of the Background Report. 77

78 These impacts are larger for the SMEs under the Cooperative measures compared to the SMEs in the core group under the Collective measures. Table 9: Other effects and impacts of the project on the SMEs (a,b) Total COLL COOP The project helped to increase the scientific and technological knowledge of my organisation's 3,7 3,4 3,77 personnel***,++ The project helped to increase my organisation's ability to network with universities or public research institutes*** The project helped to increase my organisation's ability to network with other firms*** The project helped to increase my organisation's ability to look for a solution to problems through innovation*** The project helped to increase my organisation's ability to be up to date with the latest innovative solutions in our sector*** 3,62 3,65 3,61 3,58 3,5 3,6 3,46 3,38 3,48 3,46 3,28 3,5 The project helped to increase my organisation's ability to develop innovative products or services***,++ 3,4 3,08 3,47 The project encouraged my organisation to look for public research funding more regularly*** The project helped to increase my organisation's ability to buy in scientific and technological knowledge from outside the organization 3,3 3,18 3,33 3,01 2,94 3,02 The project helped to increase my organisation's R&D infrastructure ++ 3,01 2,92 3,03 (a) To what extent to do you agree with the following statements about the expected effects and impacts of the project within you organisation/on the SME-members of your organisation? With mean: 1= Completely disagree, 2= Partially disagree, 3= Neutral, 4= Partially agree, 5= Completely agree (b) The * -signs indicate a statistically significant difference from the neutral value for that variable. *** refers to a 1% significance level, ** to a 5% significance level and * to a 10% significance level. The + sign refers to a statistically significant difference between the answers from the SME participants in the Collective versus the Cooperative measure. +++ refers to a 1% significance level, ++ to a 5% significance level and + to a 10% significance level. Source: IDEA Consult on the basis of survey results The other effects and impacts that SMEs experienced due to their project participation are ranked in Table 9 from high to low. Overall, the SME respondents agreed mostly with the statements that the project helped to increase the scientific and technological knowledge of their personnel and improved their network abilities, both with research partners as well as with other companies. On the other hand, on average the project did not help to increase the SME s R&D infrastructure or its ability to buy in scientific and technological knowledge. We tested whether the answers of the SMEs were significantly different from the neutral answer: we found that the first seven impacts described in Table 9 were indeed significantly different from the neutral value, i.e. 3 (denoted by *). When comparing the SMEs financed under the Collective (the core group members) and Cooperative measures, we note that the impacts for the SMEs under the Cooperative measures seem to be larger than those for the SMEs in the core group under the Collective measures. The difference in means between the two groups is significant in the case of three impacts (denoted by + in Table 9). 78

79 Within the Collective projects we also compared the impacts on the broader group of SME-members of the IAGs 36 with those on the SME core group members. On average, the SME members of the IAGs seem to have experienced larger other impacts than the SME-members that were part of the core group 37. This result is counterintuitive since the SME core members are the first beneficiaries of the project outcomes. Our explanation is that SMEs are more critical about the project outcomes and effects compared to the IAGs. Further analysis also shows that completed projects seem to have had larger other impacts than still ongoing projects. The cases underline the positive impact on the networking abilities and the scientific and technical knowledge of the SMEs personnel but also indicate an improved market insight, increased visibility and project management skills as a positive consequence of project participation. The findings of the survey are broadly confirmed by the qualitative reading of the case studies. In the case studies, networking is nearly always mentioned as a major benefit of participation to the SME-specific measures. Generally, international cooperation is viewed positively, because it allows to source knowhow internationally or to work with other European SMEs that share the same R&D interest without being a direct competitor. However, the case studies have also shown that international cooperation has a downside with participants struggling to communicate in English. The geographical spreading of different project participants also poses practical constraints to the organization of project meetings. As in the survey, increased knowledge is also brought up as an important impact of participation to the SME-specific measures. The case studies allowed us to specify more impacts, not touched upon in the survey. Firstly, market insight is an impact frequently referred to in the case studies, encompassing an increased understanding of competitors and potential partners and an improved insight into customer needs. Secondly, increased visibility thanks to participation in a European project also came up regularly. Finally, the projects have also increased the project management skills of the participants involved: the improvement in these management skills is considered as potentially very valuable for future international cooperation projects. The IAGs also mention an important positive impact on their networking abilities as well as on their ability to disseminate technological information to the SME members of their organization according to these IAGs This result is in line with the exercise conducted above regarding the difference between the SME members of the IAG and the SME members of the core group with respect to the impact on their economic standing. 79

80 Table 10: Other effects and impacts of the project on the IAGs (a) The project helped to increase my organisation's ability to network with universities or public research institutes** Total n 28 The project helped to increase my organisation's ability to disseminate technological 3,96 information to the SME members of my organization*** n 28 The project helped to increase my organisation's ability to network with other industrial 3,93 organisations or groupings*** n 28 The project encouraged my organisation to look for public research funding more 3,68 regularly*** n 28 The project helped to increase my organisation's ability to buy in scientific and technological 3,44 knowledge from outside the organization n 25 4,00 (a) To what extent do you agree with the following statements about the (expected) effects and impacts of the project within your organisation? Mean with 1=completely disagree, 2=Partially disagree, 3=Neutral, 4=partially agree and 5=Completely agree Source: IDEA Consult on the basis of survey results On average, the IAG respondents partially agreed that the project had helped to increase their ability (1) to network with universities or public research institutes, (2) to disseminate technological information to their SME members and (3) to network with other IAGs. All answers regarding these impacts are significantly different from the neutral value with the exception of the last impact noted in Table 10. The latter means that there is no strong support for a positive impact on the IAG s ability to buy in scientific and technological knowledge from outside the IAG Benefit-cost ratio At the end of a series of questions on the effects and impacts of project participation on their organisation, the respondents in the experimental group were also asked to compare, overall, the cost and benefits of participation in their project. The results are given in Figure

81 Figure 24: Benefit-Cost ratio (a) Benefits much higher than costs Benefits higher than costs Benefits equivalent to costs Benefits lower than costs Benefits much lower than costs 0% 10% 20% 30% 40% SME IAG RTD (a) Overall, how do you compare the costs and benefits of participation in this project? Source: IDEA Consult on the basis of survey results On average, project participants evaluate the benefits of project participation to be slightly higher than the costs. However, there are large differences by type of respondent: for the RTD-performers, the benefit-cost ratio is the highest, while for the SME-participants, it is the lowest. The total average answer indicates that the participants evaluate the benefits to be slightly higher than the costs. However, there are large differences by type of respondent. For the SME participants, the benefits are on average slightly lower than the costs. For 1 out of 3 SMEs, the benefits were (much) lower than the costs, for 1 out of 4 the benefits were (much) larger than the costs. For the RTDperformers, the benefit-cost ratio is the highest with 51 % of them indicating that the benefits are (much) higher than the costs. This result is in line with what we observed before, i.e. that RTD-performers are more positive about the reach of the project objectives than SMEs. A further explanation is provided by the fact that RTD-performers receive 100% funding. The opinion of the IAGs on the benefit-cost ratio is somewhere in between: on average, IAGs rate the benefits equal to the costs. Further analysis showed that: There is no significant difference between the benefit-cost ratio evaluation of the participants in the Collective projects compared to the Cooperative projects, nor between finished and on-going projects; The SMEs that indicated that the benefits were (much) higher that the costs experienced more positive impacts on their R&D and economic standing than the overall group of SMEs in the experimental group; also the average attribution of these trends to project participation is larger for the SMEs that considered the benefit-cost ratio positive than for the whole group of SME respondents (in the experimental group); Increased partner involvement positively affects the benefit-cost ratio. 81

82 5.7 Success factors 38 Based on the cases, the following factors contribute positively to the success of a project: Commitment of the partners Nearness to the market of what is being developed A critical amount of EC-funding for the non-research partners Regular and face-to-face interaction between the partners Flexibility to adopt a project in line with findings Clear IPR-arrangements from the beginning Strong leadership by the coordinator SMEs with the capacity to absorb the research results, RTDperformers that are (private) client oriented and focus on exploitation at an early stage in the project are factors that seem to have a positive impact on the chances of achieving commercial product/service/process results from the project output. Commitment of the partners The commitment from partners is what makes a project. All project partners need to deliver what has been agreed upon, even if this contribution to the project is overall limited, as is the case for most SMEs. Complementary expertise between the partners which therefore have a specific role to play in each stage of the project implementation seems to be supportive to acquire full involvement of all partners. However, expectations about the level of commitment are not always clear from the beginning, in particular for the SMEs. This may be the consequence of the fact that RTD-performers very often take the lead in proposal preparation and need to convince SMEs to join the consortium. This sometimes happens on the basis of the argument that it will not take too much time from the SMEs. If, at the project start, it becomes clearer for these SMEs what the expectations are and what budget is available to finance their own efforts, some withdraw. Nearness of the market of what is being developed To obtain commitment from in particular the SME partners in a project, it is important that what will be developed is sufficiently near to the market. This is an important issue, both in the genesis of the project as well as during the execution of the project. It is important that proposals submitted for funding are really driven by the needs of the SMEs. The ambition between the SMEs and the other types of partners, in particular RTD-performers, need to be compatible in this respect. But also during the execution of the project, it is important that the research activities remain on track, i.e. guided by the market-oriented interests of the SMEs. This critical point however may be in contradiction with (or hampered by) the pre-competitive nature of FP-funding. 38 More detailed information can be found in section 6.9 in the Background Report. 82

83 A critical minimum amount of EC-funding for the non-research partners Based on the cases as well as on some of the additional interviews, for instance with national contact points, a too high level of EC-funding outsourced to the RTD performers limits the involvement of the SMEs. According to various interviewees, this issue is important to explain the level of commitment of SMEs. In particular, a high level of outsourced funding (compared to the total funding available) may imply that not enough money is left for the SMEs to be sufficiently committed and involved in follow-up and leadership of the project. Many IAGs also complain about the level of funding being too low to coordinate a collective project properly. Regular and face to face interaction between the partners Also contributing to commitment are regular and face-to-face meetings between the partners. In many of the cases these are mentioned to be vital in ensuring an effective project, not only to transfer knowledge and exchange ideas, but also as an instrument for social control. Flexibility to adopt a project in line with findings R&D-projects are by definition risky business since the outcomes will not be clear from the beginning. Therefore, when a project develops and intermediate findings show that the intended results will not be achieved, it is important that the project is adopted in a flexible way and in line with the modified objectives. A thorough discussion of the IPR configuration between all partners at the beginning of the project, resulting in a clear IPR arrangement, is an important pre-requisite for a fair implementation of the project. A good, detailed definition of IPR rules ex-ante is important so that everybody plays a fair game. All IPR aspects should be carefully and exhaustively considered, discussed and defined. While the EU model of IPR ownership is used in the application process, in some projects this arrangement was afterwards modified by internal agreement among the partners. The modified internal arrangement allowed for more flexibility and reflected the fact that not all of the partners were interested in holding IPR. Strong leadership by the coordinator The role of the coordinator seems to be very important, in particular because of the international cooperation which makes the projects more complicated. The lack of a strong leadership and management by the coordinator has been the cause for the delay or low effectiveness, in a number of cases. This also relates to the final outcomes (or the lack thereof) of a project: the coordinator needs to have a commercial drive. SMEs with the capacity to absorb the research results, RTDperformers that are (private) client oriented and an early focus on exploitation are factors that seem to have a positive impact on the chances a commercial product results from the project output. These factors were derived by screening the cases of projects that resulted in a commercial output. Note however that the identification of these factors is based on a limited number of cases (4 cases). 83

84 Nevertheless, the success of projects seems to be dependent on the manager of the SME. SMEs can be characterized by their capacity to absorb (or not) new knowledge resulting from R&D. This often comes down to the owner of the small SMEs. A suggestion from the interviewees was to better assess the experience of the lead SME in managing research projects. Another suggestion that was made is to favor the participation of strong and efficient R&D partners with criteria such as received at least 30% of their funding from the private sector since at least 3 years or equivalent experience. Another interviewee made the suggestion that all projects should have someone from the IPR Helpdesk or hold an exploitation seminar at their kick off meeting as exploitation is such a long process, projects need to be thinking about IPR arrangements very early on. Early focus on the exploitation of results might also be reinforced by involving the SMEs more in contract negotiations 5.8 Collective versus Cooperative projects 39 Based on the results from the survey and from the case-studies, Table 11 below shows a few key relative characteristics or differences between the Collective and the Cooperative Measure. 39 More detailed information can be found in section 6.7 in the Background Report. 84

85 Table 11: Relative characteristics of Collective versus Cooperative Measure Focus Nature of research Link to the market Size, duration SME involvement ERA Sector Profile Target group (size) Target group (R&D capabilities) Target group (R&D cooperation experience) Match between project and SME s own objectives Impacts (R&D formalisation) Impacts (R&D and economic trends) Impacts (Other) Cooperative Scheme Strong focus, rather specific for some SMEs Problem-solving research, pre-competitive Closer to the market and closer to individual needs of SMEs Smaller, shorter, less participants Stronger involvement in both project preparation and execution Weaker contribution to structuring the ERA Relatively more SMEs from manufacturing sector Relatively more Medium- Size Enterprises (min capacity for follow-up) Relatively more SMEs with medium or high R&D capabilities Participating SMEs more experienced in R&D cooperation, especially abroad Slightly lower Weaker increase in R&D formalisation Limited but slightly larger More important and slightly larger Source: IDEA Consult on the basis of survey and case-studies results. Collective Scheme Generic, common to whole sector Problem-solving research, precompetitive, to some extent also pre-normative research Further away from market and further away from specific needs of SMEs Longer, more participants, higher co-ordination costs Weaker SME involvement in both project preparation and execution, even for the SME core group Stronger contribution to structuring the ERA Relatively more from primary sector Relatively more Small Enterprises (no or less capacity for follow-up) (core group) Relatively more SMEs with low R&D capabilities Participating SMEs less experienced in R&D cooperation Slightly higher Stronger increase in R&D formalization (catching-up) Limited and slightly weaker More important and slightly weaker The following main conclusions can be drawn: Collective projects are considered by SME participants as not as close to the market as the cooperative projects. Linked to the larger scale of the project, participants also report higher coordination costs and a relatively lower level of involvement by SMEs in both the conception and development of the idea and the actual project implementation, compared to Cooperative projects. One of the advantages of the Collective research scheme is that it fosters the pan-european inter-connectedness of national / regional / local pockets of collaboration (even more than the cooperative scheme). Cooperative and Collective projects attract different, distinctive profiles of SMEs: the SMEs in the Cooperative projects are somewhat larger, have higher R&D-capabilities and are more experienced with R&D cooperation 85

86 compared to the SME particpants in the core-group of the Collective projects. While the majority of SMEs participating in the Cooperative or in the Collective measure come from the manufacturing sector (57.5% of SME participants overall are from the manufacturing sectors), the cooperative scheme attracts relatively more SMEs from the manufacturing sector (59% of participants) than the collective scheme (51%). Conversely, the collective scheme attracts relatively more SME participants from the primary sector (22%, e.g. aquaculture) than the cooperative measure (10%). The collective measure appears also to attract relatively more small SMEs while the cooperative scheme re-groups relatively more medium-size SMEs. The cooperative scheme requires more involvement from the SMEs in the conception and follow-up of the project, while under the collective scheme this role is partly taken over by an IAG. This may explain why SMEs participating under the cooperative scheme need to be somewhat bigger, i.e. to have a minimum internal follow-up capacity. SMEs participating in the cooperative scheme have also a relatively higher level of R&D capabilities than SMEs participating in the collective scheme (26% of participating SMEs under the cooperative scheme declare having high R&D capabilities and 32% medium R&D capabilities against respectively 17% and 29% for the collective measure). Conversely, 54% of the SMEs participating in the collective measure indicate having low R&D capabilities against 42% in the cooperative measure. Logically, SMEs participating in the cooperative measure show more experience with R&D cooperation (especially abroad) than SMEs participating in the collective measure. Obviously, these differences partly result from a sectoral bias (primary sector has on average lower R&D capabilities than the manufacturing sector), but clearly the two measures attract different population groups. SMEs under the Collective measure seem to be somewhat more satisfied with their project s own objectives than the SMEs under the Cooperative measures. This is not necessarily contradicting the observation that SMEs are relatively less involved in (both the conception and the execution of) the project, since operating intermediaries (IAGs) under the Collective measure are expected to pass on and defend- the needs of their SME members to the RTD-performers. In this regard, various interviewees representing IAGs declared that more attention should be paid to the role and position of the IAGs. IAGs defend the idea that some of them do not have the financial resources to carry out the role that they should carry out (and do not receive enough funding from the EC). Some others are not enough service-provider for the benefit of the whole sector and the representative SMEs. While it is not always clear what is the value added of IAGs in collective projects, in other projects, IAGs did play an important role to bridgethe-gap between the SME members ( we know and understand your needs ) and the RTD-performers, as seems to be confirmed by the somewhat higher correspondence between project s and SME s own objectives. Participating SMEs (in both the Cooperative and the Collective measures) report an improvement of their R&D formalization i.e. they previously did not budget R&D expenditure within the company, after the project they adopt a yearly budgeting of R&D. 86

87 This increase is statistically significantly higher than the control group. SMEs having participated in Collective projects (only core-group) report an even stronger improvement in R&D formalization (budgeting R&D) than SMEs having participated in Cooperative projects. SMEs participating in the Cooperative measures however keep a higher level of R&D formalization: after completion of the project, 57% of the SMEs having participated in the Cooperative measure declare having a yearly budgeting of R&D, against 50% for the SMEs having participated as core-group in a Collective project. In this respect, one can say that Collective SMEs have been catching up with their Cooperative counterparts. The results on the trend in economic and R&D variables of the SMEs reveal limited impacts on the participating SMEs. There are however important other, less tangible impacts like increased knowledge and networking abilities. Changes in overall competitiveness and competence level of employees are described as the most positive. However, the average SME respondent attributes only a marginal fraction of these changes to their participation in the project. When comparing the SMEs under the Collective and Cooperative measures, the trend is for the large majority of the variables described as more positive under the Cooperative measures than under the Collective measures and the attribution to project participation is also larger according to the SMEs under the Cooperative measures. Similarly, SMEs experience a very limited overall change in economic standing attributed to project participation, even though the change is found to be somewhat higher among SMEs having participated in Cooperative projects than for the SME core-group of Collective projects. There are however important other, less tangible impacts such as increased scientific/technical knowledge and networking abilities. The SME-participants under the Cooperative measure agree to a larger extent on these impacts compared to the SMEs in the core group of the Collective projects. Collective projects mainly help SMEs with low R&D capabilities to reach a first level of formal organization of R&D activities. Summarising the available impact data, and even though some caution has to be kept hereon, it seems that Collective projects mainly help SMEs with low R&D capabilities to reach a first level of formal organization of R&D activities, while more R&D-experienced SMEs participating under the Cooperative scheme appear to be more able to translate benefits of participation into further improving the R&D capacity and the economic performance (according to their own judgment). The more specific character of cooperative projects (as compared to the more generic character of the collective projects) explains to some extent the higher level of economic benefits for cooperative SMEs. 87

88 5.9 Monitoring and Reviewing Process 41 The case studies in particular generated insights in the way FP-participants consider the monitoring and review process of their project by the EC services. The most important insights can be summarized as follows: There are many complaints about the limited involvement by the EC and regular switches of scientific officers. The administrative follow-up of the projects was in general appreciated and good, but participants would have appreciated more participation from the EC (e.g. participation in meetings) and a more active involvement from the content point of view. It has been mentioned that in the context of the SME specific measures of FP7, the EC scientific officers are present at some of the project team meetings which is very much appreciated by the project partners. Their presence gives more weight to these meetings and the partners will do their very best to make a good impression. In some cases, documents (reports, forms) had to be re-submitted to the new scientific officer. Monitoring by the EC is generally found to be too limited to purely administrative or legal matters. The administrative follow-up was in general appreciated and good, but participants would have appreciated more participation from the EC (e.g. participation in meetings) and a more active involvement from the content point of view. It has been mentioned that in the context of the SME specific measures of FP7, the EC scientific officers are present at some of the project team meetings which is very much appreciated by the project partners. Their presence gives more weight to these meetings and the partners will do their very best to make a good impression. The overall assessment in the review reports tends to be in line with our own assessments of the success of the projects except for projects that later turn out to be unsuccessful. From the 36 review reports that were made available (for the 45 cases that we conducted), 9 projects received a good to excellent rating. There is a consistency between the review reports evaluations and our own evaluations in that the projects that get a good to excellent project rating in the review reports are also, on average, the more succesfull projects in the context of the cases. Review reports that get an acceptable rating get lower success story scores on average. Review reports however fail to detect projects that appear to be, afterwards, a failure since they assess the implementation of the project based on the Description of Work (annex I to the grant agreement). No project is considered unsatisfactory in the review reports examined under this evaluation. 41 More detailed information can be found in section 6.8 in the Background Report. 88

89 5.10 Relevance and value added of the SME specific measures 42 In this section we summarise the conclusions of the analysis with regard to both the relevance of the SME specific measures (to what extent do they address the needs of the SMEs) and their value added (VA) compared to national/regional/european programmes that have similar objectives (if they exist) Relevance of the SME specific measures These measures are in particularly relevant if they match with the needs of the SMEs. Therefore we first inquired about the needs of the SMEs and subsequently to what extent these needs are addressed by the SME specific measures. The results can be summarised as follows: Based on both the surveys and the case-studies, it appears that financing research and innovation activities is the most important R&D and innovation need for SMEs, followed by access to new scientific and technological knowledge and solving technical problems. This top-3 of needs is the same when we distinguish between the participants in the Collective and Cooperative measure. The two most important needs are also the ones that are the best addressed by the SME-specific measures. Solving technical problems appears to be much less well addressed than the two others. Participating SMEs report that the Cooperative measure addresses innovation needs to a larger extent than the Collective measure. This may be due to the fact that cooperative projects are more specific than collective projects, which usually are of a more generic nature. The level of involvement by SMEs in cooperative projects is also proven to be on average higher than in the case of collective projects. Interestingly, buying in ad-hoc research activities and results, a main objective of the SME-specific measures, gets the lowest ranking both in terms of needs as wells as the extent to which this need is addressed and consistently so for both the participants in the Collective and Cooperative measures. So access to scientific and technological knowledge new to the company ranks higher than buying in research. According to additional interviews with National Contact Points or Innovation Agencies in the Member States, SME-specific measures are very relevant for two types of SMEs: 42 More detailed information can be found in section 6.6 in the Background Report. 89

90 - SMEs without high in-house R&D capabilities that want or need to develop a project in an international context (for instance because the solution to their problem cannot be solved by outsourcing research within the country). - SMEs with higher R&D capabilities that want to complement their R&D activities by buying in additional knowledge assets from a RTD-performer. These SMEs in general all have a high level of business maturity, e.g. they have a good knowledge of their own strengths and weaknesses (not only from a scientific or technological point of view). This allows them moving up to a European project or seeking for additional, adhoc expertise Value added of the SME specific measures To address the value added of the SME specific measures, the respondents in the different categories were first asked about the existence of similar programmes at the national or regional level. Those respondents that confirmed the existence of such programmes, were subsequently asked to evaluate the value added of the SME specific measures compared to these national or regional programmes. The results from the on-line survey, the case-studies and the additional interviews can be summarized and integrated as follows: Respondents are aware of more national programmes that are similar to the Cooperative measure compared to the Collective measure. This answer is similar when making the distinction between SMEs, IAGs and RTDperformers. The common benefits of national programmes compared to the European SME-specific measures (or the FP funding schemes more in general) are: (1) usually shorter time span and thus quicker reaction to changes in the market, (2) easier and shorter application procedures, (3) less administrative requirements and (4) higher success rate (lower competition). It should be borne in mind, however, that the requirement for international cooperation put under the FP SME-specific measures (but similar of course for the other FP funding schemes) induces as such an extra difficulty and complexity to the set up and organization of a project, which may explain the longer application procedure and higher level of administrative requirements. Compared to other relevant policy measures or programmes at the EU, national or regional level, SME-specific measures have a clear valueadded on at least three points: 1. Complementarity SME-specific measures occupy a unique niche within the whole supply of policy measures and therefore complement well other measures at regional, national and EU level. In comparison to Eureka or Eurostar, for instance, the SME-specific measures provide added value since they target rather low-tech SMEs and help them outsource research. Entities participating in Eurostar and Eureka, conversely, 90

91 have high R&D capabilities: Eurostar is specific for high-tech SMEs with a goal to be closer to the market, while Eureka is not an SME-specific measure but is targeting industrial participants more in general. In comparison with the thematic programmes, the more concrete innovation and valorization project objectives are valued, the possibility to outsource R&D and the shorter time to contract, which is however still longer than compared to the national or regional programmes. Thematic programmes have a different target group, namely that of high-tech SMEs. SME-specific measures deal with specific problems or innovation needs of SMEs, to be tackled on short or medium-term and closer to the market, while other EU instruments such as European Technology Platforms (ETPs), Joint Technology Initiatives (JTIs), Joint Programming etc either do not address specific problems of SMEs or deal with challenges or S&T advances in the (very) long term, or tend to structure the ERA. It was also found that SME-specific measures have a clear value added with regard to the existing national and regional programmes, mainly in terms of international cooperation (see below). 2. International cooperation and market expansion One of the major areas where SME-specific measures provide a clear added-value is the access to international cooperation and the widening of the own local market and strategy of the companies beyond regional or national borders. This is a major added-value when compared to national and regional programmes, or with the Structural Funds (which are usually regionally managed). There is indeed mounting evidence that, at the end of FP6, international opening up of regional / national programmes was still very marginal. On the one hand, Member States do recognise the importance of offering their researchers and companies the possibility to engage in cross border R&D collaborations and are therefore supportive of high levels of national participation in international R&D programmes. Frequent mention is made of the need to participate strongly in the Framework Programme and measures aimed at stimulating, directly (e.g. financial support for preparation applications, cofunding of successful participants) or indirectly (e.g. through strengthening centres of excellence, imitation of Framework Programme thematic priorities, establishment of national Technology Platforms or building critical mass), the participation of their own nationals in the Framework Programme are often mentioned by interviewees. On the other hand, some Member States have gone one step further and have introduced the (partial) unilateral opening up of their national research systems as part of their overall strategy, inspired by the view that knowledge spill-overs from abroad can add to the existing domestic R&D capacity, that this can increase its quality through increased competition or can give access to fields in which national capacity is limited. However, a study carried out on behalf of the European Commission in made a number of interesting observations in this respect, leading to the conclusion that the international character of regional / national measures and their opening up towards foreign participation is still very marginal 43 : 43 'Examining the design of national research programmes', December 2005, Optimat Ltd VDI/VDE-Innovation + Technik GmbH 91

92 The funding of trans-national research projects is most commonly observed as an element of opening up. Although a majority (60%) of the programmes surveyed in the study (the study covered 127 national or regional programmes in 34 countries of the ERA) reported having funded transnational projects, the actual budget spent on trans-national activities remains marginal, the majority of programmes remaining below 5% of total budget. Only a small proportion (16%) of programmes report having contributed to multilateral programmes with a central budget. Around two thirds of surveyed programmes allow participation of non-resident researchers from other EU countries. Only 23%, however, have actually paid for the participation of foreign partners and an even smaller number (16%) state that foreign participation is actively encouraged. Bi- and multilateral cooperation between Member States, driven in part by the ERA-NET (European Research Area Network) instrument which was introduced in FP6, are also apparent in many national policies, even though international cooperation is usually restricted to specific research domains (e.g. life-sciences) or to specific projects, such as the coordination of the construction of a joint facility. ERA-Nets may contribute to international opening up and convergence, but their potential should not be exaggerated as they remain composed of national / regional programmes initially designed to reflect national / regional interests. Even though ERA-Net projects are conducted on a larger scale compared to regional programmes, it remains more local than a project carried out under the SME-specific measures. Finally, international R&D cooperation via SME-specific measures can be the first move towards sustained international cooperation, as mentioned many times by interviewees. 3. Bottom-up approach A third and important added-value consists of the fact that the SME-specific measures are purely bottom-up driven, while the majority of other initiatives at the EU, national or regional level have a thematic focus. This allows SMEs, RTDperformers and supporting structures in the Member States (such as the NCPs) building up a research project a la carte for the SMEs involved. Some regional initiatives in the Member States do have a bottom-up orientation, but they miss then the international opening of their EC counterpart An outlook to FP7 Even though FP7 remains outside the scope of the evaluation, this final section regroups results on differences in the SME-specific measures between FP7 and FP6, as they are perceived by SME participants or members of other organisations. It does not include yet policy recommendations on what FP7 should look like (the policy recommendations can be found in the next chapter). The most important differences perceived can be summarized as follows: There is a large confusion among SMEs, RTD-performers, IAGs and even NCPs about the characteristics and advantages of the FP7 funding rules One of the aims of the new funding rules under FP7 is to further stimulate the share of funding outsourced by SMEs to RTD-performers, as well as to limit the 92

93 share of the budget that has to be directly funded by SMEs. Under FP7, on average ca 80% of the budget actually goes to RTD-performers (via the SMEs using the money to buy in R&D activities from RTD performers), while under FP5 / FP6, RTD-performers were expected to carry out at least 40% of the research. A lot of interviewees from various countries and organizations consider that it is not obvious that the remaining budget (20%) is sufficient to ensure an in-depth, adequate involvement by SMEs, while some others believe the funding set up had gone too far in favour of the researchers. It is even expected FP7 projects to fail as partners realise the cost implications. It is crucial that the European Commission, together with the Member States and their supporting structures, promote the real advantages of the new FP7 funding model, in the most simple and transparent way. It is obvious that in a lot of countries (and even among experienced FP-clients ), these rules have not been understood in a correct way. Mixed feelings about the administrative burden under FP7 No more auditing is needed below a certain threshold. This was a problem under FP6 because every partner had to be audited by an external auditor independent of degree of involvement. This delayed the final claims. Paperwork is more streamlined under FP7, more online system possibilities are available, there is less bureaucracy and a better focus on the scientific and technological objectives. The consortium agreement is now concluded before the start of the project. However, some problems still remain. According to some RTD-performers interviewed, there are now inconsistencies in the budgeting and accounting system. For Cooperative projects it is not clear how to invoice the subcontractors and other partners when the RTD budget is distributed by the coordinator and the beneficiary has to invoice the target SME. This assessment from some RTDperformers contrasts sharply with our observation that rules are now clearer for SMEs under FP7. There were relatively much less SMEs complaining about bureaucracy (compared to RTD-performers), but this might be biased by the fact that RTD-performers play on average a more active role in project initiation, proposal preparation and coordination. SMEs from the new member States perceive FP7 (including the SMEspecific measures) as too high-tech, which leads to the exclusion of lower-tech SMEs. Some SME participants from the new Members States find that FP7 tends to focus too much on high tech companies. According to various interviewees from the new Member States, some SMEs consider themselves as not high tech enough, and the threshold to participate to FP7 is too demanding for them. While we do not agree with these statements, certainly not with regard to the FP7 SME-specific measures, this biased perception of the reality was found in many various interviews from new Member States and should be taken seriously by the Commission Services. 93

94 6 POLICY RECOMMENDATIONS 6.1 Introduction The policy tool under investigation in this evaluation study targets SMEs with medium or low in-house R&D capabilities, but with clear innovation needs. It belongs to the family of policy measures that aim at stimulating the creation, diffusion, application and exploitation of new (technological) knowledge among SMEs and between SMEs and knowledge providers, i.e. public research organizations or universities. The basic principle of intervention is that SMEs receive funding to outsource or buy in new knowledge generated by research activities carried out by RTD-performers. The measure is organized under the Framework Programmes for Research and Technological Development (FP) of the European Commission. More in particular, the study assessed the impact of the CRAFT scheme under FP5 and of the Cooperative scheme and the Collective scheme of FP6. In order to assess the impacts of the schemes on the participating SMEs and Industrial Associations or Groupings (IAGs) according to the highest quality standards possible, different, mutually-reinforcing methodologies were combined such as desk research, surveys, in-depth case studies and an economic performance analysis based on a large micro-economic dataset. The evidence collected hereby allows us to formulate the following policy recommendations to the European Commission, which we organized according to three levels: 1) Strategic level (what is the rationale of the policy tool? Is it still pertinent to keep it in the future, and if yes under which form?); 2) Programme level (which type of programme is needed?); and 3) Implementation or project level (how should such a programme be actually designed?). 6.2 Strategic level Do we need an SME dedicated measure organized through the EC Framework programme in the future? Providing an answer to this question requires answering the following key questions: 1) Is there a rationale for offering public R&D support to this type of SMEs (regardless the level of intervention regional, national or trans-national)?; 2) If so, is there a rationale for offering this support at EU level? 3) If so, is there a rationale for offering this support through the EC Framework Programme? 94

95 Recommendation 1: There is a need to keep an SME-dedicated R&D support measure According to a large body of economic literature 44, SMEs deserve specific attention in R&D and innovation policy because they are affected by market failures to a larger extent than large companies: imperfections in the capital market affect them to a greater extent, the cost of regulation is for SMEs relatively higher, SMEs experience more difficulties to appropriate the return from innovation (due to e.g. relatively higher costs of patenting), engaging in R&D is often riskier for an SME than for a large firm because it is disadvantaged in terms of combining complementary assets to R&D etc. Moreover, SMEs represent the very large majority of the industrial texture. Supporting and enhancing their innovative activities can therefore contribute significantly to the realization of the post Lisbon strategies. The population of SMEs is, however, highly heterogeneous and these arguments do not hold to the same degree for all SMEs, which also partly explains the large variety of R&D and innovation policies and policy tools targeting SMEs in the EU. Beyond a small group of SMEs with high-tech in-house R&D capabilities, often involved in leading-edge research, the SME-population mainly consists of a much broader group of companies with medium- or low-tech capabilities, but which nevertheless occasionally develop, apply or acquire new technology (see Background report chapter 2). This broader group of medium- to low-tech SMEs is less technologically advanced, but, because of their sheer size/proportion in the industrial texture, their innovative activities may induce a higher multiplier-effect. From an evolutionary perspective, it also suggests a strategic policy objective, namely to help these SMEs to gradually and structurally develop their innovative capabilities. In this regard, their innovation needs can be met by specific intervention towards technology transfer and dissemination. Although these are clear arguments in favor of R&D and innovation policies specifically targeting medium- or low-tech SMEs, it is less clear at which level of policy governance this kind of support should be offered. Recommendation 2: An SME-dedicated measure for SMEs with limited R&D capabilities is appropriate at EU-level if it addresses (1) a technological need which is common to a community of SMEs that is broader than a single country or region, or where the critical mass needed to address it is not present at the regional or national level only, or (2) a specific technological problem which cannot be solved through the available research capacities in the home country or region, but only by having recourse to research activities abroad Bearing in mind the group of SMEs targeted, one has to recognize that their channels for transferring (absorbing) new (technological) knowledge are often regional, or even local. Some medium- or low-tech SMEs operate at an international level (or tend to), but this is a small minority of the SMEs under consideration. Therefore, the role of supportive structures at regional level is of utmost importance for this target group of SMEs. This would suggest that there are no significant economies of scale in innovation policies for these SMEs and that the regional or national level is the most appropriate level for these policies See Background report, chapter 2. See for instance Lejour, A., Koskenlinna, M., Sluismans, R., Subsidiarity and EU support for innovation, Pro Inno Europe Inno Learning Platform, June 2008, p

96 However, the evidence collected in this study reveals that there are some international needs that can be better met at EU level, suggesting, according to the subsidiarity principle, a clear value-added compared to regional or national policies. Within the target group considered here, intervention at EU level is therefore justified when: 1. It addresses a technological need which is common to a community of SMEs that is broader than a single country or region, or where the critical mass needed to address it is not present at the regional or national level only. 2. When the solution to a specific technological problem cannot be found among the available research capacities in the home country or region, but only in another EU country or region. In those two cases, intervention at EU level is fully justified because it complements the actions of regional or national policy by providing the necessary (higher) critical mass, by providing positive network externalities (such as dissemination, diffusion) that would not materialize at regional/national level, or by providing an competitive advantage at EU level. Recommendation 3: Given the needs it has to respond to and the options to organize it, this SME-dedicated measure is best organized as a Community initiative. The most realistic choice is the Framework Programme for Research and Technological Development, however better integrated (aligned) with the Competitiveness and Innovation Framework Programme (CIP). If one agrees on the fact that such needs are better covered by an SME-dedicated programme at EU-level, the next question to address is then: through which strategic policy tool should it be organized? In theory, there are three options possible. A detailed comparative assessment of these options being far beyond the scope of this study, we mainly mention them as point of departure for further inquiry. 1. The support needed could be offered through existing national / regional initiatives. In almost every Member State, there are national/regional R&D and innovation support programmes targeting SMEs. One could imagine addressing the R&D and innovation needs of low/medium tech SMEs that require a solution to be found by cross border cooperation by opening up these programmes to foreign participants or through bottom-up crossborder collaborative joint programming : several national/regional programmes targeting SMEs linking up with each other. The available evidence used in this study shows, however, that neither mutual opening nor bottom-up joint programming without Community involvement has made much progress so far 46. Thus these do not, at the moment, constitute credible options for meeting the internationally oriented R&D and innovation needs of low/medium tech SMEs. Nevertheless, in view of the 2008 Commission Communication on joint programming 47, Member State efforts in the direction of mutual opening and joint programming deserve sustained support See for instance background report section Communication from the European Commission, Towards Joint Programming in Research: Working together to tackle common challenges more effectively, COM(2008) 468 final,

97 2. The support needed could be offered via existing inter-governmental initiatives or Article 169-like measures. While bottom-up joint programming has not made much progress so far, inter-governmental initiatives (EUREKA) and joint programming initiatives involving the Community (the Article 169 initiative Eurostars) have been established. Yet it is widely acknowledged that EUREKA in its current format has run out of steam 48 while Eurostars targets a different population of SMEs (i.e. research-performing SMEs with established high in-house R&D capabilities). A revamped EUREKA or a Eurostars-bis could be viable options but more in the medium to long term since they would require a very long lead time in terms of policy preparation. 3. As far as Community funds are a valid option to organize the SME-dedicated measure at EU-level, different existing policy tools can be used, such as: the Framework Programme for Research and Technological Development (i.e. the current situation), the Structural Funds, or the Competitiveness and Innovation Framework Programme (CIP). Structural Funds increasingly play an important role in funding Research and Innovation activities in the EU. Investment in research and technological development (RTD), innovation and entrepreneurship has indeed been explicitly prioritised in the current generation of EU Cohesion Policy programmes, as part of a more general focus on investments that contribute to the Lisbon Strategy for growth and jobs ("ear-marking"). Within the field RTD, Innovation and Entrepreneurship ( 58.1 billion for the period ), the largest spending categories are investment in research infrastructure (Code 02) and actual research activities in research centres (Code 01), as well as direct investment in firms, linked to innovation (Code 07) and support for RTD activities, particularly in SMEs (Code 04). The latter category, together with expenditure category 03 (investment in technology transfer and cooperation networks between actors in the RTD system) are likely to contain the activities most closely related to (or with greatest potential overlap with) the projects supported under the current SME-specific measures of the Framework Programmes. Moreover, cross-border, transnational and inter-regional programmes supported by the "Territorial Cooperation" objective introduce a cross-border dimension. However, there is considerable flexibility regarding the precise form of projects supported within individual Operational Programmes (depending on both the OP priorities and the projects brought forward by regional or national actors). Therefore, the generic expenditure categories discussed above remain very general, and analysis of real overlaps and synergies can only happen at regional level, through consideration of individual Operational Programmes and projects supported, in the context of national and/or regional policy 48 Eureka has well-recognised shortcomings: duplication of evaluation and project monitoring procedures at inter-governmental and national level, variable levels and poor predictability of the public funding available, and long delays before starting projects have consistently weakened the effectiveness of the scheme, in: COMMISSION STAFF WORKING DOCUMENT. Annex to the Proposal for the Council Regulation on the setting up of the ARTEMIS Joint Undertaking. Analysis of the effects of a Joint Technology Initiative (JTI) on Embedded Computing Systems [COM(2007)243 final], [SEC(2007) 583], p. 7. EUREKA has probably had an important impact on the technological competitiveness of European industry, although with the exception of clusters playing the role of virtual agenciesit has lost part of its initial impetus over time, in: Papon, P. (2009), Intergovernmental cooperation in the making of European research, in: Delanghe, H., Muldur, U. and Soete, L. (eds.), European Science and Technology Policy. Towards Integration or Fragmentation?, Edward Elgar, Cheltenham (UK) Northampton (MA-USA), 2009 (forthcoming), p

98 frameworks. This also means that any decentralised implementation of an EUwide, cross-border SME-dedicated measure through Structural Funds is likely to be characterised by a great variation in objectives and implementation criteria (target groups, eligibility, financial models, evaluation procedures etc). Compared to this, the FP SME-specific measures represent a unique single entry point for all European SMEs for EU-wide access to S&T capabilities and solutions. Moreover, the "mainstream" Structural Funds primarily provide support for the development of actors in the innovation system (and cooperation between them) within individual regions. Cross-border, transnational and inter-regional programmes supported by the "Territorial Cooperation" objective are less common 49. In addition, the cross-border dimension is usually limited to a couple of (neighbouring) regions; thus it is not EU-wide. Finally, by their nature and type of regions that they target, structural funds very likely focus on funding for equipment, infrastructure and capacity building for knowledge transfer and absorption, while SME-specific measures rather imply new S&T knowledge creation in an international playing field. The Community Innovation Programme (CIP) may be considered as another valid option, given the high business relevance and the relatively high focus on innovation (as compared to scientific and technological focus of SMEspecific measures currently organized under FP7). The overall aim of the CIP is to increase Europe's productivity, innovation capacity and sustainable growth. Programme actions are therefore geared towards creating appropriate framework conditions for competitiveness and innovation. With SMEs as its main target, the Entrepreneurship and Innovation Programme (EIP) of the CIP has been designed to support innovation activities, provide improved access to finance and deliver business support services in the regions. However, there shouldn t be a trade-off between the FP and the CIP, as both instruments target different things (creation and transfer of S&T knowledge versus innovative business development). Instead, one should better exploit the potential synergies between the two instruments to tackle technical and business objectives at once, at least by better channeling projects funded under the FP7 SME-specific measures to CIP for further (innovative) work and business exploitation. Another possibility would be to fully integrate CIP and the SME-specific measures under one single, SME-friendly programme. In our opinion, the most realistic choice among community initiatives is the Framework Programme, however better integrated (aligned) with CIP. Recommendation 4: Therefore, an in-depth debate is still needed on the respective role of the relevant Community initiatives in this respect and on the synergies between them, in particular to better exploit the complementary between the Framework Programme for Research and Technological Development and the Competition and Innovation Framework Programme Taking into account the interdependence and complementarity of regional, national and EU initiatives, keeping the measure under the FPs would require better alignment and vertical integration with the existing regional and national initiatives, as well as a better use of the supportive structures in the Member States. The evidence gathered in this study shows that ca. 75% of the SMEs having participated in SME-specific measures already participated in (regional or national) publicly-funded research programmes before. This implies a strong 49 In budgetary terms, these programmes represent only a small proportion (2.5%) of the RTD, Innovation and Entrepreneurship fields of the Structural Funds. 98

99 interdependence between the various levels of governance and an evolutionary path followed by SMEs which gain first level experience in publicly-funded national or regional R&D collaboration before going over to cross-border collaboration. Such a better alignment is all the more needed because of the emerging new division of labor in the EU Research policy landscape. Since FP6, new instruments such as Era-Nets, OMC-Nets, ETP, Article 169 etc. have characterised a significant shift in EU research policy from a classical grants system (project-based, centralised) to the co-funding of existing national / regional programmes / initiatives. This evolution, however, has in a way added some complexity to the model and has blurred the boundaries of the traditional division of labour between the various levels of governance. A new debate / discussion is needed here, also with regard to the SME-specific measures. Recommendation 5: If the SME-dedicated measures are continued to be organized under the Research Framework Programmes, it would definitely need some adjustments at programme- and implementationlevel, as described in the two following sections 6.3 Programme level According to the evidence collected in this study and the innovation needs of the target group, it appears that the programme should have (keep) the following key characteristics: 1) bottom-up driven, 2) comprehensive, 3) with full recognition of the specific and complementary assets of the Collective and Cooperative schemes, 4) providing funding for all proposals positively evaluated. Recommendation 6: The bottom up character of the measure should be preserved as a programme characteristic. Together with the international cooperation opportunities, the bottom up character was among the most appreciated programme characteristics of the current SME specific measures. Compared to the thematic programmes within the FP as well as most national and regional R&D support programmes for SMEs which, according to our interviewees, are programmed top down or are sector driven, the bottom up character of the SME specific measures is quite unique and seems to fill a gap in the portfolio of SME R&D support measures at EU level. We therefore recommend preserving this characteristic. Recommendation 7: The SME-dedicated measure would benefit from a comprehensive approach in which research activities and business development get better integrated within the same project, including funding for follow up demonstration activities In many successful projects, there remains a gap to be bridged between the outcomes of the project and their potential exploitation on the market. For instance, testing has taken place but on small scale pilots only which is no guarantee for success at an industrial scale, i.e. in the real world. We therefore suggest providing the possibility to setup industrial scale demonstration activities as a follow up stage to the projects. Ideally, such a follow up demonstration activity should be integrated as a possible extension to the project from the beginning so that no new application procedure 99

100 needs to be started up which would result in further delay. A possibility to do so is to introduce kind of a bonus system. Part of the budget (say 5 up to 10%) is conditional on the positive outcome of the project to be evaluated at the end 50. The bonus can only be used for demonstration activities. It further serves as an incentive to get results. One needs to be aware however of the potential perverse effects of such a system, for instance a lower level of ambition of the projects in order to increase the chances to be awarded the bonus. Such effects should be countered during the evaluation of the proposals. Another point of attention is that, since the aim is to scale up the demonstration activities, these can probably no longer be outsourced to the RTD-performers but need to take place within companies (the SMEs involved in the project or even other partners) which changes the context of the project in a substantial way. If a separate procedure needs to be developed, we would in any case recommend making this as simple and short as possible. Recommendation 8: The budget allocations for the Research For SMEs scheme and the Research for SME Associations scheme should be decided independently from one another because they serve different target groups and have different effects. The evidence collected reveals that the collective and cooperative schemes implemented under FP6 attract different types of SMEs and deliver different types of impacts on the participating SMEs. The core group SMEs participating under the collective scheme (so-called core-group ) tend to be smaller and relatively lower tech than the SMEs participating under the cooperative measure. The SMEs in the core group in the collective measures have also a lower level of R&D formalization than their counterparts under the cooperative measures. The higher level of involvement in the research project required by the very nature of cooperative projects certainly plays a role herein. In terms of impacts, SMEs having participated in the collective measures mainly benefit from a first increase in R&D formalization. In this respect one may say that collective SMEs have been catching up with their cooperative counterparts. Conversely, SMEs having participated in the cooperative measures record mainly benefits in terms of research capabilities or in economic terms. In summary, it seems that Collective projects mainly help SMEs with low R&D capabilities to reach a first level of formal organization of R&D activities, while more R&D-experienced SMEs participating under the Cooperative scheme appear to be more able to translate benefits of participation into further improving their R&D capacity and their economic performance. In addition, the collective scheme also differs from the co-operative scheme because it foresees a broader dissemination and transfer of the new knowledge created beyond the core-group of SMEs, towards all SME members of the IAG. Therefore, we conclude that there shouldn t be any trade-off between the two schemes in the budget allocated to them: they both reach different target groups and induce different impacts and appear to be highly complementary as well. Recommendation 9: Support more projects with smaller budgets The increase in the available budget for the SME specific measures between FP5 and FP6 has been absorbed by a more than proportionate increase in the average EC contribution per project while the number of projects funded has decreased. 50 Part of the budget for projects that receive funding from the Structural Funds was in the past also blocked till the end and payment was provisional on the realization of the project objectives. 100

101 At the same time, much more proposals have been submitted (for the Cooperative projects there was an increase between FP5 and FP6 of 30 %) with lower success rates as a consequence. As a result, there are many proposals that are of sufficient quality on the basis of the evaluation criteria (i.e. above threshold) which cannot be funded because of a lack of funding. The oversubscription rate (i.e. the number of proposals evaluated above threshold but not funded, divided by the total number of proposals received) amounts to 28,4 %. for the Cooperative projects under FP6 while it was lower (5,8 %) for the Collective projects. This is a large percentage in particular for the Cooperative projects with a double loss as a consequence: the investment made in the proposal and the potential output of the project which very likely will not be realized. 66,1% of the projects evaluated above threshold but not funded were stopped completely; only 3,4% of these the projects were carried out as initially foreseen. For a given total budget, the suggestion therefore is to consider supporting more projects but to reduce the budget of the projects. A drawback of this recommendation though is the increase in the cost for the EC for the management of a larger number of contracts. Alternative (or complementary) strategies to increase the success rates are to filter the proposals more efficiently to retain and fund only projects that are most in line with real SME needs. To this end, we recommend a couple of incremental adjustments to the scheme under the following section. 6.4 Project / Implementation level Our analysis of the survey results as well as the success factors derived from the case studies analysis show that SMEs will get more out of a project if the project objectives are really in line with their needs and if there is a strong commitment and active involvement of the SMEs. The first recommendations under this operational level (recommendations nr 10 to 13) are related to these success factors. Their objective is to increase the chances, starting from the evaluation procedure but also in the course of the project, to have projects that serve the needs of the SMEs that are committed partners and are actively involved in an outsourcing context. Recommendation 10: Project proposals should first pass the SME needs test It is crucial that project outcomes serve the needs of the participating SMEs (for the cooperative measure) or the needs of a wider community of SMEs (for the collective measure). This does not necessarily mean that the original project idea should come from the SMEs nor that they should be heavily involved in the proposal preparation or the project implementation. RTD-performers can (and will) play a crucial role in the definition of the project and the development of the proposal. By definition, given the outsourcing context, they take the lead in the project implementation. But the challenge is to reconcile the ambitions and agenda of the RTD-performers with the needs of the SMEs in an outsourcing context and taking into account that (most of) the SMEs (and the IAGs) do not have the capabilities nor the time to be intensively involved and/or to steer the project. On the basis of the information collected, in particular the cases, there is room for improvement in this respect. It is therefore important to take measures that provide stronger guarantees in this 101

102 respect. These measures could be taken at the moment proposals are submitted as well as in the course of the projects. For the Cooperative projects, we would suggest to introduce a two-stage selection and evaluation procedure similar to the Collective projects. In the first stage, applicants should develop the business case for the project. This first proposal should make clear that (1) there is a need among the applying SMEs that can be solved through R&D, (2) the SMEs do not have the capability of their own to do this research and (3) if this need is solved, this will have positive consequences on the competitiveness of the SMEs involved. This limited proposal should be evaluated from a business perspective and therefore by business representatives mainly, from industry and services eventually also including venture capitalists. Those proposals that pass this SME needs test, could then be further developed into a research proposal where the evaluation is mainly done by scientific and technical experts. Another advantage of this procedure is that less money will be spent by the applicants on proposals that in a single stage procedure would not be selected and can now be stopped after the first stage, before the proposals are fully developed. The SME needs test may also serve as an entry barrier for RTDperformers that submit proposals that mainly serve their own research interest and not that much the interests of the SMEs they attract as partners. A potential disadvantage is the negative impact on the length of the procedures between submission and contracting. The Collective projects already have a two-stage application procedure. The focus of the first stage could be redirected to a business case proposal like we just described. Applicants should in particular describe the way the whole sector will benefit from the project results through for instance the dissemination activities. In an extreme form, RTD-performers could be excluded from the first stage and only allowed to enter the game in the second stage. An experimental idea would be to organize an auction-type event among interested RTD-performers for the business proposals that were selected after the first stage. They would have to bid with a research proposal that offers a technical solution to the problems of the SMEs that passed the first stage. Recommendation 11: SME-participants and IAGs must sign a letter of commitment The cases also provided examples of situations in which the SMEs (and IAGs, but less so) at the start of the project were not fully aware of the project objectives and of the budget available for them compared to the RTD-performer. As a result, there are several projects in the cases that were confronted with a drop out of a number of SMEs at the start of the project or with SME participants (and IAGs) that were no longer really motivated. Reasons are the fact that these SMEs and IAGs come to the conclusion that the project objectives do not match their (SME member) needs and/or that the budget really available for them is lower than expected. A possible solution is to let the SMEs and IAGs sign a letter of commitment which clearly states the objectives of the project, the budget distribution, the involvement of SMEs and IAGs and the IPR arrangement?. SMEs should only sign such a letter if they are convinced the project serves their needs and if the budget available for them is sufficient. 102

103 Recommendation 12: With regard to IPR configuration, there is a need to both keep flexibility and foster transparency within project consortia Under FP5 and FP6, IPR belonged exclusively to the SME participants (CRAFT and co-operative projects) or to the participating SME Associations in the case of Collective projects. Exploitation and commercialisation of the projects results were therefore also expected from the SMEs or SME Associations themselves. The evidence gathered in the context of this study tend to show that the higher flexibility introduced under FP7 in this regard (with the possibility to deviate from this default regime and to allow exclusive or joint ownership by RTD-performers) was very pertinent. The usual constraints characterising the SME population considered (limited capabilities, financial constraints) significantly hampers the creation, management and exploitation of IPR as outcomes of SME-specific projects. However, there is a clear need for more transparency and information dissemination within the project consortia during the genesis of the IPR configuration. On the one hand, the study has shown that in many projects the SME-partners were not sufficiently informed neither about the details of the IPR regime chosen, nor about their actual implications for further exploitation of the results. On the other hand, in some successful case-studies a thorough discussion and definition of the IPR regime among all partners at the beginning of the project was clearly considered by all SMEs as an important pre-requisite for a fair implementation of the project. It is therefore recommended to reduce these information asymmetries within the consortia at the beginning of the project, or even prior to contract signature, through a close interaction between all projects members and the European Commission. Recommendation 13: Funding rules should be reconsidered in such a way that RTD-performers run some risk as well. With the current funding rules, RTD-performers receive 100 % financing for their R&D-activities in the context of the project. Even in an outsourcing context such as created under FP7, where the SMEs receive the money first and subsequently pay for the services delivered by the RTD performer, the RTD-performers run little risk not to be paid. SME participants that were interviewed in the context of the cases told us that in practice it would be very difficult not to pay the RTDperformer (and switch to another) if the RTD-performer would not deliver according to the project objectives. It would moreover be very difficult for an SME to prove that the RTD-performer did not do his job. There are reasons to believe that as a consequence of these funding rules, participating in the SME specific measures is very attractive for RTD-performers. Among these reasons is the fact that for most of the projects, RTD-performers took the initiative and developed the proposal, and that some RTD-performers are involved in a very large number of projects. As the case study results indicated, RTD-performers are on average more positive about the reach of the project objectives compared to the SME participants because the former consider them from a technical perspective, while the latter 103

104 apply a business perspective. Another recommendation to bring the objectives of the RTD-performers in line with those of the SMEs would therefore be to reconsider the funding rules in such a way that the RTD-performers run some project risk as well Recommendation 14: SMEs should be encouraged to get more intensively engaged in the work through participation in project meetings and by validating the interim project results. In an outsourcing context like the one of the SME specific measures, the involvement of the SMEs and IAGs in the projects is rather limited by definition. The project meetings are in that perspective very important as one of the rare moments in the course of the project where the SMEs and IAGs can react on the project progress in relation to their needs. Therefore, it is very important that (1) there are sufficient project meetings and (2) there is sufficient budget for the SMEs and the IAGs to participate in these meetings. To keep the project on track, we would also suggest letting the SMEs and IAGs validate the interim project results as an extra guarantee mechanism that the RTD-performers serve the needs of the SMEs. Of course, research remains a risky business and SMEs need to be ware that projects can fail to deliver the expected results. Recommendation 15: Balanced requirements for international cooperation Besides the project meetings, SMEs are also involved, typically, in the final testing and trial stages of the project. The cases show that the SMEs that are involved in these tests are typically located in the country of the RTDperformer(s). Reasons for this are that these RTD-performers and SMEs know each other from before as well as distance, language and cultural barriers which make it more difficult to involve SMEs from another country. The latter types of barriers make it, more in general, sometimes also difficult to compose a partnership that satisfies the requirements in terms of international composition. These international partnership requirements have been changed between FP6 and FP7 and in a different way for the SMEs and the IAGs on the one hand and the RTD performers on the other hand. For the SMEs and IAGs they have become more stringent 51, for the RTD performers they have become less stringent 52. International cooperation between the SMEs should however in any case be a requirement because this is a major element in the value added of the SMEspecific measures compared to national or regional funding programmes. However, it would be beneficial for a more effective transfer of knowledge In Research for SMEs (FP7) there must be 3 SME proposers from 3 different countries compared to 3 SME proposers from 2 different countries under the Cooperative research measure (FP6). A similar change in requirements applies for Research for SME associations (FP7) compared to the Collective research measure (FP6). The 2 RTD performers can under FP7 come from any country while they had to come from 2 different countries under FP6. 104

105 towards SMEs to have RTD-performers from as many countries as SMEs, so that each project would have in each country covered pairs of actors. National Contact Points should be helpful in the search for partners from abroad (those that do not do this already). Recommendation 16: Pro-active monitoring of the dissemination and training activities in the collective projects For the collective projects, we could not yet really evaluate the multiplier effect. This multiplier effect is based on the hypothesis that the results of these projects will be further disseminated to all the SME members of the IAGs involved and eventually to all the SMEs in the sector. So compared to the cooperative projects in which only the participating SMEs will benefit, there should be a multiplicative effect in the Collective projects. In most of the cases conducted on Collective projects, the projects were not yet finished completely and the dissemination and training activities, important in the light of the multiplier effect to materialize, take place of course at the end of the project. The realization of the multiplier effect will be dependent on successful outcomes of project and subsequent dissemination. We suggest a pro-active monitoring of these dissemination and training activities, because in some of the cases, SME members of the core group were against wider dissemination of the results to the other SME members in the sector. The existing reporting requirements on these activities should be closely followed up and if not respected, the Commission should intervene. Recommendation 17: Additional information collection, at the right time, to be able to evaluate the effects and impacts of the SME specific measures It was one of the operational objectives of the study to develop indicators for the monitoring of the SME performance/activity and the impacts of SME specific measures under the Framework programes for future assessment purposes. The current monitoring and reviewing process has been described before (see section 5.9). One of the conclusions was that the monitoring by the EC is generally found to be too limited to purely administrative or legal matters. It is not obvious, however, that this monitoring and reviewing process should be reinforced, expanded or improved. There is indeed a trade-off between effectively monitoring / reviewing projects and ensuring that taxpayer s money is duly allocated, on the one hand, and the requirement to limit or even reduce the administrative burden on the project s coordinators and partners, on the other hand. The following recommendations therefore refer to the expected impact their implementation would have on the administrative burden for the participants. They are also ranked in that way, starting with a recommendation that has no impact on (or even results in a reduction of) the administrative burden. To evaluate the impacts of the SME specific measures, information is needed on the outcomes of the projects, both tangible (like the development of a new product or improved process) and intangible (like improved network abilities). 105

106 The timing for this information collection is also crucial since certain impacts will only become visible after the project is fully completed. Information about the impacts of project participation can be acquired ex post through surveys and case studies like in the context of this study. Disadvantages of this approach relate to a limited response, reluctance to participate in case studies, blurred memories about the starting situation, The ideal situation is to collect this information from all participants (no bias through selective responses to surveys) and at a moment when it takes the least time for the participants to provide this information (to limit the administrative burden). Our suggestion is to collect information, ideally, at the following four moments: At the moment of application/submission of the proposal: a minimum of information for which it is interesting to have it for both the participants in proposals that will be selected as well as for participants in proposals that will not be selected. The variables are those that need to be provided in the context of the selection and eligibility criteria (like the sector and size variables of the SMEs). As such there would be no impact on the administrative burden but the challenge remains to collect, process and store this information efficiently. At the kick of meeting of the project, a base measurement could take place, using a simple fact sheet, with respect to a number of variables that describe the starting position of the participants, in particular the SMEs. This information could be collected either by the Coordinator (which would imply extra coordination costs) or by EC project officer present during this meeting (with no/limited impact on the administrative burden of the participants). Variables (indicators) could include information on: the R&D and innovation profile of the participants (like R&D-budgeting, R&D-department, experience with funded R&D-projects, R&D-cooperation experience with location of partners); economic performance variables (like number of employees and turnover, expert share and markets) The final project meeting is the ideal moment to collect information on the results of the project in relation to the project objectives, on the project implementation as well as on the (expected) impacts. Again we suggest collecting this information either by the Coordinator (which would imply extra coordination costs) or by the EC project officer present during this meeting (with no/limited impact on the administrative burden of the participants). Information to be collected relates to: project results: new or improved product, process, technology, ; Reach of the project objectives; Update on the R&D and innovation profile of the participants and on their economic performance variables; Effectiveness of project implementation; (Expected) commercial outcome of the project 106

107 (Expected) economic impacts which can be attributed to project participation; Other impacts e.g. networking abilities. Because most impacts, in particular the economic impacts, will only materialize and can therefore only be evaluated after some time, we suggest collecting this impact information 3 to 5 years after the project is completed. At this moment, it would be the responsibility of the Commission or its Research Executive Agency to collect this information (either by its own services or contracted out). Information to be collected relates to: Another update on the R&D and innovation profile of the former participants and on their economic performance variables; Commercial outcome of the project; Economic impacts which can be attributed to project participation. This impact information could first of all be used to serve accountability requirements. But the outcomes should also be used for policy improvement and integrated in the policy cycle for future policy preparation at the strategic level on issues like for instance European Value Added and the subsidiarity principle. Below we have added a table with a non-exhaustive list of indicators that may be proposed to monitoring more accurately the effects and impacts generated through project participation. 107

108 Type of indicators Timing in project life-cycle Variable Indicator Source 1. Profiling and base measurement Proposal submission Nature of the organisation (no indicator, but info needed for further monitoring) Proposal forms eligibility criteria Size of organisation Average size of SMEs participating in project Size distribution of SMEs participating in project (e.g. % of SE s) Sector of activity Sector with the largest concentration of SME partners Proposal forms eligibility criteria Proposal forms eligibility criteria Role in proposal development SME co-ordinator? (Y/N) RTD-p co-ordinator? (Y/N) Proposal forms eligibility criteria IAG co-ordinator? (Y/N) 108

109 Type of indicators Timing in project life-cycle Variable Indicator Source Project meeting kick-off RDI-profile of the projects participants Share of SMEs with yearly budgeting of R&D expenses (%) Co-ordinator or EC project officer Share of SMEs with separate R&D department (%) Share of SMEs outsourcing regularly R&D to RTD-p (%) Share of SMEs participating regularly in R&D project totally independent from public funding (%) Share of SMEs participating regularly in R&D projects funded under the Framework programmes (%) Share of SMEs regularly involved in R&D cooperation (%) (further breakdowns with other companies versus with RTD-p and domestic partners versus partners abroad Average R&D capabilities of the SMEs involved (from very low-tech to very high-tech) 109

110 Type of indicators Timing in project life-cycle Variable Indicator Source Economic performance Turnover Number of employees Co-ordinator or EC project officer Share of export in turnover National market share International market share Productivity (value added per person employed or per FTE) Overall competitiveness (from very low to very high) 2. Outcomes, outputs and short-term impacts Final meeting project Project results Share of SMEs declaring that project objectives have largely been reached (%) Share of SMEs declaring that company s own objectives have largely been reached (%) Co-ordinator or EC project officer Average cost-benefit balance for the SMEs 110

111 Type of indicators Timing in project life-cycle Variable Indicator Source RDI profile (update baseline) Share of SMEs with yearly budgeting of R&D expenses (%) Co-ordinator or EC project officer Share of SMEs with separate R&D department (%) Share of SMEs outsourcing regularly R&D to RTD-p (%) Share of SMEs participating regularly in R&D project totally independent from public funding (%) Share of SMEs participating regularly in R&D projects funded under the Framework programmes (%) Share of SMEs regularly involved in R&D cooperation (%) (further breakdowns with other companies versus with RTD-p and domestic partners versus partners abroad Average R&D capabilities of the SMEs involved (from very low-tech to very high-tech) Commercial outcome Share of SMEs declaring that commercial output is realized or is expected to materialize in the near future (%) Co-ordinator or EC project officer 111

112 Type of indicators Timing in project life-cycle Variable Indicator Source Economic impacts (update baseline) Turnover (+ share of turnover growth to be imputed to project participation) Co-ordinator or EC project officer Number of employees (+ share of ) Share of export in turnover (+ share of ) National market share (+ share of ) International market share (+ share of ) Productivity (value added per person employed or per FTE) (+ share of ) Overall competitiveness (from very low to very high) (+ share of ) Other impacts Share of SMEs declaring that project contributed to increase the company s ability to network with other firms / RTD-p (%) Co-ordinator or EC project officer Share of SMEs declaring that project contributed to increase the scientific and techno logical capabilities of the own company (%) Share of SMEs declaring that project contributed to increase the company s R&D infrastructure (%) 112

113 Type of indicators Timing in project life-cycle Variable Indicator Source 3. Long-term Impacts 3 to 5 years after the project is completed Economic impacts (update end of projects) Turnover (+ share of turnover growth to be imputed to project participation) Number of employees (+ share of ) Co-ordinator or EC project officer, participants and control-group survey Share of export in turnover (+ share of ) National market share (+ share of ) International market share (+ share of ) Productivity (value added per person employed or per FTE) (+ share of ) Overall competitiveness (from very low to very high) (+ share of ) Other impacts (update end of projects) Share of SMEs declaring that project contributed to increase the company s ability to network with other firms / RTD-p (%) Co-ordinator or EC project officer, participants and control-group survey Share of SMEs declaring that project contributed to increase the scientific and techno logical capabilities of the own company (%) Share of SMEs declaring that project contributed to increase the company s R&D infrastructure (%) 113

Fact Sheet IP specificities in research for the benefit of SMEs

Fact Sheet IP specificities in research for the benefit of SMEs European IPR Helpdesk Fact Sheet IP specificities in research for the benefit of SMEs June 2015 1 Introduction... 1 1. Actions for the benefit of SMEs... 2 1.1 Research for SMEs... 2 1.2 Research for SME-Associations...

More information

demonstrator approach real market conditions would be useful to provide a unified partner search instrument for the CIP programme

demonstrator approach real market conditions  would be useful to provide a unified partner search instrument for the CIP programme Contribution by the Ministry of Industry and Trade of the Czech Republic to the public consultations on a successor programme to the Competitiveness and Innovation Framework Programme (CIP) 2007-2013 Given

More information

Brief presentation of the results Ioana ISPAS ERA NET COFUND Expert Group

Brief presentation of the results Ioana ISPAS ERA NET COFUND Expert Group Brief presentation of the results Ioana ISPAS ERA NET COFUND Expert Group Mandate of the Expert Group Methodology and basic figures for ERA-NET Cofund Efficiency of ERA-NET Cofund Motivations and benefits

More information

COMMISSION STAFF WORKING PAPER EXECUTIVE SUMMARY OF THE IMPACT ASSESSMENT. Accompanying the

COMMISSION STAFF WORKING PAPER EXECUTIVE SUMMARY OF THE IMPACT ASSESSMENT. Accompanying the EUROPEAN COMMISSION Brussels, 30.11.2011 SEC(2011) 1428 final Volume 1 COMMISSION STAFF WORKING PAPER EXECUTIVE SUMMARY OF THE IMPACT ASSESSMENT Accompanying the Communication from the Commission 'Horizon

More information

An introduction to the 7 th Framework Programme for Research and Technological Development. Gorgias Garofalakis

An introduction to the 7 th Framework Programme for Research and Technological Development. Gorgias Garofalakis An introduction to the 7 th Framework Programme for Research and Technological Development Gorgias Garofalakis Contents What & why Potential impact Scope Inputs Framework Programme Budget and duration

More information

Strengthening the knowledge base and reducing fragmentation

Strengthening the knowledge base and reducing fragmentation Strengthening the knowledge base and reducing fragmentation I3U FINAL CONFERENCE Brussels, 25 September 2018 This project is co-funded by the European Union Research objectives Main objective: to evaluate

More information

Presentation of the results. Niels Gøtke, Chair of the expert group and Effie Amanatidou, Rapporteur

Presentation of the results. Niels Gøtke, Chair of the expert group and Effie Amanatidou, Rapporteur Presentation of the results Niels Gøtke, Chair of the expert group and Effie Amanatidou, Rapporteur Purpose and scope of the evaluation Methodology and basic figures for ERA-NET Cofund Efficiency of ERA-NET

More information

R&D funding for SMEs in the 7th Framework Programme

R&D funding for SMEs in the 7th Framework Programme R&D funding for SMEs in the 7th Framework Programme Dr Bernd Reichert Head of Unit Small and Medium-Sized Enterprises Research Directorate General European Commission Why should SME participate in the

More information

Framework Programme 7 and SMEs. Amaury NEVE European Commission DG Research - Unit T4: SMEs

Framework Programme 7 and SMEs. Amaury NEVE European Commission DG Research - Unit T4: SMEs Framework Programme 7 and SMEs Amaury NEVE European Commission DG Research - Unit T4: SMEs Outline 1. SMEs and R&D 2. The Seventh Framework Programme 3. SMEs in Cooperation 4. SMEs in People 5. SMEs in

More information

PUBLISHABLE FINAL ACTIVITY REPORT

PUBLISHABLE FINAL ACTIVITY REPORT SCRATCH PHASE IV Support for SMEs Collaborative Aeronautical Technical Research Contract N : ASA3-CT-2004-510981 Specific Support Action Specific Programme: Integrating and strengthening the ERA Thematic

More information

8365/18 CF/nj 1 DG G 3 C

8365/18 CF/nj 1 DG G 3 C Council of the European Union Brussels, 30 April 2018 (OR. en) 8365/18 RECH 149 COMPET 246 NOTE From: To: Presidency Delegations No. prev. doc.: 8057/1/18 RECH 136 COMPET 230 Subject: Draft Council conclusions

More information

The 45 Adopted Recommendations under the WIPO Development Agenda

The 45 Adopted Recommendations under the WIPO Development Agenda The 45 Adopted Recommendations under the WIPO Development Agenda * Recommendations with an asterisk were identified by the 2007 General Assembly for immediate implementation Cluster A: Technical Assistance

More information

Consultation on Long Term sustainability of Research Infrastructures

Consultation on Long Term sustainability of Research Infrastructures Consultation on Long Term sustainability of Research Infrastructures Fields marked with are mandatory. 1. Introduction The political guidelines[1] of the European Commission present an ambitious agenda

More information

EUROPEAN MANUFACTURING SURVEY EMS

EUROPEAN MANUFACTURING SURVEY EMS EUROPEAN MANUFACTURING SURVEY EMS RIMPlus Final Workshop Brussels December, 17 th, 2014 Christian Lerch Fraunhofer ISI Content 1 2 3 4 5 EMS A European research network EMS firm-level data of European

More information

Terms of Reference. Call for Experts in the field of Foresight and ICT

Terms of Reference. Call for Experts in the field of Foresight and ICT Terms of Reference Call for Experts in the field of Foresight and ICT Title Work package Lead: Related Workpackage: Related Task: Author(s): Project Number Instrument: Call for Experts in the field of

More information

Committee on Development and Intellectual Property (CDIP)

Committee on Development and Intellectual Property (CDIP) E CDIP/6/4 REV. ORIGINAL: ENGLISH DATE: NOVEMBER 26, 2010 Committee on Development and Intellectual Property (CDIP) Sixth Session Geneva, November 22 to 26, 2010 PROJECT ON INTELLECTUAL PROPERTY AND TECHNOLOGY

More information

Position Paper of Iberian universities. The mid-term review of Horizon 2020 and the design of FP9

Position Paper of Iberian universities. The mid-term review of Horizon 2020 and the design of FP9 Position Paper of Iberian universities The mid-term review of Horizon 2020 and the design of FP9 Introduction Horizon 2020 (H2020), the Framework Programme for research and innovation of the European Union,

More information

GENEVA COMMITTEE ON DEVELOPMENT AND INTELLECTUAL PROPERTY (CDIP) Fifth Session Geneva, April 26 to 30, 2010

GENEVA COMMITTEE ON DEVELOPMENT AND INTELLECTUAL PROPERTY (CDIP) Fifth Session Geneva, April 26 to 30, 2010 WIPO CDIP/5/7 ORIGINAL: English DATE: February 22, 2010 WORLD INTELLECTUAL PROPERT Y O RGANI ZATION GENEVA E COMMITTEE ON DEVELOPMENT AND INTELLECTUAL PROPERTY (CDIP) Fifth Session Geneva, April 26 to

More information

Common evaluation criteria for evaluating proposals

Common evaluation criteria for evaluating proposals Common evaluation criteria for evaluating proposals Annex B A number of evaluation criteria are common to all the programmes of the Sixth Framework Programme and are set out in the European Parliament

More information

Committee on Development and Intellectual Property (CDIP)

Committee on Development and Intellectual Property (CDIP) E CDIP/10/13 ORIGINAL: ENGLISH DATE: OCTOBER 5, 2012 Committee on Development and Intellectual Property (CDIP) Tenth Session Geneva, November 12 to 16, 2012 DEVELOPING TOOLS FOR ACCESS TO PATENT INFORMATION

More information

Working together to deliver on Europe 2020

Working together to deliver on Europe 2020 Lithuanian Position Paper on the Green Paper From Challenges to Opportunities: Towards a Common Strategic Framework for EU Research and Innovation Funding Lithuania considers Common Strategic Framework

More information

2000 Edition. SMEs & EU RTD Small and medium-sized enterprises and European Union programmes for research and technological development

2000 Edition. SMEs & EU RTD Small and medium-sized enterprises and European Union programmes for research and technological development EUROPEAN COMMISSION SCIENCE RESEARCH DEVELOPMENT 2000 Edition SMEs & EU RTD Small and medium-sized enterprises and European Union programmes for research and technological development This brochure presents

More information

WIPO Development Agenda

WIPO Development Agenda WIPO Development Agenda 2 The WIPO Development Agenda aims to ensure that development considerations form an integral part of WIPO s work. As such, it is a cross-cutting issue which touches upon all sectors

More information

European Technology Platforms

European Technology Platforms European Technology Platforms a a new concept a a new way to achieve Lisbon s goals...priority for 2004-2005 put forward by the Members States and fully supported by the Commission Launching of Greek Technology

More information

Marie Sklodowska Curie Actions. Business participation and entrepreneurship in Marie Skłodowska- Curie actions (FP7 and Horizon 2020)

Marie Sklodowska Curie Actions. Business participation and entrepreneurship in Marie Skłodowska- Curie actions (FP7 and Horizon 2020) Sadržaj Marie Sklodowska Curie Actions Business participation and entrepreneurship in Marie Skłodowska- Curie actions (FP7 and Horizon 2020) Sandra Vidović, 17th November 2017 Study of business participation

More information

UEAPME Think Small Test

UEAPME Think Small Test Think Small Test and Small Business Act Implementation Scoreboard Study Unit Brussels, 6 November 2012 1. Introduction The Small Business Act (SBA) was approved in December 2008, laying out seven concrete

More information

COST FP9 Position Paper

COST FP9 Position Paper COST FP9 Position Paper 7 June 2017 COST 047/17 Key position points The next European Framework Programme for Research and Innovation should provide sufficient funding for open networks that are selected

More information

Impact assessment of the SME-specific Measures FP5 and FP6, 2009

Impact assessment of the SME-specific Measures FP5 and FP6, 2009 EUROPEAN COMMISSION European Research Area 7th Framework Programme Impact assessment of the SME-specific Measures FP5 and FP6, 2009 Studies and reports EUR 24290 EN European Commission EUr 24290 Impact

More information

Globalisation increasingly affects how companies in OECD countries

Globalisation increasingly affects how companies in OECD countries ISBN 978-92-64-04767-9 Open Innovation in Global Networks OECD 2008 Executive Summary Globalisation increasingly affects how companies in OECD countries operate, compete and innovate, both at home and

More information

Rethinking the role of Social Sciences and Humanities (SSH) in Horizon 2020: toward a reflective and generative perspective

Rethinking the role of Social Sciences and Humanities (SSH) in Horizon 2020: toward a reflective and generative perspective THE EU FRAMEWORK PROGRAMME FOR RESEARCH AND INNOVATION Horizon 2020 Societal Challenge 6: "Europe in a changing world : inclusive, innovative and reflective society" Rethinking the role of Social Sciences

More information

EU Support for SME Innovation: The SME Instrument

EU Support for SME Innovation: The SME Instrument Audit preview Information on an upcoming audit EU Support for SME Innovation: The SME Instrument April 2019 2 Traditionally, start-ups and small and medium-sized enterprises (SMEs) in the EU have faced

More information

Belgian Position Paper

Belgian Position Paper The "INTERNATIONAL CO-OPERATION" COMMISSION and the "FEDERAL CO-OPERATION" COMMISSION of the Interministerial Conference of Science Policy of Belgium Belgian Position Paper Belgian position and recommendations

More information

GZ.:BMWF-8.105/5-II/1/2010

GZ.:BMWF-8.105/5-II/1/2010 Austrian Status Report on the implementation of the Recommendation from the European Commission on the management of Intellectual Property in knowledge transfer activities and a Code of Practice for universities

More information

Innovation Management & Technology Transfer Innovation Management & Technology Transfer

Innovation Management & Technology Transfer Innovation Management & Technology Transfer Innovation Management & Technology Transfer Nuno Gonçalves Minsk, April 15th 2014 nunogoncalves@spi.pt 1 Introduction to SPI Opening of SPI USA office in Irvine, California Beginning of activities in Porto

More information

VSNU December Broadening EU s horizons. Position paper FP9

VSNU December Broadening EU s horizons. Position paper FP9 VSNU December 2017 Broadening EU s horizons Position paper FP9 Introduction The European project was conceived to bring peace and prosperity to its citizens after two world wars. In the last decades, it

More information

Position Paper of Iberian Universities Design of FP9

Position Paper of Iberian Universities Design of FP9 Position Paper of Iberian Universities Design of FP9 The Framework Programme for Research and Innovation is the most important PanEuropean programme for research and innovation, not only in size, but also

More information

From FP7 towards Horizon 2020 Workshop on " Research performance measurement and the impact of innovation in Europe" IPERF, Luxembourg, 31/10/2013

From FP7 towards Horizon 2020 Workshop on  Research performance measurement and the impact of innovation in Europe IPERF, Luxembourg, 31/10/2013 From FP7 towards Horizon 2020 Workshop on " Research performance measurement and the impact of innovation in Europe" IPERF, Luxembourg, 31/10/2013 Lucilla Sioli, European Commission, DG CONNECT Overview

More information

COUNCIL OF THE EUROPEAN UNION. Brussels, 9 December 2008 (16.12) (OR. fr) 16767/08 RECH 410 COMPET 550

COUNCIL OF THE EUROPEAN UNION. Brussels, 9 December 2008 (16.12) (OR. fr) 16767/08 RECH 410 COMPET 550 COUNCIL OF THE EUROPEAN UNION Brussels, 9 December 2008 (16.12) (OR. fr) 16767/08 RECH 410 COMPET 550 OUTCOME OF PROCEEDINGS of: Competitiveness Council on 1 and 2 December 2008 No. prev. doc. 16012/08

More information

Establishing a Development Agenda for the World Intellectual Property Organization

Establishing a Development Agenda for the World Intellectual Property Organization 1 Establishing a Development Agenda for the World Intellectual Property Organization to be submitted by Brazil and Argentina to the 40 th Series of Meetings of the Assemblies of the Member States of WIPO

More information

WG/STAIR. Knut Blind, STAIR Chairman

WG/STAIR. Knut Blind, STAIR Chairman WG/STAIR Title: Source: The Operationalisation of the Integrated Approach: Submission of STAIR to the Consultation of the Green Paper From Challenges to Opportunities: Towards a Common Strategic Framework

More information

The main FP7 instruments. Aurélien Saffroy. 6 Dec

The main FP7 instruments. Aurélien Saffroy. 6 Dec The main FP7 instruments Aurélien Saffroy 6 Dec. 2006 www.euroquality.fr 1 Summary STRUCTURE OF THE 7 th Framework Programme STRUCTURE OF THE 7 th Framework Programme 2 The main instruments of FP7 Capacities;

More information

Draft executive summaries to target groups on industrial energy efficiency and material substitution in carbonintensive

Draft executive summaries to target groups on industrial energy efficiency and material substitution in carbonintensive Technology Executive Committee 29 August 2017 Fifteenth meeting Bonn, Germany, 12 15 September 2017 Draft executive summaries to target groups on industrial energy efficiency and material substitution

More information

CAPACITIES. 7FRDP Specific Programme ECTRI INPUT. 14 June REPORT ECTRI number

CAPACITIES. 7FRDP Specific Programme ECTRI INPUT. 14 June REPORT ECTRI number CAPACITIES 7FRDP Specific Programme ECTRI INPUT 14 June 2005 REPORT ECTRI number 2005-04 1 Table of contents I- Research infrastructures... 4 Support to existing research infrastructure... 5 Support to

More information

CO-ORDINATION MECHANISMS FOR DIGITISATION POLICIES AND PROGRAMMES:

CO-ORDINATION MECHANISMS FOR DIGITISATION POLICIES AND PROGRAMMES: CO-ORDINATION MECHANISMS FOR DIGITISATION POLICIES AND PROGRAMMES: NATIONAL REPRESENTATIVES GROUP (NRG) SUMMARY REPORT AND CONCLUSIONS OF THE MEETING OF 10 DECEMBER 2002 The third meeting of the NRG was

More information

National Innovation System of Mongolia

National Innovation System of Mongolia National Innovation System of Mongolia Academician Enkhtuvshin B. Mongolians are people with rich tradition of knowledge. When the Great Mongolian Empire was established in the heart of Asia, Chinggis

More information

INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016

INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016 www.euipo.europa.eu INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016 Executive Summary JUNE 2016 www.euipo.europa.eu INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016 Commissioned to GfK Belgium by the European

More information

INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016

INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016 www.euipo.europa.eu INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016 Executive Summary JUNE 2016 www.euipo.europa.eu INTELLECTUAL PROPERTY (IP) SME SCOREBOARD 2016 Commissioned to GfK Belgium by the European

More information

Research Development Request - Profile Template. European Commission

Research Development Request - Profile Template. European Commission Research Development Request - Profile Template European Commission Research Development Request Profile The following table can be used as a template for drafting a Research Development Request profile.

More information

10246/10 EV/ek 1 DG C II

10246/10 EV/ek 1 DG C II COUNCIL OF THE EUROPEAN UNION Brussels, 28 May 2010 10246/10 RECH 203 COMPET 177 OUTCOME OF PROCEEDINGS from: General Secretariat of the Council to: Delegations No. prev. doc.: 9451/10 RECH 173 COMPET

More information

Conclusions concerning various issues related to the development of the European Research Area

Conclusions concerning various issues related to the development of the European Research Area COUNCIL OF THE EUROPEAN UNION Conclusions concerning various issues related to the development of the European Research Area The Council adopted the following conclusions: "THE COUNCIL OF THE EUROPEAN

More information

Realising the FNH-RI: Roadmap. Karin Zimmermann (Wageningen Economic Research [WUR], NL)

Realising the FNH-RI: Roadmap. Karin Zimmermann (Wageningen Economic Research [WUR], NL) Realising the FNH-RI: Roadmap Karin Zimmermann (Wageningen Economic Research [WUR], NL) Three ongoing tracks towards a FNH-RI Design studies EuroDISH: Determinants Intake Status - Health RICHFIELDS: Focus

More information

Christina Miller Director, UK Research Office

Christina Miller Director, UK Research Office Christina Miller Director, UK Research Office www.ukro.ac.uk UKRO s Mission: To promote effective UK engagement in EU research, innovation and higher education activities The Office: Is based in Brussels,

More information

Key features in innovation policycomparison. Dr Gudrun Rumpf Kyiv, 9 November, 2010

Key features in innovation policycomparison. Dr Gudrun Rumpf Kyiv, 9 November, 2010 Enhance Innovation Strategies, Policies and Regulation in Ukraine EuropeAid/127694/C/SER/UA Ukraine This Project is funded by the European Union Key features in innovation policycomparison EU and Ukraine

More information

2010/3 Science and technology for development. The Economic and Social Council,

2010/3 Science and technology for development. The Economic and Social Council, Resolution 2010/3 Science and technology for development The Economic and Social Council, Recalling the 2005 World Summit Outcome, which emphasizes the role of science and technology, including information

More information

The Policy Content and Process in an SDG Context: Objectives, Instruments, Capabilities and Stages

The Policy Content and Process in an SDG Context: Objectives, Instruments, Capabilities and Stages The Policy Content and Process in an SDG Context: Objectives, Instruments, Capabilities and Stages Ludovico Alcorta UNU-MERIT alcorta@merit.unu.edu www.merit.unu.edu Agenda Formulating STI policy STI policy/instrument

More information

7656/18 CF/MI/nj 1 DG G 3 C

7656/18 CF/MI/nj 1 DG G 3 C Council of the European Union Brussels, 6 April 2018 (OR. en) 7656/18 RECH 120 COMPET 192 NOTE From: To: Presidency Delegations No. prev. doc.: 7424/18 RECH 120 COMPET 192 Subject: Draft Council conclusions

More information

OECD Science, Technology and Industry Outlook 2008: Highlights

OECD Science, Technology and Industry Outlook 2008: Highlights OECD Science, Technology and Industry Outlook 2008: Highlights Global dynamics in science, technology and innovation Investment in science, technology and innovation has benefited from strong economic

More information

FP7 Funding Opportunities for the ICT Industry

FP7 Funding Opportunities for the ICT Industry FP7 Funding Opportunities for the ICT Industry Haitham S. Hamza, Ph.D. R&D Department Manager Software Engineering Competence Center Agenda FP7 Structure Overview and Calls Horizon 2020 SECC Role and How

More information

Report on the Results of. Questionnaire 1

Report on the Results of. Questionnaire 1 Report on the Results of Questionnaire 1 (For Coordinators of the EU-U.S. Programmes, Initiatives, Thematic Task Forces, /Working Groups, and ERA-Nets) BILAT-USA G.A. n 244434 - Task 1.2 Deliverable 1.3

More information

"Workshops on key economic issues regarding the. enforcement of IPR in the European Union"

Workshops on key economic issues regarding the. enforcement of IPR in the European Union Ref. Ares(2015)2133028-21/05/2015 Call for expression of interest: "Workshops on key economic issues regarding the enforcement of IPR in the European Union" Background With Directive 2004/48/EC on the

More information

CERN-PH-ADO-MN For Internal Discussion. ATTRACT Initiative. Markus Nordberg Marzio Nessi

CERN-PH-ADO-MN For Internal Discussion. ATTRACT Initiative. Markus Nordberg Marzio Nessi CERN-PH-ADO-MN-190413 For Internal Discussion ATTRACT Initiative Markus Nordberg Marzio Nessi Introduction ATTRACT is an initiative for managing the funding of radiation detector and imaging R&D work.

More information

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS EUROPEAN COMMISSION Brussels, 9.9.2011 COM(2011) 548 final REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

More information

A Research and Innovation Agenda for a global Europe: Priorities and Opportunities for the 9 th Framework Programme

A Research and Innovation Agenda for a global Europe: Priorities and Opportunities for the 9 th Framework Programme A Research and Innovation Agenda for a global Europe: Priorities and Opportunities for the 9 th Framework Programme A Position Paper by the Young European Research Universities Network About YERUN The

More information

Please send your responses by to: This consultation closes on Friday, 8 April 2016.

Please send your responses by  to: This consultation closes on Friday, 8 April 2016. CONSULTATION OF STAKEHOLDERS ON POTENTIAL PRIORITIES FOR RESEARCH AND INNOVATION IN THE 2018-2020 WORK PROGRAMME OF HORIZON 2020 SOCIETAL CHALLENGE 5 'CLIMATE ACTION, ENVIRONMENT, RESOURCE EFFICIENCY AND

More information

EC-Egypt Science and Technology Cooperation Agreement. Road Map

EC-Egypt Science and Technology Cooperation Agreement. Road Map EC-Egypt Science and Technology Cooperation Agreement Road Map 2007-2008 1 TABLE OF CONTENTS pp. INTRODUCTION... 3 FACILITATING COOPERATION... 3-4 ENERGY... 4 ENVIRONMENT (INCLUDING CLIMATE CHANGE)...

More information

RENEW-ESSENCE Position Paper on FP9 September Michele Guerrini, Luca Moretti, Pier Francesco Moretti, Angelo Volpi

RENEW-ESSENCE Position Paper on FP9 September Michele Guerrini, Luca Moretti, Pier Francesco Moretti, Angelo Volpi RENEW-ESSENCE 2030 Position Paper on FP9 September 2017 Michele Guerrini, Luca Moretti, Pier Francesco Moretti, Angelo Volpi Sommario Introduction... 2 Excellence in research... 4 Support to competitiveness...

More information

Innovation policy mixes and implications on HEIs - emerging conclusions from the OECD innovation policy reviews

Innovation policy mixes and implications on HEIs - emerging conclusions from the OECD innovation policy reviews Innovation policy mixes and implications on HEIs - emerging conclusions from the OECD innovation policy reviews Gernot Hutschenreiter Country Studies and Outlook Division Directorate for Science, Technology

More information

A New Platform for escience and data research into the European Ecosystem.

A New Platform for escience and data research into the European Ecosystem. Digital Agenda A New Platform for escience and data research into the European Ecosystem. Iconference Wim Jansen einfrastructure DG CONNECT European Commission The 'ecosystem': some facts 1. einfrastructure

More information

Innovation Systems and Policies in VET: Background document

Innovation Systems and Policies in VET: Background document OECD/CERI Innovation Systems and Policies in VET: Background document Contacts: Francesc Pedró, Senior Analyst (Francesc.Pedro@oecd.org) Tracey Burns, Analyst (Tracey.Burns@oecd.org) Katerina Ananiadou,

More information

Innovation in Europe: Where s it going? How does it happen? Stephen Roper Aston Business School, Birmingham, UK

Innovation in Europe: Where s it going? How does it happen? Stephen Roper Aston Business School, Birmingham, UK Innovation in Europe: Where s it going? How does it happen? Stephen Roper Aston Business School, Birmingham, UK Email: s.roper@aston.ac.uk Overview Innovation in Europe: Where is it going? The challenge

More information

Production research at European level supports regions and SMEs

Production research at European level supports regions and SMEs Production research at European level supports regions and SMEs José-Lorenzo Vallés New generation of products DG Research European Commission 1 Among the policy guidelines for the next Commission Successful

More information

Introducing the 7 th Community Framework Programme for Research and Technological Development ( ) 2013)

Introducing the 7 th Community Framework Programme for Research and Technological Development ( ) 2013) Introducing the 7 th Community Framework Programme for Research and Technological Development (2007-2013) 2013) European Commission Research DG Dr Dimitri CORPAKIS Head of Unit Horizontal aspects and Coordination

More information

EUREKA in the ERA INTRODUCTION

EUREKA in the ERA INTRODUCTION A strategy towards becoming a leading ERA innovation stakeholder to contribute to growth and job creation for the benefit of European industry Final version 27 April 2015 INTRODUCTION The objective of

More information

Spanish Fisheries and Aquaculture Technology Platform (PTEPA)

Spanish Fisheries and Aquaculture Technology Platform (PTEPA) Information about the respondent I am answering as: [individual, government body, university/ higher education, Commercial organization (less than 250 employees), commercial organization (more than 250

More information

PROJECT FACT SHEET GREEK-GERMANY CO-FUNDED PROJECT. project proposal to the funding measure

PROJECT FACT SHEET GREEK-GERMANY CO-FUNDED PROJECT. project proposal to the funding measure PROJECT FACT SHEET GREEK-GERMANY CO-FUNDED PROJECT project proposal to the funding measure Greek-German Bilateral Research and Innovation Cooperation Project acronym: SIT4Energy Smart IT for Energy Efficiency

More information

Creativity and Economic Development

Creativity and Economic Development Creativity and Economic Development A. Bobirca, A. Draghici Abstract The objective of this paper is to construct a creativity composite index designed to capture the growing role of creativity in driving

More information

SASAR POSITION PAPER ON: GREEN PAPER ON A COMMON STRATEGIC FRAMEWORK FOR FUTURE EU RESEARCH AND INNOVATION FUNDING

SASAR POSITION PAPER ON: GREEN PAPER ON A COMMON STRATEGIC FRAMEWORK FOR FUTURE EU RESEARCH AND INNOVATION FUNDING SASAR POSITION PAPER ON: GREEN PAPER ON A COMMON STRATEGIC FRAMEWORK FOR FUTURE EU RESEARCH AND INNOVATION FUNDING INTRODUCTION This position paper represents the recommendations of the Slovak Association

More information

Water, Energy and Environment in the scope of the Circular Economy

Water, Energy and Environment in the scope of the Circular Economy Water, Energy and Environment in the scope of the Circular Economy Maria da Graça Carvalho 11th SDEWES Conference Lisbon 2016 Contents of the Presentation 1. The Circular Economy 2. The Horizon 2020 Program

More information

Position Paper. CEN-CENELEC Response to COM (2010) 546 on the Innovation Union

Position Paper. CEN-CENELEC Response to COM (2010) 546 on the Innovation Union Position Paper CEN-CENELEC Response to COM (2010) 546 on the Innovation Union Introduction CEN and CENELEC very much welcome the overall theme of the Communication, which is very much in line with our

More information

Case Study HYDRO-COAT: Duly protecting research project results

Case Study HYDRO-COAT: Duly protecting research project results European IPR Helpdesk Case Study HYDRO-COAT: Duly protecting research project results September 2012 Company details Name: Politecnico di Milano POLIMI Business sector: Mining Technology 1. Background

More information

Israel s comments on the Commission s proposal for the 7 th Framework Programme

Israel s comments on the Commission s proposal for the 7 th Framework Programme המנהלת הישראלית לתוכנית המסגרת השישית למחקר ופיתוח של האיחוד האירופי Israel-Europe R&D Directorate for FP6 Israel s comments on the Commission s proposal for the 7 th Framework Programme May 2005 1 INDEX

More information

New era for Eureka - relations with ETPs

New era for Eureka - relations with ETPs New era for Eureka - relations with ETPs Dr. Aleš Mihelič EUREKA Chairman Slovenian EUREKA Chair 07/08 The past is history Established in 1985 An initiative of French President Mitterand and German Chancellor

More information

Advanced Impacts evaluation Methodology for innovative freight transport Solutions

Advanced Impacts evaluation Methodology for innovative freight transport Solutions Advanced Impacts evaluation Methodology for innovative freight transport Solutions AIMS 3rd Newsletter August 2010 About AIMS The project AIMS is a co-ordination and support action under the 7th Framework

More information

Engaging UK Climate Service Providers a series of workshops in November 2014

Engaging UK Climate Service Providers a series of workshops in November 2014 Engaging UK Climate Service Providers a series of workshops in November 2014 Belfast, London, Edinburgh and Cardiff Four workshops were held during November 2014 to engage organisations (providers, purveyors

More information

Consultancy on Technological Foresight

Consultancy on Technological Foresight Consultancy on Technological Foresight A Product of the Technical Cooperation Agreement Strategic Roadmap for Productive Development in Trinidad and Tobago Policy Links, IfM Education and Consultancy Services

More information

Transforming Consumer and Health-Oriented Society through Science and Innovation. SBRA meeting 20 June 2018

Transforming Consumer and Health-Oriented Society through Science and Innovation. SBRA meeting 20 June 2018 OPEN INNOVATION TEST BEDS - Nanotech and Advanced Materials Transforming Consumer and Health-Oriented Society through Science and Innovation Søren BØWADT-Deputy Head of Unit Advanced Materials and Nanotechnologies

More information

Horizon 2020 Towards a Common Strategic Framework for EU Research and Innovation Funding

Horizon 2020 Towards a Common Strategic Framework for EU Research and Innovation Funding Horizon 2020 Towards a Common Strategic Framework for EU Research and Innovation Funding Rudolf Strohmeier DG Research & Innovation The context: Europe 2020 strategy Objectives of smart, sustainable and

More information

Brief to the. Senate Standing Committee on Social Affairs, Science and Technology. Dr. Eliot A. Phillipson President and CEO

Brief to the. Senate Standing Committee on Social Affairs, Science and Technology. Dr. Eliot A. Phillipson President and CEO Brief to the Senate Standing Committee on Social Affairs, Science and Technology Dr. Eliot A. Phillipson President and CEO June 14, 2010 Table of Contents Role of the Canada Foundation for Innovation (CFI)...1

More information

DANUBE INNOVATION PARTNERSHIP

DANUBE INNOVATION PARTNERSHIP Horizontal flagship support activity: DANUBE INNOVATION PARTNERSHIP 1. RATIONALE As part of the Europe 2020 Strategy, the 'Innovation Union' flagship initiative sets out a comprehensive innovation strategy

More information

POSITION PAPER. GREEN PAPER From Challenges to Opportunities: Towards a Common Strategic Framework for EU Research and Innovation funding

POSITION PAPER. GREEN PAPER From Challenges to Opportunities: Towards a Common Strategic Framework for EU Research and Innovation funding POSITION PAPER GREEN PAPER From Challenges to Opportunities: Towards a Common Strategic Framework for EU Research and Innovation funding Preamble CNR- National Research Council of Italy shares the vision

More information

Committee on Development and Intellectual Property (CDIP)

Committee on Development and Intellectual Property (CDIP) E CDIP/21/12 REV. ORIGINAL: ENGLISH DATE: MAY 16, 2018 Committee on Development and Intellectual Property (CDIP) Twenty-First Session Geneva, May 14 to 18, 2018 PROJECT PROPOSAL FROM THE DELEGATIONS OF

More information

COSME Financial Perspectives European programmes and funds to foster growth Madrid 30 October/Seville 31 October 2013

COSME Financial Perspectives European programmes and funds to foster growth Madrid 30 October/Seville 31 October 2013 COSME 2014-2020 2014-2020 Financial Perspectives European programmes and funds to foster growth Madrid 30 October/Seville 31 October 2013 Vilmos Budavari European Commission DG Enterprise and Industry,

More information

Report of Visit to Agency ANI Portugal. Lisbon, 2 May 2016

Report of Visit to Agency ANI Portugal. Lisbon, 2 May 2016 Report of Visit to Agency ANI Portugal Lisbon, 2 May 2016 1 1 Recommendation to the board, Executive summary, Executive Summary: The MPG and the EWG recommends to the Board to invite ANI Portugal (Agência

More information

Lithuania: Pramonė 4.0

Lithuania: Pramonė 4.0 Digital Transformation Monitor Lithuania: Pramonė 4.0 February 2018 Internal Market, Industry, Entrepreneurship and SMEs Lithuania:Pramonė 4.0 Lithuania: Pramonė 4.0 istock.com Fact box for Lithuania s

More information

Evaluation of the Three-Year Grant Programme: Cross-Border European Market Surveillance Actions ( )

Evaluation of the Three-Year Grant Programme: Cross-Border European Market Surveillance Actions ( ) Evaluation of the Three-Year Grant Programme: Cross-Border European Market Surveillance Actions (2000-2002) final report 22 Febuary 2005 ETU/FIF.20040404 Executive Summary Market Surveillance of industrial

More information

ACTIVITY REPORT OF THE NATIONAL INDUSTRIAL COMPETITIVENESS COMMISSION PRAMONĖ 4.0 OF 2017

ACTIVITY REPORT OF THE NATIONAL INDUSTRIAL COMPETITIVENESS COMMISSION PRAMONĖ 4.0 OF 2017 ACTIVITY REPORT OF THE NATIONAL INDUSTRIAL COMPETITIVENESS COMMISSION PRAMONĖ 4.0 OF 2017 23 April 2018 Vilnius 2 I. Introduction On 19 April 2016, The European Commission (hereinafter referred to as the

More information

ClusterNanoRoad

ClusterNanoRoad ClusterNanoRoad 723630 Expert Advisory Board Meeting Brussels April 11th, 2018 WP1 ClusterNanoRoad (723630) VALUE CHAIN OPPORTUNITIES: mapping and benchmarking of Cluster-NMBP RIS3 good practices [M1-M7]

More information

EuropeAid. Sustainable and Cleaner Production in the Manufacturing Industries of Pakistan (SCI-Pak)

EuropeAid. Sustainable and Cleaner Production in the Manufacturing Industries of Pakistan (SCI-Pak) Sustainable and Cleaner Production in the Manufacturing Industries of Pakistan (SCI-Pak) Switch Asia 2008 Target Country Pakistan Implementation period 1.03.2008-29.02.2012 EC co-financing 1126873 Lead

More information

Research Infrastructures in FP6 WORKING DOCUMENT

Research Infrastructures in FP6 WORKING DOCUMENT SUPPORT FOR RESEARCH INFRAST RUCTURES IN THE FP6 SPECIFIC P ROGRAMME ON "ST RUCTURING THE E UROPEAN RESEARCH AREA" WORKING DOCUMENT This document is being widely circulated in order to generate an open

More information

COMMISSION STAFF WORKING DOCUMENT STRATEGY FOR EUROPEAN TECHNOLOGY PLATFORMS: ETP 2020

COMMISSION STAFF WORKING DOCUMENT STRATEGY FOR EUROPEAN TECHNOLOGY PLATFORMS: ETP 2020 EUROPEAN COMMISSION Brussels, 12.7.2013 SWD(2013) 272 final COMMISSION STAFF WORKING DOCUMENT STRATEGY FOR EUROPEAN TECHNOLOGY PLATFORMS: ETP 2020 EN EN COMMISSION STAFF WORKING DOCUMENT STRATEGY FOR EUROPEAN

More information