Second Interim Evaluation of CLEAN SKY Joint Undertaking PANEL REPORT. Pre-Publication

Transcription

1 Second Interim Evaluation of CLEAN SKY Joint Undertaking PANEL REPORT Pre-Publication

2 Table of Contents List of Acronyms... 3 Executive Summary Introduction Background and Implementation of the Clean Sky JU Objectives and scope of the Second Interim Evaluation Methodology of the Second Interim Evaluation Clean Sky - Overall Progress and Effectiveness Progress towards environmental targets Progress towards definitions and development of demonstrators Coordination with FP7, SESAR and National Programmes Effectiveness in promoting participation Effectiveness of ITD and TE strategies Clean Sky response to changing industrial strategies and research needs Clean Sky response to previous evaluations Complementarity with other activities in Horizon Concluding Statements Clean Sky Joint Undertaking - Organisation and Efficiency Appropriateness of the CS legal framework and governance Appropriateness of the JU internal rules and funding JU internal rules Efficiency of funding and budget Efficiency of the JU Executive Team organisation and procedures Efficiency of the JU Executive Team Efficiency of the JU organisational and control procedures Efficiency of ITD organisations and procedures Efficiency of communication Internal communication External communication Concluding Statements Quality Quality of activities Members and Partners quality Quality of Calls for Proposals Concluding Statements Clean Sky ITDs and Technology Evaluator - Progress and Effectiveness Smart Fixed Wing Aircraft (SFWA) Green Regional Aircraft (GRA) Green RotorCraft (GRC) Systems for Green Operations (SGO) Sustainable and Green Engine (SAGE) EcoDesign (ED-ITD) Technology Evaluator (TE) Evolution since 1st Evaluation Introduction Management evolution

3 6.3 Risks follow up from 1st Interim Assessment Scientific and technical comparison Smart Fixed Wing Aircraft (SFWA-ITD) evolution Green Regional Aircraft (GRA-ITD) evolution Green Rotorcraft (GRC-ITD) evolution Systems for Green Operation (SGO-ITD) evolution Sustainable and Green Engine (SAGE-ITD) evolution Eco-Design (ED-ITD) Technology Evaluator (TE) List of Recommendations (for Clean Sky 1) Key Issues and Overall Recommendations for Clean Sky SWOT Analysis Key Issues and Recommendations for CS Conclusions Annexes Composition of the 1st Interim Evaluation Panel Composition of the 2nd Interim Evaluation Panel Short Bio of the 2nd Interim Evaluation Panel Members Terms of Reference Interviews and sources of information JU Executive Team participants to interviews ITD participants to interviews Interaction with the NSRG and STAB Reference documents used in the 2nd Interim Evaluation Procedure comparison among three JUs: Fuel Cell & Hydrogen (FCH), Innovative Medicines (IMI) and Clean Sky

4 List of Acronyms ACARE AEA ANSP ATM ATS CDR CFD CfP CROR CS CSDP CSJU EASA ED ExD EDA EDS ETP FAA FP6, FP7, GAM GAP GB GFS GRA GRC GTF IAS ICAO ITD JTI JU KPI LCA Advisory Council for Aeronautics Research in Europe All Electric Aircraft Air Navigation Service Provider Air Traffic Management Air Transport System Critical Design Review Computational Fluid Dynamics Call for Proposals Contra Rotating Open Rotor Clean Sky CS Development Plan Clean Sky Joint Undertaking European Aviation Safety Agency Eco-Design Executive Director Eco-Design for Airframe Eco-Design for Systems European Technology Platform Federal Aviation Administration Framework Programme 6, 7, Grant Agreement for Members Grant Agreement for Partners Governing Board General Forum of Stakeholders Green Regional Aircraft Green Rotorcraft Geared Turbo Fan Internal Audit Service International Civil Aviation Organization Integrated Technology Demonstrator Joint Technology Initiative Joint Undertaking Key Performance Indicator Life Cycle Assessment 3

5 LNC LWC MAE MTM NC NSRG OR ORA PDR PO PPP REACH RTD SAGE SESAR SFWA SGO SME SRA STAB TE TRL WBS WP Low Noise Configuration Low Weight Configuration Management of Aircraft Energy Management of Trajectory and Mission New Configuration National States Representatives Group Open Rotor Open-Rotor Acoustics Preliminary Design Review Project Officer Public Private Partnership Registration, Evaluation, Authorisation and Restriction of Chemicals Research and Technological Development Sustainable and Green Engines Single European Sky ATM Research Smart Fixed Wing Aircraft Systems for Green Operations Small or Medium Sized Enterprise Strategic Research Agenda Scientific and Technical Advisory Board Technology Evaluator Technology Readiness Level Work Breakdown Structure Work Package 4

6 Signatures BERTOLINI Enzo BROUCKAERT Jean-François (Rapporteur) DI NUCCI, Maria Rosaria HERRERA, Ivonne QUENTIN, Francois (Chairman) 5

7 Executive Summary The report presents the results of the 2 nd Interim Evaluation of the Clean Sky Joint Undertaking (CSJU) performed between March and September In line with Council Regulation 071/2008, the 2 nd Interim Evaluation has assessed the quality and efficiency of the CSJU and the progress towards the objectives. The evaluation was performed by a Panel of five independent experts (hereinafter referred to as the Panel ) based on the Terms of Reference, defined by the Directorate General for Research and Innovation of the European Commission. Two experts out of five have participated in the 1 st Interim Evaluation as well. Part of the mandate was to further elaborate and adapt specific questions addressing the evaluation criteria: effectiveness, efficiency and quality to the CSJU and the JTI technical areas (Integrated Technology Demonstrators ITDs). Key in the assessment was the evaluation of the technical progress achieved and its contributions towards the Advisory Council for Aeronautics Research in Europe (ACARE) goals. The technical progress was made visible to the Panel thanks to visits to most of the companies involved in the ITDs. The Panel drew recommendations for the remaining activities under Clean Sky and - based on the lessons learnt - formulated recommendations for future public private partnerships under Horizon 2020 (Clean Sky 2). The present evaluation is based on a number of documents provided to the Panel by the European Commission and by the CSJU, i.e. general Clean Sky information provided at the Kick-Off Meeting, Annual Review Reports for all ITD s and meeting presentations. The Panel built its assessment on (a) internal documents and published information, (b) direct observations through the technical visits on site, (c), information gathered in interviews with a wide range of Clean Sky stakeholders e.g. representatives of Members, Partners and ITD leaders, members of CS bodies e.g. Governing Board, Scientific and Technical Board (STAB), National State Representative Group (NSRG) as well as representatives of the CSJU Executive Office. The technical visits were essential to deepen the analysis of the technical progress within CS. The Panel recognises the added value of such technical visits, which turned out to be extremely helpful for the assessment. Due to time pressure, the GRC and ED ITDs were not covered by a technical visit but their assessment is based on presentations and interviews. The structure of this report follows largely the one of the 1 st Evaluation Report for consistency reasons. The initial sections deal with the overall assessment of Clean Sky respectively in terms of overall progress and effectiveness (Section 2), organisation and efficiency (Section 3) and quality (Section 4). The bulk of the report is then devoted to the detailed technical status of each ITD mainly acquired through the technical visits on the sites where the research activities are actually performed (Section 5). A separate section describes the evolution since the 1 st evaluation (Section 6) followed by recommendations both for Clean Sky (Section 7) and for Clean Sky 2 (Section 8). Following the evaluation of the CSJU performance, a SWOT analysis (Strengths, Weaknesses, Opportunities and Threats) was performed in order to place the assessment in a broader setting, to review findings and to develop recommendations also for future activities under Clean Sky 2. The Panel is convinced that the CSJU has successfully demonstrated the viability of the Public- Private Partnership (PPP) concept for research in aeronautics. Indeed the Panel collected evidence that the CSJU has been effective in delivering on its main objectives and has been able to reinforce Europe s role for aeronautic R&D. The Panel found the research undertaken within CSJU of high quality. Today, a number of demonstrators are already running or have been tested, and in many cases, the preliminary assessments of the environmental benefits confirm the capability of achieving the overall targets at completion of the programme. The Panel acknowledges the work of the previous evaluation in 2010 and endorses a number of statements and recommendations that in spite of the progress made are still fully relevant after the 2 nd Interim Evaluation. In particular, Setting up the CSJU as an entirely new Public Private Partnership (PPP) organisation has been a significant success on its own. 6

8 The initial top-down work plan has been complemented by a detailed bottom-up work plan. The corresponding schedule foresees achieving key demonstrator targets within the Clean Sky timeframe. Furthermore, the CS timing for demonstrators seems well-synchronized with industrial deployment strategies. The CSJU has been highly successful in attracting a high level and wide participation from all EU key industries and a large number of SMEs. CS has led to new collaborations and the participation of new organisations is thus enhancing European integration. The CSJU is successfully stimulating developments towards the ACARE environmental targets. The first interim evaluation identified many strengths, but also some areas for improvement. The Panel appreciates that both the Governing Board and the CSJU have been responsive to the recommendations of the first interim review and have made much progress in implementing them. A major improvement is the substantial technical progress that has been noted, in particular during the technical visits on site. At the time of the 1 st evaluation (2010), it was noted that the gains were difficult to quantify because the CS programme was still in its infancy. The main conclusions drawn by the Panel after this 2 nd assessment are further elaborated hereafter. The Panel shares the view expressed in the stakeholders consultation in 2012 that the form of the PPP with the JU as an instrument allow for multiannual continuity and visibility. This is one of the strengths of Clean Sky in FP7 as it has enabled to avoid the fragmentation typical of smaller short term projects, and has established the appropriate pan-european structure for meeting the ACARE goals set in Vision 2020s. Overall the Panel considers that the Clean Sky governance is efficient in the management of the programme and delivery of calls and projects and is convinced that the CSJU has created an effective dialogue between industry and research around a common strategic agenda and has successfully implemented it. However, steps for reducing administrative work, increasing the organisational efficiency and enhancing internal and external communication are still required. Notwithstanding that the Executive Office has made significant progress in speeding up processes and reaching operational efficiency, the Panel recommends that some further adjustments are carried out to improve efficiency. Now that the Clean Sky JU is well established, the balance of skills between general administration and project management in the Executive Office needs to be enforced. Regarding the technical progress, the Panel agrees with the first review Panel that significant delays may have accumulated in some ITDs because of the CSJU set up time. The Panel agrees that the slow start of the CSJU can to a great extent be imputed to the lack of preparedness, both administrative and technical, when starting the Joint Undertaking. It is noted that, since then, some of the ITDs have caught up with the planning whereas others have accumulated delays especially when the research content was complex. For some demonstrators, those delays exceed two years. Overall, the Panel believes that the large Clean Sky research and demonstrators portfolio is of high quality. The Panel collected evidence that the JU is perceived as the flagship for Public Private Partnership supported aeronautic R&D in Europe. Overall the Panel was of the opinion that alongside considerable strengths and achievements of the CSJU, there were areas that needed some further attention and where opportunities should be taken. There is no doubt about the quality and the relevance of the technical activities carried out within Clean Sky, but the problems of resource allocation together with slipping schedules may jeopardize this quality is some cases. A full set of detailed recommendations is listed at the end of this report (see Section 7). According to the Panel, the most important recommendations are the following: The Panel assesses the CSJU as an ambitious European initiative with the potential to become an innovative model of a public-private-partnership. The Panel underlines that the CSJU strongly contributes to achieving the roadmaps that have been jointly agreed 7

9 between all stakeholders, considers the multi-annual approach as advantageous and recommends this to be continued in the future. The CSJU should seek to maximize the potential of its advisory bodies to gain support for the remaining calls and other activities at all levels. The Panel considers information exchange between the JU and NSRG very important and recommends that the NSRG continues to play a crucial role in ensuring coherence of national programmes with Clean Sky. The Panel recommends that the STAB involvement be preserved and enhanced for example in drafting the future updates of the SRIA. The role of the STAB is considered very significant, in particular in view of a follow-up of Clean Sky by Clean Sky 2. The Panel agrees that due to the expected change in aircraft replacement strategy, the Clean Sky targets could no longer be achieved in the original CS 2016 time frame for some demonstrators. There is no longer any clear indication about the actual time frame for the aircraft replacement strategy; it raises the question about some contributors motivation to dedicate resources for a long period of time. Some areas of CS are addressing operations which are highly affected by particular interests of stakeholder groups (the entry into service of the replacement aircraft for the A320 was initially foreseen for 2025, but due to the introduction of the A320Neo, has now been postponed to a later date). An early and close interaction with airlines, air navigation service providers, airports, etc. is recommended to ensure successful deployment. It is recommended to create a market advisory group to the CS Governing Board (GB) to better align JU decisions with the market evolution and trends and to advise the GB about the inputs the Technology Evaluator (TE) should feed back to the JU. It is recommended to deepen the existing relationship with both the ATM focused JTI SESAR and ACARE also at working group level to share a better view within the JU at large about the airlines, ANSPs and other stakeholder communities. In order to facilitate the CSJU management process, the Panel endorses the recommendations of the previous evaluation and reiterates that the Governing Board should focus on strategic decisions and increase the level of delegation of routine management issues to the Executive Director. The executive power of the Executive Director needs to be strengthened towards managing all programme activities. Responsibility for the implementation of the agreed executive team maximum budget should be fully given to the Executive Director. The Panel considers the number of the technical staff as being insufficient and recommends a review by the Governing Board of staff requirements to ensure that the Executive Team can exercise in full its coordinating and monitoring functions. At the same time the Panel recommends a review of potential horizontal services to be shared with other JUs and of administrative services that could be outsourced. The Panel considers that the existing possibilities to redistribute the budget amongst ITDs (as the transfer occurred in 2012 between ITDs) are an initial useful step to provide budget flexibility. The Panel is of the opinion that contingency budget can bring about transversal flexibility and regrets that there is no contingency budget at this stage. Therefore the Panel recommends to the Governing Board to consider introducing in the future a 5-10% contingency budget to increase flexibility. A detailed roadmap of technical progress should be established in order to compare achievements against the plan. This roadmap should include key decision-making points and technological milestones. The TE is not yet fully operational. It is not yet used to feed data back to the ITDs. This feedback is considered of great importance to contribute to the consistency of the CS activities. The sensitivity of the aircraft models and the confidentiality of the data about performance improvement associated to technologies should be acknowledged and the benefits of establishing an additional advisory group should be considered. The reader is referred to the recommendation of creating a market advisory group to the GB. 8

10 The envisaged developments involve safety-critical systems and operations. Consequently, certification issues need to be considered already at early design and development stages. The quality of the process of Call for Proposals is considered to be good, provides the appropriate flexibility to adapt to individual ITD requirements and attracts a satisfactory rate of applicants. However the Panel notes that the number of CfPs is very high in some ITDs and is not systematically related to the size of the ITDs. Some other ITDs have experienced delays in CfP preparations and unsuccessful topics. Regarding the setup of potential future PPPs (i.e. Clean Sky 2) the Panel has compiled a detailed list of recommendations (Section 8) of which the main ones are listed below: The Panel recommends that before starting a future PPP, the Commission should ensure that resources including a contingency budget and management tools are available and that an indepth review of the technical programme is carried out. The Panel recommends that the CS communication strategy allows for more efforts dedicated to communicating the broader socio-economic and environmental impacts not only to the aeronautical stakeholders, but also to the policy and decision makers at European and national levels. Both NSRG and STAB should be involved in these initiatives. The Panel believes that communication between ITDs can be improved by using to a larger extent the TE as a tool to feed back information and to discuss efficiency in technical matters. A closer relationship with the working groups of ACARE and SESAR could also improve this communication process. The JU team should be more involved in this process and additional resources need to be allocated to this task. It is noted that the TRL evaluation occurs at a late stage of the Clean Sky plan. By the time the TRL evaluation is performed, design concepts, technological developments and implementation directions have been committed to a great cost. The Panel recommends an early evaluation of the TRL potential and its environmental benefit when a technology is considered for Clean Sky. Lessons learnt from Clean Sky work should also be considered regarding technologies that have been stopped. Additionally to its higher TRL activities, Clean Sky 2 would be an appropriate framework to implement and manage industry-led projects of the size of the former FP7 Level 2 projects. It is important to devote a significant share of the budget to such projects, to bring technologies from TRL 3 to TRL 4 or at best 5, without the a priori objective of contributing to a flying full scale platform demonstrator. It is important that this type of industry-led projects is run directly by the JU without interference from higher TRL projects in Clean Sky. This report is the result of a joint effort and the Panel wishes to acknowledge the support of the European Commission and the CSJU for the organisation of the site visits, and to thank all companies involved and interviewees for their openness and valuable input. 9

11 1 Introduction 1.1 Background and Implementation of the Clean Sky JU The Clean Sky Joint Undertaking was established in 2008 as a Public Private Partnership between the European Commission and the Aeronautical Industry as a Community Body by Council Regulation (EC) 071/ on the basis of Article 187 of the TFEU 2 and in accordance with the Financial Regulation 3. The CSJU is planned to end on December 31 st, Clean Sky is supposed to be followed by a new Private Public Partnership (Clean Sky 2) which is now proposed by the Commission and is undergoing discussions with the Council. The CS2 activities are supposed to run from 2014 to 2024, therefore there will be an overlap between 2014 and 2017 with Clean Sky (CS).. The major aim of Clean Sky is to reduce the impact of aviation on the environment while at the same time safeguarding competitiveness as well as economic growth of the aeronautical sector in Europe and so to contribute to the targets defined by the Advisory Council for Aeronautics Research in Europe (ACARE) for reducing emissions and noise in air transport in Europe. Clean Sky continues to work towards objectives and targets defined in the Strategic Research Agenda of the ETP ACARE 4 and its updates. The CSJU addresses the implementation of innovative, environmentally friendly technologies in all segments of civil air transport, including large commercial aircraft, regional aircraft, helicopters, and in all supporting technologies such as engines, systems and materials life cycle. The maximum overall value of the contributions within CSJU reaches EUR 1,600 million. The Founding Members of the CSJU are the European Union, represented by the European Commission (EC), 12 Integrated Technology Demonstrator (ITD) leaders and 72 Associates. The CSJU activities are subdivided into in six technology areas Integrated Technology Demonstrators (ITDs): Vehicle ITDs: Smart Fixed Wing Aircraft (SFWA) 24% of the EC contribution, Green Regional Aircraft (GRA) 11% of the EC contribution, Green Rotorcraft (GRC) 10% of the EC contribution. Transverse ITDs: Systems for Green Operations (SGO) 19% of the EC contribution, Sustainable and Green Engine (SAGE) 27% of the EC contribution, and an ITD that is transverse to all ITDs: EcoDesign (ED) - 7% of the EC contribution. Around 2% of the budget is devoted to the Technology Evaluator (TE) with the aim of assessing environmental impact and benefits of technologies arising from individual ITDs. Most of the research, technological development and demonstration activities are carried out by the Members of Clean Sky. The Members activities are formally covered by Grant Agreements for 1 COUNCIL REGULATION (EC) No 71/2007 of 20 December 2007 setting up the Clean Sky Joint Undertaking. OJ L 30/1-20, ; see: 2 TFEU: Treaty on the Functioning of the European Union; Article 187 (ex-article 171 of the EC Treaty): The Union may set up joint undertakings or any other structure necessary for the efficient execution of Union research, technological development and demonstration programmes. 3 Council Regulation (EC, Euratom) 1605/2002 of 25 June 2002 on the Financial Regulation applicable to the general budget of the European Communities

12 Members (GAM). There is one amendment to the GAM per year and per ITD which specifies work plan, resources and budget. Subcontractors are selected by Members through Calls for Tender. A part of the Clean Sky programme using 25% of the EC contribution is performed by Partners selected through Calls for Proposals (CfP). In the evaluation period there have been on average three CfP calls per year with on average 38 topics per call and 1.7 Partners per proposal. Including Call 15, the average value for the 579 topics published is 665,000. Successful CfPs lead to the signature of Grant Agreement for Partners (GAP). The average GAP duration is 20 months. The ITD and TE activities are coordinated and integrated by the CS JU Executive Team led by the Executive Director (ExD). The CSJU supervisory body is the Governing Board (GB) with representatives from the European Commission, ITD leaders and one Associate per ITD. The GB receives technical advice from the Scientific and Technical Advisory Board (STAB). For each ITD, a Steering Committee is in charge of supervision and monitoring of the activities. The General Forum provides the platform for involving all participants of CS Members and Partners. These bodies are complemented by the National State Representative Group (NSRG) which advises the CSJU and liaises with the national programmes. 1.2 Objectives and scope of the Second Interim Evaluation The present report is the result of the work of the Independent Expert Group (hereinafter referred to as the Panel ), appointed to assist the Commission in carrying out the second interim evaluation of the Clean Sky Joint Undertaking (CS JU). The evaluation performed by the Panel is based on the Terms of Reference (see Annexes, Section 10.3) defined by the European Commission. The objective of this second interim evaluation is to assess the progress and achievements of the Clean Sky Joint Undertaking as described in the Terms of Reference. The evaluation addressed the following criteria: Effectiveness: The progress towards meeting the objectives set, including how all parties in the public-private partnerships live up to their financial and managerial responsibilities and keep an open non-discriminatory attitude towards a wide community of stakeholders. Efficiency: The extent to which the JUs are managed and operate efficiently. Research Quality: The extent to which the JUs enable world-class research that helps propel Europe to a leadership position globally, and how JUs engage with a wider constituency to open the research to the broader society. An important part of the mandate was to further elaborate and adapt specific questions addressing the above criteria to the CSJU and ITDs so as to draw recommendations for the remaining activities under CS1 and - based on the lessons learnt - formulate recommendations for CS 2. Following the evaluation of the CSJU performance, a SWOT analysis (Strengths, Weaknesses, Opportunities and Threats) was performed in order to place the assessment in a broader setting, to review findings and to develop recommendations also for future activities under CS Methodology of the Second Interim Evaluation The methodology followed by the Panel was based on the Terms of Reference, which provided a set of predefined questions under the evaluation criteria. These questions were subsequently supplemented by an additional set of horizontal as well as specific questions referring to the ITDs, which addressed the specificities of the different actors within the CSJU, and the Panel agreed on a list of people to be interviewed (see Annexes, Section 10.3). The Panel undertook a detailed review of the relevant documents. The documents surveyed can be found in Section

13 The evaluation was performed by the Panel from the 5 th of March until the 31 st of October 2013 with a combination of remote work, conference calls, six Panel meetings and several site visits. The arrangement of technical visits to the companies and facilities within several ITDs represented a novel aspect of this 2 nd Interim Evaluation. Visits allowed the Panel to collect detailed technical information and to see a representative selection of the demonstration hardware realised so far. The scope of the technical visits varied due to resources constraints. A first two-day visit was organized at Airbus, Thales and Liebherr in Toulouse, France (May 23-24, 2013) in relation to the activities within the SFWA, SGO and TE ITDs. All members of the Panel attended this meeting. A second visit was organized at Rolls-Royce in Derby, UK (June 18, 2013) as a one-day meeting and was attended by two members of the Panel. Detailed technical presentations were given to the Panel members about SAGE 1, 3 and 6, and the demonstration hardware was presented during a visit of the workshop. Since no detailed presentations were given about SAGE 2, 4, and 5, one member of the Panel attended as an observer the SAGE annual review meeting held in Trollhättan, Sweden (June 24-28, 2013). Finally, another two day meeting was organized at Alenia-Aermacchi in Pomigliano d Arco, Italy (July 4-5, 2013) and attended by two members of the Panel for the assessment of the GRA ITD. In all these meetings, the activities related to the TE ITD were addressed, so that only the GRC and ED ITDs were not covered by a technical visit and their assessment is based on presentations and interviews. This choice was made by the Commission for reasons of budget resources and time pressure. The Panel built its assessment on (a) internal documents and published information, (b) direct observations and (c), information gained in interviews with a wide range of Clean Sky stakeholders, including representatives of Members, Partners and ITD leaders, members of CS bodies such as Governing Board, Scientific and Technical Board (STAB), National State Representative Group (NSRG) as well as representatives of the CS JU Executive Office (see list in Section and ). This report is the result of a joint effort and the Panel wishes to acknowledge the support of the European Commission and the CSJU for the organisation of the site visits, and to thank all companies involved and interviewees for their openness and valuable input. 12

14 2 Clean Sky - Overall Progress and Effectiveness 2.1 Progress towards environmental targets The ACARE (Advisory Council for Aeronautics Research in Europe) performance targets for 2020 have been set in 2001 as reduction of CO 2 by 50%, of NO x by 80%, noise by 50% and to make substantial contribution in reducing the environmental impact of the manufacture, maintenance and disposal of aircraft and related products, for the overall air transport system (ACARE Strategic Research Agenda, SRA, 2002). In 2007, the CS contribution to the ACARE goals was set as -10 to 20% CO 2, - 10% NO x and -10dB noise by completion of the project in 2017 (see Clean Sky proposal 2007). CS has experienced difficulties to monitor the 2007 indicative targets. These targets were not always consistent across the range of technologies. The product objectives were not clearly defined down to CS specific technologies. The CS Development Plan (CSDP) provides a structured way to monitor and assess the achievement of the environmental goals. Three complementary measures are used. These are the maturity of technologies in terms of Technology Readiness Levels (TRL), the concept aircraft and demonstration programmes. The TRL monitors the maturity of technologies within each ITD. The CS environmental benefits are measured by comparing the existing aircraft (baseline reference Y2000 and Y2020) and a virtual concept aircraft incorporating CS technologies as defined by the aircraft ITDs. By the end of Clean Sky 1, the demonstration programmes will allow to provide evidence of integration of several technologies and to indicate the potential benefits in a relevant operational environment. The Panel notes that the ACARE goals set for 2050 are: 75% reduction of CO 2, 90% reduction of NO x, and 65% reduction of noise relative to R-2.1 CS1 and CS2 related: The current progress is reported in relation to CS objectives. The Panel recommends a more transparent traceability between the ACARE goals and CS specific contribution. R-2.2: The Panel encourages the Partners and Project Managers to provide more clarity and consistency in the figures presented as well as on the assumptions taken for the evaluation of the environmental targets in relation to the ACARE goals. 2.2 Progress towards definitions and development of demonstrators At this stage, all demonstrators have been defined in terms of detailed concepts. Some of the ground or flight demonstrations have already been achieved with success. For some of other demonstrators, unexpected difficulties emerging from the definition phase have led to rescheduling. Regarding the technical progress, the Panel agrees with the observation of the first interim review that significant delays may have accumulated in some ITDs because of the CSJU set up time. The Panel agrees that the slow start of the CSJU can to a great extent be imputed to the lack of preparedness, both administrative and technical, when starting the Joint Undertaking. It is noted that, since then, some of the ITDs have caught up with the planning whereas others have accumulated delays especially when the research content was complex. For some demonstrators, those delays are exceeding two years. 13

15 Overall, the Panel considers that the technical development of the demonstrators is making satisfactory progress. The reader is referred to Section 5 for the detailed description of demonstrator progress within each ITD. 2.3 Coordination with FP7, SESAR and National Programmes In essence, Clean Sky is targeting high TRL level activities in order to finally achieve demonstrator vehicles or hardware. In this sense, it is understandable that there is a certain amount of overlap with FP7 Level 2 projects, which are aiming at developing the underlying technologies at lower TRL level. However, the boundaries between activities carried out within FP7 Level 2 programmes and Clean Sky are not clearly defined or explained. It is difficult to assess where Level 2 projects stop and where Clean Sky starts. The Panel recognises that Clean Sky is intended to bring those Level 2 technologies to a higher TRL level but the issues of double work and duplicating funding should be monitored (see R-SAGE.6 for example). Regarding the coordination with SESAR, many interdependencies exist with several ITDs. Concerning flight management for example, and as a transversal ITD, SGO has direct and indirect interfaces with GRA, GRC, SFWA, TE and of course SESAR. However, the initial link with SESAR was not optimal. Delays on information from SESAR to CS have been identified and have affected progress for example from SESAR to SGO MTM. This problem has been improved in 2012 and common reviews between programmes have been performed. The purpose of these reviews has been to identify potential overlaps in the themes related to flight management. Important interfaces are reported to the general board and trans-itd workshops on common themes are organized. The interfaces are considered as being managed in an adequate manner. Many interdependencies are also seen among ITDs and with other national and EC activities. The Panel recommends incorporating current interface management practices into a specific interface management function. Moreover, formal exchange of information should be established among the CS, SESAR and other research programmes (e.g. Horizon 2020). This step will speed up research work and avoid a potential duplication of work. The Panel appreciates that Clean Sky is re-using existing hardware developed under previous Framework Programmes. R-2.3: It is recommended to deepen the existing relationship with both SESAR and ACARE aimimg - at working group level- to reach a better view within the JU at large about the airlines, ANSPs and other stakeholder communities. Regarding the link between Clean Sky and national research programmes, the Panel notes that the NSRG not only acts as an advisory group, but also represents an important interface with the relevant stakeholders in their respective countries and in liaising with the national programmes - where available - and in dissemination activities. It is also noted that - if compared with analogous bodies in the other JUs (see Appendix 10.4) - the NSRG has had a proactive role in the Clean Sky initiatives. However, it was evident from interviewees comments that there is some regret that the advisory role of the NSRG mostly focuses on the Call for Proposal process and that industry (and the JU executive office) hardly consults them on other matters. The Panel received information that the NSRG would appreciate an early insight in the activity plans in order to better promote results which are not confidential. R-2.4: The Panel believes that information exchange between the JU and NSRG is very important and recommends that the NSRG continues to play a crucial role in ensuring coherence of national programmes with Clean Sky. 2.4 Effectiveness in promoting participation The Panel considers the participation of SMEs and the increase of new entrants in the JU and the CfP procedures and regulations as satisfactory and appreciates the clear industry commitment to the programme. 14

16 The procedures, according to which the single entity applying is eligible for 50% or 75% and depending on the legal status (for example industry or SME), appears adequate 5. The Panel notes that the average funding rate in Calls is 65.6% and considers satisfactory that the applicants success rate is approximately 35%. The calls and the JU appear to be successful in attracting new players and the Panel notes that approximately 50% of the partners are new. Although CS-JU is not perceived as SME-friendly, for SMEs the participation in CS is obviously attractive, especially for the opportunity to enter in the supply chain. The Panel appreciates that SMEs account for 38% of participants in CfP and absorb 36% of the budget. There is a number of SMEs also amongst the associates. R-2.5: The Panel appreciates that Clean Sky does not require a consortium as a condition for participation to calls for proposals; even a single entity can apply and that there are a number of mono-beneficiaries also amongst SMEs. It however recommends making the high participation of SMEs and of new players more visible (seeing also 3.5 Efficiency in Communication). 2.5 Effectiveness of ITD and TE strategies At this stage, all ITD strategies are defined; most of them are considered relevant and effective. The effectiveness of the strategies is depending on the capability of the ITD to adapt to the changing market requirements. The reader is also referred to Section 5 (ITD progress). TE still requires a further maturity gain before playing its full role in the assessment of the environmental benefits. The Panel believes that TE approach has a very high potential for being adopted in other sectors to assess environmental benefits. R-2.6: The Panel recognises that TRL concept has been refined during CS and recommends the CSJU to disseminate the results across the R&D community. R-2.7 Lessons learnt for CS 2: The Panel recommends that a TRL check is performed before a technology is considered as a valid candidate for a CS project. This will avoid delays and difficulties due to uncertain and/or low TRL. R-2.8 CS1 and CS2 related: The visit provided evidence of very good cooperation between research development activities and flight test preparations. Detailed reviews have been conducted including multidisciplinary teams with experienced personnel in flight test. Moving from the example of the good GRA flight test preparation, the Panel recommends to ITDs to make greater efforts to communicate and disseminate best practices and encourages them to extract from successful cases of other ITDs useful lessons for own future activities. 2.6 Clean Sky response to changing industrial strategies and research needs The main change occurring in Europe during Clean Sky was probably the postponement of the new air transport short-range aircraft far into the 2020 s. As a consequence some key technologies are less under pressure to reach TRL 6 by Changes and adaptations have been related to address technological setbacks, cancellation of technologies, introduction of new technologies from on-going technological developments, reduction of TRL scope, delays in test rig building, Intellectual Property Rights issues and withdrawal of some partners. Decision-making needs to consider trade-offs between most promising technologies, its industrial applicability, schedule and budget. There are activities that 5 In case of a consortium, both funding criteria apply and the resulting funding is an average of the two percentages, weighted by the actual contributions of each partner. 15

17 have been deleted in the updated GAM, the rationale and consequences are not justified with sufficient detail. R-2.9 Lessons learnt for CS2: The traceability and evolutions of the GAM should be better documented to establish and assess its overall compliance and performance. Further, this traceability should track changes in the GAM and its impact. This action ensures the ability of the programme to adapt to new challenges and opportunities. 2.7 Clean Sky response to previous evaluations An adequate procedure to analyse and implement recommendations from the 1 st Interim Evaluation was presented to the Panel. Most of the recommendations targeting the JU and the Governing Board concerning implementation bottlenecks have been realised or are under implementation. However, the completion of a number of recommendations is still pending or is on-going. The Panel estimated that the CSJU and ITDs have implemented more than half of the recommendations from 1 st interim evaluation. The Table below shows an overview of recommendations and the status of their implementation. Table 1: Status of implementation of the recommendations Category Overview Closed In progress No action Comment GB 7 recommendations Limited staff still no action Future PPP 3 recommendations 3 In progress with H2020 definition Schedule and risk management 13 recommendation 11 2 No formal review on GAPs, No contingency plan moved to CS2 Call & GAP 5 recommendations No action due to lack of resources. No full responsibility of implementation to ExD Policy 2 recommendations is relevant to the EC Management 7 recommendations No extra staff Communication 3 recommendations 1 2 Assessment 3 recommendations 2 1 Coordination 5 recommendations NSGR coordination still needs to be improved Diverse 2 recommendation 2 Total The Panel appreciates the systematic process for evaluation and implementation. It is also noticed that the implementation of the recommendations requires time and additional efforts. 16

18 2.8 Complementarity with other activities in Horizon 2020 Clean Sky 2 industry-led projects (equivalent to FP7 Level2 collaborative projects). Clean Sky has demonstrated the ability to run projects of the size of FP 7 Level 2 research projects. The CSJU is well organised and has developed efficient processes in order to run collaborative research projects. They involve Research Establishments, SMEs and Universities. R-2.10: Additionally to its higher TRL activities, Clean Sky 2 would be an appropriate framework to implement and manage industry-led projects of the size of the former FP7 Level 2 projects. It is important to devote a significant share of the budget to such projects, to bring technologies from TRL 3 to TRL 4 or at best 5, without the a priori objective of contributing to a flying full scale platform demonstrator. R-2.11: It is important that this type of industry-led project is run directly by the JU without interference from higher TRL projects in Clean Sky. R-2.12: These projects should use the Technology Evaluator to provide inputs during the evaluation phase and to assess environmental impact and efficiency at the end of the projects. 2.9 Concluding Statements The Panel is convinced that - in spite of initial delays due to the slow start the JU has marked satisfactory progress towards meeting the objectives set and has manifested an open nondiscriminatory attitude towards a wide community of stakeholders. In particular, there has been an effective strategy (e.g. methods, processes and tools) in launching and managing the Calls for proposals, in selecting the best proposals, managing successfully level 2 like projects and promoting participation of SMEs and increasing the rate of new entrants in the JU and the CfP. The existing links with both SESAR and ACARE need being enhanced and it is important to reach a better view within the JU at large about the airlines, ANSPs and other stakeholders. Also the technical development of the demonstrators is making satisfactory progress. The Panel believes that by the end of Clean Sky 1, the demonstration programmes will allow to provide evidence of integration of several technologies and to indicate the potential benefits in a relevant operational environment. 17

19 3 Clean Sky Joint Undertaking - Organisation and Efficiency In considering the appropriateness of the organisation and the efficient use of resources, in line with the terms of reference of the second interim evaluation, several aspects were analysed. These include the clarity of the overall legal framework and the modalities for the implementation of the JU programme, the governance structure and processes, the robustness of the monitoring and control system including the level of supervision/control within the JU and the appropriateness of the available capabilities to monitor progress the use of funding and the communication and dissemination strategy. 3.1 Appropriateness of the CS legal framework and governance The Clean Sky JU as a public-private-partnership between the European Union, represented by the Commission (public partner) and Industry consisting of 12 ITD leaders and 72 Associates (private partners) represents a suitable vehicle for stimulating aeronautic research and development in Europe. The Panel considers the JU legal framework as set out in its Statutes to be appropriate. In the review period, the CS JU has increased its efficiency and has implemented most of the 1 st Interim review recommendations. The Clean Sky JU governance, based on three bodies (Governing Board, Scientific Technical Advisory Board, Executive Director with the support of the Clean Sky Executive Office) and supported by external advisory bodies (National States Representatives Group and Stakeholder Forum) appears well-suited to achieve the Clean Sky objectives. The Panel reviewed the governance documents regarding the Clean Sky JU, interviewed members of various bodies and committees, considered their roles appropriate and found all governance bodies well integrated and working efficiently. On the whole, the Panel rates the present CSJU governance form as appropriate and efficient. The STAB and NSRG mandates are clear within the governance structure and their current configuration appears adequate for these mandates. The governance structure represents a valid model to be continued also in the future. The Governing Board (GB) is working well. The industrial governance structure has proven to be sound and efficient. Criticism has been expressed about the fact that the role of Associates in the governance has been limited and fragmented and the possibility for strengthening their role has been voiced. The Panel analysed this issue and concluded that the role and achievements of associates are sufficiently considered as the Associate Member is supposed to seek advice from the other Associates and hence they are adequately represented in the Governing Board. The Panel regards amendments as not necessary. The Steering Committee for ITD: There are several activities addressing coordination among ITDs. The Panel collected evidence that potential synergies between ITDs are not fully exploited. For example, coordination with SAGE and modelling activities between TE and other ITDs are necessary to ensure timely results. The Scientific Technical Advisory Board (STAB): The STAB is part of the governance structure, but its role is primarily advisory and not in decision making. The Panel notes that the STAB can count on highly qualified members that (pro)actively participate in the Clean Sky activities, especially in the review and monitoring process. The Panel appreciates that members of the STAB participate in annual and various other reviews and that each STAB member is associated with at least two ITDs and checks the quality of the reports they deliver and that they have produced since 2012 a synthesis of the annual reviews outcomes. The Panel values the STAB working groups on socio-economic implications and follows with interest the recent development of a matrix with various criteria addressing innovation, environment, competitiveness, etc. R-3.1.1: The Panel recommends that the STAB contributions needs to be preserved and enhanced for example in drafting the future updates of the SRIA. Their role also for a CS2 is considered 18

20 significant and it is recommended to ensure that high quality individuals are willing to be involved as it is the case in Clean Sky. The National States Representative Group (NSRG): The Panel notes that the NSRG not only acts as an advisory group, but also represents an important interface with the relevant stakeholders in their respective countries and in liaising with the national programmes - where available - and in dissemination activities. It is also noted that - if compared with analogous bodies in the other JUs (see Appendix 10.6) - the NSRG has had a proactive role in the Clean Sky initiatives. R-3.1.2: Notwithstanding the valuable involvement of the advisory bodies, there is still room for a greater and more pro-active involvement of the STAB and NSRG. The CS-JU should seek to maximize the potential of its advisory bodies to gain support for the remaining calls and other activities at all levels. R Lessons learnt for CS2: The Panel believes important that a constant feedback on National Programmes to the JU takes place and that in the future the NSRG maintains a strong role and continues to exchange experiences, to advise and provide recommendations to the JU Executive team. The General Forum of Stakeholders (GFS) The General Forum of Stakeholders is working well and appears efficient; it plays an important role for partners with no direct access to the ITD steering committee. The Panel endorses plans to reshape the GFS with workshops and working groups. 3.2 Appropriateness of the JU internal rules and funding The first interim evaluation criticised that the Community Body status of the CSJU entails rules and uses procedures not common to industrial practice. These rules and procedures were alleged to be constraining and inhibiting for achieving the Clean Sky objectives. In general the JU has stepped up its efficiency over the review period, especially through implementation of most of the 1st interim review recommendations. The Panel notes that important steps have been taken to reduce red tape and to introduce some elements of flexibility. However, there is still room for improvement JU internal rules The Panel notes that there have been improvements in the JU internal rules and procedures (in line with the developments proposed within the 2008 review and the first interim review in 2010) to enable Clean Sky to reach its full potential. Since the last evaluation some flexibility has been achieved. However, the Panel considers that more flexibility in the JU internal rules and procedures are necessary to enhance Clean Sky efficiency. This regards especially attributing more autonomy to the executive director and granting a certain budgetary flexibility. The Panel also shares the view expressed in the stakeholders consultation in 2012 that the PPP in form of a joint undertaking is an appropriate instrument that allows multi-year continuity and visibility. This is one of the strengths of Clean Sky in FP7 as it has enabled to avoid the fragmentation typical of smaller short term projects, and has given the appropriate setting for meeting the ACARE goals set in Vision 2020s. R-3.2.1: The Panel underlines that the Clean Sky JU also contributes to achieving the roadmaps that have been jointly agreed between all stakeholders, considers the multi-annual approach as advantageous and recommends this to be continued in the future. The previous interim evaluation recommended reviewing the level and type of GAM-related decisions which could be delegated to the JU Executive Director and recommended that the GB should focus on strategic decisions and delegate routine management to the Executive Director. It was found important to strengthen the executive power of the Director especially towards ITDs. The Panel shares this view and believes that too little progress has been made to that extent. 19

21 The previous interim review underlined that amendments to the GAMs are negotiated every year, even though activities covered by GAMs are multi-annual. They remarked that annual budget process provokes certain rigidity in the CS multi-annual work plan, worsened by the fact that the annual budget tends to become frozen well before the start of the year. The Panel shares this view. R-3.2.2: The Panel regrets that concerning the negotiation of a multi-annual GAM, there continues to be a need for more flexibility in the management of GAMs. In general, the Panel recommends more discretionary power for the Executive Director in management matters and believes that GAM budget transfers should be initiated, negotiated and implemented by the Executive Director. This step would help speeding up the implementation of necessary decisions since it would no longer be necessary to involve the Governing Board.. R-3.2.3: The Panel is aware that recommendations have been issued concerning completeness and timing of the strategic planning (CSDP) and alignment with annual planning (AIP) and annual amendments of the GAMs. In this context a specific finding has been raised by the Internal Audit Service (IAS) concerning subsequent changes of topics compared to the approved AIP. The Panel endorses plans to delegate a number of decisions and functions from the GB to the ED for the approval of such changes to ensure the required flexibility for the JU to adapt the lists of topics to the actual needs during the year Efficiency of funding and budget The three different levels of engagement and commitment through Founding Members, Associates and Partners have demonstrated their feasibility. The Panel shares the view that the present funding procedure is adequate, though not entirely efficient. The Panel regrets that there is still little budget flexibility to shift budget from one ITD to another and that a qualified majority is needed. Also the limited flexibility to shift funds to later years can be considered as an obstacle to increase efficiency. This applies for both members activities within the ITD programme, and partners activities through CfPs. Budgetary constraints can be problematic when delays occur and there are no straightforward mechanisms to shift unfinished tasks from one year to another. The Panel notes that in the initial phase of the JU there has been under-spending, which has now been recovered. Now, certain flexibility is available and there are possibilities to shift budget according to the so called 3-years rule. R : The Panel considers that the existing possibilities to redistribute the budget amongst ITDs (as the transfer occurred in 2012 between ITDs) are an initial useful step to provide some budget flexibility. The Panel is of the opinion that contingency budget can bring about transversal flexibility and regrets that there is no contingency budget. Therefore the Panel recommends to the Governing Board to consider introducing a 5-10% contingency budget to increase flexibility. Concerning the planning processes of the JU s grant management, the Panel notes that the IAS has put forwards recommendations concerning the JU s control of the multi-annual and annual budgeting. The IAS requested the JU to ensure the overall budget re-allocation at programme level including its timely approval by the GB and to complete the information provided in the GAMs on budget distribution. The Panel understands that re-allocation of the budget to completion has been established by the JU and has been approved by the GB. The Panel endorses this recommendation and considers these actions necessary to improve the efficiency of the process. The private stakeholders in Clean Sky contribute financially to the running costs of the JU and the management costs. This creates considerable administrative burden as all Members are invoiced. In addition, Associates are expected to contribute to the management costs of the ITD leaders, without being able to charge their own management costs. These bottlenecks need to be addressed and straightforward solutions looked for. The Panel is aware that the procedures in use to verify that Members in-kind contributions to CS match the cash contribution from the EC. The verification is carried out at three levels, by audits inside the Members organisations, by a CS audit on the basis of the documents provided and by an ex-post audit of Members expenses against the specified GAM activities. 20

22 R-3.2.5: The Panel is of the opinion that the verification of in-kind contribution is still a laborious and time-consuming issue to manage and to negotiate and that the current procedure is not efficient. Therefore it recommends steps to simplify the procedure. R Lessons learnt for CS2: It has been critically remarked that the Clean Sky Financial Regulations only allow for either 20% flat rate without justification or real overheads and that there is nothing in between. For CS 2 it is recommended to verify whether there are more efficient solutions. 3.3 Efficiency of the JU Executive Team organisation and procedures Efficiency of the JU Executive Team There is an authorised ceiling of 24 members of staff. Of these there are eight project officers; 75% of staff dealing with operational activity (technical and financial); six staff on horizontal support, e.g. Executive Director, Head of Administration, secretary, Internal Auditor etc. The direct management of the research programme is carried out by eight project officers. Of the estimated 442 projects, 342 GAP (Grant Agreement for Partners) and 7 GAM (Grant Agreement for Members) each Project officers appears to manage 1 GAM and 60 GAPs on average. Given the complexity of the process, the high technical level of the work and the large number of projects this is commendable. The Panel recognises the heavy work load of the JU Executive Team by JU management tasks, CfPs, grant agreements, reviews, and ITD monitoring. These figures suggest that the JU - when measured by projects managed per project officer - is very efficient. There is however a patent imbalance between technical and administrative staff, and this may cause excessive indirect costs which can be partly explained by the small size of the organisation and an apparent need for autonomous services in administration, legal affairs, human resources, accountancy, information technology, auditing, procurement, etc. Considering that there are other JUs also located in the same premises of the CSJU it is hard to justify this extent of autonomy. The Panel is of the opinion that savings could be achievable by sharing services with other JUs 6. R-3.3.1: Notwithstanding that the Executive Office has made significant progress in speeding up processes and reaching operational efficiency, the Panel recommends that some further adjustments will be carried out to improve efficiency. Now that the Clean Sky JU is well established, the balance of skills between general administration and project management in the Executive Office needs some readjustment. R-3.3.2: The Panel considers the number of the technical staff as being insufficient and recommends a review by the Governing Board of staff requirements to ensure that the Executive Team can exercise in full its coordinating and monitoring functions. At the same time the Panel recommends a review of potential services to be shared with other JUs and of administrative services that could be outsourced. R-3.3.3: The Clean Sky Executive Office should seek further ways of reducing bureaucracy and ensure that it has the optimal organisational structure for the tasks ahead. R-3.3.4: Although participation and success rate of the applications indicate that the performance of the JU in administration of the programme, project management and programme design and implementation is adequate and capable, the Panel notes that the Time to grant is still rather high (240 days from call publication to GAP; 360 days on average for grants signed in 2012) and recommends this to be shortened. 6 There are already services that are shared as logistics (building and the IT infrastructure). There is a regular coordination between Internal Audit Functions of the three JUs (IMI, FCH and CS; see Appendix 10.6) in place for issues of horizontal nature (e.g. audit methodology, approach towards the Court of Auditors). Audit services are also shared between JUs when it is the most cost-efficient solution (e.g. common framework contract on Ex-Post audits, joint engagements, etc.). 21

23 Efficiency of the JU organisational and control procedures The Panel considers the organisational and control structure of the JU Executive Team to be adequate to fulfil its objectives. The Panel appreciates that a Management Manual describing internal rules and procedures has been operated for years. Concerning the internal control system and quality management, the Panel understands that the process structure of the internal control system and quality management including steering the JU (CS strategy, annual objectives, AIP, quality management, budget, accounts, risk management, financial reporting, etc.); Programme Management, Management of the executive team, Communication Management and Quality Management (process management, ICS, periodic progress reviews, KPIs, management of exceptions, ex-post audits) and Audits Management represents a complex, valid instruments of internal control and quality system contributing to an efficient management. R-3.3.5: The Panel recognises the value of the adopted system of 16 internal control standards representing a robust system for an efficient and effective management, notes that there is a satisfactory alignment of strategic and annual planning and recommends its systematic implementation. The Panel understands that the Clean Sky Internal Audit Officer has a focus on advisory services, risk assessment, ex-post audit process and that the AO has internal advisory function and partially management role. The Panel is convinced by the arguments of the CSJU stressing the high added value of an internal audit function and considering this a more efficient solution than outsourcing. The Panel shares the view expressed in the stakeholders consultation of 2012 that programme activities and CfPs should be implemented on a multi-annual basis and that a mechanism has to be implemented to shift funds and activities from one year to the next one. The preferred solution would be multi-annual financial commitments comparable to L1 and L2 projects in collaborative research. R-3.3.6: The Panel appreciates the intention of the JU (as in the GB meeting of ) to launch trainings for Topic Managers and endorses endeavours to increase the monitoring from the Project Officers and the administration team, to make sure that delays and problems in execution of the projects are tackled as soon as possible. These steps are important ones to address bottlenecks currently limiting the overall efficiency. R-3.3.7: The Panel appreciates that in the evaluation period ex-post audits of financial statements of CS JU beneficiaries have been implemented and recommends that the efforts undertaken to reduce the error rates will be continued. It values that the JU has put efforts in improving its exante validation process and has provided extensive guidance to its beneficiaries concerning the eligibility of costs for the Clean Sky projects Efficiency of ITD organisations and procedures The Clean Sky structure, with six separate Integrated Technology Demonstrators (ITDs) covering Green Rotorcraft, Regional Aircraft, Eco-Design, Engines, Smart fixed Wing and Green Operations, each led by two industry leaders has proven to be effective. 7 To date, 65 audits have been launched, out of which 52 have been finalised. Audit results have been implemented (i.e. overpayments were recovered) with more than 96%. The residual error rate, reflecting the remaining errors in favour of the JU - after corrective measures have been taken place- passed from 4.22% in 2011 to 1.29% in 2012, resulting in an accumulated rate of

24 However, the Panel notes that management processes and tools differ from ITD to ITD. There is no evidence of harmonized management approaches, including resource allocations, milestone achievements, deliverable measurements and budget spending. These issues will be further analysed under Section 5. R-3.4.1: The Panel appreciates that the monitoring and control tools are mature and implemented. The Panel recommends harmonised progress activity reports and technical evaluation reports across ITDs. In particular, progress reports should contain achieved progress against plans, and achieved deliverables against planned deliverables. The Panel recommends technical evaluation reports to follow the EC standard. This standard is useful in terms of evaluating in a systematic manner technical and management aspects. 3.5 Efficiency of communication In general, the Panel commends the JU Executive Team for the adoption of a Communication and Dissemination strategy, but considers communication and dissemination as key issues deserving even more attention Internal communication Communication between and within the JU Executive Team and ITDs appears to be satisfactory. Apart from channels such as the ITD Steering Committees, there are direct links between ITD leaders and the JU project officers which have proved crucial in identifying in a timely manner difficult issues and interfaces. General communication with all stakeholders is achieved through the General Forum, which is especially helpful for partners without a direct access to ITD Steering Committees. Communication between ITDs is still limited and should be enhanced. R-3.5.1: Cooperation and exchange between ITDs appear to be still limited and should be enhanced. Models and tools produced across ITDs should be analysed in the view of potential complementarities. The TE interface with other ITDs deserves careful attention to ensure timely results. R Lessons learnt for CS2: The Panel believes that communication between ITDs can be improved by using to a larger extent the TE as a tool to feed back information and to discuss efficiency matters. A closer relationship with the working groups of ACARE and SESAR could also improve this communication process. The JU team should be more involved in this process and additional resources need to be allocated to this task External communication The Panel appreciates that achievements have been made in the Communication and Dissemination strategy and notes that a number of publications have been released. However it regrets that there are no adequate metrics to document and measure outreach and satisfaction rate. The Panel values that Clean Sky has been involved in the organisation/participation in major events including also Clean Sky SME Day in May 2013 and the Paris Air Show in June However, the Panel believes that more structured outreach activities and promotion are needed in nonscientific/technical media. The Panel is of the opinion that there is a need for a communication strategy with overarching goals for increasing awareness levels and perception of Clean Sky amongst all target groups in order to reach new players and SMEs and involve them in the development process. R-3.5.3: The Panel believes that raising the profile of Clean Sky and the importance of being a PPP are key aspect of CS s communications objectives. The Panel endorses the recommendations of the previous interim evaluation and reiterates that CS should improve its visibility to the interested public. R-3.5.4: The Panel appreciates the effort on the part of the Executive Office to communicate call topics and disseminate the Clean Sky initiatives via publications. However the Panel felt that, as there have been more successes stories coming out of the projects, these could form the basis for 23

25 intensified dissemination targeted to a broader range of stakeholders, including policymakers within the Member States. R-3.5.5: The technical information on the website should be improved, with more active involvement and input from the ITDs. Moreover it is deemed necessary to find forms for communicating the activities and assessment of the TE. R-3.5.6: The Panel recommends that the CS communication strategy puts more dedicated efforts for communicating the broader socio-economic and environmental impacts not only to the aeronautical stakeholders, but also to the policy and decision makers at the European and national levels. The NSRG and STAB should be involved in these initiatives. The Panel appreciates that the JU plans to meet some Permanent Representations of Member States, in cooperation with the NSRG members and understands that the European Parliament is a major target, in particular in the preparation of CS2. Nonetheless a strategy how to address institutional actors and MEPs appears to be missing and is deemed necessary. The Panel believes that in this process the NSRG should be actively involved and should activate organisations and multipliers within their specific countries. Additionally, the members network could provide local insight and implementation for specific campaign measures. R-3.5.7: The Panel commends that Clean Sky has been successful in attracting a high level of interest from companies, well above the average participation of industrial entities in collaborative projects in FP7. However the Panel notes that although there is a remarkably high participation of SMEs, Clean Sky is still perceived as big industry and big technology and therefore recommends that success stories involving SMEs should be communicated on the website and in dedicated publications. 3.6 Concluding Statements Overall the Panel believes that the Clean Sky governance is efficient in the management of the programme and delivery of calls and projects and considers the present governance structure a valid model to be continued also in the future. However, efforts for increasing the organisational efficiency, reducing the administrative burden and enhancing internal and external communication are still required. The Panel recommends strengthening the resources of the JU alongside with the streamlining of the potential services which could be shared with other JUs. Communication and dissemination efforts are satisfactory; however it is deemed necessary that the CS communication strategy puts more dedicated efforts for communicating the broader socio-economic and environmental impacts not only to the aeronautical stakeholders, but also to the policy and decision makers at the European and national levels. The NSRG and STAB should be involved in these initiatives. 24

26 4 Quality 4.1 Quality of activities For the 1 st Interim Review of Clean Sky, the Panel stated that no in-depth assessment of the overall quality of the activities was attempted by the Panel. The assessment was based on specific technical examples for each ITD which were presented to the Panel. These examples provided evidence of the high quality of activities. For this 2 nd Interim Evaluation, a major change was the arrangement of technical visits on site, with at least a full day meeting (or more) of technical presentations. Those site visits have been appreciated by the Panel, as they provided specific technical information and the possibility to discuss - in the workshop - with engineers who directly participated in the product development. The current assessment of the quality of the research activities is therefore based on a much more empirical base. Section 5 reports in detail the status of technical progress which could be assessed in this manner for each ITD. Technical presentations, documentation and technical visits provided evidence of excellent specific technical achievements in many cases. The technical presentations provided evidence of the complexity of certain tasks in the demonstrator developments. Many promising technologies were presented by the ITDs in terms of software and hardware. There is evidence that the CSJU has contributed to progress beyond the state-of-the-art in Aeronautics. R-4.1: The Panel recognises the added value of technical visits and technical presentation meetings which provide more insight and allow a deeper analysis and enable an objective assessment. The Panel considers this as a key instrument to assess the quality of the technical developments and recommends to make site visits an integral part of the review process. There is no doubt about the quality and the relevance of the technical activities carried out within Clean Sky. However, problems of resource allocation together with slipping schedules may end up jeopardizing this quality is some cases. R Lessons learnt for CS2: The Panel recommends to all participants to carry out a realistic risk analysis and establish early mitigation plans. For large ITDs it is recommended to adopt systematically an industrial project management methodology from the very beginning of the project. In some areas, the Panel noted that the CSJU could benefit from advances in other industries e.g. ED-Design would profit from developments in the automotive industry. Overall, the Panel believes that the large Clean Sky research and demonstrators portfolio is of high quality and in some cases excellent. The Panel collected evidence that the JU is perceived as a flagship for Public Private Partnership supported aeronautical R&D. 4.2 Members and Partners quality The members and partners are of high quality as are the major players in the European aeronautical industry. They possess the critical mass needed to achieve the ambitious CS objectives. The Panel appreciates that there is a wide diversity in terms of stakeholders, including industry, academia, research organisations, and SMEs and that a good fraction of them are coming from domains other than aeronautics. However, it is noticed that some partners have not attributed strong priority to CS work causing delays on specific developments, in most cases by not allocating the required resources. 4.3 Quality of Calls for Proposals The CSJU provided information about topics and outcome from the evaluation of the CfPs. The proposals are evaluated in terms of specific criteria: technical excellence, innovative character, 25

27 adequacy and quality of the respondents and contribution to European competitiveness. These proposals are evaluated by internal ITD experts and external experts. Good examples of high quality developments produced by CfPs and used within ITDs were provided during the review. The quality of the process is good, provides the appropriate flexibility to adapt to individual ITD requirements and attracts a satisfactory rate of applicants. However the Panel notes that: The number of CfPs is very high in some ITDs and is not systematically related to the size of the ITDs: Eco-Design, for example, has launched a high number of CfPs. Other ITDs have experienced delays in CfP preparations and unsuccessful topics e.g. SGO and SAGE. R-4.3: In case of a large number of proposals for a specific ITD, the Panel recommends a flexible distribution of responsibilities in order to optimise the associated work load within the JU. The Panel is aware that there have been some complaints about the rigidity of the topic description received by the applicants and for not allowing enough room for innovation. R-4.4: It is proposed that the topics include the possibility to present a more innovative approach leading same results than the one described in the topic. The technical quality and the relevance of the CfPs have not been analysed in detail by the Panel. Technical ITD reviews do not analyse the quality of CfP in a systematic manner. Still, the Panel believes that the technical quality regarding the objectives of the research and the relevance of the content regarding the research pursued by the ITD can be improved in some instances. In some cases it also appeared that developments starting at very low TRL (1-2) were proposed as CfPs to be launched at a late stage of projects. This weakens credibility and could be interpreted as a means of using underspent budgets. R-4.5: It is recommended that the technical ITDs reviews include a systematic CfP review to monitor and contribute to the high quality of the CfP. This would establish a clear connection between CfP topic and ITDs activities thus improving the focusing of the technical activities. R-4.6: The Panel notes that, in some cases, the inappropriate choice of subcontractors has led to poor results relative to the project they are related to. The Panel therefore recommends the JU to investigate possible ways of improving the selection process of subcontractors. 4.4 Concluding Statements There is no doubt about the quality and the relevance of the technical activities carried out within Clean Sky. However, problems of resource allocation together with slipping schedules may end up jeopardizing this quality is some cases. Overall, the Panel believes that the large Clean Sky research and demonstrators portfolio is of very high quality. The Panel collected evidence that the JU is perceived as a flagship for Public Private Partnership supported aeronautical R&D. 26

28 5 Clean Sky ITDs and Technology Evaluator - Progress and Effectiveness 5.1 Smart Fixed Wing Aircraft (SFWA) The SFWA Objectives The objective of the Smart Fixed Wing Aircraft ITD flying demonstrator is to develop and validate up to TRL 6 innovative technologies which were at TRL 2 or 3 level at the time of the CS launch in order to demonstrate a step improvement in the area of fuel consumption and noise emissions. To this end the SFWA ITD integrates innovative wing and airframe concepts of different types. The objective to bring most of these technologies up to TRL6 maturity (last stage of technology maturity before development phase) will allow trade-offs between technologies and technology insertion risk management to be performed prior to insertion of these technologies on a large scale new aircraft development programme. The SFWA Structure and Research Programme The SFWA ITD programme is quite complex. It consists of major components and technology streams. The three major components paths are: Smart Wing Technology (SFWA1); New Configuration (SFWA2), linked to interfaces and technology assessment and Flight Demonstration (SFWA3). Currently, this ITD addresses eight technology streams as shown in the Figure below. Figure SFWA and technology streams Natural Laminar flow: The objective is to achieve TRL 6. The major technologies are aerodynamic wing design, structural concepts and actuation (leading edge Kruegers), antiicing, surface quality, trailing edge for both high speed and low speed operations, health monitoring. The work to be performed includes development, integration, manufacturing process demonstration, maintainability demonstration and flight test demonstration. Hybrid laminar Flow: it is considered as a fall back solution in case of disappointing results for the Natural Laminar Flow projects. Fluidic Control Surfaces: it is designed to enhance high lift performance. It uses innovative concepts to generate high lift of leading edge and trailing edge. Numerical studies have been 27

29 produced. Computational fluid analysis and wing tunnel were presented in the reviewed documents (e.g. wind tunnel test on a passive leading edge device). Load Control Functions and Architectures (TS5): it focuses on design and evaluation in wind tunnel. It covers active load control with sensors and control surfaces, passive methods using structural and aero design, vibration and damage control. Buffet Control: it is focused on design and wind tunnel tests. Developments use passive and active devices relevant for turbulent and laminar flows. Test campaigns aim to demonstrate the achievement of TRL4 by using several study concepts and possibly a large scale model. CROR, engine integration: it is designed to test interactions between engine and airframe in flight. The flying test bed selected is an A340. It is a complete design; integration and test projects include wind tunnel and flight tests. Integration of Innovative Turbofan engine to Bizjets: It is the integration of a rear fuselage section with rear mounted engines. The project will address design and ground tests. A full scale model was planned to demonstrate acoustics, thermal and vibration characteristics. Advanced Flight Test instrumentation: it is designed to support the testing phases of the Flight Demonstrators. Two approaches are used: i) low risk integrating sensors with high TRL that do not compromise the demonstrator and ii) novel sensors with the potential to improve significantly the demonstrator in terms of new knowledge. There is a need for a new technology in this field enabling fine measurements during the test phases. The technologies need to be selected and brought to TRL 6. The original plan included nine technology streams. The innovative control surfaces technology stream has been merged with the other technology streams. These technology streams contribute to the following key activities: Smart Wing Technologies encompassing development, integration and flight test demonstration on a large scale. The topics addressed by this activity are: Natural Laminar Flow, Hybrid Laminar Flow and Active and Passive load control. This activity addresses also the innovative enabling materials and innovative manufacturing technologies required to provide TRL 6 validated solutions for each of the topics. Innovative Power Plant Integration is focused on technology integration and the objective is to provide large scale flight demonstration. This activity addresses impact of airframe flow field on propeller design (acoustics, aerodynamics, vibrations) and impact of Open Rotor configuration on airframes (certification, structure, vibration modes, noise). Innovative empennage design concentrates on the validation of a structural rear empennage concept for noise shielding engine integration on business jets; eight leaders are contributing with seven associated partners. The structure of the programme is complex because it covers vertical components paths (SFWA1, 2 and 3) and technology streams. A matrix was prepared in 2011 to formalize structure, functions and management distribution between SFWA work packages and technology streams. In addition a correlation matrix was prepared to indicate input and output from each task. Key deliverables: The five major demonstrators planned are: 28

30 High speed smart wing flight demonstrator (BLADE) based on an A to validate low drag solutions for wings. Two different technologies will be tested simultaneously with two different wing tips (8 meters long). Figure High Speed Flight Demonstrator (BLADE), Low Speed Demonstrator (Ground and Flight) and Innovative Empennage Ground Demonstrator Low speed smart wing demonstrator including a smart flap large scale ground demonstrator and a low speed vibration control flight test demonstrator. Innovative empennage ground demonstrator to validate the noise shielding function on a business jet configuration. Figure Long Term Technology Flight Demonstrator and CROR engine demonstration Innovative engine demonstrator flying test bed on A of a CROR engine Long term technology flight demonstrator tested in service on an operational aircraft: A300 Beluga to test in service maintainability and performance of airframe surfaces. Contribution to ACARE s goals: The SFWA ITD is key to provide significant contribution to the CS objectives, thereby contributing to the ACARE s objectives. For example, the Smart Wing projects are intended to provide CO 2 and NO x reduction by 10% by using Natural Laminar Flow contributing for 7% and the associated weight reduction contributing to 3%; -15 to -20% fuel burn through integration of the CROR. The ITD is determinant for the success of the TE because it provides all figures at airframe level and the associated models. Airbus and its main partners have a precise understanding of each technology potential contribution to the aircraft performance. This is a core competence of airframers and it is a very sensitive element of the aircraft economical performance as well. As such, it is highly confidential. As it represents a key contribution to the TE to CS global challenges, some attention should be paid to this matter. The sensitivity of the subject should be taken into account. In order to enhance the TE efficiency, it is deemed necessary to integrate all the inputs from the various ITDs to perform the evaluation and propose adequate choices. The detailed figures which were discussed informally are attractive and show that the technologies under investigation are potentially promising. The performance assessment process is shown in Figure provided by Airbus. The Large Aircraft ITD takes into account all contributions from all relevant projects. The Panel notes some inconsistencies in the calculation of environmental targets within ITDs and SESAR (e.g. SFWA ITD presentation dated 10 April 2013 and Clean Sky Development plan version V2.01). The reader is referred to Recommendation R

31 Figure Example of SFWA calculation of environmental targets. ITD status at the end of May SFWA is late and has encountered unexpected difficulties and potential cost overruns which have led so far to several programme content and schedule redefinitions. The master schedule had to be adapted and the objectives had to be reviewed and occasionally down-sized to allow some achievement within the Clean Sky time frame and to remain within the limits of the budget allocation for Clean Sky. The figure below shows delays of 32 months in some areas at the level of the High Speed BLADE and CROR Flight Tests demonstrators. Figure and show that the others demonstrators - despite being late - are scheduled to deliver results before the termination of CS by the end of At the time of the visit in Toulouse, the maturity of the action plan to achieve a set of objectives within the budget and the CS global schedule were improving. However it was not yet stabilised and de-risked. The situation was improving in terms of stability, but remained somewhat fluid. The probability of achieving a satisfying status of the technologies by the end of CS1, despite down-sizing, looks quite good. The CROR integration is probably the least advanced project of all: the flight test of the CROR engine is not possible within CS1 for reasons due to the CROR engine definition and schedule (see the SAGE assessment) and also due to SFWA and the un-anticipated difficulties for flight testing. 30

32 Figure High-Speed Demonstrator Passive (BLADE) status May 2012 reported to be delayed more than 2 months since Oct 2010 while CROR Flight Demo is shifted to CS2 (Source: presentation at the Toulouse site visit). Figure 5.1.6: Major demonstrators planning (Source: presentation at the Toulouse site visit) Recommendations: The Panel regrets that, at the time the ITD content was defined back in 2006/7, the consequences of flight tests were not taken into account with the proper involvement of the flight test experts. Some risks were not correctly evaluated: certification difficulties to flight test the technologies on safety critical functions of the flying test beds; risks during the flight tests and the potential associated brand image damages to AIRBUS were not correctly assessed and induced a lot more work, schedule slippage and costs increases; test equipment, tools and test methodologies and processes were underestimated due to the scale of the innovation gap compared to the State of the Art practices. As a consequence, time had to be spent on these topics creating delays and additional costs. 31

33 R-SFWA.1: The Panel recommends that flight tests should be taken into account at the very beginning of the ITD. It is to be recognised as a necessary step, overlooked at the project launch but very much needed to ensure project success. The Panel assumes that competition related corporate games played by the engine manufacturers compounded with lack of key resources on some critical work packages and due to priorities allocated to aircraft programmes might have led to unexpected and abrupt change of content in some projects. The ITD had to adapt itself to this situation. Because the downstream research addressed by CS is close to programme competition it is not surprising to have such situation. Globally, the disruption was put under control despite the magnitude of the shock and the impact on the project. Some partners were weaker than expected and the screening process to validate their participation taking into account their capabilities was not effective enough and led to schedule slippage (particularly in the certification domain). This issue raises the question of an effective participant screening and selection which should be discussed at the JU level. Due to the size and complexity of the ITD, the complete bottom up evaluation of the ITD is a one year long processes running Airbus project management methodologies. It was launched late (2012) but it is the warranty of a better control and a better involvement of all contributing parties. It should have been carried out already in 2008 at the CS launch. This issue raises the question of adequate programme management methodologies. R-SFWA.2: For large ITDs, it is recommended to adopt systematically an industrial project management methodology from the very beginning of the project. The CS objective is to develop mature technologies ready for insertion in a programme. Airbus has recognised that a first task was to define with precision the meaning of the TRL maturity stages because of domain specific characteristics in aerospace. Involving EASA and FAA at this stage of developing breakthrough technologies was considered as mandatory. The relationship is working well, enables a safe progress and will allow light certification validated for flight tests. It has taken some time to bring all the parties together and to establish the working protocols. R-SFWA.3: It is recommended to secure robust commitment from the participants to find ways to prevent a lack of attention and of focus from the participating companies and to secure adequate resource allocation by all. The level of commitment and the relevance of the resources allocation plans have to be checked and validated by the CSJU. It is not a development programme and it should not be managed the same way. However, being close to development, the required resources are significant in terms of size and in critical specialist domains. The level of risk mid 2013 has not yet been reduced enough and the granularity issue is still too high. More efforts are planned to reduce the size of the risks and to restrain any negative impact on the schedule. Some key partners have underestimated the level of risks of their packages, leading the overall project to some difficulties. R-SFWA.4: It is recommended at the pre-design phase level to run an assessment of risks on the work package contents. 32

34 Clean Sky is a research programme with the associated risks. ITD leaders should not contemplate to put an obligation of results on their partners as it is the case in any development programme. However, due to the high TRL level of CS projects, significant resources are required in terms of competences. Some partners may consider CS as a low priority venture and allocate insufficient resources causing delays and reduction of scope. R-SFWA.5: The Panel recommends the JU to focus on minimising this potential risk, and to entrust the GB the responsibility of motivating the potentially defaulting partners. R-SFWA.6: Downstream research leading technologies to TRL6 maturity should achieve the following steps: performance readiness, engineering readiness, operational readiness (main tenability, stability ), manufacturing readiness. The Panel believes this recommendation is applicable to all large ITDs. The main lesson learnt from SFWA is that research aiming at bringing technologies at TRL 6 should take into account complexities and difficulties very close to those encountered in development programmes. Moreover, because it is research there are risks. As remarked in a presentation, The more innovative, the more efficient is a new technology, the higher are the potential value to the programme and the higher are the associated risks. CS is addressing TRL 6 technologies which can lead to significant performance improvement at Aircraft level. Being subject to risks, these projects can encounter delays, reduction of scope or even dead ends. SFWA concluding statements: The SFWA ITD can be considered as the reference for the Clean Sky ambitions. It is managing some very critical technologies, potentially contributing to breakthrough performance improvement for aircraft and to a step change in achieving ACARE s goals. New tools, new methodologies, new certification processes have been investigated and developed to allow progress towards TRL6. The JU and its governance bodies should review with special attention all the issues encountered during the past years in SFWA and draw all the lessons from this first phase of Clean Sky in order to avoid any repetition in CS2. ITDs need to be flexible not only technically but also in terms of budget. 33

35 5.2 Green Regional Aircraft (GRA) The GRA Objectives Green Regional Aircraft shall deliver low weight, using smart structures, as well as low external noise configurations, and the integration of technology developed in other ITDs, such as engines, energy management and new system architectures. The GRA Structure and Research Programme GRA addresses five technological domains, performed with well organised links with the relevant ITDs of Clean Sky: Low Weight Configurations (GRA1), linked to ED-Design, Low Noise Configurations (GRA2), All Electric Aircraft (GRA3), linked to ED-Design and Systems for green Operation (SGO), Mission & Trajectory Management (GRA4), linked to SGO and New Configurations (GRA5), linked to TE and SAGE (Fig ). Figure The GRA programme, showing the main interfaces with other ITDs GRA includes many industrial organisations, SME, Research Centres and Universities. In addition, the involvement in the CfPs has been very significant with 163 winners and 16 Countries involved. GRA Contribution to ACARE Goals The overall objective of Clean Sky (CS) is to develop technologies, that would allow to fly Aircrafts, including all aspects of the Industry, enabling to maintain present performance, while drastically reducing emissions and noise as compared to the standards of the year The specific product contribution to Regional A/C Y2020 expected to be delivered was -40% CO 2, -60% NO X and -20% db. CS has experienced difficulties to monitor the 2007 indicative targets. These targets were not always consistent across the range of technologies. The product objectives were not clearly defined down to CS specific technologies. The CS Development Plan (CSDP) provides a structured way to monitor and assess achievement of the environmental goals. Three complementary measures are used. These are the maturity of technologies in terms of Technology Readiness Levels (TRL), the concept aircraft and demonstration programmes. The TRL monitors the maturity of technologies within each ITD. The CS environmental benefits are measured by comparing the existing aircraft (baseline reference Y2000 and Y2020) and a virtual concept aircraft 34

36 incorporating CS technologies as defined by the aircraft ITDs. The demonstration programmess will allow to provide evidence of integration of several technologies and to determine the true potential benefit in a relevant operational environment. A realistic selection for GRA was performed with resized ATR and Embraer E-190. The 90 Pax Turboprop and 130 Pax are adapted to Y2000 reference, and CS green aircraft concept Aircraft models are prepared by GRA and provided to the Technology Evaluator (TE). The TE uses the GRA aircraft model to perform the environmental forecast for CO 2, NO X and noise at aircraft, mission and ATS level. The current objectives for GRA 90 Pax (passengers) and GRA130 Pax are respectively -25 to -30% & -27% to -35% for CO 2 & NO X. The noise objective for GRA 90 is -1 to -3.3 source noise reductions and -1 to -2 operational measure (for single operation with respect to Area %). The noise objective for GRA 130 is -4 to -7 source noise reduction and -1 to -2 operational measure (for single operation with respect to Area %). RGRA-1 CS1 and CS2 related: The current progress is reported in relation to CS objectives. The Panel recommends a more transparent traceability between ACARE goals and CS specific contribution. Regarding the three complementary measures, the technical visit to Alenia provided concrete evidence of TRL progress, environmental benefits related to the GRA aircraft and preparations towards demonstration. The Panel received evidence of a technology roadmap for each technology of GRA, TRL gate reviews and TRL progress. At the beginning of the CS GRA programme, most of the technologies started with TRL2-3; by 2013 TRL 4-5 have been achieved. This is considered as good progress given the complexity of the technological challenges and the slow start of the overall CS. Aircraft simulation models of the GRA ITD (GRASM) for the aircraft concept have been prepared and delivered to the TE. The GRASM of Green 90 Pax and 130 Pax provide a solid basis for the preparation of reference aircraft and evaluation of the environmental benefits of the green concept aircraft. Good interaction and technical reviews among partners have been performed to prepare demonstration activities. Agenda of the Meeting at Alenia in Pomigliano d Arco 4 th -5 th July 2013 The Agenda of the Meeting, which took two solid days and involved several engineers of the Alenia set up, including some from the other Alenia Factories (more than 20 engineering staff in total), is provided in the Annex. The meeting showed evidence for real commitment and priority from the ITD leader and partners. RGRA-2 CS1 and CS2 related: The visit provided evidence of very good cooperation between research development activities and flight test preparations. Detailed reviews have been conducted including multidisciplinary teams with experienced personnel in flight test. It is recommended to other ITDs to learn from the good GRA flight test preparations. Although in the meeting the status of all five technology projects was presented and briefly discussed, by far the major attention was given to Low Weight Structure and to a somewhat lesser extent All Electrical Aircraft, as agreed with Alenia representatives a the 2d meeting in Brussels (10 April 2013). Assessment on the Status of GRA ITD 1. Low Weight Structure (in the Workshop) This technology is essentially based on using composites and no metal for the structure of the aircraft (fuselage, cockpit, wings). The actual composition of the material has not been revealed, 35

37 because it is an Alenia - Industrial Partner Patent. It is black, probably made by fibre reinforced graphite (as for the re-entry protection of the nose of the space shuttle and for the first wall structure of the JET Tokamak). The required thickness (variable for different part of the structure) is build up by wrapping at 3 different angles ( ) tapes of 0.1mm thick, typical 10 layers to reach the basis structure of 1mm thickness. An extended set of well planned tests have been performed on this structural material, with panels size over one meter side. Some key tests were performed at the presence of the Panel members, namely: a. hail test (diameter up to 2.75 inc., velocity 30m/s, up to an energy of 86J ). No indentation was noticed and ultrasonic test showed no damage to the internal structure; b. a drop test was also performed, up to 30J of energy; even in this case no external or internal damage was noticed. However this material exhibited a drawback: vibrations and therefore noise in the simulation of flying conditions. This problem was successfully solved by inserting a thin damping layer in the middle of the structural material. The structural panel requires reinforcements and this is achieved by inserting shaped stringers of the same material (Fig 5.2.2). Figure Structural illustration of a section of the fuselage using the new composite material The body of the aircraft (reference ATR72) is made up by the fuselage, the cockpit and the wing, eventually all using the composite material briefly described above. The Panel visitors spent quite a considerable amount of time in the workshop, following the process of manufacturing the section (5m long, 3.5m diameter) of the fuselage and of large section of the wings. Each section is cured in an autoclave, where the stringers are pre-cured and assembled on the Panel concerned before the structure in cured in the autoclave. The fuselage sections are then closed with two end structures (prepared with a curing process outside of the autoclave). The closed fuselage section is then pressure tested. A panel of 5m length and 1.5m width is shaped to replace a standard Al section (Fig ). This modified ATR72 will then ready for ground (2014) and eventually for flying tests (September 2015). 36

38 structure on the frame and stringer, between stringer 4 LH/RH, will be removed. To reinstall them it is necessary to add new intercostals (see following figures) and plate support in accordance to the new crown panel configuration. Figure The Panel in composite material, to replace an aluminium Panel of an ATR72 for the flying tests planned by 2015 The Panel visitors could see sections of the wing Panels, where the thickness is built up with more than 10 layers, typically 26 layers. This is certainly required for the wing structure, however, no flight tests, within GRA Clean Sky, are planned. JTI CleanSky 2 The expected advantages/benefits of using nd Interim Evaluation Panel the composite approach should be the following: Visit to AleniaAermacchi (Pomigliano D Arco - Naples) Reduced Fuselage weight by 6% Reduced recurring man-hours (for maintenance) No corrosion No scheduled Inspections No fatigue Extended service operation life Marketing appeal, evolving technology 12 From what has been seen, both in the presentations and mainly in the workshop tests and manufacturing processes, the Panel believes that the GRA ITD should be able to achieve the benefits mentioned above. The table of Fig clearly indicates the full scale demonstrator for ground test and the planned Flight tests to be performed by the end of 2015, when GRA Clean Sky (1) will come to an end. Figure Programme for the ground (2014) and flying tests (2015) for the new composite structure for the body of GRA Among the benefits it is important to stress the No fatigue, since fatigue is usually what limit the life of any engineering device. Tests have been performed on relevant samples with a crack, up to cycles without any evident damage (no crack propagation). 37

39 2. All Electrical Aircraft In the context of Clean Sky an all electrical aircraft, is one in which all onboard systems are operated by electricity, while the propulsion comes from internal combustion engines (bleed-less type), that reduces significantly the fuel burn and consequently the emissions (CO 2 and NO X ). The AEA concept encompasses: Bleed-less engines Electrical environment control systems No centralized hydraulic system Electrical wing ice protection system Electrical actuator for flight control and landing gear systems High power, high speed electrical generation High voltage DC electrical distribution Electrical energy start The expected benefits of the AEA concept include: Improved engine performance Mass reduction (due to the elimination of hydraulic bleed systems) Improved on board systems utilization Improved reliability Reduced maintenance However the designers should consider the increase of mass and size of the electrical equipment, which could be the main drawback of an AEA. In the past there have already been hybrid conventional and electrical systems such ad POA (Power Optimized Aircraft) and MOET (More Open Electrical Aircraft), facing already the problem of weight, but now the challenge is to eliminate completely any non electrical power and energy supply. Figure Activities concerning a GRA - all electrical aircraft (AEA) concept major affected on-board systems 38

40 The design of an AEA should be performed according to the following guidelines: Simplification of the architecture Multi-purpose power electronics motor controller Higher power to weight ratio for power electronics Reduction of electrical load analysis budget at the aircraft level High power distribution centre integration Smart management of generators overload capability While the AEA benefits have been basically demonstrated for large aircrafts, this is not the case for Regional Aircrafts, and this is the present goal of Alenia and Partners. The on board systems involved are synthetically described in Fig The Electrical Power Generation and the Distribution System (EPGDS) design has been completed and manufacture of components is well advanced. The reference aircraft is a GRA 90pax, with two propeller engines: two electrical motors/per engine (110 Kw/generator), are installed, providing redundancy in emergency. Power distribution is at 270 VDC (Fig ). The main drawback using AEA for GRA is the weight (weight of EPDGDS is 430Kg for a 400 litres volume). A mode of operation is been optimised, by allowing to reduce the power supplied to some selected loads, when an overloading is requested elsewhere. However activities are still on at the THALES member to design an integrated generator which should allow to reach a 40% reduction in weight (by working on the DC network, by reducing the speed ratio, by limiting the overload capability and by using new magnetic materials). Figure The EPGDS architecture for AEA 90-Pax TP configuration 5 The Electric-Environmental Control System (E-ECS) is the most demanding user of power in the AEA configuration, because the suppression of conventional air sources supply implies the need of an electrically-driven air source. The results of architectural studies leads to an optimum scheme, because it requires slightly more power installation (100 KW), but has a lower weight (512 Kg). E- ECS ground and flying tests, will verify system behaviour in different operating conditions. The Hybrid-Wing Ice Protection System (H-WIPS) has been studied for the ATR 90-pax wing profile chooses a hybrid solution in order to reduce the electrical power demand. However a fully electrothermal architecture is also under study, but the resulting high weight and large volume led 39

41 to abandon this approach and no further activities are planned for Clean Sky GRA at present. However, ways to improve the efficiency of the de-icing method are still on going activities. Flight Control Systems-Electro-Mechanical Actuators (FCS-EMA), new technologies to overcome weight and reliability problems are now considered, by using brushless motors, more robust electronics and new anti-jamming systems. These activities will be performed by a partner (CESA) selected through a Call for Proposal, who will develop also an anti-jamming system for EMA. Electro-Mechanical Actuator for main Landing Gear (EMA-LG) Electro-Mechanical actuation for LG extension-retraction is under evaluation in several research programmes. The electro-mechanical braking for LG is currently in use on flying aircraft, such B787, Bombardier, etc. Alenia is considering the Main Landing Gear (MLG) actuator for the Regional Aircrafts in the framework of the Clean Sky initiative, aiming to bring the technology to TRL 5. For this purpose, a CfP was launched in 2011 for the design, manufacturing and testing of an EMA, capable of operating the MLG of the reference Turbo-Prop Regional A/C (the project is named ARMLIGHT). Main features of the project are: single DC brushless motor, anti-jamming system, no weight increase as compared with the current systems, higher reliability and lower maintenance needs. Ground and flying tests, for the electrical integration only, are planned in No further information was provided/ requested during the meeting. 3. Other significant activities 3.1. LW/AEA Technologies to assess environmental impact GRASM Activities validation The success of Clean Sky will be eventually measured on the numbers of the key parameters defining the environmental impact of the new generation of A/C, designed and manufactured using the new technologies developed by the Clean Sky ITDs. It is therefore of great interest the environmental impact studies performed by Alenia on the evaluation of the reduction of CO 2, NO X (fuel consumption) and noise level on GRA Green Y2020 A/C. This is even more interesting because TE ITD performed the same exercise (not only for GRA) obtaining similar results, with the methods of analysis appropriate to TE, but with the input of models proposed by GRA. The Panel considers this approach as a good validation rather than double work as this validation has been carried out by independent teams. In fact the objectives of GRA Simulation Models (GRASM) are to: Provide TE with effective tools to perform the Clean Sky assessment relevant for the environmental impact (noise and emission) at Mission level, at Airport level and at the Global level (i.e. on the evaluation of global fleet operation), Complement internal GRA analyses: preliminary design assessment, trade-off between configurations, design low noise take off and approach trajectory paths, Provide validated Trajectory Path (TP) model to perform optimised trajectory and optimisation studies. Assessment of AEA/LW technologies at the A/C level In order to highlight the improvement of performance towards the ACARE objectives, concerning GRA, a comparison has been made between a Reference Turboprop A/C configuration with 2000 Year Technology (ATR pax) and a GRA Green Turboprop A/C configuration (2020 Year Technology 95 pax). The 2020 Year Technology, include the outcome of the development of the GRA ITD, namely: Low Weight domain activities (composite material for the structure, load alleviation, etc) and All Electrical Aircraft domain activities (EPGDS, ECS, ICE Protection, Wire architecture, etc). 40

42 The Table of Fig show the results of the comparison of fuel consumption (CO 2 and NO X abatement), which reveal a saving of 24% in fuel consumption, close to the target values (25-30%). The improvement concerning noise reduction is expressed in the reduction of the impact area: 50% at take-off and 25% at approach (Fig ), have to be considered a very good achievement. Near future work in this area will include a sensitivity analysis, to highlight the more beneficial technologies. Figure Main results of the evaluation, using the Alenia GRASM, of the weight and emission reduction using the new technologies The GRA models (GRASMs) have been provided to the Technology Evaluator (TE) and similar preliminary results have been obtained (see TE ITD assessment). Figure Main results for the noise reduction evaluation, using the model above 3.2. Low Noise Configuration Low Noise Configuration activities refer to design/modelling and technology maturation for a Regional Aircraft 130-seats, a Regional Aircraft 90-seats and a Low Noise Landing Gear. These imply computational studies and experimental activities, including Advanced Aerodynamics, Aeroelasticity, Aero-Structures and Systems. Mainstream Technologies relate to Wing Optimisation Aerodynamics, Load Control, Load Alleviation, Low Noise High-Lift Devices and Low-Noise landing Gear. Technical solutions involve Natural Laminar Flow, Active control of wing movables, Wing Aeroelastic Tailoring, etc. 41

43 Tests are planned during 2015, just at the end of GRA Clean Sky activities: no flying tests are considered, only Wind Tunnel tests. Most of the computer design work has been done on the wings, aiming at obtaining natural laminar flow. An aerolastic finite element model has been studied. The wings are fitted with a morphing flap. Mechanical prototypes have been designed, manufactured and tested, showing their functionality in matching the target shape, while withstanding simulated aerodynamics loads. Two prototypes were shown to us during the visit to the workshops, one mechanically operated and the other operated electrically. A CAD model of the 130-seat A/C is shown in Fig A model for gust load alleviation has been produced by Politecnico of Turin. The structural material of the aircraft is of composite structures, similar to the ones briefly described in the Low Weight section of this Report. A similar analysis has been performed for the Green Regional 90-seats A/C. Figure CAD Model & CFD Pressure distribution at high-lift condition A detailed structural/mechanical CAD model has been constructed to study the optimum configuration for a Low Noise Landing Gear, Main LG and Nose Landing Gear. Following the computer studies to evaluate the sources on noise, two models (in ALLEGRA and ARTIC Projects by CfP) will be constructed and tested in two independent Wind Tunnels, respectively in the Pininfarina WT and in the DNW-LLF WT. The testing facility will be equipped with sensors to measure the noise in various locations and in a number of operational positions of the LG. Final tests are planned to be completed, as for most of the GRA ground and flying tests, for selected solutions before the end of GRA Clean Sky activities, i.e. within 2015 (ref CSDP) Mission and Trajectory Management The aim of the Mission and Trajectory Management (MTM) is to study, in coordination with SGO (System for Green Operation) avionics solutions enabling the Aircraft to reduce its environmental impact. However no flight tests are considered, but only tests with the GRA Flight Simulator. The institutions involved are Thales Avionics, University of Bologna, CIRA and ELSIS. The main interfaces are with SGO, in the various phases of the project development. The main commitment of GRA is to develop Green FMS (Flight Management System), while SGO should care about the down selection and the functional requirement definition of the Green FMS. The green functions are: Green Cost Index (release 8/2012), for the optimization of the cruise speed, Optimum Flight Level Selection, to provide the best cruise altitude (release 5/2013), Continuous Descent Path, in order to reduce CO 2 emissions and Noise level (expected release 12/2013). The aim of the GRA Flight Simulator is to assess in real time/pilot in the loop environment the benefit of the green functions. Two configurations were foreseen: a) basic configuration (ATR), completed in May 2012, b) upgrade configuration (GRA TP90 pax) to be completed by 12/2013. Two flights from Venice to Napoli have already been performed (ATR72), with the aim to verify the GCI cruise speed function. 42

44 The final assessment will start next year (by using a Flight Simulator modified for GRA TP90 pax). The final release of Green FMS will contain the following functions: GCI for optimum cruise speed, OFL, for optimum cruise altitude, CDA, for optimum descent path. Benefits are expected in terms of CO 2, while NO X reduction benefits are expected by the engine performance. Finally, only limited benefits are expected from the perceived noise, because descent optimisation is at high altitude ( feet). GRA concluding statements: The GRA ITD has a comprehensive task, dealing with the Aircraft Body (with the exclusion of the engine), All Electrical Aircraft Devices, Mission and Trajectory Management, and finally with the evaluation of the benefits for the environment as defined by ACARE. The Panel was pleased to see concrete evidence of progress in innovative technology developments, concrete contribution towards ACARE targets and to note that in the environmental assessment performed at this stage of GRA development, both GRA ITD and TE show similar quantitative results. The key issue is the reduction of weight by using composite materials. The R&D on new structure design and composite materials is supported by a wide range of laboratory tests; full scale ground demonstrators are planned to be concluded with a variety of flying tests within Flying tests require extensive preparation in terms of new technology and its suitability for an existing regional aircraft. An important aspect addressed in an appropriate manner by GRA is the combination of experienced production and R&D personnel involved in detailed planning and appropriate reviews for preparation of demonstration activities. The two days visit to the Alenia premises was instrumental to convince the Panel, that the GRA ITD will be completed on time, with more than satisfactory results. 43

45 5.3 Green RotorCraft (GRC) The GRC Objectives GRC focuses on the integration of technologies and demonstration of rotorcraft platforms (helicopters and tilt-rotor aircrafts) to drastically reduce emissions and noise while maintaining present performance. The GRC Structure and Research Programme The Programme combines seven technology subprogrammes and one management package, namely: GRC0 - ITD Management GRC1 - Innovative rotor blades GRC2 - Reduced drag of airframe & dynamic systems GRC3 - Integration of innovative electrical systems GRC4 - Installation of a Diesel engine on a light helicopter GRC5 - Environment-friendly flight paths GRC6 Eco-Design Demonstrators (Rotorcraft) GRC7 - Technology Evaluator for Rotorcraft (interface & preparation) A strong point of this ITD is the apparent strong link with other ITDs and the inclusion within GRC of specific sub-programmes linked to other ITD s (GRC5, 6, and 7): SAGE o Agreement with SAGE 5 for sharing new turboshaft characteristics (GRC5) o Involvement of SAGE 5 in GRC7 / TE SGO o Link with Aircraft Energy (MAE) for HEMAS and for starter generators (GRC3) o Link with MTM for Rotorcraft specific trajectories and missions (GRC5) o Models for GRC3 Architecture studies o Electromechanical Main Rotor Actuator (GRC3) Eco Design o with EDA for LCA and technologies (GRC6) o Interface with EDS for electrical test bench adaptation of rotorcraft requirements (GRC3) o To highlight weight impact resulting from their research for incorporation into GRC7 conceptual rotorcraft Technology Evaluator (GRC7) o Benefit assessment on the SEL (Single Engine Light) & TEL (Twin Engine Light) model helicopter accomplished and delivered to TE June 2012 o Team work with GRC4 completed. Diesel Engine Light platform deliverable Q4/2013 o Input for CleanSky reference rotorcraft for LCA to be defined (GRC6) GRC Contribution to ACARE Goals The environmental objectives derived from the ACARE objective and specific to the GRC ITD are the following: Turboshaft engine CO 2-25% -40% NO X -60% -50% Noise -10 EPNdB or -50% Diesel piston engine 44

46 These are average figures based on the individual gains expected for each platform (light helicopters with single or twin engines, medium or heavy helicopters, tilt-rotor or Diesel). As of today, the evaluation carried out within GRC7 and TE confirms those objectives. In particular, for the single engine light helicopter (SEL), the evaluation shows a reduction of 30% for CO 2 and 47% for noise compared to the targets of -25% CO 2 and -50% noise respectively, thanks to the new technologies developed within GRC. However, the evaluation of NO x reduction is not yet available. This evaluation will be complemented by the evaluation of the other helicopter models in order to obtain an average result for all helicopter categories considered within CS. Still, it has to be noted that the two helicopter models evaluated so far represent about 65% of the forecasted world fleet in This gives hope that GRC will attain its environmental objectives at project completion. The Panel considers this a very positive achievement. However, the evaluation of the environmental objectives in terms of NOx, CO 2 and noise are often based on a number of assumptions which are sometimes unclear or not sufficiently justified, and the final numbers sometimes lack consistency. R-GRC.1: The Panel encourages the Partners and Project Managers to provide more clarity and consistency in the figures presented as well as on the assumptions taken for the evaluation of the environmental targets in relation with the ACARE goals. Assessment on the Status of the GRC ITD The present assessment is based on a number of documents received from the European Commission and a number of meetings. No technical visits or attendance to review meetings were scheduled for this ITD. During the meeting of April 10, a short presentation of about ten slides covered the main aspects like master schedule, budget, technical progress, risk status, management and impact assessment. It is clear that the ITD is well run at overall level. IPR and confidentiality as well as interaction with other ITDs seem to be adequately addressed. There is a delay of +/- 12 months in GRC 1/2/3; the others ones are on track, but no impact is expected on completion of targets. Although there has been substantial technical progress, the project tends to shift to the right. The budget is late in spending, and there is a discrepancy with regard to the forecasted spending in general. However, the objectiveness of the risk assessment has to be noted. Statistics were presented about the CfP s, but there has been little detailed information about the topics. The Panel emphasizes the need to increase momentum, prioritizing work, proceed with planning, cope with problems of resources and to ensure available resources. However these issues could not be assessed in sufficient detail (from a 90 min presentation during the meeting on April 10). The Panel recognises the added value of technical visits or technical presentation meetings which would have given more insight (see R-4.1), some annual technical review reports are less positive As requested by the Panel, additional material was provided by the Project Officer. A summary slide was provided with the link with other previous European projects. It is positive that GRC builds on and integrates results from previous Framework Programme projects (mainly on FP6 OPTIMAL, GOAHEAD, NICETRIP, etc), but it is noted that delay on deliveries from FRIENDCOPTER has had a negative impact. 45

47 R-GRC.3 - Lessons learnt for CS2: The link with previous or ongoing Framework Programmes should be clearly stated in order to avoid overlap and possible double funding. This recommendation is valid for all ITDs. As requested by the Panel, the deliverables for two chosen sub-projects (GRC 1 - Gurney Flaps and GRC4 - Diesel Engine) were provided but only for Gurney Flaps (2 of light content) and not for Diesel Engine because all deliverables are confidential. Some changes in the initial workplan occured and since 2010 the following actions have been implemented: o o o o Alignment of technologies towards market opportunities, as per anticipated to date Shaping activities to clarify both outcomes with respect to Clean Sky objectives and to implement system-analysis approach, and an improved maturity of results (GRC5 for instance) Clarification of interfaces between GRC and other ITDs (mainly Eco-Design) Implementation of good practices described in the CSMM (TRL maturity assessment for instance) The overall technical progress was presented in 2 slides of general overview and the active Gurney flaps (GRC1) and the Diesel engine (GRC4) were presented as the two flagships of this ITD. The Panel member appreciated the fact that an additional 50 slides (not presented during the meeting but of pure technical content) were provided as additional material to the general presentation. Good work has been done in preparing presentations and a strong involvement of the Project Officer was noted. A clear comparison was presented between previous planning and current planning and reviewers comments from the last meeting were discussed. However, there is a need for more consistency in the figures presented in relation to target reductions. (see R-GRC.1) GRC1 - Rotor blades Design activities are continuing for model and full scale demonstrator blades with Active Gurney Flaps. The CDR for 2D wind tunnel test of AGF was passed in February The PDR for model scale AGF actuation system was held in January. A new CfP was launched to select partners to support CFD modelling of AGF and analysis of test results. An overall delay of 12 months is quoted to be without any impact on completion. Major changes are under review to increase the maturity at completion and to recover delays: in particular for Active Gurney Flaps: inclusion of flight tests to increase maturity from 5 to 6, and budget increase by transfer of budget from activities with low maturity target (laminar cover blades for instance and Active blade devices based on Piezoelectric). GRC2 - Drag reduction The second wind tunnel testing campaign has been conducted for the 1/5 scale EC135 wind tunnel model (ADHeRo) featuring optimised landing skids. The final wind tunnel test campaign at ONERA for active devices (synthetic and pulsed jets) on aft fuselage confirmed drag reduction up to 36% and almost reached TRL 4. The optimisation of the Tilt-Rotor sponsons and landing gear were successfully achieved. The kick-off of the CfP project ROD (PoliMi) for wind tunnel tests on the common H/C platform was held at the end of February The design and manufacturing is completed for the remotely controlled movable horizontal stabiliser for the GOAHEAD model. Instrumentation and calibration have started. GRC3 - Electrical Systems 46

48 The concept configuration in terms of mass and electrical data has been supplied to GR7. Technology progress has been demonstrated by formal design reviews in several areas: PDR closed for EMA for Flight Control, EMA for Rotor Brake, Electrical Conventional Tail Rotor, and Starter Generator; PDR closure pending on Piezo Power Supply; and CDR closed for EMA for Landing Gear. Definitions and test plans supplied for equipments on the ETB. GRC4 - Diesel Engine The partner consortium has been selected in 2010 (highest project in value: 9,3 M ) and the kickoff took place in On the optimal helicopter, the engine definition is almost finished (end of March 2013) and the helicopter architecture activities will continue in On the demonstrator helicopter, engine tests have started end of February 2013, and iron bird preparation is on-going, tests scheduled to start in September GRC5- Flight Path A thorough review took place in 2012, leading to a complete change of the WBS to make an efficient use of resources, and align activities towards CS targets (mainly in terms of visibility and understanding). On the technical side, the H/C procedure optimization activities are on track, the T/R performance during operations has been shared with TRAVEL partner, the low noise path optimizer has been validated (TRL4), the first low-pollutant tilt-rotor missions have been computed (estimated 7% CO 2 reduction), preliminary in-flight tests of tunnel-in-the-sky display have been performed, and finally, ground tests of pollutant measurement system achieved with MAEMRO and EMICOPTER. GRC6 - Eco Design The final design and manufacturing concept of the thermoplastic composite structure is on-going until mid 2013 but processes have to be adapted while tooling design and tooling manufacturing are to be started in the second half of Suppliers and joining process have been selected for the thermoplastic tail-cone. The tooling design has been finalized and the manufacturing has started beginning of Regarding the Tail Gearbox, the Zn-Ni coating of parts is finished since April 2013 and the manufacturing of Mg parts since June For the intermediate gearbox, the parts manufacturing and data collection is in progress and the supplier for TPC-shaft has been selected. GRC7 Techno Evaluator There is evidence of significant efforts to feed TE with the right inputs, and of mitigation of technical difficulties by reducing the number of deliveries and focusing on the most representative models (SEL for instance). Work is completed for the PhoeniX platform v3.1 and the Twin Engine Heavy (TEH) is ready for delivery to the TE ITD. Work has commenced on the PhoeniX platform v4.1 Twin Engine Medium (TEM-B). Both platforms include engine models from SAGE5 by Turbomeca and the benefits of GRC 6 Eco Design Demonstrators are to be incorporated as well. Finally, TE s second assessment for GRC is completed. GRC concluding statements: GRC focuses on the integration of technologies and demonstration of rotorcraft platforms (helicopters and tilt-rotor aircrafts) to drastically reduce emissions and noise while maintaining present performance. The Active Gurney flaps and the Diesel Engine were presented as the two flagships of this ITD. 47

49 Regarding the achievement of the ACARE goals, as of today, the evaluation carried out within GRC7 and TE confirms those objectives. In particular, for the single engine light helicopter (SEL), the evaluation shows a reduction of 30% for CO 2 and 47% for noise compared to the targets of -25% CO 2 and -50% noise respectively, thanks to the new technologies developed within GRC. The Panel considers this a very positive achievement. However, the evaluation of NO x reduction is not yet available and the evaluation still has to be complemented by the evaluation of the other helicopter models in order to obtain an average result for all helicopter categories considered within CS. The work plan shows delays in some areas but no impact is expected on completion of targets as appropriate mitigation plans have been put in place. It is clear that the ITD is well run at overall level. Some of the ground tests have been completed already and the ambition of GRC is to include flight tests as well towards the end of the programme to achieve a nominal TRL6 level. 48

50 5.4 Systems for Green Operations (SGO) Overall developments SGO focuses on developments in two independent pillars: the Management of Aircraft Energy (MAE) and the Management of Trajectory and Mission (MTM). Firstly, MAE supports the development of all-electric equipment system architectures. This allows a more fuel-efficient use of secondary power. Secondly MAE investigates the generation of electrical energy and its distribution to electrical aircraft systems. MTM aims at developing technologies to reduce emissions and noise addressing the way aircraft manage its trajectory either in flight or ground. SGO had a slow start and strategic re-planning was performed in The initial plan was considered aggressive and with a lower technological maturity than originally expected. There is an overall delay of 12 months with no impact in the final CS completion date. The Panel agrees with the remarks from technical reviews about activities being shifted towards the end of the CS project with consequent fewer margins in the schedule. It is appreciated that measures and priority is given to developments related to the main demonstration. R-SGO.1: CS1 related: The Panel recommends carefully monitoring and implementing and early warning mechanism to critical activities, success factors of SGO. R-SGO.2: lesson learnt for CS2: The Panel recommends that administrative and project management procedures are set-up before the start of technical work. The Grant Agreement for Members (GAM, 2013) highlights that technical content has been adapted and it is below the initial ambitious expectations. Changes and adaptations have been related to address technological setbacks, cancellation of technologies, introduction of new technologies from on going technological developments, reduction of TRL scope, delays in test rig building, Intellectual Property Rights issues and withdrawal of some partners. Decision-making needs to consider trade-offs between most promising technologies, its industrial applicability, schedule and SGO budget. There are activities that have been deleted in the updated GAM, the rational and consequences are not justified with sufficient detail. R-SGO.3: lesson learnt for CS2: The traceability and evolutions on GAM should be better documented to establish and assess its overall compliance and performance. Traceability should track changes and their impact. This action enhances the ability of the programme to adapt to new challenges and opportunities. The SGO programme has two cycles of validation and maturation of technologies and subarchitectures were planned. The first cycle is for demonstrations of sufficiently mature technologies. The second cycle is dedicated to demonstration of technologies investigated within CS. Demonstrations for mature technologies include large scale ground hardware tests rigs and flight tests. Recognised stakeholders of the domain are involved in the ITD: Airframers, system suppliers and research organisations. A total budget of approx. 300M is allocated to this ITD and a significant part has been allocated to partners about 25%. In the period SGO made available 2.5m EURO to the JU. By 2013, 78% of the budget has been consumed which is considered average in comparison to other ITDs. As a transversal ITD, SGO has direct and indirect interfaces with GRA, GRC, SFWA, TE as well as SESAR. Specific reviews have been performed together with SESAR to identify potential overlaps in the themes related to flight management. Deficiencies in receiving documentation from SESAR have been identified. Important interfaces are reported to the general board and trans-itd workshops on common themes are organized. The interfaces are considered as being managed in an adequate manner. R-SGO.4 lesson learnt for CS2: Many interdependencies are seen among ITDs and with other national and EC activities. The Panel recommends incorporating current interface management practices into a specific interface management function. Moreover, formal exchange of information 49

51 should be established among the CS, SESAR and other research programmes (e.g. Horizon 2020). Implementing this recommendation would speed up research work and avoid of potential duplication of work. Most of the work packages have been active in the review period; some technical achievements and challenges include: Aircraft solutions and definitions (WP1) completed the validation and verification master plan. A good TRL roadmap has been prepared. It is remarked that the TRL approach has been demanding and more complex than initially expected. The Panel appreciates a SGO efforts and positive link currently established with SESAR. In the Management of Aircraft Energy (MAE, WP2) equipment is under final preparation and/or delivered for demonstrator testing. For example a prototype of the skin heat exchanger is under final preparation. Design of the electrical power distribution centre has been completed and manufacturing was launched. Tests campaigns and assessments include icing tunnel tests, TRL gate reviews, critical design reviews. Examples of these developments were presented during the technical visit. The Mission and Trajectory Management (MTM, WP3) progress includes advance versions of multidisciplinary optimization framework GATAC (Green Aircraft Trajectories under ATM constrains) and developments of Flight Management green functions. GATAC facilitates multiobjectives optimization and minimization of conflicting objectives e.g. fuel burn vs. flight time and NO X. This software supports theoretical identification of trajectories with minimum environmental impact. The review of documentation showed that GATAC has passed TRL 4. However, the capabilities have been delivered late, future GATAC refinements are expected within CS. The FMS functions passed TRL 3 and delays are reported for TRL 5 and 6. Large-scale demonstration (WP4) consists of ground and flight demonstration activities. Evidence of progress towards demonstration has been provided. The technical visits provided evidence of progress related to the electrical ground tests, definition of scenarios for pilot in the loop interactions for demonstration of the green functions. Aircraft assessment and exploitation (WP5) started by the end of After a slow start, it is remarked that 2012 has been used establishing a private collaborative web-site, agreement on targeted benefits and on criteria for appropriate technology selection. A mix of criteria is applied e.g. TRL maturity, manufacturing readiness level, avoidance of confidentiality issues, high impact in terms of certification, standardization and certification. It is remarked that environmental benefits are not among the criteria identified in the documentation provided to the Panel. SGO contribution to ACARE goals The SGO ITD contributes to the ACARE 2020 targets by improvements in energy and mission management, new trajectories and system reduction and improved on-ground operations. ACARE targets have been decomposed to specific SGO objectives in terms of fuel burn saving and noise improvements. These reductions are calculated per flight phase and per mission. The expected contribution from SGO is a -5 to -9% CO 2 reduction, -2 to -5dB (approach and landing) and -2 to - 3dB (take-off and climb) noise improvements (GAM 2013 only mentions large a/c). MAE targets - 2% fuel burn saving for large aircraft while MTM overall targets -3 to -6 CO 2 reduction and -2 to - 5 db. SGO contributions are expressed per flight phases e.g. climb, cruise optimisations, take-off reductions. The CS objectives are broken down into individual objectives for each technology. The direct contribution of SGO to improve the environment can be expressed in terms of weight savings, energy efficiency, suppression of hydraulic fluids and other. Some contributions are expressed in a qualitative manner e.g. weight benefit. SGO developments are delivered and integrated in vehicle ITDs. Then, SGO environmental CO 2 and noise benefits can only be validated at aircraft level. It is difficult to track down the environmental benefits down to specific technology. Environmental improvements are calculated at aircraft, mission and global level. It is difficult to associated single technologies contribution. Good evidence is provided on contribution to environmental targets in 50

52 line with the technical progress. The documentation provides evidence that some technologies seem to require significant effort in development and appear to contribute to a limited extent to CS environmental targets. R-SGO.5 lessons learnt for CS2: The Panel recommends to include metrics such as weight saving, energy efficiency, maintenance environmental impacts (e.g. reduction of hydraulic fluids) and expected efforts to maturation and manufacturing individually per technology to assess the benefit for CS1 and potential candidate for CS2. R-SGO.6: SGO benefits are expressed per flight phase. It makes difficult a comparison across ITDs regarding the most promising technologies. Therefore, the Panel agrees with technical reviews about alignment of SGO environmental benefits metrics to other ITDs. A clear dashboard presenting Technology Readiness Level (TRL) advances and roadmap was presented for specific technologies. This dashboard is used to monitor the technology maturity. Good evidence is provided of maturation of technologies e.g. ice protection systems and environmental control systems from TRL3 to TRL4. Example of validation exercises were presented e.g. icing tunnel test. Flight management functions developed in MTM present progress towards TRL4. The developments and progress presented have a strong focus on technological developments and less attention is given to the interaction with other key stakeholders outside SGO and certification, which is essential to achieve the desired environmental improvements. More involvement of endusers, consequences of technology acquisition, operation, maintenance and costs is needed. R-SGO.7 CS1 and CS2 related: The Panel recommends a thorough preparation for the transition to the new developments proposed by Clean Sky. The compatibility of CS with end users expectations needs to be addressed. SGO technical visit progress evidence The technical visit provided evidence of hardware and software developments and progress towards demonstration related to the Airbus PROVEN, Liebherr GETI test benches and Thales AIRLAB simulator. PROVEN test bench will be used to test power distribution and electrical load management. The GETI test platform can be adapted to represent and simulate different aircraft and to analyse performance in terms of electrical and thermal loads. The AIRLAB simulator is used to test flight management functions for different flight phases. Other test benches are planned within SGO e.g. AVANT test facility. R-SGO.8: Demonstration activities for some equipment are foreseen in a single test platform. Back-up plans in case of delays in the test platform need to be addressed. Management of Aircraft Energy (MAE) presentation reflects utilization of results from previous EC projects such as More Open Electrical Technology (MOET, FP6) and Power Optimized Aircraft (POA, FP5). The ITD further mature and develops additional technology and improvements e.g. the overall system weights from MOET have been reduced. Evidence is provided about the validation and evaluation activities. These activities will be performed through ground physical or virtual testing rigs or flight-testing. The demonstration starts when the technology reaches certain maturity. The MAE technologies include electrical equipment, thermal management equipment and load management functions. The MTM includes green flight management system, robustness to weather and electrical taxiing. SGO targets a high number of technologies and associated tools (about 35 to 40 technologies) and several demonstration campaigns. Critical paths for each demonstration are periodically reviewed and potential risks are identified and managed in an appropriate manner. The planned electrical Flight Test Demonstrator (eftd) was presented as shown in figure below. This eftd will 51

53 enable test of wing ice protection electro-thermal or electromechanical with their associated ice detection system driven by electrical compressors. Figure Example of electrical Flight Demonstrator The visit to PROVEN test bench provides concrete evidence of developments towards the ground demonstration. PROVEN as illustrated on Figure below is an open full scale electrical test bench. PROVEN will be used to test e.g. electrical networks with high voltage and power convertors. The technical visit to the test rig PROVEN provided evidence of how the electrical equipment will be tested in different electrical configuration. This test bench is dedicated to research projects, but it is possible to adapt its level of representativeness. The control room will be used to monitor status, power centre distribution site, the electrical drive to simulate a generator, programmable loads to simulate the aircraft configuration in various flight phases, integrate some equipment representative from ECS. It is possible to see how the tests are foreseen. There are possibilities to record test and to analyse problems. Ground tests for electrical technologies such as starter generator and electrical power distribution. Control room Actual loads Electrical drives Programmable loads Power centres Actuators Mobile Loads Figure PROVEN large electrical test bench 15 The AirLab is a technical operational laboratory. This simulation environment within Thales to validate Flight Management Functions (FMS) green functions was presented to the Panel. A demonstration of specific green functions at different flight phases was provided. The functional 52

54 concept for optimization of the Noise Abatement Departure Procedure (NADP) with Multi-Criteria Departure Procedure for take-off was explained. The Panel was informed that this functionality has been implemented in the flight management and could see how the ED functionality works in the mock-up in the AirLab for specific flight phases, e.g. initial climb. The Adaptive Increased Glideslope is another FMS functionality presented to the Panel. This functionality is seen as complementary to the Continuous Descent Approach (CDA). CDA is addressed by SESAR; during the technical visit the Panel was informed that technical review with SESAR has been performed to ensure alignment between CS and SESAR developments. The airlines will be able to select a function optimised in terms of reduction of noise and fuel burn. An example of acoustic assessments for approach at Charles de Gaulle airport was presented to the Panel. The Panel was informed that Airbus pilots have been involved in the TRL3 assessment in the AirLab experimentations. All flight management functions have passed TRL3. The development and validation of FMS function required external realistic inputs such as weather information. A weather data repository and weather simulation engine provided by the SIMET CfP is currently linked to the FMS. SIMET includes worldwide weather data for the period (analysis and forecast), graphical interface and simulation capabilities (see Figure ). Scope Noise NOx Contrails CO2 Fuel Figure Advanced Increased Glideslope (A-IGS) and FMS green function The visit to Liebherr Aerospace enabled the members of the Panel to discuss and see MAE developments related to the Electrical Environmental Control Systems (E-ECS), the Wing Ice Protection Systems (WIPS), Cooling technologies and Thermal management and GETI (Gestion dynamique de la puissance Electrique et de la gestion Thermique) test platform. Examples of Liebherr technological involvement were discussed with the Panel, e.g. Vapour Cycle System, the electro thermal wing ice protection system and the E-ECS. The Panel appreciated E-ECS good overview and technological roadmap. Its scope is from model definition, components developments, ground and flight-test for large, regional and bizjet aircraft. For the bizjet no flight test is planned. TRL3 has been achieved for the overall modelling. The major key technologies were validated. Specific adaptations have been developed for regional aircraft e.g. constraints in terms of space and flight envelope are taken into account. Special architecture has been developed for regional a/c e.g. an optimized ECS architecture. Since 2012 development of different components and key technologies common between regional a/c and large aircraft has started. An ice wing protection system was presented to the Panel. The presentation included pictures of actual hardware a leading edge. Ground tests included a full demonstration in icing wing tunnel at NASA facilities. Thermal management validates components as well as system architecture. A first vapour cycle system prototype has been developed. Ground demonstration for key components such as the high cooling vapour systems and validation of thermal management architecture are foreseen at the GETI test platform. The visit to the GETI facility allowed the Panel to see real hardware pieces and preparations for demonstrations. Assessment test were conducted while the Panel was visiting the facility. SGO concluding statements: Concept defined by Airbus Green and implemented in Green cruise Airlab departure Airlab experimentations o HMI tested on fixed base simulator (Airlab) o Standard A321 autopilot used o Simulated MMR o Airbus flight test crew in nominal operation Acceptance of the A-IGS concept First benefits evaluation T/O Climb Cruise Descent Approach The transversal ITD SGO develops technologies addressing More Electrical Aircraft and Management of Trajectory and Mission. The Panel observed concrete examples of technologies, A-IGS AIGS Control Panel 53

55 architectures and software tools as well as preparations towards demonstration activities. In general, flight and ground demonstrations are foreseen for MAE technologies while ground demonstrations are foreseen for MTM technologies. The documentation, presentations and demonstrations during the technical visits provided good evidence of the SGO contribution to the ACARE goals in terms of weight, fuel savings and noise reduction. The Panel appreciates that the SGO technologies take into account results from previous FP projects and develop them further. SGO mature technologies have been adapted to regional and large aircraft ITDs. Many technologies are expected to achieve TRL 5 or TRL 6 e.g. the green take-off function and Electrical Environmental Control System. Still, the Panel notes that not all technologies will achieve TRL 6 e.g. advance weather algorithms. The assessment of TRL has been more challenging than expected. Currently, TRL monitoring and risk management tools have been implemented in a satisfactory manner. Lessons learnt from this assessment process can be transferred to other domains. SGO has a lot of interfaces internally within CS and externally e.g. with SESAR. Deficiencies in receiving documentation from SESAR have been identified. Specific reviews between SESAR and CS are carried out and members seem satisfied with the level of interaction achieved. Close involvement of EASA is still an open issue. More coordination with TE and SESAR is advised regarding models e.g. noise models and noise assessments to ensure complementarity and synergies. 54

56 5.5 Sustainable and Green Engine (SAGE) The SAGE Objectives The purpose of the Sustainable and Green Engine (SAGE) ITD is to assess, design, build and test up to five full-scale engine demonstrators for various types of aircraft. The proposed engine ITD contains 5 testing vehicles, distinguished by application (helicopter, regional, narrow-body and wide-body) and by engine architecture (2-shaft, 3-shaft, open-rotor). These demonstration vehicles are using the competencies and facilities of all the European aeroengine manufacturers complemented with those of related research establishments, academia and SMEs. The proposed demonstrations will prepare new solutions for the complete range of the market, with engines for the narrow body fleet, high thrust engines for wide body aircraft, regional aircraft engines and helicopter engines. For fixed-wing aircraft, a particular focus will be put on the novel engine architectures (open-rotor and geared-fan engine). The primary focus of engine demonstration is ground test to deliver proven architectures for advanced engines and mature ready to use technologies. The SAGE Structure and Research Programme The SAGE ITD is divided into 6 main sub-projects: SAGE1, Geared Contra-Rotating Open Rotor Demonstrator, SAGE2, Geared Contra-Rotating Open Rotor Demonstrator, SAGE3, Large 3-Shaft Light-Weight Turbofan Demonstrator, SAGE4, Geared Turbofan Demonstrator, SAGE5, Turboshaft Demonstrator SAGE6, Lean Burn Combustion SAGE Contribution to ACARE Goals The successful validation of these technologies will then facilitate the early introduction of innovative new products to significantly reduce the environmental impact of air transport. The impact on the achievement of the ACARE targets, relative to the ACARE baseline, in the context of ongoing major research programmes is shown in the Table below. Engine Sector Environmental Targets and Achievements Noise (EPNdB) CO 2 NOx Cumulative 2000 Baseline Baseline Baseline Clean Sky -14 % to 20 % TRL 6-60 % to 80 % TRL 6-16 to 20 TRL 6 ACARE -20 % -80 % -20 * NOx baseline is roughly consistent with 80% of CAEP2 ** Noise baseline is roughly ICAO Stage3 10 EPNdB Assessment on the Status of the SAGE ITD The present assessment is based on a number of documents received from the European Commission and a number of meetings, among which technical visits or review meetings. SAGE 1 (Geared Pusher Open Rotor Rolls-Royce, UK) A strategic change was made by RR to reduce activities in SAGE1 to the benefit of new activities on Lean Burn Combustion for which SAGE 6 was created. RR confirmed that there would be no demonstrator for SAGE1 within Clean Sky. Activities are now concentrated on technology areas 55

57 relevant to the design of CROR systems, such as aero-acoustics, some mechanical design and manufacture of key components, and participation in EASA/industry studies of safety issues and airworthiness requirements. The OR Demonstrator Engine preliminary concept design is available. A first analysis of extensive rig test data suggests that ACARE Noise goals can be met, which the Panel considers as an achievement that has to be underlined. Performance levels are close to predictions and shared with SFWA-AI for A/C level assessment. Aero-acoustic design methods and tools have to be further developed and ORA design and manufacturing technology acquisition has to go on. SAGE 2 (Geared Pusher Open Rotor Safran/Snecma) A strategic change was also made by Snecma to abandon the initial direct drive architecture to the profit of the geared engine architecture. The engine demonstrator concept phase was completed mid The selection of the geared configuration and of the gas generator from the M88 engine is set. However, the choice of the donor engine results in a number of additional technical challenges, in particular, the design of the power turbine due to the strong temperature gradient caused by the dual flow of the core engine. R-SAGE.1 lessons learnt for CS2: The Panel questions the appropriateness of designing a new CROR engine demonstrator, based on the non-optimal choice of an existing gas generator. It is understood that this is a cost and time limiting solution. There are doubts whether the final demonstrator is going to be fully representative of a future CROR engine. Therefore the Panel recommends strengthening the validity of the design in view of more representative demonstrators. The Engine Demonstrator preliminary design phase is on-going (Demonstrator modules architecture selection and Engine demonstrator performance). However, some of the so-called design-to-demo assumptions remain unclear. Engine technologies risks abatement plan is on-going (Propeller blades: Wind tunnel tests / Full scale composite blade manufacturing for tests, PCM: Mechanical analysis & Component tests). Despite the very challenging design of the Power Gear Box (PGB), no component tests are scheduled, which remains a major risk of failure of the demonstrator. There are several other areas of the engine where there are significant technical challenges and risks for example, the pitch change mechanism, the control system, the dynamic behaviour of the rotating parts, etc. The problems which may be encountered are likely to cause delays, which could impact significantly on the time finally available for the engine tests. Overall, the difficulty of holding the project to the current plan is considerable and the time frame to demonstrator is extremely tight, if not unrealistic. R-SAGE.2: It is highly recommended to explore the possibilities of testing the gearbox (with AVIO) in order to reduce the associated risk. SAGE 3 (Advanced Large 3-shaft Turbofan - Rolls-Royce, UK) The activities carried out in the frame of SAGE 3 are considered to be well on track with a first engine test (of a series of 3) already completed in January Advanced Dressings (Fig ) demonstrations are completed with nearly 100 hours of engine running time. Composite Fan test preparations are ongoing, a complete set of composite fan blades (Fig ) and annulus fillers have been delivered and were shown to the Panel during the visit of June

58 Figure Advanced light weight dressings on fan case (RR-UK). Figure First large size composite fan blade (RR-UK). Intercase rig testing is completed with successful series of tests investigating structural stiffness and strength, culminating in ultimate load test. 57

59 The LP turbine module design passed the PDR and the whole engine design assessments is continuing. Ground demonstration is foreseen in 2013 and flight demonstration in Early CfPs are starting to close, delivering components such as variable fuel pump, high temperature electronics, etc, but are not going to be tested on the demonstrator engine. SAGE 4 (Advanced Geared Turbofan - MTU) The first GTF engine has been certified and the donor engine is available as technology platform for Clean Sky testing. The SAGE4 ground demonstration test programme is scheduled for the first quarter of The detailed design is executed according to plan; the fourth design review has been passed in June 2013 (middle of detail design phase). Manufacturing trials for special processes and component rig tests have been started (e.g. tests of abrasive behaviour of blade stator sealings). The demonstrator test concept is currently established with the programme partners; the test concept review has been held in April Although it is presented as the flagship of SAGE 4, the Power Gear Box remains the main concern. A CfP has been launched to manufacture a test rig and the winner is a consortium led by the University of Pisa. Besides the time schedule to manufacture the rig and test the gear box, which is very tight, a new issue appeared at the beginning of this year: AVIO has been bought by GE, a major competitor to Pratt & Whitney, manufacturer of the donor engine. There is, therefore, a risk of jeopardizing the tests. However, as the decision has been taken not to test this new AVIO gearbox in the demo engine, the planned engine demonstrator running would presumably be unaffected but this issue needs clarification. R-SAGE.3: The conditions of access to the future Gearbox test rig by third parties needs to be clarified. R-SAGE.4: The planning and technology features of the SAGE 4 demonstrator need to be clarified and confirmed. R-SAGE.5 Lessons learnt for CS2: The possible influence on programmes of changes in the structure of industry should be kept under review by CleanSky officials, with the aim of identifying opportunities to prevent or to minimise adverse effects. Some activities on advanced rub system, electromechanical machining of blisks, SLM VGV mechanism were stopped and new activities like non-contact blade vibration monitoring were introduced. Work on TiAL for the LPT seems to show overlap with Level 2 projects like E-Break or ENOVAL. R-SAGE.6 Lessons learnt for CS2: In general, the boundaries between activities carried out within FP7 Level 2 programmes and CleanSky are not clearly defined or explained. The Panel recognises that CleanSky is intended to bring those Level 2 technologies to a higher TRL level but the issues of duplicate work and duplicate funding should be monitored. SAGE 5 (Advanced Turboshaft Safran/Turbomeca) This sub-project seems to be the most advanced one with the first turboshaft engine demonstrator running and tests being completed at the end of April The official celebration of the first rotation of the TECH800 turboshaft demonstrator took place on the 26 th of April 2013 in Pau (France), in the presence of Siim Kallas, Commissioner for Transport and Vice-President of the European Commission, Eric Dautriat, Executive Director of Clean Sky, Jean-Paul Herteman, Chairman and CEO of Safran and Olivier Andries, Chairman and CEO of Turbomeca (Fig ). 58

60 Partial rig test activities have been carried out: Dynamic rotor test has been completed in January HP turbine test for build2 has been performed end 2012 (earlier than expected). Demonstrator Parts Design activities for build 2 are completed. Demonstrator Parts Manufacturing Activities are on-going: Raw part for build 2 has been ordered Engine assembly for build 1 is completed. First rotation for build 1 has started in February Figure The TECH800 turboshaft demonstrator at Safran/Turbomeca. This very well managed project encountered its initial major technical difficulty with those first engine tests. Although all the rig tests on the separate components have been successfully completed, severe vibrations were encountered during the initial running of engine Build 1, started early this year. A committee of experts has been set up to investigate the situation and to identify a solution. There are no problems on the other components: the Build 1 combustion chamber will withstand the temperature of Build 2 engine tests and it has been decided to cancel the second combustion chamber manufacture. The collaboration with GRC 7 was presented; twelve (!) engine models will be delivered: three references (year 2000, year 2020, with and without CS) for four types of helicopters. Another issue raised during the SAGE 5 review is the future of unsuccessful - and only partially successful - CfPs. All CfPs within Sage 5 but one (the inlet guide vane electrical actuator which will be too late for the tests) were not necessary for the demo; most of them concerned nice to have types of technology. Several are completed by now and there is no follow on expected. This raises two questions: the future of these CfPs and the way to improve the choice of sub-contractors, as already mentioned (see Recommendations R-6.2.2, R-4.5 and R-4.6). SAGE 6 (Lean Burn Combustion Rolls-Royce, UK) SAGE 6 addresses lean burn combustion for large engines allowing a drastic reduction of NO X emissions, an objective clearly in line with the general environmental objectives of Clean Sky. The requirement for the Lean Burn ALECSYS demonstrator engine has been issued for stakeholder review, the related definition document is worked with feedback from sub-systems. Full Annular Rig testing with latest set of injectors was successful. A formal Stage 1 Exit review with the Rolls- 59

61 Royce audit team was carried out in May 2013, further reviews are on track to hold first engine pass to test end of An important development in the structure of SAGE 6 is the addition of WP 6.9 covering the testing of the lean burn system on the Rolls-Royce EFE (Environmentally Friendly Engine) core vehicle. This can provide the increased temperature and pressure at combustion chamber entry which will exist in future large engines and allow measurement of emissions at these conditions. Satisfactory results from these EFE tests will then be followed by the whole engine demonstration on the ALECSYS (modified Trent 1000 donor engine). Unfortunately, however, turbine damage due to an unresolved incident on EFE is currently causing delay, and the corresponding risk is not yet clear. Another issue in lean burn development is the possibility of adding flight testing to SAGE 6, at an additional cost of 17 M (plus 7 M already given). Flight work is not currently included in the SAGE 6 programme, though if all goes well in ground testing it might be possible to accommodate it within the timeframe of Clean Sky 1. No formal proposal for such flight work, and its extra funding, has yet been made and it appears to the Panel that the time between the end of ALECSYS ground test and the suggested FTB dates seems too short to allow any change on the combustor (as an example a delay of six month is necessary to re-manufacture the injectors). R-SAGE.7: Any proposal for a lean burn flight test within Clean Sky time scale has to be clarified in terms of schedule and financing. SAGE concluding statements: The purpose of the Sustainable and Green Engine (SAGE) ITD is to assess, design, build and test up to five full-scale engine demonstrators distinguished by application (helicopter, regional, narrow-body and wide-body aircrafts) and by engine architecture (2-shaft, 3-shaft, open-rotor). These demonstration vehicles are using the competencies and facilities of all the European aeroengine manufacturers complemented with those of related Research establishments, academia and SMEs. The main novelties in the SAGE ITD are the start of engine demonstrator tests (SAGE 3 Large Turbofan and SAGE 5 Advanced Turboshaft) and the availability of new hardware including the composite fan blades (RR) for the large turbofan, composite blades (Snecma) for CROR and the intermediate casing (GKN) for the turbofan. Another important achievement is the noise issue of CROR engines, which could be significantly mitigated by appropriate design of blades. Confidence is now expressed by both SAGE 1 and SAGE 2 leaders that CROR powerplants can achieve the reductions in external noise levels (in EPNdB) desired in future civil aircrafts. The Panel regards all these achievements as encouraging outcomes of the CleanSky work so far. However, delays in the work plan are still threatening the programmes although mitigation plans are being adopted. Changes in the industry structure in Europe may also threaten the programmes and the consequence of the acquisition of Clean Sky Partners by non-european competing firms should be considered. The high number of CfPs is promoting involvement of a large number of SMEs at European level and widening participation of companies from other industrial sectors. However, the low success rate of those CfPs in some areas remains a concern. Finally, further consideration should be given to the detailed process of estimating the benefits of the CleanSky programme in relation to contributions from other relevant programmes, and to how the benefits can most clearly and accurately be conveyed to authorities outside the specialist scientific/technical community. 60

62 Titanium 3% Steel 1% Composite 10% Miscellaneous: Copper, bronze, synthetic 5% Aluminium 81% 5.6 EcoDesign (ED-ITD) The ED objectives: ED focuses on the reduction of the environmental impact during the on-ground phases of the aircraft life cycle: design and production, maintenance and withdrawal. It aims to develop technologies for these three phases to allow a drastic reduction of waste. The objective is to reduce the environmental impact while maintaining the European Industry competitiveness. ED addresses aircraft and Helicopters, but does not address reduction of fuel consumption. The concept of the ITD is well illustrated by Figure below. The final eco-statement is an extrapolation of green technologies to real industrial conditions to evaluate its environmental impact and benchmark against current technologies. A/C Reference parts Material & Process Technologies Bill of Materials & Processes A/C Material & Process cakes LCA on Reference parts Combination for A/C extrapolation Technologies Eco-Statement (current & innovative) A/C Level Eco- Statement (current & innovative) Figure ED concept and main results The ED Structure and Research Programme: The ITD is composed of two streams of research managed in parallel: Design for Aircraft Application (EDA) and Eco-Design for small aircraft Systems (EDS). The EDA stream focuses on all relevant phases: design, manufacturing, maintenance and aircraft disposal. This stream is developing tools and investigating technologies to assess their potential contribution to the objectives (refer to figure left side). The Life Cycle Assessment relies on software tools such as GaBi LCA database extension that support design for environment and Atalys. The design model is based on a survey of green design in different industrial branches (shipbuilding, railway and car industries (EDA T ). The most promising technologies are evaluated and further developed to reach maturity. Figure (right side) illustrates the process managed by the EDA stream of the ITD. Non-expert Non-expert Interface GaBi ATALYS Expert Tools Commercial (GaBi, Aerospace Ecoinvent, etc.) (CS EDA developed) Databases State of the art Clustering and downsizing SoA where more than 150 tasks were highlighted, that corresponds to 235 single technologies Figure Databases for Life Cycle Assessment (left side) and process for selection of the most promising technologies T0+6 Trade-off based on a scoring Scoping Downsizing in which EDA consortium members were encouraged to identify their company's favourites Clustering where WP 2.1 and 2.2 technologies were grouped to stress EDA key topics and to focus the ground demonstration on these key areas T Technologies Technology Development T

63 The EDS stream focuses on enabling the removal of all hydraulic driven equipment on board small aircraft and helicopters by replacing them by electrical systems. The small aircraft domain has been selected because it is easier to address by developing high TRL technologies from the state of the art at CS launch. This stream is addressing technologies with a dual focus on functionality and thermal contribution. Test beds have been designed for both purposes: thermal simulation and functional integration. Expected deliverables: EDA is working on ten structure partial demonstrators, two cabin interior partial demonstrators and six equipment demonstrators. EDS is focusing on a small aircraft common electrical platform with an electrical ground test facility and a thermal test bench facility. This is performed in cooperation with GRC and GRA by sharing the COPPER Bird test rig. ED contribution to ACARE s goals: The ITD contribution is related to the reduction of the environmental impact during the on-ground phases of the aircraft life cycle The EDA stream targets are -20% reduction of process emission, full compliance with REACHregulations and -15% reduction in energy consumption. Concerning the EDS stream, the contribution is taken into account at the vehicle ITD level. Environmental contributions include weight benefits and energy management target is a reduction of fuel consumption by 2% and removal of hydraulic noxious fluids.. ED is designing tools and enabling technologies to mature from low TRL to high TRL. There is no explicit evidence of the ITD contribution to the objectives of CS, however the EDS contribution is used as an input into the Aircraft ITDs and then fed into the TE, thereby providing proof of the ITD contribution. The EDA contribution is still difficult to assess and work should be done to clarify and make more transparent the contribution of the ITD. The process used by the ITD is shown in figure Whilst most of the input to TE comes from the vehicle ITDs, the EDA results are fed directly into the TE. Figure EDA feeding process of TE. The ITD status: This ITD proceeds well in a structured and organized manner. The level of activity, the remaining amount of funding and the current schedule are consistent and will allow a complete execution of the programme within the allocated period of time. 62

64 This ITD was not selected for a dedicated analysis. All the information was provided in written form and during a briefing on April the 10 th. The ITD organisation: The ITD is lead by DASSAULT AVIATION and the FRAUNHOFER Institute. Eight leaders are managing key packages. Most of the relevant key stakeholders in the industry are involved. A high number of partners (in total 110) are involved and provide the content of individual work packages related to Calls for Proposals issued by the JU. Key findings and remarks: The Panel got the impression that: The amount of funding dedicated to the ITD is quite modest in comparison to most of the other ITDs (57.13M ). The ambition and scope of the ITD is quite large and diverse (EDA and EDS). The domain of research of EDA has not been investigated in depth during the past decade in the different FPs. There is little material in terms of existing practices in Aerospace. 111 technologies are investigated within EDA. The review of each technology was not possible in the allocated timeframe, however a sample was required from the ITD leaders and selected by them: Replacement of Chemical Machining by Mechanical Machining. Surprisingly, these technologies have been used for decades and are well understood. The trade-off results which were provided raise a question mark about the contribution of this specific study to real research. R-ED.1: The Panel recommends reviewing the relevance of the high number of technologies reviewed by the ITD EDA stream. The size of the ITD stream is too small to encompass so many technologies and bring them successfully to TRL 6. R-ED.2: It is recommended to check that EDA is taking into account lessons learnt by other domains such as automotive and by the emerging deconstruction eco-system. EDS is working on relevant topics providing potential contribution to the ITD s objectives. However, it is not clear how the EDS results are taken into account into the TE and how it can be used to provide data to assess trade-offs and make choices. A number of important integration tests are conducted on the different test beds; it is important to make sure that the coordination with SGO and GRA is excellent at operational level. R-ED.3: Taking into account the content of EDS, it is recommended to ensure consistency and check gaps or overlaps with SGO and GRA/ GRC ITDs related to electricity. There are synergies and potential cross fertilization opportunities. ED concluding statements: The ED ITD is focused on a very critical domain for Aerospace. Until now this domain has been insufficiently taken into consideration by research studies in the different Framework Programmes, namely to improve the environmental impact of Aircraft design, manufacturing, maintenance and withdrawal. The ITD is well managed and its contribution is notable. However, the JU should better define the concept of the ITD and indentify the potential contribution from other State of the Art domains to Aerospace (e.g. railways, automotive, etc.) in order to build a consistent and coherent approach for the domain. Clean Sky 2 offers an opportunity to launch a top-down design phase to address the domain by taking into account inputs from other areas. 63

65 5.7 Technology Evaluator (TE) The TE Objectives The TE objectives are clearly described in the document CLEAN SKY Aeronautics & Air transport JT1 Proposal March As a summary: The Technology Evaluator will provide a core activity of the project integrating the technical content across the JTI. The TE will be realised through a simulation suite that can evaluate the merit of R&T activities in the ITDs in relation to the ACARE targets (see Fig ). Figure Technology Evaluator Input /Output scheme The translation of the impact of innovative airframe, engines, systems and eco-design technologies into overall ATS performance, with respect to the environmental challenges, is the general objective of the Technical Evaluator of Clean Sky. TE also will provide feedback to the ITDs at different levels: aircraft design, aircraft operations and global ATS (see Fig Figure Technology Evaluator feedback to ITDs TE development and assessments will be a continuous effort during nearly all Clean Sky duration, with delivery of several upgraded versions. A new significant version should be delivered on a yearly basis, following new achievements in technology ITDs, for which constant support will be delivered. 64