Topic D: Practitioner-academic reports Neumann H-M. Pangerl E, Airaksinen M., Ahvenniemi H., Bosch P., DeCunto A., Homeier I., Huovila A., Jakutyte- Walangitang D. Jongeneel S., Kontinakis N., Kotakorpi E., Neralic S., Pinto-Seppä I., Sarasa D., Schnepper M., Tillie N., Vielguth S. Measuring the Performance of Smart Cities in Europe 1 Introduction and background Urban sustainability is currently closely related to the concept of a Smart City, an idea heavily supported by the European Commission and the European Innovation Partnership on Smart Cities and Communities (EIP SCC) 1. The European Commission has developed two parallel approaches to support European Cities in becoming smart: (1) Large scale demonstration of technology in cities and communities ( lighthouse projects') and (2) 'horizontal activities' to address specific challenges as in regulatory barriers, in standardisation, public procurement and performance monitoring. 2 The results presented in this paper are first outcomes of the CITYKEYS 3 project funded by the EC under the H2020 Programme, which is part of the 'horizontal activities. The goal is to provide a validated, holistic performance measurement framework for monitoring and comparing the implementation of Smart City solutions, inside and beyond Horizon 2020. CITYKEYS focuses on developing and validating performance indicators and data collection procedures in a bottom-up process with five partner cities: Vienna, Rotterdam, Tampere, Zagreb and Zaragoza. The resulting processes and procedures shall be used for the common and transparent monitoring of Smart City projects (initiatives/actions) and solutions across European cities once the project is completed. 2 Methodology The ambition of CITYKEYS is to develop an indicator framework in a bottom-up process, based on the needs of cities. To achieve this ambition, CITYKEYS started by analysing the cities and their stakeholders needs regarding a performance measurement system. To identify the needs of cities on the measurement of smart city performance, two surveys were distributed to the CITYKEYS partner cities (Tampere, Rotterdam, Vienna, Zagreb and Zaragoza) and to other members of the EUROCITIES network, a large European city network. The survey Cities needs covered topics such as Smart City and Smart City project definitions, Smart City and Smart City project performance measurement, measurement tool properties, data collection and open data, whereas the survey Citizens and stakeholders needs focuses mainly on Smart City project evaluation and acceptance of open data. In addition, several in-take workshops were held in the partner cities to gain better understanding of specific key topics by a qualitative approach. The key topics included questions regarding the potential target groups of a performance measurement framework and use cases for a performance measurement system, to provide two examples. Besides from elaborating the cities and stakeholders needs, existing frameworks for assessment and (open) data exchange, of other relevant developments e.g. research projects, data protocols etc. and KPIs are assessed. To create an inventory of existing frameworks, an extensive survey of existing KPI systems for project evaluation is carried out. The inventory covers relevant FP7 and HORIZON 2020 projects as well as national and international KPI frameworks. In order to map the KPIs efficiently, the KPIs are classified based on the themes People, Planet, Prosperity. Finally, the results of the surveys and the in-take workshops are merged with the results of the mapping of the KPIs. To fulfil this task four workshops are organised, in which practitioners as well as researchers participate. The workshops 1 European Innovation Partnership on Smart Cities and Communities 2013: Strategic Implementation Plan, Brussels: EIP SCC 2 Strategic Energy Technology (SET) Plan. 2014: Towards an Integrated Roadmap: Research and Innovation Challenges and Needs for the EU Energy System. Brussels: EC 3 www.citykeys-project.eu
aim to narrow down the number of KPIs from existing framworks,based on several criteria, such as relevance, availability, measurability and reliability in order to establish the final set of KPIs. The results are regularly adjusted with the findings from the investigation of the cities and stakeholders needs to ensure the needs of the practitioners. 3 Results & Discussion In total, 19 cities from all over Europe submitted completed questionnaires and/or offered additional input regarding their needs and expectation towards a performance measurement framework. The survey on cities needs showed, that approximately 50% of the cities do not measure their smart city performance, although almost 75% consider the topic Smart City of high importance. The top five themes, that cities would like to measure with a Smart City performance measurement systems on city level are Energy, GHG emissions, Transportation, Digital infrastructure and services, Resource management and Citizens participation. On project level the top five themes are GHG emissions, Energy, Transportation, Digital Infrastructure and services, Environment (air quality, noise, soil, green areas, ) and Quality of Life. On the contrary, the citizens and stakeholders consider Creation of innovation and knowledge, Better public transportation, Protection of the environment, Better education and skills building and Clean Energy as the five most important outcomes of Smart City projects. The surveys demonstrated that there is a high need for Smart City KPIs. Many cities consider the topic Smart City as highly relevant, but do not have an appropriate measurement systems in place. Moreover the analysis showed, that the topics Energy, Transportation and Environment / Resource management are considered relevant by cities as well as by stakeholders and citizens, on city and on project level. Topics such as Digital Infrastructure and services, Citizens Participation and Quality of life are ranked higher by cites than by citizens and stakeholders, whereas Creation of innovation and knowledge and Better education and skills building seem to be higher concerns for citizens and stakeholders than for cities. However, the amount of completed questionnaires is considered to be too low to derive representative results. Nonetheless, they provide a first overview of the cities and stakeholders needs. To improve the results a second round of questionnaires is considered at a later stage of the CITYKEYS project. The mapping of the indicators shows that there are numerous indicators available. However, it is crucial to identify the appropriate KPIs for the purpose of CITYKEYS. This selection can only be performed in close collaboration between practitioners and researchers. The great variety of indicators and the broad range of topics accompanying the Smart City discussion provide a challenge in prioritizing the indicators and KPIs. 4 Next steps The CITYKEYS framework will be tested in the 5 partner cities: Tampere, Rotterdam, Vienna, Zagreb and Zaragoza. The cities will be involved in defining the stakeholders needs and the performance system requirements (including usability requirements), in evaluating and testing the system prototype and in building recommendations for its implementation in planning and municipal decision-making processes. CITYKEYS performance measurement framework will include specific Smart City KPIs that go beyond the traditional division into categories and will measure the integration and innovation level of the smart solutions. The implementation of a common performance measurement framework based on a set of relevant indicators, open data applications and decision-support user-interfaces enables stakeholders to learn from each other, create trust in solutions, and monitor progress. This will allow cities to move towards a sustainable transformation while spending less public resources and improving services offered to its citizens.
Annex A: Figures Figure 1: Piority areas of European Innovation Partnership on Smart Cities and Communities 4 Figure 2: Work plan of the CITYKEYS project 4 European Innovation Partnership on Smart Cities and Communities 2013: Strategic Implementation Plan, Brussels: EIP SCC
Figure 3: How high is the topic of smart city in your city s agenda? Figure 4: Does your city measure its smart city performance? Figure 5: Areas where cities need indicators to measure their smart city performance
Annex B: Corresponding authors Hans-Martin Neumann hans-martin.neumann@ait.ac.at AIT Austrian Institute of Technology GmbH Energy Department Giefinggasse 2 1210 Vienna Austria Eva Pangerl eva.pangerl@extern.wien.gv.at City of Vienna Municipal Department 18 City Development and Planning Ebendorferstraße 1 1082 Vienna Austria