Assessing progress. Impact of Science. Pillar Hall. Alliances with stakeholders. 12 & 13 June 2017, Stockholm. Partners & Supporters:

Transcription

1 Impact of Science Pillar Hall 12 & 13 June 2017, Stockholm Alliances with stakeholders Assessing progress Main partners: Partners & Supporters: Networking Partners:

2 Impact of Science Pillar hall 12 & 13 June 2017, Stockholm Assessing Progress Göran Marklund Deputy Director-General external matters of Vinnova, Sweden

3 Assessing Progress How to structurally stimulate and measure how universities are connected to society Göran Marklund Vinnova

4 Science - Necessary but Insufficient for Societal Progress Science Impact Innovation Imperative!

5 Framing Progress Assessing Understanding Managing Linear Measurement Linear Understanding Linear Management Basic Research Applied Research Development (Experimental) Innovation Society R&D Statistics OECD 1963 Innovation Statistics OECD 1992

6 QUEST FOR FUNDAMENTAL UNDERSTANDEING? INTERACTION COOPERATION? Towards Improved Framing Assessments & Management INNOVATION PROCESSES? PASTEUR S QUADRANT CONSIDERATION OF USE? Source: Stokes, D.E., Pasteur s Quadrant Basic Science and Technological Innovation, 1997

7

8 Scientific Renewal Framing Scientific Impacts High Path-breaking Scientific Findings Science Paradigm Shifts Low Path-dependent Scientific Findings Path-dependent Science Progress Low High Science System Impacts

9 Fit-for-Purpose Assessments Linking Science to Society How? Interaction? Innovation? What? Assessing Science Impact for: Understanding Outcome History: Science on Science Managing Process Directions: Research Managers Governance System Design: Policy Makers

10 Science Impact Assessments Why? How? Who? When? Then?

11 SWEDISH INNOVATION AGENCY GÖRAN MARKLUND

12 Framing Science Societal Impacts Science in Innovation Processes System Renewal System Landscape Socio-Technical Paradigm Silo Management Power Structures Trust Norms Routines Culture Incentives Regulations Path-dependent Science Innovative Experiments Societal Value Time

13 Framing Science Societal Impacts Science in Innovation Processes System Renewal System Landscape Socio-Technical Paradigm Silo Management Power Structures Trust Norms Routines Culture Incentives Regulations Path-dependent Science Innovative Experiments Societal Value Time

14 Impact of Science Pillar hall 12 & 13 June 2017, Stockholm Assessing Progress Sean Newell Chief Executive Officer of ResearchFish, United Kingdom

15 Researchfish Sean Newell

16 How to structurally stimulate and measure how universities are connected to society

17 Universities and their impact today What is impact? Different things to different stakeholders To universities it s about innovation, discovery and the pursuit of excellence

18 How to measure impact? Pathways to impact Consistent and standardised question set Each stakeholder group gets to understand impact from their perspective Large number of UK funders use Researchfish increasing number of universities too.

19 Universities adding impact to strategic vision There has never been a more important time in modern history for great universities to make a full contribution to society Ed Byrne, president of King s College London

20 Government is recognising impact is key HEFCE mandates impact for REF Australia cements the link to society by making key findings, knowledge transfer and engagement activities key. Showing return on investment is the new black Evidence-based decision making is critical

21 Change can be effected through the students There are more of them than ever before They are more ambitious, more connected and more aware of global issues than their predecessors. We need to cut the waste 85% of biomedical research is wasted 70% of publications are never read Avoid the valley of death!

22 Measure, measure and measure again Connect the dots. From the funding.. To the basic research.. To the applied research.. To the societal impact

23 Impact of Science Pillar hall 12 & 13 June 2017, Stockholm Assessing Progress Steven Wooding Lead for Research and Analysis at the Centre for Science and Policy at Cambridge University

24 University research: measuring what does and how it does it 15 YEARS, 4 TOOLS AND THE FUTURE STEVEN WOODING UNIVERSITY OF CAMBRIDGE, CENTRE FOR SCIENCE POLICY AESIS 2017 STOCKHOLM JUNE 2017

25 Outline ostate of play in 2002 ohow four of the tools have developed owhere we are now owere we are going

26 The situation 10 years ago

27

28 Four tools obibliometrics osurveys oeconomic analysis ocase studies

29 Bibliometrics ocounting papers olooking at journals ocounting citations onormalising citations olinking to impacts

30 Surveys obespoke ocommon structure ocommon data collection engine, silo d analysis

31 Economic Analysis osteady state assumption oresearch contribution guesstimate otime lags and bottom up

32 Case studies osuccesses odiversity olarge scale

33

34

35 Revisiting the key questions

36 Future of the tools Bibliometrics Getting beyond zero order assumption Surveys Frameworks to capture context and the origins Economic analysis Flows and import export Case studies Representative/reusable

37 Challenges for the future ANALYSING THE DATA Open access publication We have massively more data STANDARDISING AND SHARING Proprietary collection systems Drive progress and standardisation Walled garden Funders willingness to share and transparency EXPERIMENTATION Willingness to acknowledge uncertainty Willingness to experiment (not just change things)

38 Dr Steven Wooding Lead for Research and Analysis University of

39 Impact of Science Pillar hall 12 & 13 June 2017, Stockholm Assessing Progress Laura Hillier Director Evaluation and Outcome Assessment at Canada Foundation for Innovation

40 Assessing Progress: How to structurally stimulate and measure how universities are connected to society Impact of Science 2017 Stockholm, Sweden Laura Hillier, Director Evaluation and Outcome Assessment Canada Foundation for Innovation

41 Canada Foundation for Innovation Created by the Government of Canada in 1997, to build Canada s capacity to undertake world-class research and technology development that benefits Canadians by investing in the research infrastructure necessary for our researchers to discover, develop and apply new knowledge in all areas of science, the humanities, health, engineering and the environment.

42 Who should be responsible to assess and report the impact of science? We all have a role to play S&T ecosystems are typically complex with multiple funders, multiple producers, and multiple end-users all with slightly different needs related to the impact of science but also different evidence of the impact of Science Collaboration

43 How to assess and report the impact of science? Given our funds and approach to supporting research infrastructure, we typically evaluate at a portfolio or horizontal level rather than undertaking project of program level evaluation more aligned to the way science works & the timeframe on which impacts tend to occur

44 Some examples:

45 Medical imaging research A collaborative project to assess the socioeconomic impacts achieved through a portfolio of investments in medical imaging research and research infrastructure

46 Arctic research A collaborative project to assess scientific and societal impacts of a research platform An Expert Panel found that the Amundsen platform and research program: is enabling science of the highest international quality and is facilitating the translation and application of new knowledge to address societal issues of major consequence for the Arctic regions of Canada and for other Arctic settings. has had a major impact on the productivity, reach and influence of Canadian Arctic science as shown by the strong publication record and by the seminal papers produced on such topics as sea ice and ecological research in the Beaufort Sea. has engaged a diverse set of end-users encompassing federal and provincial science-based government departments and agencies, industry, and communities.

47 Environmental research A collaborative project to explore the impact of multiple related awards at a small institution CARE:

48 Agricultural research A collaborative project to uncover pathways to impact in five important areas of agricultural research

49 How best to communicate the impact of science?

50 Thank you! Laura Hillier Director, Evaluation & Outcome Assessment

51 Impact of Science Pillar Hall Assessing Progress Chaired by: Goran Marklund Deputy Director-General external matters of Vinnova, Sweden 12 & 13 June 2017, Stockholm Recommendation: Assessing methods increasingly sophisticated. Learning across different funders and different evaluators, a challenge. End-users engagement in impact studies, a challenge. Policy learning about how to design and adapt processes for impact, a challenge.

52 Impact of Science Next up 12 & 13 June 2017, Stockholm Now Lunch Cabaret restaurant 13.45h Research councils Room 357 Impact through education Room 353 Global challenges Room 253 Collaboration and impact Pillar Hall