PARTICIPATORY DEVELOPMENT OF FUTURE-ORIENTED POLICY CONFIGURATIONS FOR CLIMATE CHANGE AND MOBILE GAMING

Transcription

1 PARTICIPATORY DEVELOPMENT OF FUTURE-ORIENTED POLICY CONFIGURATIONS FOR CLIMATE CHANGE AND MOBILE GAMING AHTI SALO 1, ROBIN GUSTAFSSON 2, ILKKA KAAKKOLAMMI 3, TOMMI GUSTAFSSON 1 1 Helsinki University of Technology, Systems Analysis Laboratory, P.O. Box HUT, Finland 2 Helsinki University of Technology Institute of Strategy and International Business P.O. Box 5500, HUT, Finland 3 Tampere Technology Centre Hermiankatu 1, Tampere, Finland ABSTRACT: Further to the Kyoto treaty, many nations have committed themselves to challenging targets for reducing CO 2 emissions as a step towards the mitigation of harmful consequences of climate change. In this paper, we report a policy workshop where Finnish experts took stock of a national research and technology development programme on climate change and employed Internet-based decision support tools in the development of policy recommendations: for example, they used multidimensional weighting procedures to characterise policy measures across four application domains. At a more general level, we argue that analogous weighting procedures can assist in the development of policy configurations which link private and public actions for the attainment of future targets. Towards this end, we structure relevant dimensions in the fostering of an industrial cluster of mobile gaming and present an illustrative application of how the resulting framework can be plausibly applied to derive priorities for research and technology development. KEYWORDS: Technology foresight, decision analysis, multi-criteria methods, energy policy, industrial cluster, mobile services. I. INTRODUCTION Many nations have set themselves targets for reducing the emissions of greenhouse gases, with the aim of mitigating the adverse impacts of climate change. The enormous environmental and societal issues at stake have motivated an unparalleled surge of concentrated research efforts which seek to clarify how the reduction targets can be attained through complementary policy measures (Leaf et al., 2003). These efforts build, in part, on incompatible assumptions about an immensely complex industrial and environmental ecosystem, wherefore the interpretation of resulting research findings in terms of policy implications tends to pose difficulties (Foray and Grübler, 1996). Owing to its utmost importance, climate change has acted as a major catalyst for methodological advances in integrated assessment (Dowlatabadi, 1995). In their discussion of such advances, Greening and Bernow (2004) argue that conventional approaches like cost-benefit analyses suffer from several shortcomings which can be SESSION 3 MODELS AND VOICES 1 1

2 PARTICIPATORY DEVELOPMENT OF ORIENTED POLICY CONFIGURATIONS partly addressed through multi-criteria methods, whereby the value judgements behind unavoidable trade-offs among impact classes are laid open. Still, multi-criteria methods are not a panacea, because their straightforward application does not capture the full complexity of issues at hand. Bell and et al. (2003), for example, report a participatory workshop where experts assessed policy measures with regard to several pre-defined criteria. Although successful in several regards, approaches like this are geared towards an assessment of individual policy measures in isolation, albeit most policies for climate change consist of a portfolio of measures. In this paper, we report experiences from a strategic policy workshop where leading researchers, industrialists and policy makers in Finland were invited (i) to assess the results of ClimTech, a national research and technology development (RTD) programme on climate change and (ii) to put forth policy recommendations supported by this assessment. Methodologically, the workshop was assisted by novel combinations of different approaches. For example, separate taxonomies were built for (i) the policy options (e.g., research vs. technology vs. competition policy) and (ii) the application domains in which these policies will be implemented (e.g., systems of energy production; industrial use of energy). These taxonomies were then employed to build a two-dimensional matrix for assessing the importance of policy options across the application domains. The workshop also benefited from the use of Internet-based decision support tools for voting and anonymous commenting, which made it possible to solicit multiple perspectives in a systematic and balanced manner. In general, the preparation of co-ordinated action plans among industrial firms, research organizations and policy makers involves several dimensions of which technology is but one. Motivated by this realisation, we expand ClimTech s two-dimensional weighting procedure towards a methodological and processual framework for development of coordinated policy configurations. We also demonstrate the applicability of the resulting framework called multi-weighting for short by outlining how the formulation and assessment of novel concepts for mobile gaming can be plausibly carried out, in order to derive generic priorities for RTD efforts, among others. The rest of this paper is structured as follows. Section 2 briefly discusses the use of multi-criteria methods in integrated assessment, research evaluation and technology foresight. Section 3 describes the ClimTech program and the evaluation workshop that was organized to generate future-oriented recommendations. Section 4 describes an illustrative application of multi-weighting in the context of mobile gaming. Section 5 concludes. II. MULTIPLE CRITERIA IN INTEGRATED ASSESSMENT It is now widely accepted that human activity provokes changes in the earth s climate, to the effect that many regions will become warmer, some will suffer from aridity and salinity, while the ferocity of many natural events (e.g., hurricanes) will increase (see, e.g., Foray and Grübler, 1996). This phenomenon which is seen as a major threat to sustainable development has motivated broad and well-funded research agendas, ranging from the improvement of surveillance techniques and computational techniques for enhancing the predictive accuracy of forecasts to the continual development of approaches such as scenario techniques which help address the inherent uncertainties in climate change. SESSION 3 MODELS AND VOICES 2

3 In this setting, a growing research stream has sought to develop methodologies for the integrated assessment of predicted impacts of climate change, with the aim of supporting the shaping of policy responses (Dowlatabade, 1995). In particular, the need to accommodate several impact classes has lead to the recognition that multi-criteria methods hold considerable potential, even if their application is not yet prevalent (Greening and Bernow, 2004). Nevertheless, some insightful accounts on applied work have been reported: Bell et al. (2003), for example, describe a real-life application of multi-criteria methods where climate change experts and policy makers evaluated several policy measures (e.g., introduction of global taxes, relaxed SO 2 standards, promotion of nuclear power and biomass energy) with regard to six criteria (i.e., temperature increase, ecosystem stress, sea-level rise, annualised SO 2 emissions, annualised nuclear waste generation, annualised cost) in view of the time horizon from 1990 to Apart from integrated assessment, multi-criteria methods have found uses in research evaluation, priority-setting and technology foresight, too. Salo et al. (2003, 2004) present a case study where the participants of fifteen participatory workshops employed a two-level value tree in the evaluation of a major cluster programme for Finnish forestry and forest industries. Braunschweig et al. (2001) report an application where an extensive value tree consisting of more than 32 criteria was used to develop of research priorities for the Chilean agricultural sector. In their account of a priority-setting exercise for the Scandinavian forest industries, Salo and Liesiö (2004) argue that such priority-setting exercises may benefit from preference programming methods (Salo, 1995) which admit incomplete information about preferences and consequences. Keenan (2003), in turn, reports experiences from the UK foresight program, noting that while such methods are useful in soliciting evaluative judgements, organizational considerations may curtail possibilities for contrasting such judgements at the aggregate level. III. THE CLIMTECH PROGRAM Issues in Climate Change Policies National strategies on climate change and related RTD activities involve multiple perspectives and objectives at different levels of decision making. Hence, the stakeholders objectives are not necessarily aligned; they may even point into opposite directions, which may reduce the efficiency of policy design and implementation. These kinds of complex contexts have been examined within several disciplines of innovation studies. Marin and Meyntz (1991), for example, refer to them as multi-level/multi actor arenas that can be approached with the help evolutionary theories of innovation system (Lundvall, 1992; Nelson, 1993). Despite their differences, the relevant disciplines recognise that the development of open systems calls for close stakeholder interaction. In broad terms, the objective for government intervention in climate abatement policy is to reduce the harmful consequences of climate change while creating opportunities for industrial competitiveness, economic growth and societal well-being at large. From the viewpoint of industrial strategists and government officials, this decision context is SESSION 3 MODELS AND VOICES 3 3

4 PARTICIPATORY DEVELOPMENT OF ORIENTED POLICY CONFIGURATIONS exceedingly complex, not least due to major uncertainties and far-reaching implications for several industrial sectors. From the viewpoint of structuring the main objectives, problems and opportunities in climate change, it is helpful to distinguish between endogenous and exogenous change factors. Exogenous factors include, for instance, political decisions at the national and international level, as well as macroeconomic trends and incentive mechanisms that are beyond the influence of individual innovators or an industrial sector. Endogenous factors, on the other hand, are concerned with specific regions, technologies, market segments or industrial sectors. They include national policy measures for altering the level of domestic energy production and consumption, changes in energy prices due to specific production technologies, and the diffusion of innovations within industrial clusters and sectors; as a rule, endogenous change factors are contingent on exogenous ones. In this setting, decision makers are faced with the challenge of constructing a mental model of short and long term concerns in view of both kinds of factors. A National Program on Technology and Climate Change The National Technology and Climate Change Program (ClimTech) was initiated by the National Technology Agency of Finland (Tekes, in co-operation with ministries, with the aim of supporting the joint development of climate and technology strategies among policy-makers and industry. Throughout its duration , ClimTech sought to generate information on the cost efficiency, market structure and anticipated commercialisation difficulties of technology options in the near future and beyond the Kyoto treaty. By doing so, ClimTech helped build a comprehensive and analytically grounded picture of the envisaged impacts and opportunities of climate change, covering both exogenous and endogenous factors. This was seen as a major step towards the identification of promising technologies for domestic investments and global markets (Savolainen et. al., 2003). Within ClimTech, 27 research projects were started with a total funding volume of 5 million Euro, most of which was provided by Tekes. The projects focused on topics that were predefined by the program steering-group: specifically, the program had six focus areas, which included four technology topics (renewable energy sources; distributed energy production; energy efficiency and industry; non-co 2 greenhouse gases, capture and utilization of CO 2 ), a topic on commercialization aspects, as well as a topic on pollution models and systems (Soimakallio and Savolainen, 2002; Gustafsson and Ahola, 2003). With the aim of enhancing the formulation of policy strategies for climate change and associated RTD efforts, an evaluative future-oriented workshop was held among high level experts in April 2003 (Gustafsson and Ahola, 2003). The workshop was attended by (i) four industrial experts (two consultants, two industrial association workers, a CEO of a major company), (ii) government officials from the Ministry of Environment and the Ministry of Trade and Industry and (iii) three representatives from agencies for research, technology and energy policy. The large majority of participants were well informed about the activities of ClimTech and had participated in at least some of its activities. The workshop lasted six hours in a conference room where all participants had access to a personal computer. It was jointly facilitated by two experts from the National Technology Agency. SESSION 3 MODELS AND VOICES 4

5 Structuring the strategy options There exists a vast range of policy measures for enhancing industrial competitiveness, fostering economic growth and reducing adverse climate change impacts. The relevance and efficiency of these measures depend, among others, on the maturity and costeffectiveness of relevant technologies and the infrastructures which contribute to the desired outcomes. Also, the functioning of markets and even industry-specific features such as innovativeness are relevant to the design and implementation of incentives. We sought to overcome problems of varying scale and time horizons by not requesting the participants to assess the importance of individual policy measures with regard to predefined (multiple) objectives; nor did we request them to choose among technological solutions. Rather, the assessment model was built with the aim of revealing shared objectives and views on future policy needs, whereby a distinction was made between (i) the application domains which entail diverse needs for technological and infrastructural development and (ii) the appraisal of policy options through which these needs can be partly fulfilled. In the recognition that energy production and consumption is embedded in larger systems, we created two-dimensional taxonomy and an associated value matrix for the examination of alternative policy options. In effect, the systemic nature of energy production and consumption implies that different options are often complementary, whereby the development of a explicit framework can uncover additional values and interrelationships (see Keeney, 1992). At the highest level of aggregation, opportunities in application domains were divided into the two classes of energy production and energy consumption. Energy production was further divided into (i) the development of new distributed energy technologies and the integration thereof into energy systems and (ii) efficiency improvements in existing energy production and distribution systems. Energy consumption, in turn, was divided into the consumption of energy in (iii) energy intensive industry and (iv) other societal uses (households, traffic and transportation). In the workshop, the participants were asked to provide weights (A) for indicating the relative importance of the four applications domains for the overall strategy portfolio; an aggregate measure for the relative importance of each domain was obtained by averaging these responses. This also provided one of the outputs for focusing further workshop discussions. The assessment of policy options (B) was carried separately for each application domain. Towards this end, the following five predefined options were defined: (1) research policy (2) technology policy (3) market forces, (4) competition policy, and (5) communication policy; these options also approximated the participants role and expertise in the energy system. Within each application domain, the participants were asked to assess the relative importance of the five policy options by allocating 100 points among them (i.e., weights B1-B5; B6-B10; B11-B15: B16-B20). SESSION 3 MODELS AND VOICES 5 5

6 PARTICIPATORY DEVELOPMENT OF ORIENTED POLICY CONFIGURATIONS Efficiency of present energy production system Dev t of new and distributed technologies Efficience of industry processees Dev t of the use of energy in the society Research policy Technolog y policy Market forces Competitio n policy Communic ation policy A1*B1 A1*B2 A1*B3 A1*B4 A1*B5 A2*B6 A2*B7 A2*B8 A2*B9 A2*B10 A3*B11 A3*B12 A3*B13 A3*B14 A3*B15 A4*B16 A4*B17 A4*B18 A4*B19 A4*B20 Table 1. Value matrix on policy options. Finally, averages of the participants weighting judgements (A and B) were computed to obtain a two-dimensional value matrix with real numbers in the range [0,1] in each cell. This matrix offered a visual top-down representation which conveyed priority areas for the further elaboration of strategy options in the workshop discussions and also later on. In effect, these discussions provided opportunities for interpreting the resulting priorities in terms of concrete implementation plans. Decision Support for Process Implementation The workshop participants were encouraged to share information openly while participating in a collaborative information structuring and evaluation activity. In effect, much of the information was provided and processed through face-to-face discussions. These, in turn, were supported by a mind-mapping software and a Group Support System (GSS) which allowed the participants to vote and put forth anonymous comments. Apart from contributing to the uptake of results from ClimTech, this approach seemed well-motivated, because equivocal expert statements are often best assimilated in face-to-face discussions (cf. Daft and Lengel, 1986). The workshop activities included the assessment of the programme implementation and the utility of research results, as well as the identification of change drivers and potential conflicts on national strategy formulation (see Table 2). The GSS was used in formal evaluations (e.g., votes on the results of ClimTech), and made it possible to synthesise participants statements for immediate display to the group as a whole. SESSION 3 MODELS AND VOICES 6

7 Phase Activities Persons Assessment of programme implementation and the utility of research results a. Presentation b. Discussion c. Assessment Facilitator All Identification of change drivers and potential conflicts in national strategy conflicts Table 2. The workshop agenda. a. Presentation b. Discussion Industrial experts Facilitator All Activities before, during and after the workshop were structured with the aim of aligning the participants mindsets. Before the workshop, the participants and several other experts responded to an Internet survey which supplied relevant background information. For example, the respondents were asked to assess on a five-point Likert scale the ClimTech research reports in terms of their plausibility, reliability, information content and insightfulness. They were also asked to identify change drivers and possible conflict topics in the formulation national strategy, and to submit written comments about how national energy, science, technology, environmental, and competition policies might be improved. Building on multiple perspectives and information sources, the first part of the workshop drew attention to the complexity of the context within which the participants were requested to formulate statements on strategic options. This was essentially a relatively loose process of knowledge acquisition and structuring, based on visual contact, voice and intensive interaction. Rochlin (1997) has referred to such loose and unstructured processes of information transfer as creating the bubble, which is crucial for the operation of complex high-reliability systems (e.g., nuclear power plants, air traffic control centres, power distribution systems), among others. The second part of the workshop which was also started by presenting relevant results from the Internet survey contributed to learning from past through detailed questions on programme implementation and the utility of research results. The participants responded to many questions through computer mediated anonymous voting, the results of which were then presented to the whole group. Finally, informal discussions during the closing phases of the workshop sought to reveal the collective mind (Weick, 1993), shared beliefs (Zafonte and Sabatier,1998) and participants common views, also called cognitive maps, on complementary policy issues and roadmaps for future interventions. Overall, the experiences from the ClimTech workshop suggest that a top-down multiweighting procedure helps communicate the relative importance of aggregate policy options, even if the resulting weight profiles must be further refined and interpreted to account for complex interrelationships among specific policy measures. Another conclusion is that the participatory process is important, not least because it contributes to a shared perception of future opportunities, whereby it offers strategic policy intelligence through reduced uncertainties about possible development paths (see, e.g., Kuhlmann, 1999). SESSION 3 MODELS AND VOICES 7 7

8 PARTICIPATORY DEVELOPMENT OF ORIENTED POLICY CONFIGURATIONS IV. BUILDING AN INDUSTRY FOR MOBILE GAMING The need for co-ordinated shaping of policy options is by no means confined to climate change; rather, the need for the co-ordination of policies and action plans is vital to the fostering of mission-oriented concepts which entail elements of architectural innovation. One such context is mobile gaming which can be seen as a part of a larger concept mobile entertainment defined as any leisure activity undertaken via a personal technology which is, or has the potential to be, networked and facilitates transfer of data (including voice, sound and images) over geographic distance either on the move or at a variety of discrete locations (MGAIN3, 2003). This definition is broad enough to include portable modes of gaming where there is no actual interaction with the environment or other players. From a broader perspective, gaming is of considerable interest, because it exploits new technologies rapidly and constitutes the fastest growing field of the entertainment industry globally (Arc Group, 2001). Also, because there are close links between mobile games and adjacent industries, researchers and industrial firms need to decide to what extent they wish to focus on key technology areas or, alternatively, on enabling and supporting technologies. Elements of mobile gaming Building on the above characterisation of mobile gaming, it is useful to construct a taxonomy of different types of mobile gaming along the dimensions of (i) format categories for content presentation (MGAIN3, 2003) and (ii) utilization of mobility; taken together, these two dimensions largely define how a particular game utilizes its mobile environment. Moreover, a third dimension can be associated with (iii) commonly-known game types (e.g., quizzes and stand-alone games; see ArcGroup, 2002). The resulting three-dimensional taxonomy offers an applicable framework for the analysis of mobile gaming (see Table 3). Game type Presentation format Mobility function Stand alone SIM Audio, image Portability Quiz Messaging Portability, interaction Downloadable stand alone Audio, image Download Downloadable on-line Audio, image, messaging Download, interaction, group gaming Cross media Audio, image, video, Download, group gaming, (cell phone, internet, TV) Ubiquity, context awareness Table 3. Dimensions of mobile games. messaging, browsing Audio, image, video, messaging, browsing interaction Group gaming, interaction, positioning For the purpose of priority-setting, it is useful to distinguish between technologies and content. For example, the ten technology areas identified in MGAIN4 (2003) (e.g., display technologies; battery technologies; integration of digital services) can be structured to obtain a shortlist of five core technologies (i.e., terminal technologies; networking technologies; provisioning technologies; reality enhancing technologies; SESSION 3 MODELS AND VOICES 8

9 standardisation). The production of game content which provides the actual entertainment functionality usually takes place through a ten-phase process where three modes of content production are prevalent (i.e., narratives; graphics and audio; design; see Manninen, 2004). Based on the above concepts, we outline a framework for future-oriented prioritysetting in the context of Neogames, the cluster development programme for the Finnish gaming industry. This programme was established at 2003, largely in response to the recognition that technological development of new media (e.g., digital TV), enhanced capabilities of cellular phone systems and advances in content creation provide a fertile ground for new types of innovative products and services. Also, the Finnish content production industry has remained relatively small, even though the ICT sector has grown to one of the largest industries in Finland. Neogames seeks to create long-term industrial competencies for computerised gaming, including mobile gaming. It differs from many large RTD programmes in that plenty of attention is given to the development of results in the shorter term, not least because this is warranted by the dynamism of the industry (Salmenkaita and Salo, 2004). Neogames is organized into four working groups (i.e., research and education; business development and internationalisation; technology; communications). The joint projects are funded mostly by the National Technology Agency (Tekes). Multidimensional assessment framework Successful products and services in mobile gaming are essentially manifestations of viable concepts that are implemented through mission-oriented RTD activities and related efforts. Consequently, fresh inputs to cluster development can be obtained by soliciting suggestions for new concepts from industry and research community, with the remit that the proponents must elaborate their concepts from the users perspective so that a meaningful assessment can be made. The concepts can then be evaluated in view of their novelty, technological risks and business potential (see, e.g., Salo et al., 2003), with the aim of deriving aggregate performance measures that help in the identification of most promising concepts. More specifically, each concept can be mapped onto activities that should be carried out towards its successful realisation within the programme. At the aggregate level, these activities can be examined from the viewpoint of content and technology subcategories. For each concept, the importance of subcategories can be determined through a weighting task where the experts (such as the participants of the ClimTech workshop) allocate points among the relevant subcategories (see Table 4). Taken together, the two assessments (i.e., evaluation of concepts, weighting of subcategories) can be combined to derive aggregate priorities for content and technology development. Judgemental adjustments to these priorities may be called for, particularly if (i) there are considerable synergies between the concepts or their underlying technologies, or (ii) if the concepts are not representative of the envisaged programme scope. The framework can be expanded through additional descriptive categories, in order to illustrate how the concepts correspond to different games types and applications. The need for additional categories depends on the number of submitted proposals: the larger SESSION 3 MODELS AND VOICES 9 9

10 PARTICIPATORY DEVELOPMENT OF ORIENTED POLICY CONFIGURATIONS the number of potential concepts, the more time and effort will be needed to structure them meaningfully. In Table 4, for example, some indicative X s (on a qualitative scale from X to XXX) indicate the degree to which a given concept fits a particular subcategory. Concept Concept Concept Concept Priorities Assessment (0-10 pts) 8 pts 3 pts Content Narratives (3x40) Graphics & audio (8x10+3x20) Game design Sum Technologies Standardisation 5 40 Terminal technologies Networking technologies Provisioning technologies Reality enhancing technologies Sum Game types Stand alone SIM X Quiz XX Downloadable Cross media XX Ubiquitous Applications Music X Short gaming XXX Education XXX Social benefits Table 4. A multi-weighting framework for the assessment of mobile gaming concepts. Apart from the concept-oriented bottom-up approach outlined above, also other kinds of priority-setting processes can be carried out within the framework of Table 4. For example, one may seek to assign weights to the rivalling technologies in a top-down manner, much in the same way as the participants of the ClimTech workshop allocated points to the four application domains. One may even fix a given application class, and evaluate technologies and modes of content production from the viewpoint of the selected application class. Overall, a potential benefit of the concept-oriented approach is that tangible proposals can more readily subjected to an ex ante evaluation while the generation of such proposals promotes creativity. The detailed examination of these concepts may also reveal implications for innovation activities (thus contributing to enhanced co-ordination), which may not happen if technologies are dealt with in the abstract. In its current form, the above framework is geared towards the analysis of concepts that are viable in the shorter and intermediate term which is not a major limitation given SESSION 3 MODELS AND VOICES 10

11 the relatively short product developments cycles in the gaming industry (Salmenkaita and Salo, 2004). The framework can also be extended to incorporate more futuristic concepts as well, with the aim of generating fresh perspectives and longer-term action agendas. Even in this case, the systematic assessment of concepts is likely to contribute to an enhanced understanding of focal questions in the development of technologies and competencies. V. DISCUSSION Experiences from the ClimTech workshop and our tentative discussion of mobile gaming suggest that multidimensional weighting can assist in the development of priorities which seek to co-ordinate the actions of industry, research community and public administration in support of mission-oriented targets for innovative activity. Many of the benefits of such procedures accrue from following observations: Validity of statements: The explication of relevant dimensions provides a systematic framework for the delivery of comprehensive analyses. At the same time, policy options and other actions can be examined in recognition of several conditioning dimensions (cf. application domains in the ClimTech workshop), which helps the experts to put forth statements that reflect the specific concerns in each particular context. This can be contrasted with the difficulties of assessing policy options at a high level of generality which, for instance, forces the experts to average the viability of policy options across several application domains. Contrasting perspectives: The use of multi-weighting for top-down priority-setting does not preclude the use of complementary approaches. As we have argued in the context of mobile gaming, both top-down and bottom-up approaches can be combined within the same priority-setting exercise, whereby the examination of differences may yield further insights. Support for policy co-ordination: Even though the multi-dimensional frameworks we have discussed are not expressly intended for the explication of links between interdependent policy options, the consideration of alternative contexts (as defined by combinations of dimensions) does support the development of plans that reflect the characteristics of each context. Concept-oriented bottom-up approaches, in particular, seem suitable for this purpose, because each concept can be taken as a unit of analysis which is potentially implemented through co-ordinated actions. Yet, an adequate balance between top-down and bottom-up approaches is needed, because the big picture may be lost if the focus remains on individual concepts only. VI. REFERENCES 1. Arc Group Mobile Entertainment Worldwide Market Analysis and Strategic Outlook , July 2001, ( 2. Arc Group Mobile Gaming, October 2002, ( 3. Bell ML, Hobbs BF, Ellis H The Use of Multi-Criteria Decision-Making Methods in the Integrated Assessment of Climate Change: Implications for IA Practitioners. Socio-Economic Planning Sciences 37/4, Braunschweig T, Janssen W, Rieder P Identifying Criteria for Public Agricultural Research Decisions. Research Policy 30/5, Daft RL, Lengel RH Organizational Information Requirements, Media Richness and Structural Design. Management Science 32/5, SESSION 3 MODELS AND VOICES 11 11

12 PARTICIPATORY DEVELOPMENT OF ORIENTED POLICY CONFIGURATIONS 6. Dowlatabadi H Integrated Assessment of Climate Change: An Incomplete Overview. Energy Policy 23, Foray D, Grübler A Technology and the Environment: An Overview. Technological Forecasting and Social Change 53, Greening LA, Bernow S Design of Coordinated Energy and Environmental Policies: Use of Multi-Criteria Decision-Making. Energy Policy 32, Gustafsson R, Ahola E A Prospective Evaluation of the ClimTech Operations Model and the National Climate and Technology Strategy. Technology Programme Report 20/2003. Tekes, Helsinki (in Finnish). 10. Keenan M Identifying Emerging Generic Technologies at the National Level: the UK Experience. Journal of Forecasting 22/2-3, Keeney R Value Focused Thinking. Harvard University Press, Cambridge. 12. Kuhlmann S. et al. (1999). Improving Distributed Intelligence in Complex Innovation Systems, Final report of the Advanced Science & Technology Policy Planning Network (ASTPP). A Thematic Network of the European Targeted Socio-Economic Research Programme (TSER). Fraunhofer ISI, Karlsruhe. 13. Leaf D, Verolme HJH, Hunt WF Jr Overview of Regulatory/Policy/Economic Issues Related to Carbon Dioxide. Environmental International 29, Lundvall B-Å National Systems of Innovation: Towards a Theory of Innovation and Interactive Learning. Pinter Publishers, London. 15. Manninen, T Rich Interaction Model for Game and Virtual Environment Design, University of Oulu, Oulu. 16. Marin B, Mayntz R (ed.) Policy Networks: Empirical Evidence and Theoretical Considerations. Campus Verlag, Frankfurt am Main. 17. MGAIN Mobile Entertainment Industry and Culture, A European Commission user-friendly Information Society, (IST) Accompanying Measures project IST , WP3 Mobile Entertainment Concepts and Culture, Deliverable D3.1.1, Mobile Entertainment in Europe: Current State of the Art 18. MGAIN Mobile Entertainment Industry and Culture, A European Commission User Friendly Information Society, (IST) Accompanying Measures project IST , WP4 Mobile Technologies, Deliverable D4.2.1, Emerging and Future Mobile Entertainment Technologies. 19. Nelson RR (ed.) National Innovation Systems: A Comparative Analysis. Oxford University Press, New York. 20. Rochlin GI Trapped in the Net: The Unanticipated Consequences of Computerization, Princeton University Press, Princeton. 21. Salmenkaita J-P, Salo A Rationales for Government Intervention in the Commercialization of New Technologies. Technology Analysis and Strategic Management 14/2, Salmenkaita J-P, Salo A Emergent Foresight Processes: Industrial Activities in Wireless Communications. Technological Forecasting and Social Change (to appear). 23. Salo A Interactive Decision Aiding for Group Decision Support. European Journal of Operational Research 84, Salo A, Gustafsson T, Mild P Prospective Evaluation of a Cluster Programme for Finnish Forestry and Forest Industries. International Transactions on Operations Research 11/2, Salo A, Gustafsson T, Ramanathan R Multicriteria Methods for Technology Foresight. Journal of Forecasting 22/2-3, Salo A, Liesiö J A Case Study in Participatory Priority-Setting for a Scandinavian Research Programme. Unpublished manuscript, Helsinki University of Technology, Systems Analysis Laboratory. 27. Savolainen I, Ohlström M, Kärkkäinen A Climate - A Challenge for Technology. Tekes, Finland (in Finnish). SESSION 3 MODELS AND VOICES 12

13 28. Soimakallio S, Savolainen I (eds.) Technology and Climate Change CLIMTECH Final Report. Technology Programme Report 14/2002, National Technology Agency of Finland, Helsinki. 259 p. 29. Weick EK, Roberts KH Collective Mind in Organizations: Heedful Interrelating on Flight Decks. Administrative Science Quarterly 38, Zafonte M, Sabatier P Shared Beliefs and Imposed Interdependencies as Determinants of Ally Networks in Overlapping Subsystems. Journal of Theoretical Politics 10/4, SESSION 3 MODELS AND VOICES 13 13