A Cognitive Case Study of a Product/Service-System Design Using Protocol Analysis

Transcription

1 A Cognitive Case Study of a Product/Service-System Design Using Protocol Analysis Tomohiko Sakao * Department of Management and Engineering, Linköping University Linköping, 58183, Sweden tomohiko.sakao@liu.se * Corresponding author John Gero Department of Computer Science and School of Architecture, University of North Carolina at Charlotte Charlotte, 28223, USA john@johngero.com Hajime Mizuyama Department of Industrial and Systems Engineering, Aoyama Gakuin University Fuchinobe, Chuo-ku, Sagamihara-shi Kanagawa , Japan mizuyama@ise.aoyama.ac.jp Abstract This article presents results of a cognitive case study of a Product/Service System (PSS) design session using protocol analysis. It is motivated by the current lack of cognitive studies of PSS design that provide empirical support for understanding of conceptual design of PSS. Hypotheses related to where and how designers expend their cognitive design effort that are considered to characterize conceptual design of PSS are proposed. They are examined using the results from the protocol analysis based on Function-Behaviour-Structure (FBS) coding scheme. The results show that, in this case study, PSS designers put most of their cognitive design effort into behaviour and that PSS design is a co-evolutionary process. This article also introduces new measurement techniques for cognitive study of PSS design. Keywords design behaviour, design cognition, design process, engineering design, conceptual design 34 1

2 Highlights Case study of a PSS design using protocol analysis with FBS coding scheme is presented. Four hypotheses derived from existing literature on PSS were examined. High cognitive design effort was spent on purposes and expectations in the design case. Analysis and evaluation were found to be the dominant processes. Co-evolutionary process between the problem and solution spaces was observed. 41 2

3 (from this point, 9948 words incl. references plus illustrations; 8360 excl. references) Submissions to this journal should normally be within the range words plus illustrations Today, manufacturers in developed countries regard service activities as increasingly important (Meier, Roy, & Seliger, 2010). Some manufacturers earn more than half of their revenue from services (e.g. aerospace by Rolls-Royce (2015)). Further, some firms are strategically shifting from being a product seller towards being a service provider. One reason is that they face intense competition to continue to develop products and services with higher value. In line with this trend, Product/Service Systems (PSS) (Mont, 2002; Tukker & Tischner, 2006) is much debated as a promising concept for a design object in academia as well as industry (Roy & Baxter, 2009). A definition of PSS is tangible products and intangible services designed and combined so that they jointly are capable of fulfilling specific customer needs (Tischner, Verkuijl, & Tukker, 2002). According to the definition of PSS, in designing PSS, service is addressed as part of the design object, which has been often dominated by physical products in manufacturing industries. This may impact substantially PSS-design process, as the design object may substantially influence the design process (Hubka & Eder, 1987)(Visser, 2009). However, there are insufficient insights, especially insights based on empirical research, into conceptual design of PSS. For instance, only a handful of descriptive studies has been carried out quantitatively on how PSS design is actually carried out (Bertoni, 2013; Sakao & Mizuyama, 2014; Sakao, Paulsson, & Mizuyama, 2011; Shimomura, Nemoto, & Kimita, 2015), and there is little literature on an empirically-based understanding of PSS design processes. The processes of PSS design are not sufficiently grounded in scientifically derived data. Currently it is not possible to answer whether or not PSS designing is different from other designing, and, if so, how it is different. Were this information available to PSS designers, it could be effectively used to develop PSS design support methods and tools. Motivated by this gap in our knowledge, the research reported in this article aims to provide a greater understanding of conceptual design processes of PSS based on quantitative data. To do so, the research analyses the details of the entire process of a PSS design case using protocol analysis (Ericsson & Simon, 1993). The remainder of the article is structured as follows: Section 1 presents knowledge gaps in existing research, the aim of this article with its significance, and the hypotheses to be examined; Section 2 describes the research method; Section 3 presents the setting for the PSS design case; Section 4 shows the results of the analysis; Section 5 discusses the analysis and concludes the article. 1. Literature analysis 1.1 Need of descriptive study in PSS design research For more than a decade, interest in integration of services with products in the manufacturing industry has grown, and, as a result, both theory and practice for the integration has evolved (Baines, et al., 2007; Meier, et al., 2010; Oliva & Kallenberg, 2003). Services here include e.g. monitoring, inspection, operation, maintenance, repair, upgrade, overhaul, take-back, training, and consultation. Existing literature about this integration suggests methods and strategies for PSS, although they tend to be generic in terms of insights provided (Tukker, 2015). Design is crucial in PSS as is implied by the definition given in the previous section and there is a small but growing research-based literature on PSS design (e.g. (Roy & Baxter, 2009)). The research conducted thus far includes descriptive and prescriptive studies. Literature states, for instance, characteristics expected for PSS design. The literature is based on different types of design research methodology and some are based on empirical data from different sectors. Some prescriptive models for PSS design have been proposed. For instance, Morelli (2003) explored PSS design by analysing a concrete development of a PSS (an urban tele-centre) as a case study, and, based on the case study, proposed a PSS design process model. It consists of the following ordered activities: value proposition, market analysis, product/service definition, use-case analysis, tentative architecture, test, and final definition. Alonso-Rasgado et al. (2004) proposed a design process for Total Care Products (Functional Products), which means integrated systems comprising hardware and support services and 3

4 is understood as PSS, partly based on literature from the service design domain. It produced the following ordered process: (1) business ambitions of the client, (2) potential business solutions, (3) core definition of PSS plus PSS options, (4) enhanced definition of the potential PSS, and (5) risk analysis and evaluation of business cases. Aurich et al. (Aurich, Fuchs, & Wagenknecht, 2006) proposed a service design process for technical PSS, and illustrated it by means of an example from the investment goods industry. Sakao and Shimomura (2007) proposed a design process of products and services for Service Engineering and verified it with two cases one from the service and the other from the manufacturing sector. Maussang, Zwolinski, and Brissaud (2009) proposed a methodology to provide designers with technical engineering specifications for PSS by using functional analysis, which is originally developed for product design. Other models for PSS verified with computer aided design (CAD) tools are available (Hara, Arai, & Shimomura, 2009; Sakao, Shimomura, Sundin, & Comstock, 2009). Another CAD tool explicitly describing and simulating dynamic relations between products and service in the life cycle has been proposed (Komoto & Tomiyama, 2008). Further, Amaya et al. (Amaya, Lelah, & Zwolinski, 2014) proposed a general methodology to model PSS in order to help designers to quantify the environmental benefits. These models intended to be used for support of PSS design have been developed largely based on reasoning using existing design theories and methods for product design or service design. Only a few articles report descriptive and quantitative study of PSS design processes. For instance, Sakao and colleagues (Sakao & Mizuyama, 2014; Sakao, et al., 2011) carried out protocol analysis of a PSS design and uncovered lifecycle activity is a central notion addressed within the design case. Bertoni (2013) reports, based on protocol analysis of eight PSS-design sessions, that design teams using colorcoded CAD models as carriers of value-related information made a more extensive use of information during problem analysis and followed a more structured design process than teams using spreadsheets. Shimomura, et al. (2015) found, from protocol analysis with three PSS-design sessions, that a session having produced a better design solution spent more time on value proposition. The earlier research gives some indication of the characteristics of PSS design, however, none of them answers clearly whether or not PSS designing is different from other designing, and, if so, what are the differences. In this sense, our current understanding of PSS design process is insufficient, and the lack of this understanding can be a problem for the research area - producing a number of methods and tools to support PSS design without an adequate understanding of the PSS design processes. This prevents researchers from developing design support methods, as well as any basis for improving the efficiency of the design process. This gap in our knowledge provides the motivation for the authors to conduct this research in order to better understand the PSS design processes through a quantitative study of PSS designing. As an initial step we carried out a case study, which is reported here. 1.2 Aim and scope The aim of this article is to identify characteristics of PSS designing and thereby to enhance the current understanding of PSS design processes. It is part of a larger study comparing PSS design with product design. The driving research questions of this article are: What are the characteristics of PSS design in terms of distribution of design issues and design processes? The research reported in this article focuses on conceptual design in PSS design, because of the following reasons. First, conceptual design is less well understood than other aspects of designing and requires further research. Second, conceptual design in PSS design, where a realization structure for a purpose is not necessarily fixed as a product or service, is peculiar to PSS design. Once each realization structure is decided as product or service, design is then more about that of a pure product or pure service, to which more insights are available. Thus, it is more interesting to research conceptual design in PSS design. The article also focuses on designing without customers involvement (Neale & Corkindale, 1998), as it is less complex but still lacks insights. The insights produced by this research, if generalizable, would provide implication for evaluating existing PSS design methods, developing new PSS design methods, and identifying various future research issues in PSS design. 4

5 Hypotheses This section first analyses literature on PSS to derive characteristics of PSS. Based on the characteristics, it further analyses literature on design in general and PSS design to derive their implication on the conceptual design of PSS. To do so, characteristics of PSS based on literature review from the perspective of information flows (Durugbo, Tiwari, & Alcock, 2011) are adopted and more characteristics are added from the design perspective (as shown in Table 1). The characteristics are categorized into their properties of PSS identified (ibid.). Then, implication on the conceptual design of PSS are derived. Table 1. Key properties and characteristics of PSS and their implication on conceptual design of PSS Property Characteristics Implication on conceptual design Open process systems Business model Social construct System architecture System components System behaviour (INCOSE, 2006) Inputs and outputs Processes and functions Uncertainty (Erkoyuncu, et al., 2011) Human activities (Alonso-Rasgado & Thompson, 2006) Heterogeneity (Regan, 1963) Domain of application Nature of business Customer orientation (Tukker & Tischner, 2006) Value proposed (Sakao & Shimomura, 2007) Performance of asset (Alonso-Rasgado, et al., 2004; Baines, et al., 2007) Available resources Actors, roles, and scenarios Technological and socio-cultural interactions Relationship between customer and provider (Baines, et al., 2007) Apply system thinking (Baines, et al., 2007). Consider various types of uncertainty. Analyse behaviour as a system. Analyse if the adopted business model works in the domain. Analyse customers (Sakao & Shimomura, 2007). Include value proposition (Morelli, 2003). Analyse performance of products and services. Apply co-creation process between customer and provider (Alonso-Rasgado, et al., 2004; Baines, et al., 2007; Morelli, 2003; Smith, 2013). Note: The three properties are taken from (Durugbo, et al., 2011), while the characteristics adopt those in (ibid) and others added by the authors with references. Implication on conceptual design comes from the authors own elaboration. The first property of PSS is open process systems. This means that PSS is a system with input and output flows. Output flows are determined by processes in PSS, which can be used to describe human activities, and functions. The human activities (Alonso-Rasgado & Thompson, 2006) are characterized by heterogeneity inherited from generic characteristics of pure service (Regan, 1963). Further, PSS is characterized by interdependency between product and service (Meier, et al., 2010) and thus interaction between them (Komoto & Tomiyama, 2008). These mean conceptual design of PSS is more complex than that of pure product or service. These imply need of system thinking in conceptual design of PSS (Baines, et al., 2007). For designing a system, behaviour as a system needs to be analysed. Behaviour of elements is relevant to design in general (Love, 2000), however the higher level of complexity of PSS makes the behaviour as a system especially relevant in conceptual design of PSS. The next property is business model, which takes into account the nature of business in the concerned domain. Business model is often defined to include value as its construct (Mason & Spring, 2011; Osterwalder, Pigneur, & Smith, 2010). Therefore, value proposed is a crucial characteristic (Sakao & Shimomura, 2007). In addition, customer orientation is a PSS characteristic (Tukker & Tischner, 2006). As an implication, in carrying out conceptual design of PSS, value proposition to PSS recipients including customers needs to be included (Morelli, 2003) and for value proposition analysing customers is crucial (Sakao & Shimomura, 2007). As value often lies in the performance of products and services of PSS instead of the ownership as such (Alonso-Rasgado, et al., 2004; Baines, et al., 2007), the performance needs to be analysed as well. Last, the social construct involving more actors than in e.g. pure product design is a PSS property. Further, Baines, et al. (2007) assert the relationship between the customer and the provider plays a key role in PSS design, which is indeed reported with the case of Rolls Royce (Smith, 2013). This implies 5

6 co-creation between customer and provider is recommended in PSS design. The implication on conceptual design of PSS is here transformed into hypotheses to be examined in this paper. Having the driving questions of the article (Section 1.2) in mind, the issues used in PSS design are addressed first. As discussed above, behaviour is a crucial issue to be addressed in particular in conceptual design of PSS. This leads to Hypothesis 1 (H1). H1. In conceptual design of PSS, the behaviour of the design is the dominant design issue. As discussed above in relation to the second property, a customer is to be analysed to define value proposed in PSS design. Alonso-Rasgado et al. (2004) assert a PSS customer pays only for a functional performance to be expected at the customer s own settings, i.e. the customer s purposes, and does not buy the hardware (a partial solution). In line with this, the PSS design process model proposed by Morelli (2003) further introduces a step for value proposition to define the needs expected to be fulfilled by the PSS. The design process proposed by Sakao & Shimomura (2007) includes a step for describing purposes of concerned stakeholders. These statements in literature leads to H2. H2. Conceptual design of PSS addresses purposes and expectations of a design more than solutions. Concerning the processes of PSS design, analysing a concerned system, performance, and customers is raised as important from the first and second properties in Table 1. In design in general, analysis of a design solution is regularly followed by evaluation. Evaluation is carried out against the expectation for a solution and is thus an activity to reason about a design solution and a design problem to be solved (Pahl & Beitz, 1996). Reasoning between the solution and problem spaces, which corresponds to evaluation, is also implied to be substantial in PSS design by Morelli (2003). Komoto and Tomiyama (2008) even state that PSS design is to find a mapping between activities in a service environment and value. Therefore, H3 is created as below. H3. In conceptual design of PSS, analysis and evaluation are the dominant processes. The third property of PSS in Table 1 implies that the co-creation process between customer and provider is recommended in PSS design. Alonso-Rasgado, et al. (2004) suggest an iterative process between the PSS provider and recipient in the conceptual design of PSS. The reason is that the options available are not readily apparent to the PSS recipient and a comprehensive picture of client needs and requirements cannot be built up before conceptual design begins (ibid). When carrying out conceptual design without customers directly involved, this could be replaced with co-evolution process between problem and solution (Maher, Poon, & Boulanger, 1996). This sort of design process model is proposed by (Morelli, 2003), consisting of an iterative sequence of phases in which problems generate solutions, which, in turn, redefine new problems. This leads to H4. H4. Conceptual design of PSS is conducted in a co-evolutionary process between the problem and solution spaces. 2. Method 2.1 Protocol study with a PSS design case This research adopts protocol analysis as the method to provide empirically-based quantitative evidence to examine the four hypotheses. Protocol analysis is a rigorous methodology for eliciting verbal reports of thought sequences as a valid source of data on thinking. It is a well-developed, validated method for the acquisition of data on thinking (Ericsson & Simon, 1993; van Someren, Bardard, & Sandberh, 1994). It has been used extensively in design research to assist in the development of the understanding of the cognitive behaviour of designers (Atman & Bursic, 1996; Badke-Schaub, Lauche, Neumann, & Ahmed, 2007; Christensen & Schunn, 2007; Gericke, Schmidt- Kretschmer, & Blessing, 2007; Kavakli & Gero, 2002; Mc Neill, Gero, & Warren, 1998; McDonnell & Lloyd, 2009; Purcell & Gero, 1998; Suwa, Purcell, & Gero, 1998; Tang & Gero, 2002). The motivation for the selection of the case study method (Yin, 2013) is to develop new knowledge based on a rich empirical description of PSS designing to begin to fill the knowledge gap outlined in Section

7 FBS (Function-Behaviour-Structure) scheme Overview This article makes use of a method of determining and describing design cognition, based on the Function Behaviour Structure (FBS) ontology (Gero, 1990). This is a design ontology that is independent of the design task, the designer s experience and the design environment and hence produces commensurable results from different experiments (Gero, 2010; Gero & Kannengiesser, 2014; Jiang, 2012; Kan, 2008). It is therefore suitable for use in studying PSS designing and also for later comparing the results to other studies of designing. The FBS ontology provides a uniform framework for classifying cognitive design issues and cognitive design processes, and includes higher level semantics in their representation. The FBS ontology (Gero, 1990) models designing in terms of three classes of ontological variables: function, behaviour, and structure plus two variables that are expressible in terms of function, behaviour or structure, requirements and design description, as shown in Figure 1. In this view the goal of designing is to transform a set of functions, driven by the client requirements (R), into a set of design descriptions (D). The function (F) of a designed object is defined as its intended purpose, expectations or teleology; the behaviour (B) of that object is either derived (Bs) or expected (Be) from the structure, where structure (S) represents the components of an object and their relationships. A design description is never transformed directly from the function but is a consequence of a series of processes among the FBS variables. These processes include: formulation which transforms requirements and functions into a set of expected behaviours (process 1 in Figure 1); synthesis, where a structure is proposed to fulfil the expected behaviours (process 2); an analysis of the structure produces derived behaviour (process 3); an evaluation process acts between the expected behaviour and the behaviour derived from structure (process 4); documentation, which produces the design description (process 5). There are three types of reformulation: reformulation I reformulation of structure (process 6), reformulation II reformulation of expected behaviour (process 7), and reformulation III reformulation of function (process 8). Figure 1 shows the relationships among the eight transformation processes and the three basic classes of variables, which claim to be the fundamental processes for designing. The FBS ontology has been used in numerous protocol studies of designers ((Gero, 2010; Jiang & Yen, 2009; Kan, 2008; Lee, Gero, & Williams, 2012; Williams, Gero, Lee, & Paretti, 2011)).[ADD other non-gero refs] R Figure 1. The FBS ontology with its consequential ontology of design processes, labelled 1 through 8. The FBS ontology has been referenced extensively as an ontology of designing that has been used in multiple disciplines and one that transcends individual designers, the design task, the design environment, and whether designing individually or in teams (Branki, 1995; Hofmeister, et al., 2007; Jiang, 2012; Kruchten, 2005; Robin, Rose, & Girard, 2007; Van Wie, Bryant, Bohm, McAdams, & Stone, 2005; Visser, 2006) Interpretation and use of FBS scheme Table 2 shows a match between the design issues in the FBS scheme and frequently addressed dimensions in PSS design. There is no commonly agreed set of dimensions for PSS as a design object, and thus here the nine dimensions in (Müller, Kebir, Stark, & Blessing, 2009) are adopted as a 7

8 working set. This matching is used as a basis for the protocol analysis shown in Section 4. Table 2. FBS design issues applied in the PSS context FBS scheme Explanation PSS dimensions Requirement What is required by the client Needs stated by the client Function What it is for Client s needs as interpreted by the designers and those added by the designers Values Expected Behaviour What it is expected to do Life cycle activities Structure What it is Core product Peripheral product Actors Contract elements (in documents) Payment model Structure Behaviour What it does Life cycle activities Document What it is documented as Contract Sketches Deliverables (e.g. service manual) The results from an FBS coded protocol can be measured in multiple ways to provide an increased understanding of PSS designing. This article uses the following quantitative measures. Tabular statistics: this produces the statistical distributions of the design issues and the design processes and provides quantitative measurements of where designers cognitive design effort is expended. This can be visualized with cumulative curves (see Section 2.2.3). Problem-Solution index: this is a macro-measure that describes whether the designers are spending more of their cognitive design effort on the problem or the solution across time during the design session (see Section 2.2.4). Problem-Solution discontinuity ratio: this represents the frequency of transitions between the problem and solution spaces in a given period (see Section 2.2.5) Cumulative occurrences, curves and their shapes! The cumulative occurrence (C) of design issue (x) at segment (n) is C! =!!! x! where (x i ) equals 1 if segment (i) is coded as (x) and 0 if segment (i) is not coded as (x). Plotting the results of this equation on a graph with the segments (n) on the horizontal axis and the cumulative occurrence (C) on the vertical axis will visualise the occurrence of the design issues. Figure 2 shows a general representation of such a graph, where a curve with its shape shows characteristics of the occurrences over segments. Similarly C x, the cumulative occurrence (C) of!!! syntactic process (y) is C! =!!! y! where (y i ) equals 1 if the transition from segment (i) to segment (i+1) is coded as (y) and 0 if it is not coded as (y) Figure 2. Graphical representation of the cumulative occurrence of design issues in a design protocol 8

9 Problem-Solution index The Problem-Solution index (P-S index), whether for issues or processes, is a measurement to characterize the overall cognitive style of designing. It is determined by calculating the ratio of the total occurrences of the design issues/processes concerned with the problem space to the sum of those related to the solution space, as shown in Equations (1) and (2). The problem-related processes are formulation C 1, reformulation 2 C 7 and reformulation 3 C 8. The solution related processes are synthesis C 2, analysis C 3, evaluation C 4 and reformulation 1 C 6. The process documentation C 5, is not coded using information that allows it to be placed into either category and is hence not used in the calculation of the P-S index. P-S indexes with a single value facilitate comparisons across multiple sessions and across sessions involving different situations. P-S index (cognitive issues) = P-S index (syntactic cognitive processes) =!"#$%&'!!"#$%"&!""#$"!"#$%&"'!!"#$%"&!""#$" =!!!!!!!!"!!"!!! (1)!"#$%&'!!"#$%"&!"#$%&$'&!"#$%&&%& =!!!!!!!! (2)!"#$%&"'!!"#$%"&!"#$%&$'&!"#$%&&%&!!!!!!!!!!! When the P-S index =1 the cognitive design effort is equally divided between problem and solution. For values of P-S index < 1 more cognitive design effort is expended on the solution than the problem and for values of P-S index >1 more cognitive design effort is expended on the problem than the solution Problem-Solution discontinuity ratio The Problem-Solution discontinuity ratio (Yu, Gu, Ostwald, & Gero, 2015) represents the frequency of transitions between the problem and solution spaces in a given period. It is calculated as: Discontinuity ratio (%) =!"#$%&'&($!"#$%&!"#$%&&!"#$"%&!!"#$%& 100 (3) Higher values of this ratio indicate that interactions between the problem space and solution space occur more frequently. 3. Setting for PSS design case study The task of this design was to find improvement options at a conceptual level for an existing PSS provided by a company who develops, manufactures and delivers drilling equipment with its related services such as training, spare parts delivery and MRO (maintenance, repair and overhaul) for the construction industry. In addition, the designers were asked to represent the improvement options with nine dimensions to describe a PSS (Müller, et al., 2009). The reason why a conceptual level was set as an end point is the article s focus on conceptual design. This task was given to a group of three designers and required to be conducted within approximately one hour. Prior to the session, the designers were provided with different types of information. First, relevant stakeholders of the PSS were listed: the client of the PSS provider is a construction company who makes tunnels for roads in mountains. The client of the construction company is a governmental ministry of infrastructure. Suppliers of the construction company include construction services at the tunnel site. Second, the designers were provided with information about the PSS through materials such as brochures and web sites of the PSS provider. In addition, how the equipment and its related services work was shown at a tunnel construction site. Third, they were instructed to think aloud. The benchmark and budget of the PSS were not provided. The designers were not taught any particular methods for PSS designing in order to avoid influencing the design process. After these types of information were given, they were requested to individually prepare one or two improvement ideas and describe it/them with the nine dimensions, and an expert on PSS design commented to each designer on his improvement idea(s). The reason is to make the design session more likely to focus on conceptual design. The designers were not allowed to exchange information about the design with the others in order to prevent any co-development in advance of the design session. Three designers were graduate students from a master course majoring in mechanical engineering. Each had basic knowledge about PSS in addition to knowledge in mechanical engineering. 9

10 The language was Japanese, the mother tongue of the three designers. A poster-sized paper with postits and pens were used to describe and share discussed information. In addition, a whiteboard and pens were used for complementary communication. They were asked to and did collaborate with each other in developing improvement options together. The equipment used for both audio and video recording consisted of two video cameras with mobile microphones to provide suitable sound recording. The design session produced distinguishable ideas for improving the PSS. These were all effective solutions with respect to the information given to the designers. Thus, the given design session can be regarded as an effective design session. 4. Results - Analysis of a PSS design episode 4.1 Coding The design session was transcribed and translated into English. Then, the transcription was segmented and coded by two independent coders. The results of each coder s segmentation and coding were compared and arbitrated. When the two coders were unable to arbitrate to an agreement a third coder was consulted for a final decision. The episode eventually consisted of FBS coded 242 segments. Some segments are shown as examples in Table 3. The average of the two coders agreement with the final arbitrated coding was 83%, which is above the threshold for reliability. Table 3. A part of segmented protocol Segment number Designer Utterance Design issue 135 MK This is important but how easy is it to understand "Cost"? If it can be estimated from the beginning F 136 MK Shorter time, productivity and F 137 MK And cost. F 138 ( writes) D 139 MK Also another thing is safety, F 140 MK how can that be achieved? Be 141 MK (points to his chart) these three or so... are only "Needs," originally. Be 142 MK After that we could have what was mentioned before, that feedback is fast. Bs RK MK Yeah. Feedback is fast and so on...also I think, what I was talking about before 143 MK Skill is... low, how about that? Be 144 MK In the end... the customer should have a need to build up skills over time, so F 145 MK And then how much skills are built up on this job, and Be 146 ( writes) D 147 MK Well and feedback...those are the other two. Be 148 ( writes) D 4.2 Design Issue distribution The distribution of each design issue s occurrence from the entire episode is shown in Table 4. Bs (33.9%) and Be (27.3%) are the two highest occurring issues. The two issues together represent behaviour and account for more than 60% of the total cognitive design effort. These are followed by S (14.0%) and F (13.2%). Their differences to Be are large; S and F each are only approximately one half of Be. These are followed by D (9.9%). The P-S Issue Index for the entire design session was calculated to be 0.88, meaning that across the design session more cognitive design effort is expended on the solution than the problem but the difference between this value and an equal distribution is relatively small

11 Table 4. Issue distribution [%] and P-S Issue Index Requirement (R) 1.7 Function (F) 13.2 Expected Behaviour (Be) 27.3 Behaviour derived from Structure (Bs) 33.9 Structure (S) 14.0 Description (D) 9.9 P-S Issue Index 0.88 Figure 3 shows the moving averages chronologically across the design session of each design issue with a window of 61 segments, corresponding to a quarter of the entire session. The graph begins and ends with the 30 th and 212 th segment, respectively, as a moving average is plotted at the middle point of its window. Figure 3 shows that the cognitive design effort for the design issues vary substantially over time and provides a graphical basis for a qualitative interpretation of the results. From Figure 3 the high percentages for both Bs and Be can be seen with the transition over segments. More cognitive design effort was expended on Be after the middle of the session than at any other time. The cognitive design effort expended on Bs is more in the earlier and later parts of the design session. S is addressed more in the early and final parts, similar to Bs. F is addressed also in the early and later part, but this later part of F occurred earlier than the final part of S. number of occurrences D S Bs Be F R segments Figure 3. Moving average of cognitive design effort expended on design issues (window of 61 segments) Breaking the data into each segment, i.e., looking into more details than Figure 3, Figure 4 presents a graphical representation of the cumulative occurrence of design issues in the protocol. The graphs at the segment 242, i.e. the final points of the episode, correspond to Table 4 and show that Behaviour derived from Structure (Bs) received the highest number of efforts. The graphs shapes in Figure 4 give an intuitive understanding of transition of cognitive design effort over time. In each graph in a part with the higher slope, the issue is more frequently addressed. The design issues are different in terms of which parts of the entire design session the issues are addressed more in, as represented by the different shapes. For instance, the high effort received by Be found after the middle (as described above) of the session in Figure 3 can be seen between 100 th and 165 th segments in Figure 4. The reason of the lag between the middle and the 100 th segment lies in the different ways of measurement; an envelope containing 61 segments is used in Figure 3. In addition, increase of effort in F followed by that in S, as stated in the last sentence of the previous paragraph about Figure 3, can be seen between 160 th and 230 th segments in Figure 4. 11

12 Bs Be F S R Figure 4. Cumulative cognitive design effort expended on design issues In order to quantify the shape of each graph, a linear approximation was conducted for each design issue s cumulative effort across the session. Figure 5 shows, as an example, the result for design issue Bs. The coefficient of determination was calculated as in this case and shows a relatively high linearity. Other coefficients are shown in Table 5. The linearity of Bs, Be, and F is sufficiently high with the threshold for linearity for R 2 being Those for D and S are very close to the threshold for linearity. Only R clearly fails to meet the threshold for linearity. This means that the design issues Bs, Be, and F can be regarded as being constantly focused on during the design session Figure 5. Result of linear approximation of the cumulation of design issue Bs Table 5. Coefficients of determination from linear approximation of the transition Requirement (R) Function (F) Expected Behaviour (Be) Behaviour derived from Structure (Bs) Structure (S) Description (D) Syntactic design process distribution The distribution of each syntactic process aggregated from the entire episode is shown in Table 6. The percentage of each process is a ratio of its occurrence over those of the eight processes, with the sum of all the eight percentages being 100%. Note that Be Bs (4. Evaluation) is a bidirectional process unlike the others, which are uni-directional as indicated by à. Evaluation, referring to the process between Be and Bs, occurred with by far the highest frequency (45.5%). Since Be and Bs sit in the problem space and solution space, respectively, this shows the 12

13 high frequency of transition between these two spaces. Considering this, one could infer that evaluation is a characterizing process of this PSS design. Table 6. Syntactic process distribution [%] and P-S Process Index 1: Formulation (Fà Be) : Synthesis (Beà S) 7.9 3: Analysis (Sà Bs) : Evaluation (Be Bs) : Documentation (Sà D) 1.0 6: Reformulation 1 (Sà S) 1.0 7: Reformulation 2 (Sà Be) 5.0 8: Reformulation 3 (Sà F) 1.0 P-S Process Index 0.24 The second highest frequency is that of analysis, referring to the process from S to Bs, (25.7%). The total of the frequencies of these top two, evaluation and analysis, is 71.2% and one can say these two are dominant processes. Analysis is followed by formulation, referring to the process from F to Be, (12.9%). The top three distributions of evaluation, analysis, and formulation indicate that behaviour are the dominant design issue within the syntactic processes as well as that the behaviour is at the end point of the processes rather than the starting point. Figure 6 shows moving averages of each syntactic process with a window of 61 segments. The reason why the total number of occurrences per each window is not 61 is that these eight syntactic processes are not collectively exhaustive. For instance, the transitions from F to S occurred but are not counted as any syntactic process. number of occurrences S-F S-Be S-S S-D Be-Bs S-Bs Be-S F-Be segments Figure 6. Moving average of cognitive design effort expended on syntactic processes (window of 61 segments) The majority syntactic processes change over time, and the whole session could be divided into four phases (divided by three dotted lines in Figure 6). From the beginning to approximately the 90 th segment, the major syntactic processes are; Fà Be (1. Formulation), Beà S (2. Synthesis), Sà Bs (3. Analysis), and Be Bs (4. Evaluation). After this and up to approximately the 120 th segment, Be Bs (4. Evaluation) and Sà Bs (3. Analysis) are dominant. Then, up to 160 th segment, they are dominated by Be Bs (4. Evaluation) and Fà Be (1. Formulation). In the last phase, they are Be Bs (4. Evaluation), Fà Be (1. Formulation), Sà Bs (3. Analysis). Interestingly, Be Bs (4. Evaluation) occurred substantially throughout the session, though the second and third phases include more occurrences. Except for Be Bs (4. Evaluation), the whole session could be understood in this way: the first phase is occupied with Fà Be (1. Formulation), Beà S (2. Synthesis), and Sà Bs (3. Analysis). The second is occupied with Sà Bs (3. Analysis), the third is with Fà Be (1. Formulation), and then the fourth is with Fà Be (1. Formulation) and Sà Bs (3. Analysis). Shifting to a more microscopic view of syntactic processes occurrences, Figure 7 shows numbers of 13

14 occurrences of each syntactic process on the vertical axis at each segment in a cumulative form. The values of the graphs at segment 241 correspond to Table 6, showing e.g. Be Bs occurred with the highest number. From the shapes of the graphs the following steeper slopes are observed: Be Bs (4. Evaluation) from the 92 th to 145 th and from the 155 th to 178 th ; S à Bs (3. Analysis) from the 50 th to 75 th and from the 220 th to 240 th ; Fà Be (1. Formulation) from the 140 th to 165 th ; and Beà S (2. Synthesis) from the 45 th to 65 th. These observations are a set of the processes most frequent occurrences within shorter windows and give a different view from that in Figure 6 because of the difference in granularity. cumulative occurrences S-F S-Be S-S S-D Be-Bs S-Bs Be-S F-Be S-Be Be-Bs S-Bs F-Be Be-S S-F S-S segments Figure 7. Cumulative cognitive design effort expended on processes 4.4 Problem-Solution index series The Problem-Solution issue index for the entire session is 0.88, as is shown in Table 4. The P-S issue Indexes from session deciles is found to vary over time as shown in Figure 8. The deciles with the index greater than 1 are the first, fifth, sixth, and seventh deciles. It means that the problem space is focused on more than the solution space in those deciles. Interestingly from the graph s shape it can be seen that the sixth decile has by far the highest P-S Index. This corresponds to a window right after the middle in Figure 3, where Be has its peak and F is also discussed substantially. In addition, it coincides with the third phase discussed with Figure 6, where Fà Be and Be Bs are dominant syntactic processes. In addition, the index increases from the third to the sixth decile, while it decreases from the sixth to the eighth decile. It means the space addressed shifts from the solution to the problem towards the six decile and then shifts back to the solution Figure 8. P-S Index in deciles over the design session Continuing to investigate the Problem-Solution space transition, Figure 9 shows the transition with 14

15 the finest granularity. I.e., this figure shows the space of every segment over time if it is in the problem or the solution. Comparing Figure 8 and Figure 9, it is evident that the blue, lighter colour (problem) is dominant in Figure 9 when the P-S index is high in Figure 8. Note that the number of segments in Figure 9 is not 242, as segments of Documentation belong to neither solution nor problem. In qualitative terms, some patterns can be observed in Figure 9. Apparently, a number of chunks of several segments remaining in the same space (problem or solution) can be identified. This kind of chunk is often followed by a smaller set of segments (often between one and three segments) in the other space. In addition, this combination is repeated several times. This means that one of the spaces is dominant in a given period and this dominant space switches over time, which corresponds to the observation given with Figure 8. In sum, new observation enabled by this figure is the repetition of the combination of bigger and smaller sets in the same order. Solution Problem Figure 9. Movement between Problem and Solution Spaces 4.5 Problem-Solution discontinuity ratios Using Eq. (3), the discontinuity ratios between the design problem and solution spaces were calculated. The discontinuity ratio given as a percentage from P to S is 49/242 = 20.25%, and from S to P is 50/242 = 20.66%. Figure 10 shows the moving averages of discontinuity ratios (on both directions from P to S and from S to P) with a window of 61 segments. In the beginning, the ratios stay between 30 and 50%, but in the later part they are beyond 50% for a while before going down below 50% in the end. This means the transitions between P and S occurred more often in the middle of the session. However, the ratios are primarily between 35 and 55% throughout the entire session. discontinuity ratio (%) segments Figure 10. Moving average of Problem-Solution discontinuity ratios (both from P to S and from S to P) (window of 61 segments) 5. Discussion and conclusion 5.1 Examining hypotheses This section examines the four hypotheses using the results presented above about this PSS design episode. 15

16 H1. In conceptual design of PSS, the behaviour of the design is the dominant design issue. Distribution of the design issues (see Figure 1) is used to examine H1. The ratio (%) of Be and Bs in total is calculated based on Table 4 as: Be + Bs = = 61.2 This means behaviour was addressed for 61.2% of all the design issues. In addition, the high linearity of the cumulative occurrence of Bs (with an R 2 = in Figure 5) and that of Be (with an R 2 = in Table 5) indicate that the behaviour is discussed constantly during the entire process. Therefore, H1 is supported. H2. Conceptual design of PSS addresses purposes and expectations more than solutions. The Problem-Solution issue index in the FBS scheme is used to examine H2. As explained in Section 2.2.4, where this index is greater than 1, the problem space corresponding to purposes and expectation is more addressed than the solution space. According to the index from the entire episode, which is 0.88 as shown in Table 4, H2 is not supported for the entire episode. However, looking at Figure 8, at four of the ten deciles of the episode, the P-S issue Index exceeds 1. Therefore, H2 is only supported for some parts of this PSS design episode. H3. In conceptual design of PSS, analysis and evaluation are the dominant processes. Distributions of the syntactic processes of the FBS scheme, which include analysis and evaluation (see Figure 1) are used to examine H3. The distributions of analysis and evaluation from the entire episode were calculated as 25.7% and 45.5%, respectively (see Table 6), i.e., about 70% for both. In addition, these two processes occur nearly constantly throughout the session (see Figure 6). With these, H3 is supported. H4. Conceptual design of PSS is conducted in a coevolution process between the problem and solution spaces. The Problem-Solution discontinuity ratio (3) is used for examining H4. As defined in Section 2.2.5, this represents a probability that an utterance changes the spaces to the next utterance. The Problem-Solution discontinuity ratios from the entire episode were 20.25% and 20.66% for the transition from P to S and that from S to P, respectively, as shown in Section 4.6. This means that about 40% of the utterances on average shift the space to the other space, i.e., the focus shuttles between the problem and solution spaces approximately about every 2.5 utterances. As Figure 10 shows, the moving averages of the ratios were mostly around 35-55% throughout the entire session. These mean that the transition between P and S occur throughout the entire session. With this, this PSS design episode is presumed to be a coevolution process between the problem and solution spaces. In addition, the distribution on evaluation from the entire episode is calculated as 45.5% (see Table 6). Because evaluation is defined as a process shifting between Be in the problem space and Bs in the solution space, its high distribution also supports the coevolution process. With these, H4 is supported. 5.2 Implication of the examination results The analysis showed that more than 60% of cognitive design effort was expended on behaviour. This is in line with earlier research (Sakao & Mizuyama, 2014; Sakao, et al., 2011) stating that lifecycle activity is a central notion addressed in PSS design. In addition, other earlier research states positive effects of addressing issues related to value; spending more time on value proposition (Shimomura, et al., 2015) and receiving value-related information via a CAD tool (Bertoni, 2013) had positive influences on the design processes. This might be related to the result in this session; high cognitive design effort spent on purposes and expectations, which are closely linked to value, of a design object during several parts (see H2 examined in Section 5.1). These indicate that the results from the case study presented in this article do not conflict with the existing PSS design literature but rather present empirically grounded and more detailed, quantitative 16