Sources of Inspiration? Making Sense of Scientific References in Patents.

Transcription

1 Sources of Inspiration? Making Sense of Scientific References in Patents. Julie Callaert, * Maikel Pellens ** and Bart Van Looy *** * Julie.Callaert@econ.kuleuven.be (corresponding author) ECOOM, KU Leuven, Waaistraat 6, 3 Leuven (Belgium) ** Maikel.Pellens@econ.kuleuven.be; MSI, Faculty of Business and Economics, KU Leuven, Naamsestraat 69, 3 Leuven (Belgium) *** Bart.Vanlooy@econ.kuleuven.be ECOOM, KU Leuven, Waaistraat 6, 3 Leuven (Belgium) and MSI, Faculty of Business and Economics, KU Leuven, Naamsestraat 69, 3 Leuven (Belgium) Abstract Scientific references in patent documents can be used as indicators reflecting science-technology interaction. Whether they reflect a direct knowledge flow from science to technology is subject of debate. Based on interviews with inventors at firms and knowledge-generating institutes, we analyze the extent to which scientific references in patents reflect sources of inspiration. Our results indicate that science acts as a source of inspiration for about 5% of the inventions. At the same time, the scientific references in patents do not provide the full picture: 3% of patents that were inspired by science do not contain scientific references. Moreover, 2% of cited scientific references are evaluated as unimportant by the inventor. Differences are uncovered between KGI and firm inventors. The former rely more heavily on science as a source of inspiration and attribute more importance to the cited scientific references, implying that the validity of NPRbased indicators for capturing scientific sources of inspiration may be more straightforward for KGI patents than for firm patents. Overall, the findings urge for care with interpreting scientific references as representing direct knowledge flows between scientific and technological activity realms. Introduction Previous work has highlighted the importance of basic science for economic growth (Jaffe, 1989; Adams, 199) and the positive externalities that can arise from academic research towards private R&D activities (Acs & al., 1992; Mansfield, 1995; Cohen & al., 22). Interactions between firms and scientific actors manifest themselves in multiple ways, including universityindustry collaboration and contracting (e.g. Zucker & al., 22), industry financing of university research (OECD, 26), university spin-offs and licensing activity (Thursby & al., 21), and mobility of university researchers (Kim & al., 25). Science-technology interactions have been studied through indicators based on large datasets of patents, publications, and the implied references: patenting by knowledge generating institutes (KGIs: universities and research centers), scientific publishing by firms, co-application and co-authorship between firms and knowledge-generating institutes of respectively patents and scientific publications, and the citation of scientific literature in patent documents (or vice versa).

2 This paper focuses on the interpretation of scientific non-patent references (SNPRs), i.e. references to the scientific literature, cited as prior art in patent documents. These are often conceptualized as a reflection of knowledge flows between scientific and technological actors. The number of citations to academic literature in industrial patents in the United States has grown threefold around the mid-nineties (Narin & al., 1997; see also Van Looy & al., 27), which is held indicative of growing industry-science interactions over time (Narin & al., 1997; Hicks & al., 21). This, along with the increased recognition of the importance of these interactions for innovation, has inspired further development and application of NPR-based indicators. Examples include the development of science-technology concordance schemes (Tijssen & al., 2; Verbeek & al., 22) which allow for analyzing science-technology relations over space and time. 1 NPR-based indicators also allow to assess the relation between science-technology interactions and innovative performance, at the level of patents (Cassiman & al., 28; Fleming & Sorensen, 24; Harhoff & al., 1999), firms (Cassiman & al., 21; Subramanian & Soh, 21) and countries (Van Looy & al., 23, 27). Although NPR-based indicators have been used increasingly over the last decades, extending on some caveats seems appropriate. Apart from technical issues regarding the large scale identification of cited scientific sources, there is ongoing debate about what is actually reflected in scientific NPRs and whether their use for the measurement of science-technology interactions is valid. In this regard, some insights from studies about patent-to-patent citations can be adopted. Trajtenberg (199) states that patent-to-patent citations reflect spillovers and pathways of innovative trajectories, where downstream patents cite those upstream patents on which they build. He concludes that citing patents would bear a sort of causal relationship to the cited patent, with citations being the overt manifestation of such a link (Trajtenberg, 199: p. 185). Likewise, Jaffe et al. (1993, p. 578) argue that knowledge flows do sometimes leave a paper trail, in the form of citations in patents. Thus, a citation between two patents means that the cited patent represents a piece of previously existing knowledge upon which the citing patent builds. A survey of inventors later verified that the likelihood of actual knowledge spillovers is significantly higher if there is a citation (Jaffe & al., 2), leading to the conclusion that patent citations signal spillovers and can be used to track knowledge flows. However, Jaffe & al. (1993) present some issues concerning the validity of interpreting patent citations as knowledge spillovers. Not all spillovers are captured in citations, as the majority of research output is never patented 2 ; and not all citations represent spillovers. The authors also warn about differences in citation practice between patents and scientific literature: citations in patents are meant to serve legal purposes, and are the result of a search for and selection of prior art by the examiner, and not by the inventor (see also Tijssen & al., 2). This means that there might be citations referring to documents of which the inventor was not aware, leading to misinterpretation of patent citations as reflections of direct knowledge spillovers (see also Alcacer & Gittelman, 26; Breschi & Lissoni, 21). Indeed, much of the caveats related to the interpretation of NPR-based indicators stem from the fact that a large portion of cited prior art is added by the patent examiner and not by the applicant / inventor. The presence of examiner versus applicant given references has been part of several studies (Alcacer & al., 29; Narin & al., 1989; Sampat, 24; Tan & Roberts, 21; Tijssen & al., 2; Vanderbeke, 26). Whereas for US patents, it has been found that around 2-25% of all citations in the full texts of US patents 1 For some applications, see e.g. Guan & He (27), Lo (21), Ribeiro & al. (21) and Tijssen (2, 21). 2 By including the scientific non-patent references, this issue can at least partly be addressed. 167

3 (applicant-given) are also listed on the front page (examiner-given), this share is much lower for EP patents (less than 5%, see Vanderbeke, 26); which is primarily due to different citation and examination practices in both patent offices (Michel & Bettels, 21). A limited number of studies has focused on the role and the meaning of non-patent references in patent documents. In a small scale case study of nanotechnology patents, Meyer & Persson (1998) find that most cited articles do not reflect the original source of the patented idea, and that tacit knowledge plays a more important role in the inventive process. Scientific NPRs therefore do not represent a direct link between the citing patent and the cited article, and scientific literature plays more an indirect role as source of relevant background information. Tijssen et al. (2) note that NPR-based indicators do not include information about the nature of their contribution to the invention or the knowledge transfer involved, and acknowledge that the rationale underlying the selection of citations remains unclear. However, they assume that citations are primarily meant to indicate significant contributions of scientific research to elements of the invention (Tijssen & al., 2, p.394) and claim that citations represent genuine, direct, empirical links between research and technical inventions (Tijssen & al., 2, p. 394). Sternitzke (29) considers the roles of patent references versus scientific references. Based on his analysis of examination reports of rejected European patents, the author suggests that patent references serve as a source for qualifying novelty, whereas scientific references rather qualify the inventive step (i.e. the non-obviousness of the invention). In addition, his study reveals that scientific knowledge constitutes more than general background information that aids the technical knowledge generation process, as in many cases, it is of commercial relevance. In conclusion, the literature on science-technology links often conceptualizes scientific NPRs as reflecting knowledge flows between scientific and technological actors. In its most far-reaching interpretation, cited articles are represented as sources of inspiration on which the patented invention was based. Few studies provide an in-depth examination of scientific NPRs and the extent to which these sources are captured in reference lists on patent documents. This study provides evidence on the role of scientific references as sources of inspiration through interviews with inventors and assesses the relevance of the scientific non-patent references included in the search report of the patent. Data For gaining in-depth insight into the sources of inspiration guiding inventive processes and for evaluating the extent to which these sources are covered in the reference lists available in patent documents, interviews were conducted with 36 inventors. Inventors were sampled from 3 Belgian knowledge generating institutes (KGI s: universities and research centers 2 inventors) and from 6 Belgian firms (16 inventors). To assure the relevance of science-technology linkages and related (NPR-based) indicators, institutes and firms were selected that are active in the fields of Nanotechnology, Biotechnology and Life Sciences which are generally known as scienceintensive technologies. For each institute, patents were identified (EPO or PCT patents) that were applied for in the last 1 years (between 21 and 211) and that contained one or more references to prior art. Inventors affiliated to the targeted institutes were identified on the selected patent documents and were contacted and interviewed. Each inventor was interviewed about one specific invention which resulted in a patent. The interviews, which lasted approximately one hour, were semi-structured. A first part contained open-ended questions about 168

4 the inventive process with a focus on the sources of inspiration. In a second part, questions pertained to the citations referred to in patent document. By means of rating scales, inventors were asked to rate the importance of the cited documents. In what follows, we first synthesize the inventors stories about their sources of inspiration, whereby we distinguish between inventors affiliated to KGI s and firms. We evaluate whether there is a relation between the mentioned sources of inspiration and the cited references in the search report. Second, we report on inventors assessment of the relevance of the cited references in their patent documents which allows to evaluate to what extent the reference lists, available in patents, represent an accurate picture of the sources of inspiration underlying the inventions. Results Sources of inspiration This section summarizes the sources of inspiration based on the descriptions of the inventive processes that have lead to patents, as provided by the interviewed inventors. The question about sources of inspiration was open-ended and inventors were asked to describe their sources before being shown the extracted reference lists of the patent. We relied on the following categorization: - Professional expertise: (tacit) knowledge, gained from previous activities and experience inspired the inventive process. - Scientific literature: (codified) knowledge obtained from scientific articles or databases inspired the inventive process. This category was only coded as such if a specific document was indicated by the respondent. - Patent literature: other patent documents (whether or not invented by the respondent) inspired the inventive process. This category was only coded as such if a specific patent was indicated by the respondent. - Firm contacts inspired the inventive process: whether embedded formally (e.g. contract, cooperative project, ) or informally. - Contacts with knowledge institutes inspired the inventive process: whether embedded formally (e.g. contract, cooperative project, ) or informally. Table 1. Inspiration sources in the inventive process. Professional background/ Expertise Scientific literature Firm contacts KGI contacts Patents Total Calculation unit: inventions (N = 36) Share, calculated on number of inventions (n=36) 8,56% 47,22% 16,67% 41,67% 5,% For many inventions, more than one inspiration source was pointed out (see table 1), hence the indicated total is higher than the number of inventions (N=36) covered in the interviews. The category professional expertise was mentioned for more than 8% of the inventions. This is in line with Meyer & Persson s observations (1998), where inventors pointed out that their inventions were based primarily on specific domain knowledge and experience. Also, Tijssen & 169

5 al. (2) observe that inventors mention in-house research as important or very important to arrive at the patented invention (94% of the cases analyzed). 3 Scientific knowledge was an inspiration source for approximately half of the inventions. This finding underscores the relevance of scientific input to technological development. In line with our observation, Tijssen & al. (2) found that external scientific knowledge under the form of co-operation with KGI s or outsourcing of research was mentioned as important or very important in over 4% of their analyzed cases. Table 1 shows that patent documents are mentioned less frequently than scientific ones, although contacts with firms appear to be highly relevant as well within the invention process. Our findings can be compared to those from the PatVal inventor 4 survey (see Giuri & al., 27), which covers 9,17 patents granted by the European Patent Office (EPO) between 1993 and Inventors were asked to rate the importance of several predefined sources of knowledge in the invention process on a scale from 1 to 5. Scientific literature was rated 2,5, which comes down to a 5% relevance, which is in line with our findings. Universities and PRO s (analogous to our category KGI contacts ) was scored 1,4/5, suggesting a lower attributed importance than the 5% that we observed. Regarding patent documents, PATVAL survey results indicate a score of 2,6/5. The PATVAL categories competitors and suppliers which can be considered as analogous to firm contacts reveals a score (1,9/5) similar to our observations. Discrepancies between the results from the PATVAL survey and our results may be due to several reasons. First, for patents as sources of inspiration, we scored this item only if the inventor spontaneously cited a specific patent. This is a more conservative way of verifying whether patent literature acts as a source of inspiration. Second, the PatVal survey considered all technological domains, whereas our results focus on science-intensive domains. Finally, most of the inventors of the PatVal survey are firm-affiliated (93% versus 44% in our sample). Assuming that knowledge spillovers within one s own activity realm are occurring more frequently, inventors from knowledge generating institutes may be more prone to be inspired by the scientific literature and from contacts with KGI s, whereas firm inventors may be more inspired by the technological (patent) literature and by firm contacts. Table 2 summarizes the results of the Chi-square test in this respect (sources of inspiration versus firm/kgi affiliation). It should be noted that in our sample, the nature of the professional expertise category is affiliation-dependent: scientific for KGI inventors, and of a technical nature for firm inventors. As we will be focusing further on the distinction between scientific and technical sources of inspiration, this ambiguous category is not included in the following tables. Table 2. Sources of inspiration broken down by firm and KGI inventors. Calculation unit: inventions Scientific literature Patents Firm contacts KGI contacts Total (N = 36) Firm (n = 16) 24 (7,3) (2,6) (6,4) (7,7) Share 31.25% 31.25% 43.75% 43.75% Knowledge Institute (n = 2) 32 (9,7) (3,4) (8,6) 1,3 Share 6.% 5.% 4.% 55.% Chi-2 test: insignificant (p=.14) Expected values between brackets 3 Based on a sample of 5 Dutch USPTO patents ( ). 4 Located in France, Germany, Italy, the Netherlands, Spain and the United Kingdom. 17

6 While the figures indeed suggest that inspiration sources from the inventors own activity realm are more prominent, the overall result is not significant. As such, these findings rather confirm the presence of spillovers across activity realms. Adopting NPR-based indicators to measure the science-base of technology, in its strictest sense, implies an assumption about cited NPRs representing a source of inspiration in the inventive process. If this assumption would hold, then scientific inspiration sources (scientific literature and contact with knowledge institutes) would be mentioned more frequently for patents containing NPRs. This is tested in table 3. Table 3. Sources of inspiration broken down by patents with and without scientific NPRs. Calculation unit: inventions (N = 36) Scientific literature Patents Firm contacts KGI contacts No scientific NPRs in Search Report 5 9 (n=15) (6,4) (2,3) (5,6) Share 33,33%,% 6,% 46,67% Scientific NPRs in Search Report (n=21) (1,6) (3,8) (9,4) Share 57,14% 28,57% 28,57% 52,38% Chi-2 test: significant at the 1% level (p=.6) Expected values between brackets Total 7 (6,8) (11,3) 35 The results confirm to some extent (p <,1) that inspiration sources differ between patents with scientific references versus patents without scientific references. It can be seen that scientific literature is mentioned as an inspiration source for 57% of the patents with scientific references. For patents without scientific references, this share is considerably lower (33%), suggesting that the presence of scientific references in patents indeed coincides with a higher probability of scientific literature having served as a source of inspiration. At the same time although the numbers are small patents containing scientific references appear to be more inspired by the patent literature as well. Finally, the results show that firm contacts figure more prominently as sources of inspiration for patents that contain no scientific references. Tentatively, one could argue from these results that patents containing scientific references are based more on the codified (scientific and technical 5 ) literature, whereas patents without scientific references are based more on formal and informal contacts, especially with firms. 6 Overall, our findings reveal that scientific knowledge is an important inspiration source of inspiration for inventors, working for firms and KGI s alike. At the same time, the reported shares show that around one third of patents that are inspired by the scientific literature contain no scientific references at all. Moreover, for over 4% of patents with scientific references, the inventor did not explicitly report on scientific literature having contributed to the inventive process. As such, equaling the presence of scientific references in patents with inventions inspired directly by science introduces both the presence of false positives and false negatives. 5 This appears to be additionally confirmed by a positive correlation between the number of patent references and the number of scientific references in patents. 6 An additional check reveals that the patterns observed in table 3 are similar when the sample is split between firm versus KGI patents. 171

7 Relevance of references in search report In this section, we consider the importance of the scientific references that are cited in the search reports, as evaluated by the inventors. Inventors were asked to assess the relevance of these references as sources of inspiration, by rating them on a scale from 1 (very important) to 4 (background). A category not known was added to account for any cited references that the inventor was unfamiliar with. Table 4 presents the distribution of cited scientific references in terms of their importance as perceived by the inventor. Table 4. Rated importance of cited scientific non-patent references as inspiration sources. Calculation unit: cited Very Not Not scientific references 7 Important Important Important Background known Total Share 25.% 18.75% 2.31% 34.38% 1.56% 1% A first observation is that inventors are familiar with most of the scientific references mentioned in the search report (category not known accounts for only 1,5%). In a previous study, Jaffe & al. (2) found that, when submitting the patent application, inventors knew about 7% of all references contained on the front page of US patents. The difference may at least partly be explained by differences between the US and the EPO patent systems. As Michel and Bettels (21) point out, the average USPTO search report has the characteristics of a documentary search, whereas the EPO search reports reflect patentability searches. The patentability search is not exhaustive in the same sense as the documentary search in that it should be limited to what is directly relevant to patentability. As such, the volume of references in EPO patents might be lower and more focused than the volume of references in USPTO patents (see also Callaert & al., 212), implying that the propensity that some of the listed references in USPTO patents are not known to the inventor is higher within the USPTO system. Second, it can be seen in table 4 that less than half of the references (44%) were evaluated as important or very important source of inspiration, i.e. having directly contributed to the inventive process. Most cited references are considered as background information meaning that they are not important as a source of inspiration per se. The share of scientific references that are rated by inventors as not important, amounts to a considerable 2%. Whether or not inventors from knowledge institutes consider scientific references to be more important than inventors who are affiliated to firms is examined in Table 5. Academic inventors attribute a significantly higher importance to scientific references as sources of inspiration than firm-affiliated inventors: 67% of scientific references are evaluated as very important or important sources of inspiration, compared to 14% for firm inventors. 7 Note that the proportional distribution of patents over importance categories follows the pattern of shares of references. For the total sample, it concerns 21 patents, 9 of which are firm patents and 12 are KGI patents. 172

8 Table 5. Rated importance of cited scientific non-patent references as inspiration sources broken down by references in firm and KGI patents. Calculation unit: cited scientific references 7 Firm Very Important 3 (7) Important 1 (5,25) Not Important 1 (5,69) Background 14 (9,63) Not known Total (,44) 28 Share 1.71% 3.57% 35.71% 5.%.% Knowledge Institute (9) (6,75) (7,31) (12,38) (,56) 36 Share 36.11% 3.56% 8.33% 22.22% 2.78% Chi-2 test: significant at the 1% level (p<.1) expected values between brackets For inventors situated within firms, half of the cited scientific references are considered as relevant background information. Finally, the share of scientific references that is evaluated as not important is considerably lower among KGI inventors (8%, compared to 36% for firm inventors). The interpretational difficulties implied by these observations could be solved if there were a way to distinguish between those references that are important / relevant for the invention and those that are not. Several researchers consider examiner-assigned citation categories as proxies for the importance of references cited in patents (Akers, 1999; Criscuolo & Verspagen, 28; Schmoch, 1993; Sternitzke, 29). Citation categories that are used by examiners imply the following: X: Highly relevant documents, prejudicial to the novelty and/or inventive step of the claims, when considered in isolation. Y: Documents which preclude that a claimed invention can be considered as involving an inventive step, when they are combined with one or more other documents of the same category and when this combination was obvious to a trained person. A: Documents that define the technological state of the art, without destroying the novelty or the inventiveness. T: Documents about the theory or principal underlying the invention. Document that can permit better understanding of the principle or theory which lies at the base of the invention, or that can show that the reasoning or the facts underlying the invention are not accurate. D: Documents that were already cited in the description of the application being examined. If these categories indeed reflect the importance / relevance of cited source documents not only in the search, but also in the inventive process, then one would e.g. expect X references to represent more important sources than Y references, and A references would be considered as references denoting background knowledge rather than sources having directly led to the development of the patented invention. This assumption is tested in table

9 Calculation unit: cited scientific references Citation category X Table 6. Rated importance of cited scientific non-patent references as inspiration sources by citation category. Very Important 8 (7,34) Important 4 (5,51) Not Important 5 (5,51) Background 12 (1,28) Not known Total (,37) 29 Share 27.59% 13.79% 17.24% 41.38%.% Citation category A (7,59) (5,7) (5,7) (1,63) (,38) 3 Share 16.67% 26.67% 26.67% 26.67% 3.33% Citation category D (3,54) (2,66) (2,66) (4,96) (,18) 14 Share 28.57% 21.43% 7.14% 42.86%.% Citation category Y (1,27) (,95) (,95) (1,77) (,6) 5 Share 4.%.% 2.% 4.%.% 1 Citation category T (,25) (,19) (,19) (,35) (,1) 1 Share 1.%.%.%.%.% Chi-2 test for citation category: no relation (p=.87) expected values between brackets The results reported in table 6 do not support the validity of citation categories to denote the importance of scientific references as evaluated by the inventor. To summarize, the findings in this section suggest that researchers should be careful with interpreting scientific references in search reports as delineating the relevant sources of inspiration of the inventive process. Although almost half of them are indeed important or very important, a major share captures relevant background information that did however not contribute directly to the inventive process. A considerable 2% consists of references that are evaluated as not important, not even as background information. In addition, these observations suggest that the information about citation category made available in patent databases does not coincide with references that are evaluated as important by the inventor. Discussion and intermediate conclusions Varying interpretations regarding indicators based on (scientific) non-patent references (SNPRs) in patent documents have been put forward to denote science-industry interactions, ranging from scientific knowledge as inspiration sources to more conservative interpretations suggesting relatedness between the cited science and the citing technology. Within this contribution, we examined whether scientific non-patent references present in patent documents, reflect a source of inspiration from the viewpoint of the inventor. In a first part, we consider the inventors point of view on their sources of inspiration within the inventive trajectory. More than half of the inventors mention scientific literature or contacts with KGI s as having directly contributed to the inventive process. Technical literature (patents) inspired 17% of the inventions covered. The composition of inspiration sources, as outlined by the inventors, appears to be disconnected from the types of references that examiners introduce 174

10 during their search process (i.e. what ends up in the search reports and the patent databases), as examiners have been shown to tend more towards citing patent literature rather than scientific literature (Narin & Noma, 1985). When broken down by inventor affiliation, our findings reveal that KGI inventors are more likely than firm inventors to mention the scientific literature as an inspiration source; the opposite holds for inventors within firms who cite other patents more often as such. When considering whether the presence of scientific references in patent documents is indicative of science as a source of inspiration, we found that patents with scientific references are indeed more likely to have been inspired by the scientific literature (but also by the technical literature) than patents without scientific references. At the same time, using the presence of scientific references in patents for identifying science-based patents implies distortions in two ways. Around one third of patents that are reportedly inspired by the scientific literature contain no scientific references to reflect this (false negatives). Moreover, for over 4% of patents with scientific references, the inventor did not explicitly report on scientific literature having contributed to the inventive process (false positives). When analyzing the cited scientific references and their contribution towards the invention, most of the SNPRs present in the search report are indeed relevant sources: either as source of inspiration or as background material. At the same time, a considerable share of cited scientific references (2%) was evaluated by the inventor as not important. The uncovered differences between KGI and firm inventors (with the former relying more heavily on scientific inspiration sources and attributing higher importance to the cited scientific references), suggest that the validity of NPR-based indicators for capturing scientific sources of inspiration may be more straightforward for KGI patents than for firm patents. Finally, our findings reveal that the citation categories provided in patent systems do not provide a reliable indication to distinguish between important and unimportant references defined in terms of inspiration as assessed by inventors. Within the discussion about the exact meaning of scientific references in patents, our findings provide empirical support to the argument that these references should not be interpreted as signaling direct links between or flows from science to technology (see also Meyer, 2; Tijssen, 21). At the same time, we find that 8% of scientific references are considered relevant by the inventors, at least as background to the patented invention, which implies opportunities for analyses of patterns of relatedness between science and technology, e.g. on the level of fields. References Acs, Z., Audretsch, D. & Feldman, M. (1992). Real Effects of Academic Research: Comment. The American Economic Review, 82 (19), Adams, D. (199). Fundamental Stocks of Knowledge and Productivity Growth. Journal of Political Economy, 98 (4), Akers, N. (1999). The European Patent System: An Introduction for Patent Searchers. World Patent Information, 21 (3),

11 Alcacer, J. & Gittleman, M. (26). How Do I know What you Know? Patent Examiners and the Generation of Patent Citations. Review of Economics and Statistics, 88, p Alcácer, J., Gittelman, M. & Sampat, B. (29). Applicant and Examiner Citations in U.S. Patents: An Overview and Analysis. Research Policy, 38 (2), Breschi, S. & Lissoni, F. (21). Knowledge Spillovers and Local Innovation Systems: A Critical Survey. Industrial and Corporate Change, 1 (4), Callaert, J., Grouwels, J. & Van Looy, B. (212). Delineating the Scientific Footprint in Technology: Identifying Science within Non-Patent References. Scientometrics, In Press. Carpenter, M.P. & Narin, F. (1983). Validation Study: Patent Citations as Indicators of Science and Foreign Dependence. World Patent Information, 5 (3), Cassiman, B., Veugelers, R. & Zuniga, M. (28). In Search of Performance Effects of (in)direct Industry Science Links. Industry and Corporate Change, 17 (4), Cassiman, B., Veugelers, R. & Zuniga, M. (21). Diversity of Science Linkages: A Survey of Innovation Performance Effects and Some Evidence from Flemish Firms, Economics, 4 (33), Cohen, W., Goto, A., Nagata, A., Nelson, R. & Walsh, J. (22). R&D Spillovers, Patents and the Incentives to Innovate in Japan and the United States. Research Policy, 31 (8-9), Criscuolo, P. & Verspagen, B. (28). Does it Matter where Patent Citations come From? Inventor vs. examiner citations in European patents. Research Policy, 37 (1), Fleming, L. & Sorenson, O. (24). Science as a Map in Technological Search. Strategic Management Journal, 25, p Giuri, P., Mariani, M., Brusoni, S., Crespi, G., Francoz, D., Gambardella, A., Garcia-Fontes, W., Geuna, A., Gonzales, R., Harhoff, D., Hoisl, K., Le Bas, C., Luzzi, A., Magazzini, L., Nesta, L., Nomaler, Ö., Palomeras, N., Romanelli, M. & Verspagen, B. (27). Inventors and Invention Processes in Europe: Results from the PatVal-EU Survey. Research Policy, 36 (8), Harhoff, D., Narin, F., Scherer, M. & Vopel, K. (1999). Citation Frequency and the Value of Patented Inventions. Review of Economics and Statistics, 81 (3), Hicks, D., Breitzman, T., Olivastro, D. & Hamilton, K. (21). The Changing Composition of Innovative Activity in the US A Portrait based on Patent Analysis. Research Policy, 3 (4), Jaffe, A. (1989). Real Effects of Academic Research. The American Economic Review, 79 (5), Jaffe, A.B., Trajtenberg, M. & Fogarty, M.S. (2). The Meaning of Patent Citations: Report on the NBER/Case-Western Reserve Survey of Patentees. NBER Working Paper No Jaffe, A., Trajtenberg, M. & Henderson, R. (1993). Geographic Localization of Knowledge Spillovers as Evidenced by Patent Citations. The Quarterly Journal of Economics, 18 (3), Kim, J., John Lee, S. & Marschke, G. (25). The Influence of University Research on Industrial Innovation. NBER Working Paper no Mansfield, E. (1995). Academic Research Underlying Industrial Innovations: Sources, Characteristics, and Financing. The Review of Economics and Statistics, 77 (1), Meyer, M. (2). Patent Citations in a Novel Field of Technology: What Can they Tell about Interactions between Emerging Communities of Science and Technology. Scientometrics, 48 (2), Meyer, M. & Persson, O. (1998). Nanotechnology-Interdisciplinarity, Patterns of Collaboration and Differences in Application. Scientometrics, 42 (2),

12 Michel, J. & Bettels, B. (21). Patent Citation Analysis: A Closer Look at the Basic Input Data from Patent Search Reports. Scientometrics, 51 (1), Narin F., Hamilton, K. & Olivastro, D. (1997). The Increasing Linkage between U.S. Technology and Public Science. Research Policy, 26 (3), Narin, F. & Noma, E. (1985). Is Technology Becoming Science? Scientometrics, 7 (3-6), Narin, F., Rosen, M. & Olivastro, D. (1989). Patent Citation Analysis: New Validation Studies and Linkage Statistics. In Van Raan, A.F.J. & al. (Eds.), Science and Technology Indicators: Their Use in Science Policy and their Role in Science Studies. Leiden: DSWO Press. OECD. (26) OECD Science, Technology, and Industry Outlook. Paris: OECD Publishing. Sampat, B.N. (24). Examining Patent Examination: An Analysis of Examiner and Applicant Generated Prior Art. Working Paper Georgia Institute of Technology. Schmoch, U. (1993). Tracing the Knowledge Transfer from Science to Technology as Reflected in Patent Indicators. Scientometric, 26 (1), Sternitzke, C. (29). Patents and Publications as Sources of Novel and Inventive Knowledge. Scientometrics, 79 (3), Subramanian, A.M. & Soh, P. (21). An Empirical Examination of the Science Technology Relationship in the Biotechnology Industry. Journal of Engineering and Technology Management, 27 (3-4), Tan, D. & Roberts, P.W. (21). Categorical Coherence, Classification Volatility and Examiner- Added Citations. Research Policy, 39 (1), Thursby, J., Jensen, R. & Thursby, M. (21). Objectives, Characteristics and Outcomes of University Licensing: A Survey of Major U.S. Universities. The Journal of Technology Transfer, 26, p Tijssen, R. (21). Global and Domestic Utilization of Industrial Relevant Science: Patent Citation Analysis of Science-Technology Interactions and Knowledge Flows. Research Policy, 3 (1), Tijssen, R.J.W., Buter, R.K., & Van Leeuwen, T.N. (2). Technological Relevance of Science: Validation and Analysis of Citation Linkages between Patents and Research Papers. Scientometrics, 47 (2), Trajtenberg, M. (199). A Penny for your Quotes: Patent Citations and the Value of Innovations. The Rand Journal of Economics, 21 (1), Van Looy, B., Magerman, T. & Debackere, K. (27). Developing Technology in the Vicinity of Science: An Examination of the Relationship between Science Intensity (of patents) and Technological Productivity within the Field of Biotechnology. Scientometrics, 7 (2), Van Looy, B., Zimmermann, E., Veugelers, R., Mello, J. & Debackere, K. (23). Do Science- Technology Interactions Pay Off? An Exploratory Investigation of 1 Science Intensive Fields. Scientometrics, 57 (3), Vanderbeke, S. (26). Rol en betekenis van non-patent references : Indicator van science-technology interacties? Eindverhandeling, KULeuven, Faculteit Economie en Bedrijfswetenschappen. Verbeek, A., Debackere, K., Luwel, M., Andries, P., Zimmermann, E., & Deleus, F. (22). Linking Science to Technology: Using Bibliographic References in Patents to Build Linkage Schemes. Scientometrics, 54 (3), Zucker, L., Darby, M. & Armstrong, J. (22). Commercializing Knowledge: University Science, Knowledge Capture, and Firm Performance in Biotechnology. Management Science, 48 (1),