Science and technology interactions discovered with a new topographic map-based visualization tool

Transcription

1 Science and technology interactions discovered with a new topographic map-based visualization tool Filip Deleus, Marc M. Van Hulle Laboratorium voor Neuro-en Psychofysiologie Katholieke Universiteit Leuven Campus Gasthuisberg Herestraat 49 B-3000 Leuven, BELGIUM Tel.: Fax.: {filip.deleus, marc.vanhulle}@med.kuleuven.ac.be Abstract A new tool for discovering and visualizing interactions between scientific publication domains and industrial patent domains is introduced. The tool is applied to a database of the United States patent data (USPTO) from 1980 till 1995 and the Science Citation Index (SCI) bibliographic databases. The core algorithm behind our tool is the kernel-based Maximum Entropy Rule (kmer), a learning scheme for developing topographic maps of formal neurons. The Gaussian kernels that correspond to each neuron are used for constructing a density map on which a hillclimbing algorithm is applied for locating and visualizing clusters in the topographic map. For each set of clustered data, a new topographic map is developed, and so on. A new method is introduced in order to visualize this hierarchical structure into a single condensed cluster map. The procedure is applied to patent and bibliographic data separately in order to build two hierarchical structures: one containing clusters of technology classes and the other clusters of science classes. The two data structures are then merged into a single (pseudo-)colored linkage map, colored by the co-occurrences of patents and the scientific publications referred in them. Various zooming facilities are available for visual inspection. After the linkage map for a particular time period is developed, data from other consecutive time periods can be projected onto the map in order to monitor the evolution of the interactions. Finally, in order to quantify the dynamics of the interaction, structural equation modeling, a causal modeling technique, is applied to the most significant co-occurrences in the linkage map. 1 Introduction We introduce a new tool for discovering and visualizing interactions between scientific publication domains and industrial patent domains. These interactions are an invaluable and objective basis for policy makers to decide which scientific research fields qualify for governmental support [8]. However, although a lot of sciencetechnology interaction studies have been done in the last decades [2, 3, 6, 7, 9, 5], all of them focus on particular technology classes or on particular science classes. Our goal is to capture the overall landscape of the S&T relations. As a case study, we consider the US Patents database (USPTO) and the Science Citation Index (SCI) bibliographic database, and focus on the bibliographic citations in patents. The tool starts with a clustering of patents into technology domains, based on their bibliographic referencing behaviour, and a clustering

2 of the scientific articles into science domains, based on the technology domains that refer to them. The core algorithm behind the clustering is a hierarchical density-based clustering algorithm that relies on topographic maps, which have recently attracted the attention of the data mining community. The advantage of our clustering procedure is that clusters can be detected autonomously, without prior knowledge of their number. The clustering results are visualized graphically ( cluster maps ) as well as the interactions between scientific publication domains and industrial patent domains ( linkage maps ). Finally, since the database covers a period of 20 years, structural equation modeling is used in order to examine the temporal dynamics of these interactions which are accordingly plotted on the maps. The clustering analyses are performed using the Synes Data Prospector TM Suite [11] which implements our topographic map-based clustering algorithm. 2 Data Description 2.1 Databases For our analysis, we use two types of databases. First, the US Patents database (USPTO) offers the most complete bibliographical information about US patents, including abstracts and the full texts of all claims and all citations to other patents and to the science and technology literature. The patents are indexed using IPC-codes (International Patent Classification). Currently, 644 different IPC-codes are defined on a 4 digit level, which will be called technology classes in the remainder of this paper. The patent data is available from 1976 until The second database we are using is the Science Citation Index (SCI) database. The SCI database covers over 3500 of the world s leading scholarly journals, divided into 182 science fields, which from now on will be called science classes. 2.2 Data preprocessing The interaction between science and technology is studied by examining the bibliographic references in the patents, i.e., the so-called nonpatent references (NPR). The latter are a mixture of references to journal papers, meetings, books and many non-scientific sources such as disclosures, manuals and standards. For our purpose, only the references to journal papers are relevant. A specific parsing algorithm [13] was implemented in order to extract these references and to match them to the journal papers listed in the SCI-database. By recording for each patent the technology classes to which it belongs, together with the science classes to which the patent refers, a 644 rows by 182 columns linkage matrix was built. This linkage matrix is in fact a histogram of the number of citations from specific science classes, used in particular technology classes. This matrix can be built for a particular time period, for all available data, or for several consecutive years in case one is interested in the evolution of the linkages. 3 Clustering methodology and visualization 3.1 Theoretical Foundation Our clustering methodology is based on kernelbased topographic maps which are developed using the kernel-based Maximum Entropy Rule (kmer) as described in [12]. A kernel based topographic map is a lattice of neurons which correspond to radially-symmetric Gaussian kernels in data space. The kernels are characterised by the neurons weight vectors and by the neurons radii, which is in fact a scalar associated to each neuron. When data vectors are presented to the network, the weights are adapted so as to produce a topology-preserving mapping. The radii are adapted such that all the neurons have an equal probability to be active (equiprobabilistic map). From this equiprobabilistic map, a density map is constructed using the Gaussian kernels and a Hill climbing strategy searches for high density peaks in this map so as to define clusters and their extent. After the assignment of data points to the different clusters, this procedure is repeated in a hierarchical way until a stopping criterion is met to end up with a clus-

3 tering tree. A stopping criterion that is based on the overlap between the density estimates of the individual subclusters is described in [1]. 3.2 Defining technology and science domains We apply this procedure on the patent data from 1992 till 1995 using 7x7 neural maps with as inputs 182-dimensional feature vectors consisting of patents and the scientific publications referred in them. This leads to a clustering tree, shown in Figure 1, having 24 leaf nodes, which we call from now on technology domains. Next, we use the algorithm for hierarchically structuring the science classes. As inputs we use 24- dimensional feature vectors of co-occurrences between patents and scientific publications of given technology domains and a given science class respectively, which results in a tree having 12 leaf nodes, each one representing a new science domain. Figure 1: Clustering tree defining new technology domains, the leaf nodes are colored by traversing the tree in a depth first manner as described in section Condensed visualization of hierarchical clustering results We develop a new technique with which the clustering maps, contained in the clustering tree, can be visualized in terms of a single, condensed (2D-) cluster map. We replace the top cluster map in our clustering tree by a new, enlarged map in order to accommodate the positions of the neurons contained in the leaf maps. The weight vectors corresponding to the (newly found) positions in the map are obtained by linear interpolation. This is done in the following way. First, we traverse the tree in a depth first manner. The leaf clusters are provided with a numeric label and a unique color in our cluster map by using the label in an indexed RGB table. Since to each position in the new cluster map corresponds a position in input space (like a weight vector), we can label and color the map according to the label and color of the nearest weight vector in the original leaf maps. Due to the topographic properties of the original maps in the clustering tree, and due to the depth first labeling strategy, we end up with a new 2D-cluster map in which positions belonging to nodes of the same branch in the tree have more similar colors than those belonging to nodes of different branches. Finally, the data points can be represented in this new cluster map by looking for the position with the nearest weight vector. The new cluster map can be shown with all leaf clusters projected or with a selection of the leaf clusters, e.g., up to a given level in the hierarchy. Furthermore, the method can also be applied on a particular branch of the underlying tree, in which case the top node of the selected branch is treated as the root node for interpolation. The procedure is illustrated for the clus-

4 tering of technology classes, Figure 1 shows the clustering tree, Figure 2 shows the leaf clusters projected on the top map after interpolation of the 7x7-map to a 187x187 map. Figure 2: Leaf clusters of the tree shown in Figure 1 projected on the top node. 4 Linkage maps Interesting exploratory facilities arise when the technology maps and the science maps are linked. This allows a user to select a region on one map, and then discover the linked regions in the other map. For example, consider technology domain 4. It consists of several technology classes, as shown on the corresponding cluster in the cluster map shown in the left panel of Figure 3. Next, we have found, for technology domain 4, a number of science classes to which it is linked with varying strengths, as shown by the grayscaling of the labels in the right panel of Figure 3: a white corresponds to a strong link; a black label to a weak link. Furthermore, the zooming facilities previously described remain active, hence, the user is able to explore neighbouring regions in one map and, at the same time, inspect their links with the other map. This interactive exploration of the links between the two maps can be summarized into a single colored linkage map, colored by the co-occurrences of patents and scientific publications. This is illustrated in the centre map of Figure 4, in which each row represents a technology domain and each column stands for a science domain. In fact this figure captures the whole linkage structure for a particular time period. Furthermore, a row, a column or a single cell in the linkage map can be selected for further analysis, as also shown in Figure 4. The evolution in linkage between technology domains and science domains can be visualized by projecting data from consecutive time periods onto the previously defined linkage map. As an example, in Figure 5 the patent and publication co-occurrences for the periods 1989, 1990, 1991 and 1992 respectively, are displayed on the linkage map defined for the period It is clear that the concept of linkage maps as a visualization tool for interactions between various cluster maps may not only be applied in the context of science-technology studies as in this paper, but may be applied in any case in which data can be handled along different points of view. Moreover, due to the hierarchical nature of the underlying clustering, these linkages can be represented on a rather high level of detail and afterwards be focussed for further exploration of interesting regions, which guarantees the scalability of the linkage maps also to problems in which there are in the order of a few thousand classes.

5 Figure 3: Map of technology domains(left panel) and of science domains(right panel). Example of the linkage between science and technology: the activation of a technology domain (white labeled items in the left panel) causes the selection of several science domains (items labeled in grayscales in the right panel). Figure 4: Linkage maps; Middle panel: total linkage map, rows represent technology domains, colums represent science domains; Left panel: detailed linkage map for technology domain 4, rows represent technology classes of this technology domain, columns represent science domains; Right panel: detailed linkage map for technology domain 4 and science domain 13, rows and colums represent technology classes and science classes of the respective domains.

6 Figure 5: Evolution of a linkage scheme over time, from left to right: data from 1989 till 1992 respectively projected on the linkage map for Dynamic Linking Analysis The linkage maps, as they are described up to now, only present a general overview of the interactions between science and technology. We will now take the analysis further and present a new technique for discovering causally related linkage patterns. Techniques like these have been introduced before but they are computationally too intensive when applied to large, real-world data sets (see [10] for a review). Our new technique is based on path analysis, i.e., a statistical technique for quantifying the influence exerted by one variable on another in such a way that a network of interactions appears; the path coefficients represent the quantified influences (for a detailed description, see [4]). We perform a path analysis on the time series of the number of patents and the number of publications, that belong to a user-selected technologyand science domain, respectively. Path analysis will then quantify how strongly the time series of a particular patent cluster and a particular publication cluster are correlated, also as a function of a time delay between the two. The latter is particularly important to discover causally related links between science and technology. The interactions (i.e. path coefficients) are visualized per time delay. The set-up considered for our case study is shown in the network depicted in Figure 6. The independent variables are the number of publications in the classes of a selected science domain, for different time delays, while the number of patents in the classes of a selected technology domain are the dependent variables. In a similar way, we could quantify the interactions between different science and technology domains (clusters) as a whole. As an example, we take the time series of the publications and the patents of science domain 13 and technology domain 4 respectively (Figure 7), since these are potentially interesting ones (see the corresponding cell in the linkage map).the results are summarized in Figure 8. It is now easy to determine which science and technology classes co-evolve or in which science classes changes are followed by similar or opposite changes in particular technology classes. For example technology class B23K seems to be co-evolving with science class S370 (case 1 in Figure 8). Technology class H01J follows changes in science class S370 after one year (case 2), while this technology class seems to react negatively on changes in science class S368 (case 3: increases in S368 are followed by decreases in H01J in the next year and vice versa).

7 Figure 6: Network modeling of influences between science and technology. The first subscript denotes the index of the science or technology domain; the second subscript denotes time. Figure 7: Time series of the number of publications in the classes of science domain 13 (left panel) and of the number of patents in the classes of technology domain 4 (right panel)

8 Figure 8: Visual inspection of path coefficients

9 6 Conclusions We have introduced a new tool for discovering interactions between patents and the scientific publications referred in them. As a case study, we have considered the USPTO patent database, and the SCI bibliographic database. We have introduced a number of new concepts and graphical tools in this field, such as clusterand linkage maps, both of which are based on topographic maps. The cluster map visualizes the result of a hierarchical clustering analysis, and the linkage map the interactions between various cluster maps. It is clear that these tools can be applied to other types of data mining problems analyzed with topographic maps. Furthermore, not only static interactions were discovered but also dynamical ones, by modeling the causal interactions with the structural equation modeling technique. These interactions are important indicators for policy makers to decide which scientific fields should be supported by their expected impact on industrial research developments. In the future, we intend to apply our tool to the European patent database EPO. 7 Acknowledgements F.D. is supported by a scholarship from the Flemish Ministry for Science and Technology (VIS/98/012). M.M.V.H. is supported by research grants received from the Fund for Scientific Research (G N), the National Lottery (Belgium) ( ), the Flemish Regional Ministry of Education (Belgium) (GOA 95/99-06; 2000/11), the Flemish Ministry for Science and Technology (VIS/98/012), and the European Commission, 5th framework programme (QLG3-CT ). References [1] Deleus, F., De Maziere, P., Gautama, T., Van Hulle, M. (2001). A hierarchical density-based clustering analysis of auditory fmri activation data, IEEE Neural Network for Signal Processing Workshop 2001, Massachussets. [2] Freeman, C. (1982). The Economics of Industrial Innovation, Pinter, London. [3] Griliches, Z. (1990). Patent Statistics as Economic Indicators: A Survey, Journal of Economic Literature, 28, [4] Loehlin, J. C. (1992). Latent variable models. Lawrence Erlbaum Associates. [5] Meyer, M. (2000). Does science push technology? Patents citing scientific literature, Research Policy, 29 (3), [6] Narin, F., Noma, E. (1985). Is technology becoming science?, Scientometrics, 7 (3-6), [7] Narin, F., Olivastro, D. (1992). Status Report: Linkage Between Technology and Science, Research Policy, 21, [8] Narin, F., Hamilton, K., Olivastro, D. (1997). The increasing linkage between U.S. technology and public science, Research Policy, 26 (3), [9] Narin, F., Olivastro, D. (1998). Linkage Between Patents and Papers: An Interim EPO/US Comparison, Scientometrics, 41, [10] Silverstein, C., Brin, S., Motwani, R., Ullman, J. (2000). Scalable Techniques for Mining Causal Structures, Data Mining and Knowledge Discovery, 4, [11] Synes, Next Generation Data Mining, [12] Van Hulle, M. M. (2000). Faithful Representations and Topographic Maps. John Wiley & Sons. [13] Verbeek, A., Debackere, K., Luwel, M., Van Looy, B., Andries, P., Van Hulle, M., Deleus, F. (2001). Linking science to technology using bibliographic references in patents to build linkage schemes, Proceedings of the 8th International conference on scientometrics & infometrics, Sydney.