A Patent Quality Classification System Using a Kernel-PCA with SVM

Transcription

1 ADVCOMP 05 : The nth Internatonal Conference on Advanced Engneerng Computng and Applcatons n Scences A Patent Qualty Classfcaton System Usng a Kernel-PCA wth SVM Pe-Chann Chang Innovaton Center for Bg Data & Dgtal Convergence and Dept. of Informaton Yuan Ze Unversty Taoyuan Tawan epchang@saturn.yzu.edu.tw Jheng-Long Wu Innovaton Center for Bg Data & Dgtal Convergence and Dept. of Informaton Yuan Ze Unversty Taoyuan Tawan vceytsao@gmal.com Cheng-Chn Tsao Innovaton Center for Bg Data & Dgtal Convergence and Dept. of Informaton Yuan Ze Unversty Taoyuan Tawan vceytsao@gmal.com Meng-Hsuan Ln Innovaton Center for Bg Data & Dgtal Convergence and Dept. of Informaton Yuan Ze Unversty Taoyuan Tawan syuan47@gmal.com Abstract Data mnng (DM) approaches such as clusterng and classfcaton are employed n ths paper to dentfy and classfy the patent qualty. We develop an effectve and automatc patent qualty classfcaton system. Frst, the Selforganzng map (SOM) s used to cluster patents automatcally nto dfferent qualty groups wth patent qualty ndcators nstead of va expert dentfcaton. Then, the Kernel prncpal component analyss (ernel-pca) s used to extract ey ndcator to mprove classfcaton performance. Fnally, the Support vector machne (SVM) s used to buld the qualty classfcaton model. The proposed classfcaton model s appled to classfy patent qualty automatcally n solar ndustres. Expermental results show that our proposed approach KPCA-SVM can mprove the performance of the patent qualty classfcaton when compared wth the tradtonal method. Another advantage s that the computatonal tme s largely reduced. Keywords- patent qualty classfcaton, self-organzng maps, support vector machne, ernel prncpal component analyss, solar ndustres. I. ITRODUCTIO An mportant ssue of patent analyss s patent qualty. The hgh qualty patent nformaton can ensure success for busness decson-mang process or product development [-]. Ths study revewed the patent analyss approaches that can understand patent status le patent qualty, novelty, ltgaton, trends and so on [3]. In addton, tradtonal patent analyss requres spendng much tme, cost and manpower. Therefore, the potental for hgh qualty patent determnng approach need to shorten the tme for busness or producton. Recently, the self-organzng map (SOM) s used to analyss patent for patent trend [4] and regonal nnovaton systems [5]. It can analyse patent current stuaton and trend, but t doesn t provde a soluton for determne future patent qualty. The future patent recognton s a ey research for present tme because patent mpact on the ndustry. The future patent recognton s a ey research for the tme because patent mpact on the ndustry need to response qucly. Therefore, support vector machne (SVM) forecastng model can solve patent classfcaton problem for predct future unnown patent classfcaton such as qualty [6]. In ths study, we propose a KPCA-SVM patent qualty classfcaton system that combnes three data mnng (DM) methods such as SOM, Kernel-PCA and SVM. The SOM s used to cluster patents nto several groups for qualtes accordng to ther data characterstcs. Then, the qualty ndctor can compute the qualty levels for delmt patents qualty on each groups. The ernel prncple components analyss (ernel-pca) s based on prncple components analyss and used to transform patent data nto new features set by nonlnear ernel mappng. SVM s used to buld classfcaton model for patent qualty problem. Ths methodology helps experts ran and set values on patent qualty n solar ndustry. Therefore, a traned model can evaluate unnown patents qualty, better enable engneers and product desgners forecast patent potental for product development. II. LITERATURE REVIEWS A. Patnet Analyss There are varous tools utlzed by organzatons for analyzng patents. These tools are capable of performng wde range of tass, such as forecastng future technologcal trends, detectng patent nfrngement and determnng patent qualty and so on [3]. Moreover, patent analyss tools can free patent experts from the laborous tass of analyzng the patent documents manually and determnng the qualty of patents. The tools assst organzatons n mang decsons of whether or not to nvest n manufacturng of the new products by analyzng the qualty of the fled patents []. The eventually may result n mprecse recommendaton of patents. However, the larger data and ndcators for patent qualty forecastng are needed. B. Patnet Qualty Indcator The prmary patent qualty ndcators are related to nvestment, mantenance, and ltgaton, whch form a bass for assessng patent qualty, when the evaluaton focuses on the potental patents for busness. One nd of ndcator of patent qualty s legal status (LS) that t can show whch technologes are hot and whch are not for busness ntellgence. The legal status and search tools on the nternet are very senstve that emphass gven to the ssues related to the date of avalablty to the publc of an Internet dsclosure, ts conformance and ts possbly non-preudcal nature [7]. The legal status change that suggestons on how these changes may be traced are provded, specfcally resoluton Copyrght (c) IARIA, 05. ISB:

2 ADVCOMP 05 : The nth Internatonal Conference on Advanced Engneerng Computng and Applcatons n Scences s also complcated by the frst to nvent concept n US patent law [8]. C. Self-Organzng Maps The SOM s a two-layer neural networ that maps multdmensonal data on to a two dmensonal topologcal grd. The data are group accordng to smlartes and patterns found n the data set, usng some form of dstance measure whch use the Eucldean dstance. The results are dsplayed as nodes on the map, whch can be dvded nto dfferent clusters based upon the dstances between the clusters. Snce the SOM s unsupervsed, no target outcomes are provded, and the SOM s allowed to freely organze tself, so the SOM s an deal tool for exploratory data analyss. The authors [4] used SOM to dentfy patent trends that they analyze patent nowledge to dentfy research trends. They tested on patents from the Unted States Patent and Trademar Offce (USPTO) and result both an overvew of the drectons of the trends and a drll-down perspectve of current trends. Another algorthm usng SOM that s evolvng self-organzng map (ESOM), whch features an evolvng networ structure and fast on-lne learnng. Ther result shows that ESOM acheved better or comparable performance wth a much shorter learnng process [9]. D. Kernel Prncple Component Analyss Prncpal component analyss (PCA) s very useful to extract nonlnear features for many research applcatons. The Kernel-PCA s an extenson of PCA usng the ernel mappng before the Egen-problem. Kernel-PCA s as a nonlnear alternatve to classcal PCA of combuston composton space s nvestgated. PCA s mathematcally defned. The PCA s wdely used n many feld researches. The research [] wants to dentfy the ey mpact factors usng PCA and they selected a lot of varables accordng to frst fve components ndcate. The Kernel-PCA s used to crtcal feature extracton n stoc tradng model and capture best performance compared to PCA, ICA and so on [0]. E. Support Vector Machne Support vector machne s a machne learnng algorthm and wdely used for classfcaton problems [5]. Ths method ams to develop an optmal hyper-plane as a decson functon usng the maxmum margn hyper-plane between class vectors on both sdes of the hyper-plane. Support vector machne map nput vectors nto the hgh dmensonal feature space va the non-lnear mappng. An effectve decson hyper-plane s developed to dstngush the correct tranng data. An approach s proposed ntegrated wth a hybrd genetc-based support vector machne (HGA-SVM) model for developng a patent classfcaton system []. But they are needed expert s nowledge to analyss. The authors clam that they use these models n real-world cases of patent classfcaton rather than only use for Internatonal Patent Classfcaton (IPC). The study ntegrated the honey-bee matng optmzaton algorthm wth SVM (HBMOSVM) for patent document categorzaton. In ther results show that the HBMOSVM could result n better patent documentaton accuracy and better F-measure performance as an evaluaton ndex than GASVM model n patents document categorzaton []. III. PROPOSED METHODOLOGY: KPCA-SVM PATET QUALITY CLASSIFICATIO SYSTEM Ths study proposes an automatc patent qualty classfcaton system that ntegrated methodology as KPCA- SVM; the components used are SOM, ernel-pca and SVM approaches. Fg. shows that, frst, we collect the patent data related ndustres from the patent database, use SOM approach to cluster patens nto several groups and use patent qualty ndcators to compute potental qualty on each group for qualty dentfcaton; second, the ernel-pca extracts ey ndcators nto nonlnear feature space; fnally, SVM forecastng model s used to buld patent qualty classfcaton model usng nonlnear feature space by ernel- PCA n order to predct qualty of future patent who has potental effectveness. Then, we evaluate patent qualty classfcaton system and forecast qualty level for each paten by our proposed system. Our system s developed as follows: Fgure. The system framewor of KPCA-SVM. A. Patent Qualty Analyss and Identfcaton by SOM wth Qualty Indcators The SOM approach s adopted to cluster the patents, to dentfy patent qualty and to explore hdden patterns among these patents. Accordng to SOM, these patents wll be splt nto several groups (clusters) and each cluster ncludes number of patents wth a smlar patent qualty. Ths SOM analyss process contnues untl all nput vectors are processed. Convergence crteron utlzed here s n terms of epochs, whch defnes how many tmes all nput vectors should be fed to the SOM for analyss. Detals of the SOM algorthm are lsted as follows: Step : Set-up the parameters n the SOM networ. Step : Intalze each neuron weght T w [ w, w,.., w ] R. In ths study, neuron weghts are ntalzed by drawng random samples from nput dataset. T Step3:Present an nput pattern x [ x, x,.., x ] R. In ths case, the nput pattern s a seres of varables representng current patent status. Calculate the dstance between pattern x, and each neuron weght w, and therefore, dentfy the wnnng neuron or best matchng unt c such as Copyrght (c) IARIA, 05. ISB:

3 ADVCOMP 05 : The nth Internatonal Conference on Advanced Engneerng Computng and Applcatons n Scences d { } x w c mn d ( x w ) () () Step 4: Adust the weght of wnnng neuron c and all neghbor unts. w ( t + ) w ( t) + h ( t)[ x( t) w ( t)] where s the ndex of the neghbor neuron and t s an nteger, the dscrete tme coordnate. The neghborhood ernel h c (t) s a functon of tme and the dstance between neghbor neuron and wnnng neuron ch c (t) defnes the regon of nfluence that the nput pattern has on the SOM and conssts of two parts: the neghborhood functon h(, t) and the learnng rate functon t, c c h ( t) (3) t h( rc r, t) (4) where r s the locaton of the neuron on two dmensonal map grds. In ths wor we used Gaussan eghborhood Functon. The learnng rate functon (t) s a decreasng functon of tme. The fnal form of the neghborhood ernel wth Gaussan functon s rc r hc ( t) exp ( t) σ ( t) (5) Step 5: repeat step 3 and 4 untl the convergence crteron s satsfed. Average value for each varable of each clustered group was calculated after the patent cases were clustered, and the average value for each varable of each group would be the bass when fndng the most matchng group for the new case. After the set of patent data has been processed by SOM, a new case can be categorzed nto a predefned group. The patents of each group wll compute the qualty by qualty ndcators such as legal status. The qualty levels of each group are calculated as follows: Qualty( Groupg ) m n m n, Group where q denotes value of qualty of th varable of th patent n gth group. B. Extractng Key Patent Indcators by Kernel-PCA In order to compute dot products of the form, we use ernel representaton of the form. g q (6) K ( x, x ) ( Φ ( x ), Φ ( x )) (7) whch allows us to compute the value of the dot product n F wthout havng to carry out the map Φ. A number of ernel functons exst as been chosen before we apply the algorthm. The representatve Gaussan ernel functon K s descrbed as follows: x x K( x, x ) e( σ Gven a set of m-dmensonal normalzed patent m ndces x R, we compute the ernel matrx K R from two ernel methods, ) (8) K Φ( x ), Φ( x )) [ ( x, x )] (9) ( Carry out mean centerng n the feature space, for Φ( 0 K ' K C* K K * C + C* K * C (0) C. Buldng Patent QualtyClassfcaton Model by SVM The patent data of dentfed qualty wll be splt nto two datasets,.e., tranng data set and testng data set. The new feature spaces of tranng data tranng(tr ) and testng data testng(ts ) are represented by the Φ( and. For patent varables x n tranng perod, we extract a nonlnear component va Tranng ( tr ) ( ν, Φ( K ( x, ( Φ( x ), Φ( () where Φ( s the mean centered. The testng data s unnown data as well as the future data. We cannot drectly use the testng data to compute the mean centerng Φ( and egenvalues n PCA processes. In order to avod ths problem, we use the Φ( and from the tranng data to extract a nonlnear component. Testng ( ts ) ( ν, Φ( K ( y, ( Φ( y ), Φ( () Copyrght (c) IARIA, 05. ISB:

4 ADVCOMP 05 : The nth Internatonal Conference on Advanced Engneerng Computng and Applcatons n Scences m where y denotes the normalzed varables y R n testng data. The nput vector of SVM tranng employ the new nonlnear feature space tranng(tr ) by ernel-pca and the output vector of qualty level s gven by SOM wth qualty ndcators. The traned model of SVM s then used to evaluate the testng data testng(tr ) for patent qualty forecastng. Thus, the proposed model can be appled to buld the patent qualty model for evaluatng the potental of patents. IV. EXPERIMETAL RESULTS In ths paper, these patents were dvded nto seven groups n order to dscrmnatng qualty levels nto seven degrees. The parameters of SOM ncludng epochs and cluster are set up to 5,000 and 7, respectvely. In SVM model, the ernel s radal bass functon (RBF), whle the cost s 56 and gamma s 0.5. The patent data of solar ndustry are collected from patent database of Thomson Innovaton whch has 60,000 patents n eleven patent offces. Seven dfferent qualty groups are used n SOM and ts qualty levels are sorted by legal status of qualty ndctor. The Table shows that the hghest legal status s n group 7, the second s n group 6 and the lowest s n group. The legal status on group 7 s 0.0 and the number of patents s,35. We observed that the hgher of patent qualty s, the less of patent number s. The other way around s for low patent qualty patent. TABLE I. THE LEGAL STATUS STATISTICS O EACH GROUP Group G G G3 G4 G5 G6 G7 o. of patent,43 6,49 9,796 4, ,089,35 Avg. of legal status Fg. shows that the dstrbuton on patent applcatons for patent offces. The two larger shares of patents are US and C. Ther patents exst n dfferent qualty levels because ther marets are the most mportant economc regon attractng many patent applcatons. Fgure. Dstrbuton on patent applcatons for patent offces We focus on group 7 snce the patent offces of US; C; EP; JP and AU nclude 88% shares as shown n Fg. 3. Other offces are much less nvolved n solar ndustry. In addton, the patent apples n EP patent offces are hgh qualty whch are n Group 4, 5, 6 and 7. However, the qualty levels are evenly spread n C patent offces. Fgure 3. Dstrbuton on patent applcatons on hghest qualty group (Group 7) The results show the forecastng performance for KPCA- SVM and decson tree (DT). The optmal parameters were decded from the tranng data for ernel-pca and SVM. Table shows that our proposed model has best performance on three measure ndcators and they are 99.7% on accuracy, 99.8% on average precson of all classes, and 99.3% on average recall of all classes. TABLE II. FORECASTIG PERFORMACE FOR DIFFERET CLASSIFICATIO MODEL Classfer Accuracy Precson Recall KPCA- SVM 99.9% 99.8% 99.3% DT 96.86% 95.4% 95.08% V. COCLUSIOS We proposed the KPCA-SVM patent qualty system whch combned SOM, ernel-pca and SVM data mnng approaches n solar ndustry. The expermental results showed that the proposed approach has a better performance when compared wth other tradtonal approaches n terms of tme consumng, cost and manpower. The proposed approach tae a shorten tme to determne patent qualty and has 99.9% accuracy. In addton, the proposed system performs even better for a larger patent data and mang fast and accurate recommendatons. In the future research wor, we wll consder the relatonshp between patent qualty and patent value creaton. Therefore, dfferent patent qualty can be closely related to dfferent patent values va accurate classfcaton system. REFERECES [] Trappey, A.J.C., Trappey, C.V., Wu, C.Y. & Ln, C.L, A patent qualty analyss for nnovatve technology and product development, Adv Eng Inform, 6(), pp. 6-34, 0. [] Trappey, A.J.C., Trappey, C.V., Wu, C.Y.W., Fan, C.Y. & Ln, Y.L., Intellgent patent recommendaton system for Innovatve desgn collaboraton, J etw Comput Appl, 36, pp , 03. [3] Abbas, A., Zhang, L. & Khan, S.U., A lterature revew on the state-of-the-art n patent analyss, World Pat Inf, 37, pp. 3-3, 04. Copyrght (c) IARIA, 05. ISB:

5 ADVCOMP 05 : The nth Internatonal Conference on Advanced Engneerng Computng and Applcatons n Scences [4] Segev, A. & Kantola, J., Identfcaton of trends from patents usng self-organzng maps, Expert Syst Appl, 39(8), pp , 0. [5] Hae, P., Henrques, R. & Haova, V., Vsualsng components of regonal nnovaton systems usng selforganzng maps Evdence from European regons, Technol Forecast Soc Change, 84,pp.97-4, 04. [6] Ercan, S. & Kayautlu, G., Patent value analyss usng support vector machnes, Soft Comput, 8, pp , 04. [7] Archontopoulos, E., Pror art search tools on the Internet and legal status of the results: a European Patent Offce perspectve, World Pat Inf, 6(), pp. 3-, 004. [8] Smmons, E.S. & Spahl, B.D., Of submarnes and nterference: legal status changes followng ctaton of an earler US patent or patent applcaton under 35 USC 0 (e), World Pat Inf, (3), pp. 9-03, 000. [9] Deng, D. & Kasabov,., On-lne pattern analyss by evolvng self-organzng maps, eurocomputng, 5, pp , 003. [0] Chang, P.C. & Wu, J.L., A crtcal feature extracton by ernel PCA n stoc tradng model, Soft Comput., DOI 0.007/s [] Wu, C.H., Ken, Y. & Huang, T., Patent classfcaton system usng a new hybrd genetc algorthm support vector machne, Appl Soft Comput, 0(4) pp , 00. [] Chu, C.Y., & Huang, P.T. Applcaton of the honeybee matng optmzaton algorthm to patent document classfcaton n combnaton wth the support vector machne, Int.. autom. smart technol, 3(3), pp.79-9, Copyrght (c) IARIA, 05. ISB: