Measuring innovative performance: is there an advantage in using multiple indicators?

Transcription

1 Reserch Policy 32 (2003) Mesuring innovtive performnce: is there n dvntge in using multiple indictors? John Hgedoorn, Myrim Cloodt MERIT, Fculty of Economics nd Business Administrtion, University of Mstricht, P.O. Box 616, 6200 MD Mstricht, The Netherlnds Received 29 My 2002; received in revised form 21 October 2002; ccepted 23 October 2002 Abstrct The innovtive performnce of compnies hs been studied quite extensively nd for long period of time. However, the results of mny studies hve not yet led to generlly ccepted indictor of innovtive performnce or common set of indictors. So fr the vriety in terms of constructs, mesurements, smples, industries nd countries hs been substntil. This pper studies the innovtive performnce of lrge interntionl smple of nerly 1200 compnies in four high-tech industries, using vriety of indictors. These indictors rnge from R&D inputs, ptent counts nd ptent cittions to new product nnouncements. The study estblishes tht composite construct bsed on these four indictors clerly ctches ltent vrible innovtive performnce. However, our findings lso suggest tht the sttisticl overlp between these indictors is tht strong tht future reserch might lso consider using ny of these indictors to mesure the innovtive performnce of compnies in high-tech industries Elsevier Science B.V. All rights reserved. Keywords: High-tech industries; Innovtive performnce; Ptents 1. Introduction The mngement nd pplied economics literture on innovtion nd relted topics hs long history of struggling with the mesurement of the innovtive performnce of compnies. Both generlly vilble mesures such s R&D inputs, ptent counts, ptent cittions, or counts of new product nnouncements, nd more specific survey-bsed mesurements of this prticulr performnce by compnies hve been used in trying to cpture this innovtive performnce Corresponding uthor. Tel.: ; fx: E-mil ddresses: j.hgedoorn@mw.unims.nl (J. Hgedoorn), m.cloodt@mw.unims.nl (M. Cloodt). of compnies. 1 Mny studies use single indictor (R&D, ptents, ptent cittions, or new product nnouncements) rguing tht the specific indictor tht is pplied hs fewer shortcomings thn the other indictors. Some studies use two or more indictors to generte one construct. Given the vriety in constructs, mesurements, smples, dtbses, industries nd country settings nd inconsistency in definitions, 1 In the following, we will minly consider these generlly vilble innovtion mesures found in the public domin or in lrge dtbses nd not those obtined from survey-reserch. Bibliometric indictors could lso be used for mesuring reserch output in terms of the yield of scientific ppers. However, there re some serious methodologicl problems in relting clssifictions of fields of scientific output to technology nd industry clssifictions. Moreover, this pper focuses on the mesurement of innovtive performnce nd not on scientific performnce /$ see front mtter 2002 Elsevier Science B.V. All rights reserved. PII: S (02)

2 1366 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) it is of no surprise tht there ppers to be hrdly ny cler understnding of the concept nd mesurement of innovtive performnce. The min purpose of this pper is to estblish better understnding of the innovtive performnce of compnies, considering the possible use of both number of single indictors or vrious combintions of multiple indictors in composite mesure of performnce. This exercise implies tht we ttempt to mesure the qulity of performnce by mens of ltent vrible nd tht we nlyse the usefulness of multiple indictors, constructing composite mesure of the innovtive performnce of compnies (see lso Lnjouw nd Schnkermn, 1999). The dvntge of such multi-indictor pproch is tht, insted of ssuming the correctness of single indictor, probbly tken for resons of convenience, n nlysis of multiple indictors might llow us to mesure innovtive performnce through more complex, more informtive, composite mesure. In ddition to this, composite mesure cn be nlysed in detil, in terms of the individul indictors nd with reference to the contribution of ech indictor to the ltent vrible innovtive performnce. This prticulr pproch enbles us to mesure the informtion gins from using prticulr indictors. This ltter spect of the nlysis lso indictes tht our reserch does not imply tht multiple-indictor pproch is by definition superior. In cse of strong sttisticl ssocition between indictors, ech indictor could replce ny other mesure used in the nlysis. Given the purpose of this pper, it should be cler tht this pper is methodologicl in nture s it presents n nlysis of the mesurement of innovtive performnce. It does not outline theory or model nlysing nd explining differences in the innovtive performnce of compnies. In tht sense, the following nlysis is self-referentil, i.e. innovtive performnce is exclusively defined in terms of individul mesures or the common fctor linking the different indictors used in the nlysis. The min objective of this pper, therefore, is to better understnd the possible dvntges of specific single indictors or of composite mesures, using multiple indictors relted to ltent vribles, tht conditionl on the observed chrcteristics of these indictors, might generte greter insight. In the following, we will first discuss number of concepts tht re relevnt in the context of mesuring innovtive performnce nd tht lso enble us to picture the possible reltionships between the different indictors of this performnce. The next section presents n overview of the literture on indictors such s R&D input, ptent counts, ptent cittions nd new product nnouncements. In tht context it turns out tht R&D inputs could be resonble indictor of innovtive effort, ptents could be more thn cceptble indictor of innovtive output, ptent cittions could be used to mesure the qulity of innovtive output nd new product nnouncements could indicte the level of product innovtion. We will lso discuss the possible sttisticl reltionship between these indictors s found in number of studies. Bsed on our understnding of previous reserch, we will formulte three reserch questions. This is followed by description of the lrge interntionl smple of compnies in four high-tech sectors nd description of the specific vribles tht we will nlyse. We lso offer brief explntion of the rtionle behind the choice of industries. The four high-tech studies for which we will investigte the pproprite mesures of innovtive performnce re erospce nd defense, computer nd office mchinery, phrmceuticls nd electronics nd. The empiricl nlysis of the different indictors of innovtive performnce, bsed on fctor nlysis, will be presented in the next section. This is followed by discussion of the results nd the conclusions tht cn be drwn from this study. 2. The reltionship between inventive, technologicl nd innovtive performnce Concepts such s invention, technology nd innovtion nd the relted performnce of compnies hve to be clerly defined in order to improve our comprehension of some technicl issues relted to the mesurement of innovtion. Therefore, we will briefly outline our understnding of ech of these concepts. We define inventive performnce s the chievements of compnies in terms of ides, sketches, models of new devices, products, processes nd systems (Ernst, 2001; Freemn nd Soete, 1997). As discussed further below, inventive performnce is frequently mesured in the context of ptents where both rw counts of ptents nd ptent cittions re tken s the ctul mesures. Following Griliches (1998),

3 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Fig. 1. Venn digrms representing the reltionship between R&D, ptents nd new products in two hypotheticl industries or two industry-verge compnies (A nd B) with different degrees of overlp (see lso Bsberg, 1987). technologicl performnce cn be defined s the ccomplishment of compnies with regrd to the combintion of their R&D input, s n indictor of their reserch cpbilities, nd their R&D output in terms of ptents. We cn lso differentite between innovtive performnce in the nrrow nd in the brod sense. Innovtive performnce in the nrrow sense refers to results for compnies in terms of the degree to which they ctully introduce inventions into the mrket, i.e. their rte of introduction of new products, new process systems or new devices (Freemn nd Soete, 1997). In tht cse new product nnouncements cn be pplied s n indictor of innovtive performnce. A broder understnding of innovtive performnce encompsses the three previous mesures of performnce s it indictes the chievement in the trjectory from conception of n ide up to the introduction of n invention into the mrket (Ernst, 2001). This brod innovtive performnce therefore overrches the mesurement of ll stges from R&D to ptenting nd new product introduction. In other words, this definition of innovtive performnce in the brod sense focuses on both the technicl spects of innovtion nd the introduction of new products into the mrket, but it excludes the possible economic success of innovtions s such (see lso Acs nd Audretsch, 1989; Ahuj nd Ktil, 2001; Archibugi, 1992; Ernst, 2001; Freemn nd Soete, 1997; Grupp, 1994; Lnjouw nd Schnkermn, 1999; Sturt, 2000; Trjtenberg, 1990). Our understnding of the different mesures, which might interct in the innovtive performnce of compnies, cn be explined further using Venn digrms. These Venn digrms picture the possible overlp between R&D, ptents (both counts nd cittions) nd new product nnouncements s indictors of innovtive performnce, see Fig. 1. It is well known tht the overlp between R&D, ptents nd new products is not necessrily complete. Full (100%) overlp is usully not expected becuse not ll ptents re directly relted to R&D nd the sme pplies to new products of which only prt of the totl number re protected by ptents. Also, the spce in the squres tht surround the Venn digrms cn represent non-mesurble technologicl chnge or technologicl chnge not ccounted for by these mesures. The ctul intersection between R&D, ptents nd new products in the Venn digrms depend on the specific nture of the industry. We submit tht the degree of overlp between R&D, ptents nd new product nnouncements determines the usefulness of single or multiple indictors. The more n industry or its verge compny is chrcterized by high R&D intensity, high ptenting intensity nd high rtio for new product introduction, the lrger the intersection in the Venn digrm. In cse the overlp is complete or lmost complete, i.e. in cse of (nerly) complete ssocition of mesures, the ssessment of innovtive performnce cn probbly be limited to one of the overlpping indictors (see Fig. 1B). In cse the overlp is very smll, one hs to clerly differentite between inventive performnce, technologicl performnce nd innovtive performnce in the nrrow sense (see Fig. 1A). In other cses with vrying degrees of overlp between R&D, ptents nd new products, the

4 1368 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) use of multiple indictors might provide better understnding of the ctul innovtive performnce of compnies. 3. Indictors of innovtive performnce As discussed briefly in the bove, we cn nlyse technologicl performnce, inventive performnce, nd innovtive performnce in the nrrow sense in terms of R&D inputs, ptent counts, ptent cittions, nd new product nnouncements. These indictors cn be used individully or combined in multi-dimensionl setting to mesure innovtive performnce in the broder sense. The current literture on the mesurement of innovtion points towrds some interesting perspectives on using these different indictors. In the following, we will present brief overview of this literture tht will be helpful in providing some bckground to our reserch questions R&D inputs The literture usully tkes R&D expenditures primrily s n input indictor of the efforts tht compnies mke in estblishing R&D tht might eventully led to output. However, prt from the ctul correltion of R&D input with R&D output through ptents (Griliches, 1990, 1998; Husmn et l., 1984; Hitt et l., 1997), R&D efforts cn lso indicte the innovtive competences of compnies tht re found to ffect the performnce of compnies, prticulrly in high-tech industries (Duysters nd Hgedoorn, 2001; Henderson nd Cockburn, 1994). In tht context, R&D inputs of compnies re prt of the broder routines tht compnies follow in their innovtive efforts, where previous R&D expenditures ffect subsequent R&D inputs (Nelson nd Winter, 1982). Successful R&D input t previous stge will increse the commitment to the lloction of future R&D resources. In other words, within the routines tht most compnies hve developed in mnging their R&D inputs, the ctul R&D efforts not only reflect current input but lso their previous successes tht re n integrl prt of the skills tht compnies develop in creting n R&D strtegy with stble set of long-term projects (Hgedoorn, 1989). We expect tht, prticulrly in high-tech industries, R&D inputs re s much ffecting future output s they re prt of the current technologicl performnce of compny in terms of generting new ides, new blue prints nd new models, prt of which will eventully led to new ptents nd new products (Griliches, 1998; Bysinger nd Hoskisson, 1989; Hll, 1990) Ptents As most of the other indictors, the ptent mesure, in prticulr the use of rw ptent counts, is subject to longstnding debte regrding its bis nd shortcomings (Archibugi, 1992; Cohen nd Levin, 1989; Dosi, 1988; Griliches, 1998). Interntionl nd sectorl differences in ptenting behviour, differences in ptenting between lrge compnies nd smller firms, the identicl weight given to some very importnt ptents s well s to run-of-the-mill ptents, the fct tht ptents only cover prt of the overll trjectory from R&D to innovtion, ll of these shortcomings feture in the literture tht discusses the use of ptents. However, it ppers tht, certinly in lrge prts of the economics literture, rw ptent counts re generlly ccepted s one of the most pproprite indictors tht enble reserchers to compre the inventive or innovtive performnce of compnies in terms of new technologies, new processes nd new products (Acs nd Audretsch, 1989; Aspden, 1983; Bresmn et l., 1999; Cntwell nd Hodson, 1991; Freemn nd Soete, 1997; Griliches, 1998; Npolitno nd Sirilli, 1990; Ptel nd Pvitt, 1995; Pvitt, 1988). Even uthors who re somewht criticl of the overll use of ptents s performnce indictor, such s Arundel nd Kbl (1998) nd Mnsfield (1986), dmit tht ptents cn be n pproprite indictor in the context of mny high-tech sectors. These litertures lso suggest tht the less ptents re used for brod cross-sectionl nlysis tht ignores inter-sectorl differences in the propensity to ptent, the better ptents reflect the performnce of compnies in one or few sectors (see lso Ernst, 2001) Ptent cittions Incresingly reserchers re using ptent cittions s n indictor of inventive performnce of compnies.

5 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Compred to rw counts of ptents, which generte purely quntittive mesure, ptent cittions lso include mesure of the qulity of ptents. The bsic ssumption in using this indictor is tht there is positive reltionship between the importnce of ptent nd the degree to which ptent is cited in lter ptents. This informtion on previous ptents cn be trced in ech ptent ppliction s it is required tht ech ptent cites erlier ptents with somewht similr or relted technicl clims, lso known s the relevnt prior rt. The number of ptent cittions for prticulr ptent indictes its importnce or impct. Ptent specilists re somewht scepticl bout the counting of ptent cittions without the necessry in-depth knowledge necessry to interpret the underlying cittion reports tht ccompny ech ptent (Michel nd Bettels, 2001). However, quntittive nlysis of lrge numbers of ptents nd ptent cittions is frequently used in economics (for instnce Hrhoff et l., 1999; Jffe et l., 1993; Trjtenberg, 1990) nd the mngement literture (for instnce Rosenkopf nd Nerkr, 2001; Sturt, 2000). Evidence of the vlidity of ptent cittions s n indictor of the qulity of inventions or innovtions, in terms of the correltion between the inter-subjective ssessment of the importnce of ptents by technicl specilists nd the number of cittions, is found in number of studies (Albert et l., 1991; Crpenter et l., 1981; Krki, 1997; Nrin et l., 1987; Pvitt, 1988) New product nnouncements Some contributions use new product nnouncements, trced through vrious sources nd dtbses, s n indictor of the innovtive performnce of compnies. In their study on 250 US compnies, Hitt et l. (1996) pply the degree to which new product nnouncements of compnies were found in Dilog s NPA/plus dtbse s prt of combined indictor of the innovtiveness of compnies. Devinney (1993) shows tht the positive reltionship between the number of ptents nd new product nnouncements is primrily found t the level of industries nd not t the level of individul compnies, where less thn 3% of the vrince in individul firm new product nnouncements is explined by ptent intensity. One of the min problems with using new product nnouncements, s for instnce found in vrious dtbses, is tht these nnouncements re bsed on the press releses of the mrketing deprtments of compnies nd little or no screening ppers to be undertken by the dtbse opertors themselves. In other words, new product nnouncements by compnies themselves re simply ccepted s long s compnies identify them s such. Ptents, on the other hnd, in prticulr those registered in n dvnced economy such s the USA, re screened for their originl contribution both during the pre-ppliction period nd during the ctul ppliction period by in-compny engineers, ptent lwyers nd by ptent office officils. Therefore, more thn with some of the other indictors, creful screening of the dt might be necessry with new product nnouncements in order to void mjor problems with the vlidity of the results The sttisticl ssocition of R&D, ptents nd new product nnouncement As so mny studies pply different indictors, it ppers interesting to get better understnding of the likely sttisticl ssocition between these different indictors. The mjority of studies use only one single indictor but number of studies use multiple indictors or report on the correltion between some of the indictors mentioned in the bove. However, the literture reporting more thn one indictor ppers to present somewht diverse picture when it comes to the possible reltionship between R&D inputs, ptent counts nd cittions, nd new products. Seminl studies by Pkes nd Griliches (1984) nd Bound et l. (1984) find rther strong reltionship between R&D nd the number of ptents cross firms nd industries. Ahuj nd Ktil s (2001) study of the interntionl chemicl industry reports correltion of nerly 0.9 between R&D inputs nd ptent counts. In their nlysis of the interntionl computer industry, Hgedoorn nd Duysters (2002) mention correltion of over 0.5 between the R&D intensity of compnies nd their ptent counts. However, in their study of US compnies involved in nerly 200 cquisitions, Hitt et l. (1991) show low correltion of only bout 0.2 between the R&D intensity of these compnies nd their ptent intensity. In one of their erly smll-scle studies, Nrin et l. (1987) find tht the qulity of reserch inputs of compnies is highly correlted with rw ptent counts nd

6 1370 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) ptent cittions. Trjtenberg (1990) demonstrtes tht cittion-weighted ptent numbers of compnies re more closely correlted with their output mesures of innovtion, wheres unweighted ptent counts re more closely relted to R&D expenditures of compnies. Sturt (2000) reports very high correltion of over 0.8 between rw ptent counts nd ptent cittions, the ctul mesure of innovtion in smple of 150 semiconductor firms. However, Rosenkopf nd Nerkr (2001), who studied smller smple of opticl disk mnufcturing compnies, find reltively low correltion of bout 0.3 between number of ptents nd ptent cittion indictors. In study of smple of 250 US compnies, Hitt et l. (1996) estblish substntil correltion of 0.5 between the R&D intensity of compnies nd their new product intensity, i.e. the number of new product nnouncements controlled for size of compnies. Brouwer nd Kleinknecht (1999) indicte some correltion between ptenting nd sles of innovtive products, lthough this correltion is fr from perfect. As lredy indicted in the bove, Devinney (1993) shows tht the specific reltionship between the ptenting of compnies nd their new product nnouncements turns out to be quite wek. 4. Reserch questions It is obvious from the bove tht the literture genertes rther mixed bg of findings, to sy the lest. Some studies suggest tht, given the rther high correltion between some indictors, ny of these correlted mesures could be used s prtil or overll indictor of performnce. Yet, the lck of ny sttisticl ssocition found for the sme indictors in other studies, ppers to suggest tht the results from studies using ny of these prticulr mesures could be indictor-driven. However, the level of intermedite sttisticl ssocition between number of mesures, s found in yet nother set of studies, indictes tht one composite construct with vrious mesures could cpture ltent vrible tht expresses the degree of innovtive performnce in the broder sense. The somewht uncler nture of the mesurement of innovtive performnce nd other performnce indictors, s found in lrge prts of the literture, ppers influenced by lck of systemtic reserch in multiple industries, while frequently using only reltively smll smples of compnies in domestic context. Our understnding of the literture, s outlined in the bove, suggests number of reserch questions. In order to overcome the mjor shortcomings of previous reserch, we will investigte these questions empiriclly in n interntionl, lrge-scle, multi-sectorl setting. These reserch questions re: 1. Is there systemtic disprity between different indictors such s R&D inputs, ptent counts, ptent cittions nd new product nnouncements? 2. Is there sttisticl communlity between two or more indictors, indicting tht composite construct could cpture ltent vrible? 3. Is the sttisticl ssocition between two or more indictors of such nture tht ech of these indictors cn be pplied s representtive indictor of innovtive performnce? 5. Reserch methodology 5.1. Smple Our lrge, interntionl smple of compnies covers four high-tech industries: erospce nd defense (SIC-codes 372 nd 376), computer nd office mchinery (SIC-code 357), phrmceuticls (SIC-code 283) nd electronics nd (SIC-code 36). These high-tech sectors re selected becuse it is known tht prticulrly in these industries R&D expenditures, ptents nd new products ply role in indicting importnt spects of innovtive performnce (OECD, 1997). Our smple consists of 1194 compnies of which 51.17% is found in the electronics nd industry, 23.45% is in computer nd office mchinery, 20.44% is in phrmceuticls, nd 4.94% belongs to the erospce nd defense sector. The literture (Brouwer nd Kleinknecht, 1999; Devinney, 1993; Ernst, 2001; Griliches, 1998) suggests tht sectorl differences cn ply role in explining the vrious outcomes with respect to different indictors of innovtive performnce. Therefore, we will present the results of our nlysis for the smple of compnies s whole s well s for ech of the sub-smples for the different industries. As suggested by Ernst (2001), we lso differentite interntionlly

7 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Tble 1 Distribution of size clsses of compnies in the totl smple nd for industry sub-smples (1998 nnul revenues in US$ millions) Size clsses Totl Aerospce nd defense Computers nd office mchinery 0 < 100 (%) < 1000 (%) >1000 (%) Totl (%) N between North Americn sub-smple (70.02%) nd compnies from other countries, minly in Europe nd Asi (29.98%). Some detils regrding the sectorl nd interntionl brekdown of this smple re presented in Appendix A. Our smple is lso diverse in terms of the distribution of the size of compnies, representing mny size-ctegories of innovting firms (see Tble 1). Most of the compnies in our smple (61.39%) re reltively smll with nnul revenues of up to mximum of US$ 100 millions. A substntil number of compnies (14.74%) cn be chrcterized s very lrge with nnul revenues of over US$ 1 billion. Nerly one qurter of the smple (18.59%) cn be found in n intermedite size-clss. Obviously, there re differences between the industry sub-smples but for ech sector we do find diverse picture in terms of the size-ctegories for compnies. It is well known tht there is no officil dtbse with world-wide, industry-level list of ll compnies from which one cn drw rndom smple. Our smple is tken from the Securities Dt dtbnk on interntionl compnies, which provides informtion on compnies in vrious industries, including those tht were selected for this study. We concentrted on mnufcturing sectors becuse some of the indictors more effectively cpture spects of technicl innovtion rther thn of service innovtion (Kleinknecht, 1999). Additionl informtion on revenues nd R&D expenditures of compnies in this smple ws identified through vrious other dtsets such s Amdeus, Compustt, the Fortune 500 list, nd Worldscope. Compnies entered the smple if the vlue of t lest one of the vribles used in the nlysis hd vlue higher thn zero. In other words, compnies with zero R&D, zero ptents, zero ptent cittions nd zero new products were excluded Vribles nd dt sources R&D expenditures of compnies re mesured nnully nd stndrdized by converting the finncil dt from ntionl currencies to US dollrs. Dt on R&D expenditures were obtined through the well-known dtbses mentioned in the previous section nd through dditionl internet serches. We utilize the yerly number of ll ptents grnted to ech compny s second indictor. The dt on ptents re tken from the US Ptent nd Trdemrk Office (USPTO) dtbse. We use US ptent dt for both US nd non-us firms. Although these US dt could imply bis in fvour of US compnies ginst non-us firms, it is mentioned in the literture tht given the importnce of the US mrket, the rel ptent protection offered by US uthorities, nd the level of technologicl sophistiction of the US mrket, it is often compulsory for non-us compnies to file ptents in the USA (Ptel nd Pvitt, 1991). Furthermore, to mintin certin level of consistency, relibility nd comprbility it is necessry to choose one ptenting system insted of severl ptenting systems cross ntions (Ahuj nd Ktil, 2001). 2 Ptent cittions for ech firm re counted s the yerly ptent cittions of their ptents in ll ptent clsses. Dt on ptent cittions re lso tken from the USPTO dtbse. New products or new product nnouncements re counted nnully for ech compny in sectors such s erospce nd defense, computers, office equipment, phrmceuticls, semiconductors nd telecom, 2 With hindsight it turned out tht there is little indiction of mjor differences between North Americn (lrgely US) nd non-north Americn compnies, see the correltion for ptents nd other mesures in Appendix B.

8 1372 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) s found in the RDS Business & Industry dtbnk owned by the Gle Group. This dtbse contins the full text of press releses from ll industries covering nnouncements relted to products, with focus on new products nd services. In ddition to product descriptions, the press releses generlly contin key detils bout new products nd technologies, including technicl specifictions, vilbility, uses, licensing greements, distribution chnnels, nd prices. The dtbse covers more thn 1500 technicl nd non-technicl journls nd cn be found on For this study, we seprted nnouncements of product development projects nd new services from new product introduction nd only use the ltter informtion in the nlysis. For nnul R&D expenditures nd yerly ptent counts dt re obtined for the period , nnul ptent cittions re registered for the yers , nd yerly new product nnouncements re counted for the period We mesured the performnce of compnies in the smple s whole nd for ech sub-smple on ech indictor with lternting time periods. The first period covers the yers 1997 nd 1998 for ll indictors without ny time lgs between vrious indictors. All indictors in the dt set shre the yers 1997 nd The second time period ( ) hs n verge lg of 1 yer between indictors nd the third time period ( ) hs n verge lg of 2 yers between indictors. The fourth period is bsed on different time lgs s mentioned in the literture (Cincer, 1997; Hll et l., 1986, 2000; Kondo, 1999; Npolitno nd Sirilli, 1990; Pkes, 1985; Pkes nd Griliches, 1984; Scherer, 1984,b). For this period, we took lgs of 3 yers between R&D nd ptents grnted, 2 yers between ptents grnted nd ptent cittions, nd 1 yer between ptent cittions nd new products. 6. Results Our reserch questions suggest tht we re bsiclly mking n ttempt to understnd innovtive performnce in terms of the possible cusl structure mong number of indictors (vribles), considering set of common underlying dimensions or fctors. Therefore, fctor nlysis is pplied to exmine our reserch questions. The results of this nlysis will only be reported Tble 2 Mens nd stndrd devitions for the smple s whole nd sub-smples Vribles Men Stndrd devition Totl R&D expenditures Ptent counts Ptent cittions New products Aerospce nd defense R&D expenditures Ptent counts Ptent cittions New products Computers nd office mchinery R&D expenditures Ptent counts Ptent cittions New products R&D expenditures Ptent counts Ptent cittions New products R&D expenditures Ptent counts Ptent cittions New products Note: R&D expenditures re in thousnds of US$. for the overll smple nd its sub-smples for the period The results for the nlyses with different time lgs turned out to be lmost identicl with no significnt differences. Some descriptive dt on the mens nd stndrd devitions for the overll smple nd sub-smples re presented in Tble 2. Some dditionl descriptive dt on the reltionship between indictors for the overll smple nd the sub-smples re given in Appendix F. These dt suggest tht there re inter-sectorl differences between the rtios for the different indictors. In tht sense, despite the fct tht ll four sectors re high-tech industries, there re differences with regrd to the degree to which, for instnce prticulr levels of R&D led to ptents nd new products. This phenomenon stresses the importnce of understnding inter-sectorl differences, it does not question the use of this prticulr group of indictors. In order to discover the usefulness of these

9 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Tble 3 Correltions Vribles Totl Aerospce nd defense Computers nd office mchinery Ptents nd ptent cittions Ptents nd R&D expenditures Ptents nd new products Ptent cittions nd R&D expenditures Ptent cittions nd new products R&D expenditures nd new products different indictors, we will first hve to consider the outcomes of the fctor nlysis, presented below. Obviously fctor model cn only be pproprite if vribles re to some extent relted to ech other. 3 If correltions between vribles re lower thn 0.30, it is unlikely tht they shre common fctors. In our smple nd sub-smples correltions re greter thn 0.30, suggesting tht fctor nlysis is pproprite (see Tble 3). Similr correltions re found for the sub-smples for North Americn compnies nd other compnies (see Appendix B). An exception is found for the erospce nd defense industry in non-north Americn countries. The irregulr pttern of correltions, found for this industry nd these countries, is probbly cused by the reltively smll number of 21 compnies. 4 A somewht more dvnced method of determining the ppropriteness of fctor nlysis is the Kiser Meyer Olkin (KMO) mesure. The KMO mesure is n index for compring the mgnitudes of the observed correltion coefficients to the mgnitudes of the prtil 3 Some of the vribles (i.e. ptent counts nd ptent cittions) could be too strongly dependent on ech other if lrge number of compnies hve only zero ptents which, ignoring time, will by definition led to zero ptent cittions. This will hve n upwrd effect on the correltions. However, within given period of time (e.g ) or even in ech of the other lgged periods, ptent cittions do not necessrily refer to the ptent counts from the sme or lgged periods. Furthermore, 47.65% of the compnies in the smple do not hve ptents wheres 38.11% of the compnies hve no ptent cittions. Or to put it differently, 36.38% of the compnies in this smple tht do not hve ptents in ny of these periods, still hve ptent cittions. 4 As noted in the literture (Hir et l., 1995) fctor nlyticl methods re not well suited in cse of smll smple nd low rtio of observtions to vribles. Even seemingly meningful results, obtined from such smples, should be interpreted cutiously. correltion coefficients (Norusis, 1993). Higher vlues for this mesure indicte the degree of ppropriteness of using fctor nlysis. In ddition to this we cn lso clculte mesure of smpling dequcy (MSA) for ech individul vrible. Agin, vlues for this mesure indicte whether fctor nlysis is pproprite or not. For our smple, both the KMO nd the MSA mesures indicte tht fctor nlysis is n pproprite method to test if R&D expenditures, ptent counts, ptent cittion counts, nd new products shre one or more common fctors (see Appendix C). This implies tht we cn continue with the second stge of the sttisticl nlysis, i.e. explortory fctor nlysis by mens of principl component nlysis (Hir et l., 1995). The results of the initil unrotted fctor nlysis re very strightforwrd s the nlysis genertes only one fctor. Given this one-fctor solution there is no further need for rottion of fctors. The results of the initil unrotted fctor nlysis re presented in Tbles 4 nd 5 nd Appendices D nd E. Tble 4, lbelled communlity mesures, is n indiction of the strength of the liner ssocition mong vribles. Squring the correltion coefficient shows the mount of the vrible s totl vrince ccounted for by the fctor (Norusis, 1993). Smll squred multiple correltion coefficients suggest tht the vrible should be eliminted from the nlysis. Over 75% of the communlities re greter thn 0.80, suggesting tht 80% or more of the vrible s totl vrince is ccounted for by the fctor. In generl there re not mny differences between the communlities clculted for the smple s whole nd the industry sub-smples (see Tble 4). The reltively lrgest differences involve the vribles R&D expenditures nd new products. The vrible R&D expenditures hs somewht lower communlities for the smple s whole, for the phrmceuticl industry nd

10 1374 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Tble 4 Communlity mesures Vribles Totl Aerospce nd defense Computers nd office mchinery R&D expenditures Ptent counts Ptent cittions New products Tble 5 Results of the initil unrotted fctor nlysis for component 1 Vribles Totl Aerospce nd defense Computers nd office mchinery R&D expenditures Ptent counts Ptent cittions New products Totl Eigenvlue % vrince explined for the electronics nd industry. The vrible new products hs lower communlities for the erospce nd defense industry nd for the phrmceuticls industry. An interesting finding is tht, compred with the other industries, the communlities for the phrmceuticl industry show reltively low vlues. As indicted by Appendix D the results for compnies from North Americ nd for those from other countries re not significntly different. Tble 5 nd Appendix E present the results of the fctor nlysis. Ech row in the tble nd the ppendix contins fctor loding indicting the correltion of tht specific vrible nd the fctor. High lodings mke the vrible representtive of the fctor. The signs of the fctor lodings re ll positive nd the fctor lodings re ll sttisticlly significnt. 90% of the fctor lodings re bove 0.80, while the lowest fctor loding is still 0.774, indicting tht ech of the four vribles is representtive of the fctor. Not surprisingly, the fctor lodings exhibit the sme overll pttern s the communlities with respect to the smple s whole, the smple with North Americn compnies, nd the smple consisting of compnies from other countries. Furthermore, the fctor lodings revel the sme pttern s the communlities for the vribles ptent cittions nd new products in the phrmceuticl industry nd for the vrible new products in the erospce nd defense industry. Tble 5 nd Appendix E lso give the totl vrince explined by ech fctor. For the smple s whole nerly 85% of the totl vrince is explined by one component, tht we lbel s innovtive performnce in the brod sense bsed on its ppropriteness for representing the underlying dimension of this prticulr fctor. In the computer nd office mchinery sector more thn 90% of the totl vrince is explined by this component. The phrmceuticl industry hs the lowest percentge explined by the component, nmely 73.23%. The difference of 19.5% between the verge of both the computer nd office mchinery sector nd the phrmceuticl industry is minly bsed on the lower fctor lodings for ptent cittions nd new products in the phrmceuticl industry. Results for compnies from North Americ nd from other countries re lmost identicl (see Appendix E). 7. Discussion nd conclusions Our study covers lrge smple of nerly 1200 compnies found in four high-tech industries. About

11 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) % of these compnies re registered in the USA nd Cnd, most of the others re from Europe, Jpn, South Kore nd some south est Asin newly industrilized countries. Our findings regrding the different indictors of innovtive performnce in this smple re rther unmbiguous, leving very little spce for subtle re-interprettions of the pros nd cons of the individul indictors. Also, using different time lgs or no time lg t ll does not ffect the outcome of the nlysis. Hence, we cn present set of cler-cut conclusions regrding the possible dvntge of multiple indictors for mesuring innovtive performnce in high-tech industries. First, our reserch suggests tht there is no mjor systemtic disprity mongst R&D inputs, ptent counts, ptent cittions nd new product nnouncements (reserch question 1). A systemtic disprity ws found for neither the smple s whole, nor for individul sectors, or when tking the interntionl distribution into ccount. Second, the sttisticl communlity of the four indictors of innovtive performnce is high for the smple s whole nd for ll but one sub-smple (erospce nd defense outside North Americ). This indictes tht composite construct bsed on these indictors ctches ltent vrible innovtive performnce tht mesures the performnce of compnies in high-tech industries (reserch question 2). The ltent vrible innovtive performnce mesures the overll, brod interprettion of innovtive performnce of compnies in terms of their reserch input, the size of their inventive ctivities, the qulity of their inventive output nd their level of new product introduction (see lso Mohnen nd Dgenis, 2002; Miresse nd Mohnen, 2002). Third, the results of the nlysis, however, lso indicte tht the overlp between ech of these four indictors is tht gret (similr to Fig. 1B) tht in high-tech sectors ny of these four indictors could be tken s mesure of innovtive performnce in the brod sense (reserch question 3). Certinly in the electronics industry t lrge, with sub-sectors such s computers, office mchinery nd electronics nd, nd lso in the erospce nd defense sector, ech indictor ppers to cpture the innovtive performnce of compnies. Although the results of our sttisticl nlysis certinly do not dictte tht single indictor pproch is invlid for the phrmceuticl industry, composite construct tht overrches ll stges of the innovtion process could be used for the mesurement of innovtive performnce of compnies in this prticulr sector. The fctor lodings in the nlysis of this sector specify slightly higher degree of multidimensionlity where R&D inputs nd ptent counts represent somewht different spects of innovtive performnce thn ptent cittions nd new products (see lso Trjtenberg, 1990). Finlly, our findings do not imply tht there is no need for dditionl, context-informed nlyses. Future reserch could consider number of topics relevnt for understnding the mesurement of innovtive performnce. An obvious item for further reserch is to investigte the usefulness of the current indictors in other industries thn these high-tech sectors. We expect tht in number of industries with little R&D, indictors bsed on ptents nd new products could still be used lthough the ctul distribution for these mesures might be different. As prt of the continuous effort to build better theories nd improved models to understnd innovtive performnce, it is ppeling to focus on the effect tht the strtegic behviour of compnies hs on their innovtive performnce, explining how this might influence the different mesurements of innovtive performnce. Also, the inclusion of other indictors might be useful for nlysing high-tech industries, lthough it ppers s if the current line-up of mesures is quite exhustive. Furthermore, it should be cler tht the mesurement of innovtive performnce in non-mnufcturing sectors is in need of rther different set of indictors. Obviously, future reserch should consider some lterntive mesures of innovtion tht re more pproprite in the context of wide rnge of service industries. Acknowledgements The uthors would like to thnk prticipnts t the ECIS seminr t Eindhoven University of Technology, Anthony Arundel, Pierre Mohnen nd two nonymous referees for their suggestions nd comments on n erlier version of this pper.

12 1376 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Appendix A. Distribution of compnies in the smple, numbers nd % distribution for sectors nd regions Industry North Americ Others Totl Aerospce nd defense 38; 4.55%, 64.41% 21; 5.87%, 35.59% 59; 4.94%, 100% Computers nd office mchinery 227; 27.15%, 81.07% 53; 14.81%, 18.93% 280; 23.45%, 100% 152; 18.18%, 62.30% 92; 25.70%, 37.70% 244; 20.44%, 100% 419; 50.12%, 68.58% 192; 53.62%, 31.42% 611; 51.17%, 100% Totl 836; 100%, 70.02% 358; 100%, 29.98% 1194; 100%, 100% Appendix B. Correltions Vribles Totl Aerospce nd defense Computers nd office mchinery North Americ Ptents nd ptent cittions Ptents nd R&D expenditures Ptents nd new products Ptent cittions nd R&D expenditures Ptent cittions nd new products R&D expenditures nd new products Others Ptents nd ptent cittions Ptents nd R&D expenditures Ptents nd new products Ptent cittions nd R&D expenditures Ptent cittions nd new products R&D expenditures nd new products Correltion mtrix is not positive definite. Appendix C. Results for the KMO nd the MSA mesures Vribles Totl Aerospce nd defense Computers nd office mchinery KMO mesure MSA mesure R&D expenditures Ptent counts Ptent cittions New products

13 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) Appendix D. Communlities for North Americ nd others Vribles Totl Aerospce nd defense Computers nd office mchinery North Americ R&D expenditures Ptent counts Ptent cittions New products Others R&D expenditures Ptent counts Ptent cittions New products The correltion nd covrince mtrix re not positive definite. Appendix E. Results of the initil unrotted fctor nlysis Vribles Totl Aerospce nd defense Computers nd office mchinery North Americ, component 1 R&D expenditures Ptent counts Ptent cittions New products Totl Eigenvlue % vrince explined Others, component 1 R&D expenditures Ptent counts Ptent cittions New products Totl Eigenvlue % vrince explined The correltion nd covrince mtrix re not positive definite. Appendix F. Rtios for the different indictors in the smple s whole nd for sub-smples Industry Ptent counts/r&d expenditures Ptent cittions/r&d expenditures New products/r&d expenditures Aerospce nd defense Computers nd office mchinery Totl Note: R&D expenditures re in millions of US$. New products/ptent counts

14 1378 J. Hgedoorn, M. Cloodt / Reserch Policy 32 (2003) References Acs, Z.J., Audretsch, D.B., Ptents s mesure of innovtive ctivity. Kyklos 4, Ahuj, G., Ktil, R., Technologicl cquisition nd the innovtive performnce of cquiring firms: longitudinl study. Strtegic Mngement Journl 22, Albert, M.B., Avery, D., Nrin, F., McAllister, P., Direct vlidtion of cittion counts s indictors of industrilly importnt ptents. Reserch Policy 20, Archibugi, D., Ptenting s n indictor of technologicl innovtion: review. Science nd Public Policy 6, Arundel, A., Kbl, J., Wht percentge of innovtions re ptented? Experimentl estimtes in Europen firms. Reserch Policy 27, Aspden, H., Ptent sttistics s mesure of technologicl vitlity. World Ptent Informtion 5, Bsberg, B.L., Ptents nd the mesurement of technologicl chnge: survey of the literture. Reserch Policy 12, Bysinger, B.D., Hoskisson, R.E., Diversifiction strtegy nd R&D intensity in multi-product firms. Acdemy of Mngement Journl 34, Bound, J., Cummins, C., Griliches, Z., Hll, B.H., Jffe, A., Who does R&D nd who ptents? In: Griliches, Z. (Ed.), R&D, Ptents, nd Productivity. University of Chicgo Press, Chicgo, pp Bresmn, H., Birkenshw, J., Nobel, R., Knowledge trnsfer in interntionl cquisitions. Journl of Interntionl Business Studies 30, Brouwer, E., Kleinknecht, A., Innovtive output, nd firm s propensity to ptents: n explortion of CIS micro dt. Reserch Policy 28, Cntwell, J., Hodson, C., Globl R&D nd UK competitiveness. In: Csson, M. (Ed.), Globl Reserch Strtegy nd Interntionl Competitiveness. Blckwell Scientific Publictions, Oxford, pp Crpenter, M.P., Nrin, F., Woolf, P., Cittion rtes to technologiclly importnt ptents. World Ptent Informtion 3, Cincer, M., Ptents, R&D, nd technologicl spillovers t the firm level: some evidence from econometric count models for pnel dt. Journl of Applied Econometrics 12, Cohen, W.M., Levin, R.C., Empiricl studies of innovtion nd mrket structure. In: Schmlensee, R., Willig, R. (Eds.), Hndbook of Industril Orgniztion, vol. 2. Elsevier, Amsterdm, pp Devinney, T.M., How well do ptents mesure new product ctivity? Economic Letters 41, Dosi, G., Sources, procedures, nd microeconomic effects of innovtion. Journl of Economic Literture 26, Duysters, G., Hgedoorn, J., Do compny strtegies nd structures converge in globl mrkets? Evidence from the computer industry. Journl of Interntionl Business Studies 32, Ernst, H., Ptent pplictions nd subsequent chnges of performnce: evidence from time-series cross-section nlyses on the firm level. Reserch Policy 30, Freemn, C., Soete, L., The Economics of Industril Innovtion. Pinter, London. Griliches, Z., Ptent sttistics s economic indictors: survey. Journl of Economic Literture 28, Griliches, Z., R&D nd Productivity: The Econometric Evidence. The University of Chicgo Press, Chicgo. Grupp, H., The mesurement of technicl performnce of innovtions by technometrics nd its impct on estblished technology indictors. Reserch Policy 23, Hgedoorn, J., The Dynmic Anlysis of Innovtion nd Diffusion. Pinter, London. Hgedoorn, J., Duysters, G., The effect of mergers nd cquisitions on the technologicl performne of compnies in high-tech environment. Technology Anlysis nd Strtegic Mngement 13, Hir, J.F., Anderson, R.E., Tthm, R., Blck, W., Multivrite Dt Anlysis with Redings. Prentice-Hll, Englewood Cliffs, NJ. Hll, B.H., The impct of corporte restructuring on industril reserch nd development. Brookings Ppers on Economic Activity 3, Hll, B.H., Griliches, Z., Husmn, J., Ptents nd R&D: Is there lg? Interntionl Economic Review 27, Hll, B.H., Jffe, A., Trjtenberg, M., Mrket Vlue nd Ptent Cittions: A First Look. NBER Working Series Pper, Working Pper Hrhoff, D., Nrin, F., Scherer, F.M., Vopel, K., Cittion frequency nd the vlue of ptented inventions. Review of Economics nd Sttistics 81, Husmn, J., Hll, B.H., Griliches, Z., Econometric models for count dt with n ppliction to the ptents R&D reltionship. Econometric 52, Henderson, R., Cockburn, I., Mesuring competence: exploring firm effects in phrmceuticl reserch. Strtegic Mngement Journl 15, (specil issue winter). Hitt, M.A., Hoskisson, R.E., Irelnd, R.D., Hrrison, J.S., Effects of cquisitions on R&D inputs nd outputs. Acdemy of Mngement Journl 34, Hitt, M.A., Hoskisson, R.E., Johnson, R.A., Moesel, D.D., The mrket for corporte control nd firm innovtion. Acdemy of Mngement Journl 39, Hitt, M.A., Hoskisson, R.E., Kim, H., Interntionl diversifiction: effects on innovtion nd firm performnce in product-diversified firms. Acdemy of Mngement Journl 40, Jffe, A., Trjtenberg, M., Henderson, R., Geogrphic locliztion of knowledge spillovers s evidenced by ptent cittions. Qurterly Journl of Economics 108, Krki, M.M.S., Ptent cittion nlysis: policy nlysis tool. World Ptent Informtion 19, Kleinknecht, A., Mesuring Product nd Service Innovtion: An Assessment of Alterntive Indictors. Rotterdm Institute for Business Economic Studies, Ersmus Universiteit, Rotterdm, The Netherlnds. Kondo, M., R&D dynmics of creting ptents in the Jpnese industry. Reserch Policy 28,