IN their seminal paper on knowledge spillovers, Jaffe,

PATENT CITATIONS AS A MEASURE OF KNOWLEDGE FLOWS: THE INFLUENCE OF EXAMINER CITATIONS Juan Alcácer and Michelle Gittelman* Abstract Analysis of patent citations is a core methodology in the study of knowledge diffusion. However, citations made by patent examiners have not been separately reported, adding unknown noise to the data. We leverage a recent change in the reporting of patent data showing citations added by examiners. The magnitude is high: two-thirds of citations on the average patent are inserted by examiners. Furthermore, 40% of all patents have all citations added by examiners. We analyze the distribution of examiner and inventor citations with respect to self-citation, distance, technology overlap, and vintage. Results indicate that inferences about inventor knowledge using pooled citations may suffer from bias or overinflated significance levels. I. Introduction IN their seminal paper on knowledge spillovers, Jaffe, Trajtenberg, and Henderson (1993, p. 578) write that Krugman... perceives that [k]nowledge flows... are invisible; they leave no paper trail by which they may be measured and tracked... But knowledge flows do sometimes leave a paper trail, in the form of citations to patents. Since that pioneering work, patent citations have been utilized extensively to measure the diffusion of knowledge across a variety of dimensions: geographic space, time, technological fields, organizational boundaries, alliance partnerships, and social networks (see, for example, Almeida & Kogut, 1999; Peri, 2005; Gomes-Casseres, Jaffe, & Hagedoorn, 2006; Jaffe & Trajtenberg, 2002). The principal assumption driving this research is that citations trace out knowledge flows and technological learning: a citation from patent B to patent A indicates that inventors on B knew about and used A in developing B. However, patent examiners government agents who approve patent applications are also involved in drafting the contents of patents, and their citations are unlikely to reflect knowledge flows. In a survey of inventors, Jaffe, Trajtenberg, & Fogarty (2000) show that the influence of examiners on citations is considerable, and that inventors were fully aware of less than one-third of the citations on their patents. Until recently, examiner citations were not separately reported from inventor citations, so the empirical literature has proceeded under the assumption that examiners add noise to Received for publication August 19, 2004. Revision accepted for publication November 17, 2005. * Stern School of Business, New York University. We acknowledge the helpful comments of Daron Acemoglu, Luis Cabral, Wes Cohen, Iain Cockburn, Bill Greene, Tom Hemnes, Glenn Hoetker, Anita McGahan, Kevin Oliver, Gonçalo Pacheco-de-Almeida, Joe Porac, Claire Preisser, Rob Salomon, Rachelle Sampson, Til Schuermann, Brian Silverman, Edlyn Simmons, Jasjit Singh, Scott Stern, Don Walters, Bernard Yeung, Minyuan Zhao, Rosemarie Ziedonis, two anonymous reviewers, and seminar participants at the University of Michigan, NYU, Washington University, the 2004 NBER Summer Institute, the 2004 Strategy Research Forum, and the 2004 Academy of Management. Thanks also to our research assistants Nisha Bhalla, Jignasa Doshi, Jack Nguyen, Aakash Patel, and Neeti Shah. Errors remain our own. inventor citations, but that pooled citations nonetheless provide a good signal of knowledge flows. We test whether patent citations provide a good measure of knowledge flows, or whether inferences about knowledge flows made from the pooled data would change if examiner citations were removed. We utilize a recent (2001) change in the reporting of U.S. patent data that shows whether each citation on a patent was added by inventors or examiners. We find that examiners are responsible for 63% of citations on the average patent. The common assumption in the literature is that examiners introduce noise into inventor citations, and that their additions are unrelated to inventor citations. To test this, we estimate the probability that a citation is generated by an examiner or an inventor, conditional on a set of variables that are frequently employed in the empirical literature, and draw implications of our findings for results using pooled data. Examiner citations introduce bias for some variables; for others, bias is not an issue, because examiner and inventor distributions track each other closely. More broadly, we show that common assumptions about the differences between examiner and inventor citations are not borne out by the data for a number of variables, including self-citation and geographic distance. Our results contribute to the emerging literature on the nature of knowledge contained in citation data and institutional factors that shape patent practices (Cockburn, Kortum, & Stern 2004; Lemley, 2005; Sampat, 2005; Thompson & Fox Kean, 2005). II. Statistical Implications of Alternative Distributions of Inventor and Examiner Citations We begin with an assumption that is common in much of the literature on knowledge diffusion: that inventors knowledge of technological antecedents is imperfect specifically, that inventors are more aware of closer technologies than they are of more distant technologies, even after controlling for any actual localization of technology. Examiners, in contrast, are specialists in their technological fields, and are presumed to be more objective in their knowledge of the relevant prior art. Figure 1A shows a hypothetical distribution of inventor citations; the x-axis measures distance between a citing and cited patent, where distance can be conceptualized across a variety of dimensions: geographic, temporal, social, organizational, or technological. We describe two possible scenarios by varying examiners behavior with respect to inventor citations. In the first scenario, examiners fill gaps, adding citations that the inventor has left out, deliberately or otherwise. Under an assumption of localized knowledge, the true distribution The Review of Economics and Statistics, November 2006, 88(4): 774 779 2006 by the President and Fellows of Harvard College and the Massachusetts Institute of Technology

PATENT CITATIONS AS A MEASURE OF KNOWLEDGE FLOWS 775 FIGURE 1. DISTRIBUTIONAL ASSUMPTIONS FOR AGGREGATE, INVENTOR, AND EXAMINER CITATIONS of possible citations is less localized than those cited by inventors; therefore, examiners add more distant citations, as shown in figure 1B. This is the most common assumption made about inventor and examiner citations in the literature (Jaffe et al., 1993). Statistically, the gap-filling scenario would bias estimates of inventor knowledge. The direction of the bias, however, works against a finding of localization in the pooled data. If significant localization effects are nonetheless found, this pattern will increase confidence in the finding. 1 Figure 1C shows a second scenario, in which inventor citations remain the same, but examiner citations closely track the distribution of inventor citations. In this scenario, no bias arises from using the pooled data, although standard errors could change if only inventor citations were used, with potentially adverse consequences for inferences made from the pooled data. Moreover, identical distributions raise theoretical questions about the degree to which citations are representative of knowledge flows. Such a pattern suggests that inventors search globally and that their citations anticipate, with some error, what examiners would add. This is a plausible assumption, for most firms employ patent attorneys many of whom were formerly patent examiners to draft patent applications and to maximize the chances of an approval by the examiner. In our data (described below) 93% of citation dyads list an attorney on the citing patent s front page. In figure 2, we plot the distribution of examiner and inventor citations in geographic space, using actual data on all citation dyads occurring between January 2001 and August 2003. The x-axis measures miles between the locations of inventors on each citing-cited dyad. The graph strongly supports a tracking scenario, and the similarity in the shapes is remarkable, given the multimodal distributions of each group. The pattern casts doubt on the assumption that inventor and examiner citations are unrelated. However, portions of the graph do show differences, and we explore these further, along with other measures of distance, below. III. Data and Empirical Analysis We collected the front page images of all patents granted between January 2001 and August 2003, yielding 442,839 citing patents and 5,434,483 cited patents. The magnitude of examiner citations is high: examiners account for some 40% of all citing-cited dyads; on the average patent, 63% of citations are added by the examiner. A substantial proportion of citations do not contain any signal of inventor knowledge: approximately 40% of citing patents have all citations imposed by examiners. Only 8% of patents have no examiner-added citations. 2 Our approach is to test for distributional differences between examiner and inventor citations along a variety of dimensions of interest to the knowledge diffusion literature. To do this, we estimate the following model: Prob Y ij 1 X ij F X ij ε i, (1) where Y ij is equal to 1 if the citation of patent i to patent j is imposed by the examiner, and 0 if it was added by the inventor, and X ij is a vector of variables that identify a range of characteristics specific to each citation dyad. 2 See Alcácer and Gittelman (2004) for more detailed information on the data set. FIGURE 2. K-DENSITY GRAPH FOR DISTANCE IN MILES: EXAMINER VERSUS INVENTOR CITATIONS 1 The standard approach in localization studies has been to use a case-control method to compare the geographic distribution of actual and potential citations. Our analysis of examiner citations cannot say how those results would change if examiner citations were removed, because they depend on the distribution of the control sample. Thompson and Fox-Kean (2005) show that the results are sensitive to selection of the control groups.

776 THE REVIEW OF ECONOMICS AND STATISTICS TABLE 1. VARIABLE DEFINITIONS Distance Variable Name Definition Organizational/ individual same_examiner_all dif_company same_inventor 0 if all examiners in citing and cited patents are different, 1 otherwise 1 if assignees for citing and cited patents are different, 0 otherwise 1 if at least one inventor has the same name in citing and cited patents Temporal years Number of years between the grant dates of citing and cited patents post-grant citation 1 if cited patent was granted after citing patent was applied for Technological dif_technology 0 if there is at least one common IPC code in citing and cited patents, 0 otherwise Geographic distance Minimum distance in miles for all pairs of inventors (A,B) where A is inventor in citing patent, B in cited patent distance[0, 25 miles] Equal to 1 if the distance for all pairs of inventors (A,B) are less than 25 miles, 0 otherwise distance(25, 50 miles] Equal to 1 if the distance for all pairs of inventors (A,B) are between 25 and 50 miles, 0 otherwise distance(50, 100 miles] Equal to 1 if the distance for all pairs of inventors (A,B) are between 50 and 100 miles, 0 otherwise dif_country_all Equal to 1 if all inventors are in different countries, 0 otherwise dif_state_all Equal to 1 if all inventors are in different states, 0 otherwise dif_ea_all Equal to 1 if all inventors are in different economic areas, 0 otherwise Patent data contain a great many inconsistencies and errors; construction of our variables requires extensive cleaning of the data to identify matching elements, such as individual names. 3 To reduce this to a manageable task, we created a random sample of 1,500 citing patents. After removing incomplete and unusable records, our data set consists of 1,456 patents and 16,095 citations. 4 We construct several variables that measure different dimensions of distance between citing and cited patents: geographic distance, common inventors, organizations, technology fields, and patent examiners. We also include a variable measuring the years elapsed between the grant dates of cited and citing patents, adding a dummy variable (post-grant citation) for dyads where the cited patent was granted after the application date of the citing patent. These may occur because of administrative delays in the granting process, such that some citations are added after a citing patent is filed; thus, we expect they are more likely to be added by examiners. Table 1 shows how each independent variable is constructed. Because many citation dyads belong to the same citing patent, we wish to control for unobservable fixed effects at the patent level. Performing logistic regression would cause us to drop a large number of dyads (27% of our sample) belonging to citing patents that have all or no citations 3 For instance, when we clean and check assignee names, the number of unique assignees in our sample drops by 28%. See Alcácer and Gittelman (2004) for further details. 4 We randomly sampled 500 patents from each of the three years in the data to achieve a random distribution of patents across time. We remove patents that cite no prior art or have incomplete location data (44 citing patents) or that were granted before 1976 (1,776 cited patents). Standard distributional tests (such as the Mann-Whitney and Kolmogorov-Smirnov) show that our sample is random and representative of the full data set regarding number of citations per patent, percentage of examiner citations, application year, and technology classes. See Alcácer and Gittelman (2004) for further detail. added by examiners. We therefore estimate linear probability models with fixed effects at the patent level. 5 IV. Results All models are shown in table 2. We begin with selfcitations, where we expect a strong localization effect: inventors should be more likely to add citations to their own patents than examiners. Dyads are coded to indicate if citing and cited patents are assigned to different companies (diff_company), if they share common inventors (same_ inventor) or if they were granted by the same examiner (same_examiner). The coefficient for diff_company is positive and significant (p 0.05) across all models, implying that cross-firm citations are more likely to be associated with examiners than inventors. 6 This is behaviorally consistent with localized knowledge, and consequently measures using pooled data will be biased toward finding more cross-firm citations than if only inventor citations were used. The coefficient on examiner self-citation is positive and significant (p 0.01) across all models. Although it is not surprising that examiners are more likely than inventors to cite patents they previously examined, it is worth noting that examiner-added citations reflect the work histories of individual patent examiners, consistent with a prior finding that examiners frequently have favorite citations they add to prior art (Cockburn, Kortum, & Stern, 2004). Individual self-citation should presumably be one of the most direct indicators of knowledge flows; hence, it is curious that the coefficient for the variable same_inventor in model 1 is positive and significant (p 0.01), indicating that self-citations are, at the margin, more likely to be associated with examiners rather than inventors. One pos- 5 We also estimate random-effects models with controls for industry and country of the citing patent, and random-effects logistic regressions with similar results. Results are available from the authors. 6 We also estimate these models for self-citation at the corporate parent level, with similar results.

PATENT CITATIONS AS A MEASURE OF KNOWLEDGE FLOWS 777 TABLE 2. RESULTS OF LINEAR PROBABILITY MODEL WITH FIXED EFFECTS (1) (2) (3) (4) (5) (6) (7) (8) dif_company 0.079 0.094 0.079 0.078 0.071 0.08 0.051 0.042 [0.012]** [0.013]** [0.015]** [0.013]** [0.016]** [0.012]** [0.017]** [0.017]* same_examiner_all 0.075 0.072 0.102 0.075 0.102 0.075 0.107 0.107 [0.012]** [0.012]** [0.015]** [0.012]** [0.015]** [0.012]** [0.018]** [0.018]** same_inventor 0.047 0.085 0.045 0.05 0.055 0.047 0.042 0.039 [0.015]** [0.019]** [0.018]* [0.015]** [0.019]** [0.015]** [0.019]* [0.020]* dif_technology 0.039 0.039 0.04 0.039 0.008 0.039 0.03 0.031 [0.007]** [0.007]** [0.008]** [0.007]** [0.001]** [0.007]** [0.010]** [0.010]** years 0.007 0.007 0.007 0.007 0.041 0.007 0.008 0.008 [0.001]** [0.001]** [0.001]** [0.001]** [0.008]** [0.001]** [0.001]** [0.001]** distance 2.07e 06 1.41e 06 5.30e 06 [1.46e 06] [1.47e 06] [1.90e 06]** distance[0, 25 miles] 0.01 0.031 [0.010] [0.012]* distance(25, 50 miles] 0.026 0.042 [0.019] [0.027] distance(50, 100 miles] 0.015 0.026 [0.017] [0.025] dif_country_all 0.008 [0.007] dif_state_all 0.025 [0.011]* dif_ea_all 0.042 [0.012]** [0.012]** post-grant citation 0.19 0.188 0.239 0.19 0.239 0.19 0.268 0.268 [0.015]** [0.015]** [0.020]** [0.015]** [0.020]** [0.015]** [0.023]** [0.023]** Constant 0.404 0.391 0.303 0.411 0.325 0.404 0.312 0.307 [0.012]** [0.013]** [0.015]** [0.014]** [0.017]** [0.012]** [0.017]** [0.017]** Observations 16,095 15,769 10,205 16,095 10,205 16,095 7,627 7,627 Number of groups (citing) 1,456 1,454 716 1,456 716 1,456 720 720 Log likelihood 3,972.29 4,144.43 2,980.14 4,294.07 2,721.19 4,296.96 2,109.57 1,899.02 F 85.27 82.49 70.22 66.46 54.84 85.24 55.01 55.95 Dependent variable is equal to 1 if citation comes from examiner, 0 otherwise. Model 2 excludes self-citations with mobile inventors. Model 3 excludes non-u.s. citing patents. Models 4 and 5 measure distance as discrete segments. Omitted dummy variable is for pairs that are separated by more than 100 miles. Models 5, 7, and 8 include only dyads in the U.S. Standard errors in brackets. * Significant at 5%; **significant at 1%. Coefficients and standard errors for distance are shown in scientific format. sibility is that because many individuals share the same name, matching names creates false self-citations; however, the positive and significant coefficient remains even after we reestimate model 1 with a more restrictive namematching criterion. 7 Because we have difficulty accepting that inventors forget about their own past patents, we explore whether inventors strategically omit self-citations. The mobility of engineers has been shown to be a primary mechanism for generating knowledge flows and, by extension, citations across firms (Almeida & Kogut, 1999). Yet firms can prosecute ex-employees for theft of intellectual property. For this reason, mobile employees may choose to suppress citations to prior work, to avoid signaling that knowledge was transferred from one firm to another via job mobility, leaving it to examiners to find those citations. On the other hand, recent work suggests that firms 7 We create two more restrictive matching criteria: we code a dyad as a self-citation only if the individuals share both the same name and same city in the first case, and the same name and same company in the second. Results are available from the authors. patent more aggressively when threatened by knowledge leakage through employee mobility (Kim & Marshke, 2004) and by potential infringement litigation by owners of related patents (Ziedonis, 2004). In this case, employee mobility could increase the propensity to self-cite, as firms seek to establish property rights and reduce the threat of holdup by owners of related patents. We test whether inventors who change employers suppress citations to their own patents to a greater extent than inventors who do not change employers. We break out inventors self-citation dyads into those that also cite the same company and those where different assignees are listed. Excluding the latter group and reestimating our model nearly doubles the coefficient on self-citation for same_inventor, from 0.047 (model 1) to 0.085 (model 2). In other words, examiner-added self-citations are much more likely for inventors who do not change assignees than for mobile inventors. The results are consistent with our second explanation, and suggest that firms are more likely to self-cite on patents involving mobile employees.

778 THE REVIEW OF ECONOMICS AND STATISTICS A topic of interest in the diffusion literature is the rate at which knowledge flows decay (or grow) over time. The rate at which patents are cited locally fades over time, suggesting that local knowledge spillovers eventually diffuse more broadly (Jaffe et al., 1993; Peri, 2005). If examiners are more likely to add older (younger) citations, this could bias the effects of time on the citation rate. The negative and significant coefficient (p 0.01) on the variable year indicates that citations to older patents are more likely to be associated with inventors than with examiners, indicating that citations would be on average older if examiner citations were removed. This result is not driven by administrative delays, which we control for with the variable postgrant citation, where (as expected) the coefficient is positive and significant. The scope of technology cited has also been employed as a variable of interest as well as a control variable. The number and variety of technology classes listed on cited patents can be used to measure the breadth of prior art on a citing patent. The negative coefficient on diff_technology suggests that inventors add greater breadth of technologies than examiners, who are specialized by technological field. Inasmuch as examiners simultaneously classify patents and search for prior art within classes, it is plausible that they match citing and cited patents on technology class to a greater degree than inventors. Estimates of technology scope using pooled data are likely to be biased toward finding more within-class technology citations than if only inventor citations were used. We next turn to geographic distance. In models 1 through 3, distance is measured as miles between citing and cited patents. There is no significant difference between examiner and inventor citations. This is consistent with the pattern shown in figure 2. An advantage of this measure is that it can be calculated for all dyads, regardless of their locations. However, patent examination practices differ markedly across countries. In the European Union, for instance, examiners add most citations, and criteria for including citations are narrower than in the United States. Countries also differ greatly in size and communication patterns, so the likelihood of information flowing across geographic space, as well as our other distance measures, is to some extent a function of the location of the inventors. We therefore reestimate model 1 including only citing patents for which both the assignee and the first inventor show a U.S. address (model 3). Results are largely unchanged from model 1, indicating that our findings are not driven by cross-country variations. An exception is the variable distance, where the coefficient is now significant (p 0.01), indicating that geographic differences between examiner and inventor citations are more pronounced within the United States. We explore this result further below. We employ several alternative measures of distance between citing and cited patents. We break distance into discrete mileage segments, estimating models with all patents (model 4) and with only patents with U.S. assignees and inventors (model 5). The only significant coefficient is for distances under 25 miles within the United States (p 0.05). At greater distances, and in the model including all dyads (model 4), there is no significant difference between the likelihoods of examiner and inventor citation. In models 6 through 8, we measure distance in terms of discrete administrative units. In model 6 the variable diff- _country_all takes a value of 1 if no pair of inventors on the citing and cited patents were located in the same country. The coefficient is not significant, indicating that within- and across-country citations are just as likely to be added by examiners as by inventors. Estimates of cross-country citations using pooled data will not be biased, but significance levels could change if examiner citations were removed. In the models that follow we measure administrative units within the United States. 8 We report results for two frequently used units, state and economic area. 9 The results do show a gap-filling pattern, with citations to same-state and same-economic-area patents more likely to be made by inventors. This finding is consistent with those of Thompson (2004), who also finds that inventor citations are more localized than examiner citations, though by a small margin. Again, this creates bias against a finding of localization using pooled data. Overall, our findings on geography are mixed. We see far more tracking between the two citation streams than previously assumed. For specific geographic segments we find that inventor citations are more localized than examiner citations, consistent with a gap-filling pattern. However, this result is sensitive to the specification of distance and is only present for models that exclude non-u.s. dyads. V. Conclusions Researchers using patent citations to measure knowledge flows have always acknowledged that examiner citations add measurement error, and we show that this is indeed the case: in many instances, estimates using pooled citations would change if examiner citations were removed. The bias introduced by examiner citations is not necessarily bad, however; that depends on the specific hypothesis being tested. A common assumption is that examiner citations are less localized than inventor citations. Our models show that this assumption is violated in several instances (including several distance-based measures), where there is no significant difference between the distributions. In other cases, examiner citations are more localized than inventor citations, contrary to the expectation that inventors preferen- 8 In these models we only include dyads that have U.S. addresses for at least one pair of inventors. 9 We also estimated models using counties, metropolitan areas, and cities. The results are quite similar to those reported in models 7 and 8 in table 1, with the exception of same city, where the coefficient is not significant.

PATENT CITATIONS AS A MEASURE OF KNOWLEDGE FLOWS 779 tially cite proximate technologies whereas examiners citations are more comprehensive. Apart from the statistical implications, our findings raise broader questions about the meaning of patent citation data. The most challenging question is raised by our finding on individual self-citation. If examiners are more likely to add the self-citations of individuals, can a strong claim be made that citations reflect inventor knowledge? Our results suggest that self-citation may be driven in part by the threat of litigation by owners of related patents. The close tracking in geographic space between the two citation streams also suggests that inventors and examiners have similar citation patterns. In addition to examiner citations, several other unmeasured factors play into the generation of citations that are unrelated to knowledge flows and these may be driving our results. Firms and inventors patent strategically, choosing their citations with respect to potential infringement and holdup threats. Attorneys anticipate citations most likely to be added by examiners, so that examiner and inventor citations may come to resemble each other closely. Examiners and inventors exchange information during the application process, and examiners themselves are prone to biases in favor of citing particular patents. We suggest that studies that explore these behaviors and explicitly incorporate them into their design represent the best direction for this evolving literature. REFERENCES Alcácer, J., and M. Gittelman, How Do I Know What You Know? Patent Examiners and the Generation of Patent Citations, SSRN working paper (2004). Almeida, P., and B. Kogut, Localization of Knowledge and the Mobility of Engineers in Regional Networks, Management Science 45:7 (1999), 905. Cockburn, I., S. Kortum, and S. Stern, Are All Patent Examiners Equal? Examiners, Patent Characteristics, and Litigation Outcomes, in W. M. Cohen and S. Merrill (Eds.), Patents in the Knowledge- Based Economy (Washington D.C.: National Academy Press, 2004). Gomes-Casseres, B., A. Jaffe, and J. Hagedoorn, Do Alliances Promote Knowledge Flows? Journal of Financial Economics 80, 5 33 (2006). Jaffe, A., and M. Trajtenberg, Patents, Citations and Innovations: A Window on the Knowledge Economy (Cambridge, MA: MIT Press, 2002). Jaffe, A. B., M. Trajtenberg, and M. S. Fogarty, Knowledge Spillovers and Patent Citations: Evidence from a Survey of Inventors, The American Economic Review 90:2 (2000), 215 218. Jaffe, A. B., M. Trajtenberg, and R. Henderson, Geographic Localization of Knowledge Spillovers as Evidenced by Patent Citations, The Quarterly Journal of Economics 108:3 (1993), 577 598. Kim, M., and G. Marshke, Labor Mobility of Scientists, Technological Diffusion, and the Firm s Patenting Decision, Rand Journal of Economics, Summer 2005, 36(2) pp. 298 317. Lemley, Mark A., and Carl Shapiro, Probabilistic Patents, The Journal of Economic Perspectives 19:2 (2005), 75 98. Peri, G., Determinants of Knowledge Flows and Their Effect on Innovation, this REVIEW, 87:2 (2005), 308 322. Sampat, B. Determinants of Patent Quality: An Empirical Analysis, Columbia University working paper (2005). Thompson, P. Patent Citations and the Geography of Knowledge Spillovers: Evidence from Inventor- and Examiner-Added Citations, this REVIEW, Forthcoming. Thompson, P., and M. Fox Kean, Patent Citations and the Geography of Knowledge Spillovers: A Reassessment. American Economic Review 95:1 (2005), 450 460. Ziedonis, R. H., Don t Fence Me In: Fragmented Markets for Technology and the Patent Acquisition Strategies of Firms, Management Science 50:6 (2004), 804 820.