HYPERSPECTRAL IMAGING A NOVEL NON- DESTRUCTIVE ANALYTICAL TOOL IN PAPER AND WRITING DURABILITY RESEARCH

HYPERSPECTRAL IMAGING A NOVEL NON- DESTRUCTIVE ANALYTICAL TOOL IN PAPER AND WRITING DURABILITY RESEARCH 1 J.H. Scholten, 1 M.E. Klein, 2 Th. A.G. Steemers, 2 G. de Bruin 1 Art Innovation BV, Zutphenstraat 25, 7575 EJ Oldenzaal, The Netherlands Phone: +31 541570720; Fax: +31 541570721; Website: www.art-innovation.nl 2 Nationaal Archief, Den Haag, The Netherlands Reference Author: M.E. Klein, E-mail: info@art-innovation.nl, Fax: +31 541570721 Hyperspectral imaging (HSI) is a novel technique for fast, simple and truly non-destructive analysis of historic documents. HSI provides a series of calibrated digital images of a document at a large number (50-100) of different, well-defined optical wavelengths in the ultra-violet, visible and near-infrared. Through a quantitative analysis of these spectral reflectance images, HSI may be used, in particular, for identifying various types of ink and for mapping their distribution on the paper in order to determine the condition and enhance the legibility of a historic document. We discuss the applicability of various HSI concepts in paper and writing durability research and the setup and functionality of the instrument we are currently developing. Using a laboratory setup of this HSI instrument operating in the 380 1100 nm wavelength range, we performed initial experiments on distinguishing and mapping of inks and enhancing the visibility of hidden features on historic documents. The unique combination of high spectral (10-15 nm) and high spatial (<0.1 mm) resolution obtained from these HSI measurements is exploited using contrast enhancing and mapping software algorithms, whose numeric results are visualized with false-colour images. 1 / 14

1. Introduction Non-destructive optical techniques have always belonged to the most important investigation methods applied in paper and writing durability research. Visual inspection of a document is a fast and inexpensive way of detecting critical areas, where e.g. ink corrosion occurs in an advanced state, or where discolouration or staining jeopardizes the legibility. From visual inspection alone, the experienced researcher can get a qualitative impression of the general condition of the inspected material. For example, based on visible characteristic features of ink writing under daylight and UV fluorescence conditions a model has been developed for rating the condition of documents, and giving directions for storage and handling according to their condition [1]. However, methods based on visual inspection have several severe disadvantages. Firstly, the rating of the document condition is inherently subjective, as it relies heavily on the individual visual capabilities and on the experience of the person at the time when inspecting the document. Secondly, the features determining the overall condition of a document are not recorded in a quantitative way, so that an objective comparison with other documents, or the same document at a later time is not possible. And thirdly, visual inspection is limited by the variation of the sensitivity of the human eye over the visible spectral range (and its insensitivity e.g. in the near-infrared), and by the normally insufficient control of the illumination conditions. Therefore, visual inspection utilizes only a tiny fraction of the optical information that can be retrieved from a historic document. For a quantitative analysis and comparison of (local) material properties, and their relation e.g. to paper degradation and document authenticity, researchers have successfully employed a number of sophisticated optical techniques, including conventional reflectance [2] and fluorescence spectroscopy [3], high-resolution infrared Fourier transform spectroscopy [4], as well as laser-based spectroscopy [5]. These techniques enable an accurate measurement of material properties such as the spectral reflectance, however, most of them are per se non-imaging. This means that each measurement represents the material properties only at a single object point, averaged over an area ranging in size from sub-square millimeter to several hundred square millimeters. This results in a number of major problems that are typical for such single-point measurements. First, in order to obtain meaningful results, one has to ensure that the sampled area has a homogenous response, or else the optical data e.g. from a writing is diluted with background data from the paper substrate. 2 / 14

Second, one can never be sure that the sampled area is representative for the entire document. In addition, it is extremely difficult to make repeated measurements at exactly the same location on the document, in order e.g. to determine the progress in degradation or the influence of an applied treatment. Although a single-point measurement technique can in principle always be converted into an imaging technique by scanning the measurement area over the object, the extremely long measurement time required for a sufficiently high spatial resolution make the scanning approach in most cases totally impractical. As compared to this, imaging systems based on digital cameras have the advantage of yielding optical information over a significant object area with high a spatial resolution. However, most of the currently used digital imaging techniques are restricted to visible colour imaging, i.e. they record at three more or less independent wavelength bands in the visible range. Using an advanced multi-spectral imaging system, such as the ARTIST camera [6], a number of additional spectral channels in the near UV and infrared, i.e. outside the range of human perception, become available to high-spatial resolution digital recording. While overcoming the limitations of single-point techniques and of conventional digital colour imaging, at present multi-spectral imaging systems are almost exclusively used in a non-quantitative way. Even in cases where multi-spectral imaging is used quantitatively, the comparatively small number of wavelength bands (about 5-10) and their bandwidth of typically 100 to 200 nm are not suitable for analytical reflectance spectroscopy. We are currently developing a so-called hyperspectral imaging instrument for truly nondestructive paper and writing durability research, which combines the high spatial resolution of a digital camera with the large number of wavelength bands required for high-resolution spectral reflectance and accurate colour measurements. The instrument is to be used for application studies aiming at a reliable identification of different types of ink, early-detection of ink corrosion and local substrate discolouration, and an objective quantification of the resulting degree of document degradation. In this contribution, we discuss the operating principle of this so-called hyperspectral imaging system and we present initial experimental results obtained with a laboratory setup. These first measurements already indicate the huge potential of the hyperspectral imaging technique for distinguishing and mapping different types of inks, in order to quantify areas of paper degradation and enhance the legibility of faint writings. 3 / 14

2. The operation principle of hyperspectral imaging The term hyperspectral imaging (HSI) refers to the acquisition of a series of digital images at a large number (50-100) of different, well-defined optical wavelengths in the ultra-violet, visible and near-infrared. By applying proper calibration procedures, HSI results in the simultaneous, precise determination of the reflectance spectra from all areas of a document with a high spatial resolution, which is also referred to as optical reflectance imaging. In order to determine the spectral reflectance independently for the large number of narrow spectral bands, wavelength tunable spectral filtering has to be applied somewhere in the light path between the broadband illumination light source and the broadband image sensor. As indicated in Fig. 1, there are two basic concepts, namely either to apply spectral filtering in the imaging light path (Fig. 1A) or in the illumination light path (Fig. 1B) [7]. For hyperspectral imaging of delicate, light-sensitive objects such as historic documents, the latter concept has a decisive advantage: at any time during a measurement, the object is only illuminated with the minimum amount of light, which is required for recording a spectral image. This is only a small fraction (typically in the order of one-hundredth) of the light power incident on the object in the case illustrated in Fig. 1A, where the object is illuminated with all spectral components simultaneously and only the reflected light is filtered. For developing our hyperspectral imaging instrument for historic documents, we therefore chose concept 1B, where no degradation of the document is to be feared. The setup of the hyperspectral imaging system that we are currently developing is shown schematically in Fig. 2. The document under investigation is placed in a light-proof cabinet and illuminated from two sides under an angle of 45º with two identical, wavelength tunable light sources. These TULIPs (TUnable Light Projectors) project monochromatic light (spectral bandwidth 10-20 nm), the center wavelength of which can be tuned in steps of 10 to 20 nm (corresponding to the bandwidth) over the entire wavelength range from 380 to 1100 nm. A high-resolution monochrome CCD camera takes at each wavelength an image of the document. The wavelength tuning of the light sources and the image acquisition of the CCD camera is controlled and synchronized by a computer. The spatio-spectral illumination and detection efficiency of the system is calibrated by taking reference images at all wavelengths with a standard target with spatially absolutely homogenous reflectance properties and exactly known spectral response. Using this calibration data, dedicated software converts the raw image data from the historic document into calibrated reflectance images. The entire set of 4 / 14

these reflectance images corresponds to the so-called hyperspectral data cube, which contains for each image pixel a complete spectral reflectance curve. For applications such as mapping the distribution of different types of inks on a document, the hyperspectral instrument has to be able to resolve also fine features of thin ink handwritings, ink drawings, prints, etc., in order to obtain undiluted spectral reflectance data without interference from the surrounding paper surface. Therefore, our hyperspectral imaging instrument is designed for a resolution of more than 300 dpi, so that under ideal circumstances pure spectral data of lines thinner than 0.1 mm can be obtained. The spectral data contained in the visible wavelength portion of the hyperspectral data cube can be used e.g. for calculating CIE colour values for any standard illuminator at each point on the document. This can help to recognize, quantify and record visible changes, such as yellowing and staining of the document. In the research that we are going to perform with the instrument, all calibrated reflectance images contained in the hyperspectral cube (i.e. also outside the visible wavelength range) will be analysed in order to find spectral signatures, which can be related e.g. to different types of inks, ink corrosion, or other types of paper degradation. 3. Initial experimental investigations For initial experiments, we built a laboratory hyperspectral imaging setup, where a document area of about A8 size was illuminated with a single wavelength tunable light source (bandwidth <15 nm, tuning range 380 1100 nm in steps of 10 nm), under an angle of ~45. A modified version of our ARTIST camera was used to image the document area at each wavelength. Each of the 1360 1036 camera pixels represents to an area of 55 55 µm on the document, corresponding to a spatial sampling density of 460 dpi. 3.1 Mapping of different types of ink The first experiment aimed at investigating the differences in the spectral reflectance curves of different types of inks, and exploiting such differences for mapping the individual distributions of these inks on the document. We overlapped two manuscripts in the field-ofview (FOV) of the camera and imaged them at 57 different wavelengths in the range of 540 to 1100 nm (step width 10 nm). From the calibrated reflectance images shown in Fig. 3, it can be seen that the writings in both documents have a high contrast at a wavelength of 570 nm, 5 / 14

however, at longer wavelength the contrast in the lower document is considerably reduced while the writing in the upper document virtually disappears at 1000 nm. For a quantitative spectral analysis, we extracted from the hyperspectral image series the spectral reflectance curves at 4 small ink areas (size 10 10 pixels corresponding to ~0.3 mm 2 ) in each document. As shown in Fig. 4, the spectral reflectance curves of these 8 regions of interest (ROI) clearly fall into two groups. ROIs 1-4 in the upper document exhibit very low reflectance values at short wavelengths, and a steep increase at >700 nm. As compared to this, the ROIs 5-8 from the lower document feature somewhat higher reflectance values at short wavelength, and a more gradual increase towards longer wavelengths. These spectral signatures of the two writings were exploited for mapping their distributions in the false colour image shown in Fig. 3. This was generated by calculating for each pixel the ratio of the spectral reflectance values at 840 and 570 nm, and choosing blue colour to indicate ratio values 4, red for ratio values <4 but >1, and gray tones for ratio values <1. By applying this very simple algorithm, the resulting false-colour image shows the writing in the upper document mainly as blue pixels, and in the lower document only as red pixels. This demonstrates the potential of the hyperspectral imager for distinguishing between different inks and mapping their distribution. 3.2 Enhancing legibility using the Spectral Angle Similarity (SAS) technique The second experiment with the laboratory setup aimed at investigating the potential of the hyperspectral imaging technique to detect hidden features and/or enhance their legibility. Our sample was a paper document dated 1899, which has ink handwriting in Dutch and English language on the (imaged) front side and on the reverse side, respectively. On the front side, one can see faint features of a purple colour, which turned out to be machine written words transferred (contact-copied) from another document that had been stored in contact with our sample document. In order to make these transferred words (the target feature) readable, we applied the hyperspectral imaging technique in the following way. We took 73 calibrated reflectance images of a section (size 75 57 mm 2 ) of the document in the wavelength range from 380 to 1100 nm (step width 10 nm). Fig. 5A shows the resulting grayscale reflectance images at the wavelength 580 nm mirrored at its vertical axis, in order to compensate the effect of the transfer, so the target text runs now from left to right. Two lines of the target text are marked with red boxes. 6 / 14

In order to exploit the hyperspectral information for contrast enhancement, 6 ROIs were defined in document areas with the pure paper substrate (3 ROIs), with a letter of the targeted transferred text, with ink handwriting on the document reverse and on the front side, respectively (1 ROI each). For these ROIs the entire spectral reflectances curve were extracted from the hyperspectral data cube, as shown in Figure 6. The reverse (blue line) and front side (red line) ink handwritings show a much lower reflectance than paper (black lines) over the entire visible and most of the infrared range, corresponding to their high contrast, dark appearance for the human eye. As opposed to this, the reflectance curve of the target text deviates in the visible only slightly from typical paper curves, and not at all in the infrared. The (faint) purple colour appearance of the target text is caused by the reflectance dip in the green-yellow band (leaving high blue and red reflectances), which has a minimum around 580 nm. At this wavelength, the grayscale reflectance image has the highest contrast for the target text, which is why this had been selected for Fig. 5A. From the visible portion of the reflectance curve (i.e. 380 780 nm), one can calculate the CIE colour values for any combination of colour space and standard illuminator [8]. As a quantitative measure of the visibility of the target feature, we calculated the colour value difference?e that the target ROI would have with respect to the uncovered paper substrate, if a standard illuminant D55 light source was used for illuminating the document (CIE Lab colour space, 2 degree 1931 observer). The result is a colour difference value of about?e = 5 for the selected letter of the target feature, which still has a relatively good visibility. For making more of the targeted transferred text readable, we computed its distribution on the document with the so-called spectral angle similarity (SAS) mapping technique [9]. This technique is based on the assumption that it is mainly the relative strengths of the spectral reflectances, not their absolute values, which corresponds to the probability that a given pixel belongs to the target feature or is part of the background. In other words, the SAS technique compares the shapes of the spectral reflectance curves of the pixel and the target feature, and ignores differences in the total reflectance value integrated over the entire spectrum. In particular, the SAS technique treats any set of spectral reflectances (belonging to the target feature or to an image pixel) as the components of a vector in a multi-dimensional space. Thus, the spectral properties of the target feature and of each each image pixel are represented, according to its own set of spectral reflectances, as individual vectors in that space. The length of each vector corresponds to the total reflectance and the (multi- 7 / 14

dimensional) pointing direction is given by the relative values of the spectral reflectances, i.e. the shape of the spectral reflectance curve. According to the assumption of the SAS technique, the vectors of image pixels that point in similar directions as the target spectral vector are likely to belong to the target feature. The SAS mapping technique computes the angle between each pixel spectral vector and the target spectral vector, using the canonical mathematical definition of the angle between two multidimensional vectors. An image pixel, for which the calculation yields a small angle value, probably belongs to the target feature namely the hidden text. For defining the spectral target vector and the pixel spectral vectors, we used a set of 20 spectral reflectance values in the wavelength range from 460 to 710 nm, where the contrast between target and background pixels (paper substrate, handwriting) is highest. The distribution of the spectral angles as calculated in this 20-dimensional space was rendered into the false colour image shown in Fig. 5B in the following way: First, a grayscale image was generated, where dark pixels correspond to small, and bright pixels to large spectral angles, respectively. The visibility of features defined by the distribution of the dark pixels was enhanced, by marking all pixels with a spectral angle below a certain threshold value green. The threshold value of the spectral angle was determined by trial and error, in order to maximize the readability of the target text on the computer screen. In the false colour mapping image Fig. 5B, the potential of hyperspectral imaging for analyzing historic documents is clearly demonstrated. The just-visible target text does not only become quite well readable, but even fainter copies of the same text show up. Using the visible portion of the hyperspectral cube, we calculated between the faintest, readable letters and the surrounding paper substrate a colour difference (CIE Lab, 2 degree 1931 observer, standard illuminant D55) of about?e = 1, which is similar to the colour variation of the paper substrate itself and at the border of human colour vision, even under optimal circumstances. Note that in the false-colour mapping image, several faint versions of the same text have appeared, which are slightly displaced from the best-visible version. For example, words of the line I have, sir, the honor to transmit show up in four different places, which means, that the stack of documents had not been undisturbed, but the document from which this text copied had been moved several times with respect to our sample document. These initial experiments thus indicates the high potential of the hyperspectral imaging technique for enhancing the legibility of faint texts, which may have important applications in historic research and in forensic science. 8 / 14

4. Summary and conclusion In summary, we report on the development of a hyperspectral imaging instrument dedicated to research in paper and writing durability. The wavelength-dependent reflectance is measured with high spatial resolution by illuminating documents with a wavelength tunable, monochromatic light source and imaging with a high-resolution digital camera. The advantage over alternative approaches is that the total light intensity on the document is minimal at any time, and that there is no risk of heat or UV radiation induced damage. Initial experimental results obtained with a laboratory setup show that such an instrument can be used successfully for distinguishing between different types of ink. By exploiting the spectral information extracted from the hyperspectral data cube curves of two types of ink, a false colour image could be generated, where the distribution of two types of ink within the camera field of view is mapped with two different colours. In a second experiment, the Spectral Angle Similarity (SAS) technique was applied to the hyperspectral data set to achieve a considerable enhancement of the legibility of a text part of which was virtually invisible for the naked eye even under optimum circumstances. In conclusion, our investigations indicate that the hyperspectral imaging instrument has a huge potential to become a standard research tool for a fast, non-destructive, spatially resolved analysis of historic documents. 5. References 1. B. Reißland, Iron-Gall Ink Corrosion Progress in Visible Degradation, in: Tennent, N.H. and Mosk, J., eds., Contributions of the Netherlands Institute for Cultural Heritage to the field of conservation and research, Amsterdam, ICN (2000) 2. M. Missori, M. Righini, S. Selci, Optical reflectance spectroscopy of ancient papers with discoloration or foxing, Optics Communications 231 (2004) 99-106 3. M. Bicchieri, G. Pappalardo, F.P. Romano, F.M. Sementilli, R. De Acutis, in: K.G. Saur (ed.), Characterization of Foxing Stains by Chemical and Spectrometric Methods, Restaurator 22, Verlag GmbH, Munich, Germany, 2001, p. 1-19 4. Choisy, P., De La Chapelle, A.; Thomas, D.; Legoy, MD, in: K.G. Saur (ed.), Non Invasive Techniques for the Investigation of Foxing Stains on Graphic Art Material, Restaurator 18, Verlag GmbH, Munich, Germany, 1997, p 131-152. 9 / 14

5. L. Burgio, and R. J. H. Clark, Identification of pigments by Raman microscopy: Relevance to the authentication or otherwise of Egyptian papyri, in: C. Fotakis, T. Papazoglou, C. Kalpouzos, eds., Optics and Lasers in Biomedicine and Culture OWLS V, Springer-Verlag Berlin Heidelberg 2000, p.201 6. J.B.G.A. Havermans, H. Abdul Aziz, H. Scholten, Non destructive detection of irongall inks by means of multispectral imaging, Part 2: Application on original objects affected with iron-gall-ink corrosion, in: Restaurator; volume 24 (2003), p. 88-94 7. Note that in cases, where the imaged object converts certain wavelengths of the incident light into different wavelengths (e.g. due to fluorescence), spectral filtering has to be applied in both light paths in order to obtain valid spectral reflectance data for all wavelength bands independently. 8. R.W.G. Hunt, Measuring Colour, Fountain Press Kingston-upon-Thames England, Third Ed. 1998, 344 pages 9. S. Homayouni, and M. Roux, Hyperspectral image analysis for material mapping using spectral matching, Proceedings of the XXth ISPRS Congress, 12-23 July 2004 Istanbul, Turkey, Commission 7 papers, in: Proceedings volume IAPRS, Vol.XXXV, ISSN 1682-1750; url: http://www.isprs.org/istanbul2004/index.html 10 / 14

monochrome camera monochrome camera white light spectral filter spectral filter monochromatic light 1A 1B Figure 1: Two basic concepts for realizing a hyperspectral imaging system. Either, narrowbandwidth wavelength filtering is applied in the imaging light path (1A), or in the illumination light path (1B). to computer monochrome CCD camera wavelength tunable light source wavelength tunable light source document Figure 2: Schematic setup of a hyperspectral imaging system to be used for paper and writing durability studies. 11 / 14

Figure 3: Calibrated relfectance images for wavelengths 570 nm, 840 nm, 1000 nm, and false colour image generated by ratioing the reflectance images at 840 nm and 570 nm. Blue: ratio 4; red: 4 > ratio >1; gray: ratio >1. spectral reflectance (a.u.) 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 570 nm 840 nm ROI 5-8 { 1-4 { 500 600 700 800 900 1000 1100 optical wavelength (nm) Figure 4: Spectral reflectance curves of ink areas in the upper manuscript (ROIs 1-4, blue) and in the lower manuscript (ROIs 5-8, red) shown in Fig. 2. The differences in the curves can be exploited for distinguishing between the different writings. 12 / 14

Figure 5A: Spectral reflectance image at 580 nm, where the target feature shows the highest contrast. To improve the readability of the target feature, which is a machine written text transferred (contact-copied) from a document stored together with our sample, the image was mirrored horizontally, and two fairly well readable target text lines are marked with red boxes. Figure 5B: As shown in green, hidden parts of the target text were made visible by exploiting the hyperspectral image information with the spectral angle similarity (SAS) algorithm. 13 / 14

1.0 optical reflectance (norm.) 0.8 0.6 0.4 0.2 0.0 paper substrate target text ink, reverse ink, front side 400 500 600 700 800 900 1000 1100 wavelength (nm) Figure 6: Normalized optical reflectance spectra for 6 ROIs, representing the paper substrate (black lines), the targeted text feature (green), ink handwriting on the reverse (blue) and on the front side (red) of the document. 14 / 14