Multi-spectral Image Acquisition and Spectral Reconstruction using a Trichromatic Digital Camera System associated with absorption filters Francisco H. Imai Munsell Color Science Laboratory, Rochester Institute of Technology Abstract This report summarizes researches performed to evaluate the feasibility of using the IBM PRO\3000 Digital Camera System associated with Kodak Wratten filters to reconstruct the spectral reflectances of artwork images and analyzes the performance and discuss possible enhancements. Introduction The traditional techniques of image capture used to archive artwork in most of the museums of the world rely on conventional photographic processes. Photography has the advantages of high-resolution and optimal luminance (tone) reproduction and the disadvantage of poor color accuracy. The exception is the VASARI imaging system developed at the National Gallery, UK which employs a seven-channel multispectral 12 bit digital camera attached to a scanning device that traverses across the painting. 1 After appropriate signal and spatial processing, 20K x 20K 10-bit L*, 11-bit a* and b* encoded images result. The National Gallery has been very successful in developing colorimetric image archives and using them to provide the European community with accurate color reproductions in both soft-copy and hard-copy forms under a defined set of illuminating and viewing conditions (i.e., colorimetric color reproduction). We have an interest in drawing upon the European experiences and making two enhancements. The first is alleviating the need to scan across the painting. This will greatly reduce the cost and complexity of the image acquisition system. The second is to define images spectrally and use the spectral information to provide printed color reproductions that are close spectral matches to the original objects producing highquality color matching under different illuminations and observers. The advantages of spectral systems have been summarized by Berns 2 and Hardeberg et al. 3 Technical issues concerned with multi-spectral image acquisition have been exhaustively studied. 4-9 In particular, König and Praefcke 10 analyzed practical problems of designing and operating a multi-spectral scanner using a set of narrow-band interference filters and a monochrome CCD camera, the most common configuration for multi-spectral image capture. When using interference filters for image acquisition, a major problem is caused by the transmittance characteristic of the filters that depends on the angle of incidence. For example, in order to image a painting with horizontal dimensions of 1 meter with a distance of 2 meters between the painting and the filter, there is angle of incidence ~14 for points in the extremities. Simulations 10 have shown that this causes color differences of 2 E* ab units in relation to the image obtained at 0 angle of incidence. Another problem is that the surfaces of the interference filters are not exactly coplanar resulting in spatial shift and distortion of the captured image. We also need to consider that there are inter-reflections caused by reflections between the spectral filters and the original image, and between the interference filters and the camera lens. These technical problems make it unrealistic and impractical for image acquisition using interference filters in museums without a considerable degree of expertise in multi-spectral imaging. We believe that a conventional trichromatic digital camera combined with absorption filters can provide an alternative way to capture multi-spectral images. The spectral reflectance of each pixel of the image can be calculated by the camera signals using some non-linear iterative method. It makes the image acquisition easier and with relatively low cost since the performance-cost relation of commercial digital cameras has increased rapidly. In this research we also want to simplify the image acquisition system developed in Europe in order to avoid scanning across the painting. 1
A possible solution to the desired enhancements mentioned above can be achieved by the fusion of a high-spatial resolution lightness image with a low-spatial resolution multi-spectral image. It is possible to interpolate each pixel of each multi-spectral image to high-spatial resolution while maintaining its color information and changing the original lightness for the lightness data of the corresponding high-spatial resolution image subpixels. This can be accomplished without noticing the expected decrease of tonal resolution in the hybrid image, because the modulation of the light in the eye becomes progressively smaller as the spatial frequency increases, 11 due to optical limitations and features of the retinal mosaic. As a consequence, the chromatic channels have much lower spatial resolution than the luminance channel. This visual feature of the human eye has been applied in broadcast television, and to devise very effective compression algorithms such as JPEG. 12 In fact, pyramidal data structures that exploit the eye s contrast sensitivity can be used for efficient data storage in the proposed system. Image fusion of a multi-spectral image with a high-resolution image has been intensively researched in the field of remote sensing. 13-15 The lightness and color information can be codified respectively as L* and a*, b* in order to allow the system to be easily optimized to have the least color difference in CIELAB E* ab or E* 94. Therefore, it is important to assure that the high-spatial resolution image capture system produces very accurate L* and the multi-spectral image capturing system provides high-accurate chromatic information. Conventional photography followed by high-quality scanning results in very high spatial resolution, large dynamic range and low noise, and the photometric response of the film-scanning system can be easily determined using gray-scale targets alongside the imaged painting. A commercial digital camera with a set of filters can be used for the low-resolution chromatic data acquisition. This hybrid approach eliminates the need for scanning across artwork. We can divide the hybrid multi-spectral generation into four parts: image acquisition, spectral analysis, image fusion, and spectral reconstruction. In the image acquisition system shown in Figure 1, the lightness information, L*(x,y) is calculated for each (x,y) pixel of the high-resolution image from a scanned photograph after proper photometric and spatial calibration, where (x,y) denotes the coordinates of the high-resolution image pixels. After proper photometric and spatial calibration, the digital counts of the multi-spectral camera t i, i = 1 to m, where m is the number of filters, are used to estimate the spectral reflectance, R(x,y,λ), and the colorimetric values of the image, L*a*b*(x,y ), where (x,y ) denote the coordinates of the low-resolution image pixels. One can model multi-spectral image acquisition using matrix-vector notation. 9 Expressing the sampled illumination spectral power distribution as s 1 0 s 2 S =, (1) O 0 s n and the object spectral reflectance as r=[r 1, r 2,... r n ] T, where the index indicates the set of n wavelengths over the visible range and T the transpose matrix, representing the transmittance characteristics of the m filters as columns of F f 1,1 f 1,2 L f 1,m F = M M L M (2) f n,1 f n,2 L f n,m and the spectral sensitivity of the detector as d 1 0 d 2 D =, (3) O 0 d n then the captured image is given by t=(df) T Sr and the color vector can be represented as c=at=(x, Y, Z) T where X, Y, Z are the CIE tristimulus values. The CIELAB L*, a*, b* are given by the non-linear transformation ξ, where ξ( X, Y, Z ) = L*, a*, b*. As shown in the figure 1, the spectral reflectance R(x,y ) of the low-resolution image can be estimated using interpolation techniques such as cubic spline, 8 modified-discrete-sine-transformation (MDST), 7 or 2
spectral reconstruction methods based on statistical analyses such as principal-component analysis (PCA). 16-18 The PCA method uses a set of a priori measured reflectance-based basis functions, E(λ,k), where k denotes the basis vector. Burns and Berns compared interpolation methods with PCA and found that PCA is more accurate than interpolation methods. 19 However, König and Praefcke 7 found that certain interpolation techniques such as the smooth inverse may produce acceptable results and should be considered. The accuracy of spectral reconstruction depends on the number of basis functions. 20 The number of basis functions necessary for accurate spectral reconstruction also depends on the database used for PCA. However, 5 to 8 basis vectors seem to be sufficient for an accurate spectral reconstruction of artwork. It is possible to optimize the filters 7 but it is not considered in this stage of the research. The L*a*b*(x,y ) for the low-resolution image is calculated from the estimated spectral reflectance R(x,y ) and the performance can be compared with direct linear transformation from the digital counts of the acquired multi-spectral image. 19 As a pilot experiment the IBM PRO/3000 Digital Camera System (IBM DCS), a trichromatic digital camera with good colorimetric performance, sufficient resolution (4,920 by 3,072 pixels), 12 bits quantization and geometric stability for imaging is used. We need to test if this trichromatic digital camera combined with absorption filters can provide an alternative way to capture multi-channel images. This report will focus on the description, characterization and performance of the IBM digital camera system using Kodak Wratten filters. Figure 1. Image acquisition and spectral analysis flowchart 3
Description of the IBM DCS The IBM DCS Pro/3000 scanner 21,22 is composed by copy stand, side lights, column, scanner head and a controller as shown in Figures 2, 3 and 4 that connects to a PC and a high resolution color monitor shown in Figure 5. It also has a light for scanning transparencies on the copy stand. Figure 2. Pro/3000 copy stand and controller Figure 3. Pro/3000 side lamp Figure 4. Pro/3000 scanner head and column. Figure 5. IMB DCS PC and color monitor The IBM Pro/3000 DCS supports 3072 by 4096 or 2048 by 3072 resolution pixels by a camera mounted on a variable height copy stand with both reflective and transmissive illumination. This digital camera system has a sensor technology that samples the same pixel many times and performs a chip summation and averaging allowing much more sensitivity than conventional CCD sensors. The CCD image sensor is mounted on a moving slide located inside the scanner head. A color filter wheel with 5 positions: dark, clear, red, green, and blue, is used to filter the incoming light. There is a software that runs in the OS2 operational system called PISA95 that is used to set parameters, calibrate and image using IBM Pro/3000 DCS and produces 8 bit and 12 bit gray image as well as 8 bit color TIFF image calibrated using a Macbeth Color Chart under D50 illuminant and displayed on the IBM high-resolution monitor shown in Figure 5 with D65 white point. It is possible to set the scanner s integration rates (analogous to exposure time) for red, green and blue channels and they should be kept constant for all the calibration and imaging processes. It is also possible to extract 12 bit raw color images. 4
Experiments Some experiments were conducted to characterize the tone reproduction and camera spectral components to determine the feasibility of using this digital camera system to reconstruct the spectral reflectance of artwork paintings. 1. The effect of quantization (bit depths) on tone reproduction. In a normal operation IBM DCS yields automatically 8 bit quantization R, G, B images with dark current and spatial white correction. It is possible to extract 12 bit R,G,B with dark correction but still without white spatial correction. A 12 bit R,G,B image of a white chromalin sheet was scanned by the IBM DCS in order to perform a pixel by pixel white spatial correction for the 12 bit color image. In this experiment, a Macbeth Color Checker was imaged and the digital counts of gray patches were plotted against intensity (Y) shown in Figure 10a and 10b for red channel respectively for 8bit and 12 bit quantizations. Figure 10a. Tone reproduction curve for 8 bit Figure 10b. Tone reproduction curve for quantization 12 bit quantization 2.Characterization of the spectral components of IBM DCS The knowledge of the spectral components of the camera systems gives us more flexibility in terms of yielding simulations with different illuminants and allows us to use the camera as a spectrophotometer that gives us the spectral reflectance of each pixel of the image from the digital counts of the image. Then, the spectral characteristics of the systems should be measured (spectral power distribution of the sources and spectral sensitivity of the camera). A)Measurement of the side lamps spectral radiant power The IBM DCS uses four halogen light bulbs (Sylvania J688 ENH, 350W 120V, color temperature of 3,200 K) as side lamps. At first, the spectral reflectance factor of a sheet of white chromalin was measured by BYK-Gardner TCS-35 spectrophotometer and the obtained curve is plotted in Figure 6. 5
Figure 6 Spectral Reflectance of the sheet of white chromalin The PhotoResearch PR-704 SpectraScan Spectroradiometer was set vertically below the IBM DCS scanner head, 29.5 cm from the stand base, hold by a bridge of metal sticks mounted on tripods (strong enough to support its 14.5 kg) as shown in Figures 7a and 7b. The measured spectral radiant power of the side lamps reflected on the center of white chromalin was performed after 20 minutes warm up and plotted in the Figure 8. The camera system uses 45 / 0 geometry to digitize images and this geometry was kept during the measurements. Figure 7a Measurement of spectral radiance of the side lamps Figure 7b Positioning of SpectraScan For the measurement of the spectral radiance of the side lamps. The resulting spectral radiant power of the illuminant was calculated considering a priori measured spectral reflectance of the white chromalin surface. The absolute spectral radiant power of the illuminant is shown in Figure 9. 6
Figure 8. Measured spectra radiant power of the side lamps reflected on white chromalin Figure 9. Absolute spectral radiant power of IBM DCS side lamps B)Measurement of the transmissive light spectral radiant power There is a square hole at the top of the copy stand of IBM DCS where an integration chamber for the transmissive lamp source (Sylvania J688 ENH (350W 120V)) is inserted. A diffuser that provides a quite uniform illumination as shown in Figure 10 covers the integration chamber. A black metal with a rectangular hole of 4 by 6 inches was used to mask the illumination. The PhotoResearch PR-704 SpectraScan Spectroradiometer was set vertically with a distance of 20 cm between its lens and the copy stand top and the lens of the SpectraScan. The spectroradiometer was hold by a bridge mounted on tripods like in the previous experiment. Figure 11a, and 11b show the measured absolute and normalized spectral radiant power of the transmissive light respectively. 7
Figure 10. Measurement of spectral radiance of the transmissive lamp source Figure 11a. Absolute spectral radiance of Figure 11b. Normalized spectral radiance of the transmissive lamp source the transmissive lamp source C) The measurement of IBM DCS PRO/3000 Spectral Sensitivities. To estimate the spectral sensitivities of the IBM DCS two methods were employed: Monochromator method and Interference filters method. Monochromator method Measurement of the spectral sensitivity of the IBM DCS was accomplished using a light source Module Model 740-20 (serial 8553) in conjunction with a double monochromator, part of the Optical Radiation Measurement System Model 740 A (serial 185268-5) from Optronics Laboratories Inc. Hewlett Packard Hamsom 6274A DC Power Supply (0-60V 0-15A) was set to provide 0.06A current to the light source. This light source with monochromator provided narrowband illumination at the 10 nm exit slit, over a range 380-780 nm at 10 nm intervals. The monochromator, the light source, and the power supply are shown in Figure 12. The IEC TC-100: Audio, Video and multimedia systems and equipment had a project team 91966: 8
Colour measurement and management in multimedia systems and equipment and their Part 9 was dedicated to digital cameras. The IEC recommends the use of optical fiber to connect the monochromator to a black box with a diffuser in front of the digital camera to measure its spectral responsivity. Otherwise, Burns also measured a digital camera spectral sensitivity keeping a working distance of 70 cm from the exit slit of the monochromator to the surface of a digital camera lens. 9 In the case of the IBM DCS, it is not easy to modify the orientation of the scanner head to be positioned in front of the monochromator such as in the experiment conducted by Burns, and a optical fiber was used to connect the monochromator output to the IBM DCS scanner head. The extremity of the optics fiber was kept attached to a holder hanging very close to the camera lens (shown in Figure 13) and the diffuser recommended by IEC TC-100 was not used because it would attenuate the weak signal from the monochromator. Figure 12. Monochromator (white Figure 13. Optical fiber Figure 14. Image (8 bit) of the box), light source (black box) extremity hold below the monochromator adjusted at 600 nm. over the power supply. scanner head. The integration rate was set (R=20, G=18, and B=150) for miscellaneous 3072 by 2460 pixels resolution, using Rodesnstock 105 mm lens, fstop 5.6, scanner head positioned 858 mm from the copy stand and focus adjusted at 297.5 mm. Using these setting, a pilot experiment was performed digitizing 9 images over the range of 400-720 nm in intervals of 40 nm. Figure 13 shows a scanner head with open view finder showing the light from the optical fiber for monochromator adjusted at 440 nm and Figure 14 shows how an 8 bit image for monochromator adjusted at 600 nm looks like on the high resolution monitor of IBM DCS. Although this image is not uniform it is possible to crop the center portion of the image to get a quite uniform image. It is important to stress that 12 bit image is used in this experiment instead of the 8 bit image because the 8 bit image suffer automatic white spatial correction and other colorimetric corrections that are not desirable to determine the camera spectral sensitivity. Twelve bit images of the monochromator light were taken in a dark environment over the range of 380-780 nm, in intervals of 10 nm. Then, each image was cropped in the same position centered in the light spot producing a 200 by 200 pixels image. The averaged digital counts for each channel over the wavelength range are shown in Figure 15. 9
Figure 15. Averaged digital counts of a 200 by 200 image generated by a monochromator. Next, the camera was replaced by a calibrated detector (Optronics Laboratories, Inc. OL730-5C Silicon Photo detector SIN:1152 Hex Key with calibration certification date: May 30 1998) shown in Figure 16 whose responsivity is given in Figure 17 and the source spectral irradiance was measured over the same range used above using Optronics Laboratories Inc. radiometer 730 A (Serial # 850190). Figure 16. Optronics Laboratories Inc. calibrated Figure 17. Responsivity of the calibrated Silicon Photodetector SIN:1152 photodetector SIN:1152 The experimental arrangement to measure the source irradiance is shown in Figures 18 a,b. Figure 18a also shows the IBM high-resolution monitor displaying a cropped image for wavelength 520 nm. Figure 18b shows the optical fiber from the monochromator attached to the calibrated detector that sends signals to the radiometer. 10
Figure 18a. Experimental arrangement to measure the source irradiance (general view) Figure 18b. Optronics Laboratories Inc. Radiometer 730 A From the averaged digital counts and the measured irradiance it is possible to calculate the spectral sensitivity of R, G, and B channels. The absolute and normalized spectral sensitivity is plotted in figures 19a and 19b respectively. Figure 19a. Absolute spectral sensitivity of the IBM DCS camera. 11
Figure 19b. Normalized spectral sensitivity of the IBM DCS camera. Interference Filter Method It is possible to use the light filtered by interference filters put on the transmissive lamp at the copy stand level of the IBM DCS to produce monochromatic light that is imaged by the camera. The averaged digital counts of the image of each filtered light source is divided by the measured radiance for each filter giving the spectral sensitivity of each channel of the camera. The relative height for the spectral sensitivity of each channel can be adjusted comparing with the results obtained using monochromator method. At first, the surface of the transmission lamp diffuser was covered by a 35 mm film mask and the filters were put on the mask as shown in Figures 20a, b. Figure20a. Measurement set. Figure 20b. Detail of the interference filter Sixteen interference filters manufactured by Ealing centered over the range 400 to 700 nm in intervals of 20nm were imaged in 12 bits using the same camera settings used in the monochromator method. The used filters are listed in the Table 1. 12
Table1. Ealing Interference Filters used to determine the spectral sensitivities of the IBM DCS. Filter Number Passband range (nm) Peak wavelength (nm) 35-3219 385-415 401.6 35-3292 405-435 420.9 35-3359 425-455 439.9 35-3417 445-475 459.9 35-3458 465-495 481.4 35-3516 485-515 500.0 35-3599 505-535 520.5 35-3656 525-555 540.2 35-3714 545-575 561.7 35-3771 565-595 580.9 35-3839 585-615 601.2 35-3870 605-635 619.6 35-3938 625-655 640.8 35-4019 640-675 658.6 35-4076 665-695 679.6 36-4175 685-715 699.0 Figure 21 shows how looks like a digitized image (in 8 bits) for filter 35-3839. The spectral radiance of the transmissive source passing each filter was measured using the same setting for the Photo Research PR-704/714 SpectroScan Spectroradiometer used to measure the transmission lamp spectral radiance. A distance of 15 cm was set between the spectroradiometer lens and the filter surface. It is illustrated in the figures 22a and 22b. It is interesting to notice that the actual color of filter when it s seem from the top is red like shown in Figure 22a and not green as shown in Figure 22b. This change of color is due to the influence of the angle of view on interference filter spectral properties as mentioned in the introduction. Figure 21. Digitized Figure 22a. Spectral radiation measurement Figure 22b. Spectral radiation image for filter 35-3839 of the light leaving interference filters measurement The averaged digital counts for each channel for the images taken using interference filters can be seen in figure 23. 13
Figure 23. Averaged digital counts (12 bits) for the images taken using interference filters. The measured spectral radiance of each filter is given in figure 24. The transmittance of each filter can be either measured directly or can be calculated from the measured spectral radiance of the transmission source and the spectral radiance of the light leaving each filter. The calculated transmittance is given in figure 25. Figure 24. Measured spectral radiance for each filter. 14
Figure 25. Calculated transmittance of interference filters The absolute and relative spectral sensitivities of the IBM DCS given by this interference filters method is shown respectively in figure 26a and 26b. Figure 26a. Absolute spectral sensitivities of the IBM DCS (by interference filters method). 15
Figure 26b. Relative spectral sensitivities of the IBM DCS (by interference filters method). Comparison of the spectral sensitivities obtained by monochromator method and interference filters method The spectral sensitivities curves obtained by both monochromator method and interference filters method are plotted together in Figure 27. Figure 27. Comparison between monochromator and interference filters method to determine the spectral sensitivity of IBM DCS It is possible to notice from the curves of figure 27 the curves match except for a shift in the lower frequencies of blue channel. In order to check the accuracy of the wavelength in the monochromator- 16
optical fiber system, a mercury vapor lamp (manufactured by Oriel Corp.) was used to illuminate the input tube of the monochromator and its output was connected to a optical fiber that was linked to a photodetector (Optronics Laboratories Model 740A-D Number 387) as shown in figure 28a. The mercury vapor lamp was turned on using a Special Lamp Power Supply Model 65150 (serial number 244) manufactured by Oriel Corp. The same radiometer previously used to measure the radiant power of the light from the optical fiber was used to determine the wavelength of the monochromator which gives peaks of radiant power as shown in Figure 28b. The experimental peak wavelength was compared to the wavelength that gives the peaks of radiant power for a typical mercury vapor lamp. 23,24 This comparison is summarized in Table 2. Figure 28a. Monochromator with a mercury Vapor lamp source. Figure 28b. Measuring the radiant power of the light from the fiber optics Table 2. Comparison of measured and expected wavelengths of radiant power peaks for mercury vapor lamp Expected Wavelength (nm) Measured Wavelength (nm) 404.7 404.4 435.8 435.4 546.1 546.1 579.1 579.4 From Table 2 it is possible to see that the monochromator gives quite accurate wavelength. I believe that the mismatch in the curves of the figure is due to the interference of noise in the low digital counts values in the lower portion of spectra in the interference filter method. The spectral sensitivity determined by monochromator method will be used in the following experiments. 3.Spectral reconstruction from IBM DCS digital counts The IBM PRO/3000 trichromatic digital camera system and a set of Kodak Wratten gelatin filters number 38 (Light blue) were used to capture two trichromatic images that combined with the three channels without filters yielded six channels. Figure 29 shows the Filter number 38 and its holder. Figure 30 shows how the filter was attached in front of the camera lens. The transmittance of the filter 38 is given in Figure 31.The spectral reflectance factors of the Macbeth color checker was measured and principal component analysis were performed to calculate the first nine eigenvectors. The chart was imaged in 12 bits and the resulting digital counts were used to estimate a linear transformation matrix from digital counts to the eigenvalues corresponding to the eigenvectors of the sampled spectral reflectances. The resulting matrix was used to predict the spectral reflectance for the chart and the spectral and colorimetric accuracy was calculated for illuminant D50 and CIE 10 degree standard 17
observer. Figure 32 shows the comparison between measured and estimated spectral reflectance of the Cyan patch of the Macbeth Color Checker. The average spectral mean error was 0.17 and the average RMS error was 0.2. The mean E* ab was 6.9. Figure 29. Kodak Wratten filter number 38 and holder. Figure 30. Filter holder attached to the scanner head. Figure 31. Wratten filter number 38 transmittance The Wratten filter number 44 (Light green-blue) was added to give three additional signals yielding nine channels. In this case, the average spectral mean error was 0.029 and the average RMS error was 0.049. The mean E* ab was 2.2. As a comparison, Burns and Berns 19 performed spectral reconstruction 18
by PCA from camera signals using seven interference filters and a monochrome camera. Their mean E* ab was 2.2 for the Color Checker presenting similar performance to the proposed scheme that combines a trichromatic digital camera with Wratten filters. 0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 400 450 500 550 600 650 700 Wavelength (nm) Measured Spectral Reflectance Estimated Spectral Reflectance Figure 32. The spectral reconstruction from camera signal values for the Cyan sample of Color Checker. At the present stage simulations using iterative and non-linear methods are been performed to reduce the number of channels and this methods will be applied for Macbeth Checker as well as paintings. Paint patches (shown in figure 32a) can be imaged and their digital counts (the extraction of the digital counts from the 12 bits image using IDL is shown in figure 32b) used to establish a relationship between the measured spectral reflectance and the digital counts. This a priori processing will be used to reproduce artwork image that uses the same pigments used in the patches (figure 32c). As a pilot experiment to test image fusion, instead of using scanned photography, the original image will be blurred to produce a lower spatial resolution image and the estimated colorimetric values and spectral reflectance will be fused with the lightness information of the original image. Figure 32a. Paint patches Figure 32b. Extraction of the Figure 32c.Artwork painting digital counts of the imaged patches using the same pigments of a priori imaged patches 19
Future Research Image Fusion Figure 33 shows a flowchart of image fusion. 25 Once the high-resolution L*(x,y) and the low-resolution L*a*b*(x,y ) are obtained, the image fusion is performed. The initial step of this process is the geometric registration of the image (rotation, scaling, and translation, for example) that can be performed by a variety of commercial software such as ENVI. Among the fusion methods developed in the field of remote sensing, we decided to combine high-resolution lightness images with low-resolution colorimetric images in analogy to algorithms that use the hue, intensity and saturation (HIS) color space in remote sensing. We consider CIELAB as a reasonable first-order approximation to a vision model. Spectral Reconstruction of Hybrid Image The estimation of high-resolution spectral reflectance R(x,y,λ) from low-resolution reflectance R(x,y,λ) is based on the Wyszecki hypothesis that any stimulus can be decomposed into a fundamental stimulus (with tristimulus values equal to the stimulus) and a metameric black (with tristimulus values equal to zero) whose mathematical technique, known as Matrix R, was developed by Cohen. 26 The metameric black from R(x,y ) will be fused with the fundamental stimulus from L*a*b*(x,y,λ) resulting in a highresolution spectral image R(x,y,λ), and the same techniques used to combine CIELAB images will be used to merge spectral information. Figure 33. Image fusion and spectral reconstruction flowchart 20
Discussions Based on preliminary experiments the IBM DCS was characterized. The experiments suggest that a practical solution to the inherent problems of using interference filters can be solved using trichromatic camera and absorption filters as an alternative way to perform multi-spectral imaging. This camera input system should be able to make a priori spectral analyses used as a spectrophotometer or spectroradiometer estimating spectral reflectance factors in a pixel basis presenting advantages over direct measurements. This camera system will be used in a pilot experiment of a hybrid image capture system. It consists of a high-resolution conventional photographic and digital scanning system and a low-resolution trichromatic digital camera system. Using a priori spectral analyses, linear modeling techniques, and exploiting the human visual system's spatial properties, high-resolution spectral and colorimetric images can be generated. These techniques should be able to be used in many of the photographic departments of typical museums. Future research is aimed at further experimental optimization, verification, and testing within a museum context. References 1.Saunders, D., Cupitt, J., Image processing at the National Gallery: The VASARI Project, National Gallery technical bulletin, 14:72 (1993). 2.Berns, R. S., Challenges for color science in multimedia imaging, Proc. CIM 98 Colour Imaging in Multimedia, University of Derby, 123-133 (1998). 3.Hardeberg, J. Y., Schmitt, F., Brettel, H., Crettez, J-P. and Maitre, H., Multispectral imaging in multimedia, Proc. CIM 98 Colour Imaging in Multimedia, University of Derby, 75-86 (1998). 4.Maitre, H., Schmitt, F. J. M., Crettez, J.-P., Wu, Y., Hardeberg, J. Y., Spectrophotometric image analysis of fine art paintings, Proc. IS&T/SID Fourth Color Imaging Conference: Color Science, Systems and Applications, 50-53 (1996). 5.Haneishi, H., Hasegawa, T., Tsumura, N., Miyake, Y., Design of color filters for recording artworks, IS&T s 50th Annual Conference, 369-372 (1997). 6.Miyake, Y., Yokoyama, Y., Obtaining and reproduction of accurate color images based on human perception, Proc. SPIE 3300: 190 (1998). 7.König, F., Præfke, W., A multispectral scanner, Proc. CIM 98 Colour Imaging in Multimedia, University of Derby, 63-73 (1998). 8.Vent, D. S. S., Multichannel analysis of object-color spectra, Master Degree Thesis, R.I.T., 1994. 9.Burns, P. D., Analysis of image noise in multi-spectral color acquisition, Ph.D. Thesis, R.I.T., 1997. 10.König, F., Præfke, W., The practice of multispectral image acquisition, in International symposium on electronic capture and publishing, Proc. SPIE 3409 (1998). 11.Boyton, R. M., Human Color Vision, Optical Society of America, 1992. 12.Hunt, R. W. G., Bits, bytes, and square meals in digital imaging, Proc. IS&T/SID Fifth Color Imaging Conference: Color Science, Systems, and Applications, 1-5 (1997). 13.Haydn, R., Dalke, G. W., Henkel, J., and Bare, J. E., Application of IHS color transform to the processing of multisensor data and image enhancement, Proc. International symposium on remote sensing of arid and semi-arid lands, Cairo, Egypt, 599-616, (1982). 21
14.Braun, G. J., Quantitative evaluation of six multi-spectral, multi-resolution image merger routines, Ph. D. Thesis, R.I.T., 1992. 15.Gross, H. N., An image fusion algorithm for spatially enhancing spectral mixture maps, Ph. D. Thesis, R.I.T., 1996. 16.Vrhel, M. J., Trussel, H. J., Color correction using principal components, Color Res. Appl. 17: 26 (1992). 17.Præfke, W., Keusen, T., Optimized basis functions for coding reflectance spectra minimizing the visual color difference, Proc. IS&T/SID 1995 Color Imaging Conference: Color Science, Systems and Applications, 37-40 (1995). 18.König, F., Reconstruction of natural spectra from a color sensor using nonlinear estimation methods, Proc. IS&T s 50th annual conference, 454-458 (1997). 19.Burns, P. D., Berns, R. S., Analysis multispectral image capture, Proc. IS&T/SID 1995 Color Imaging Conference: Color Science, Systems and Applications, 19-22 (1996). 20.Vrhel, M. J., Gershon, R., Iwan, L. S., Measurement and analysis of object reflectance spectra, Color Res. Appl. 19:4 (1994). 21.Pro/3000 Digital Imaging System PISA95 Operator s Manual Version 6.0, IBM, 1996. 22.Pro/3000 Digital Imaging System Reference Guide, IBM, 1996. 23.Wyszecki, G., Stiles, W. S., Color Science: Concepts and Methods, Quantitative Data and Formulae, 2 nd Edition, John Wiley & Sons, Inc., 1982. 24.Handbook of chemistry and physics, 62 nd edition, CRC press, Boca Raton, 1981. 25.Imai, F. H., Berns, R. S., High-Resolution Multi-Spectral Image Archives A Hybrid Approach, Proc. IS&T/SID 1998 Color Imaging Conference: Color Science, Systems and Applications, (to be published). 26.Cohen, J. B., Kappauf, W. E., Metameric color stimuli, fundamental metamers, and Wyszecki s metameric blacks, Am. J. Psychol. 95: 537 (1982). 22