Multispectral image capture using a tunable filter

Transcription

1 Multispectral image capture using a tunable filter Jon Y. Hardeberg, Francis Schmitt, and Hans Brettel Ecole Nationale Supérieure des Télécommunications, Paris, France ABSTRACT In this article we describe the experimental setup of a multispectral image acquisition system consisting of a professional monochrome CCD camera and a tunable filter in which the spectral transmittance can be controlled electronically. We have performed a spectral characterisation of the acquisition system taking into account the acquisition noise. To convert the camera output signals to device-independent data, two main approaches are proposed and evaluated. One consists in applying regression methods to convert from the K camera outputs to a device-independent colour space such as CIEXYZ or CIELAB. Another method is based on a spectral model of the acquisition system. By inverting the model using a Principal Eigenvector approach, we estimate the spectral reflectance of each pixel of the imaged surface. Keywords: Liquid Crystal Tunable Filter, spectral reflectance, multispectral imaging, spectral match, metamerism. INTRODUCTION Already in 853 Grassmann stated that three variables are necessary and sufficient to characterize a color. This principle, the three-dimensionality of color, has since been confirmed by thorough biological studies of the human eye. This is the reason why digital color images are composed of three channels or layers, typically red, green and blue. However, for digital image acquisition and reproduction, three-channel images have several limitations. First, in a color image acquisition process, the scene of interest is imaged using a given illuminant. Due to metamerism, the color image of this scene under another illuminant cannot be accurately estimated. Furthermore, since the spectral sensitivities of the acquisition device generally differ from the standardized color matching functions, it is also impossible to obtain device-independent color. By augmenting the number of channels in the image acquisition and reproduction devices we can remedy these problems, and thus increase the color quality significantly. Several research groups worldwide are working on these matters, for example at the University of Chiba (Japan), RIT (Rochester, NY), RWTH (Aachen, Germany), and ENST (Paris, France). Multispectral imaging systems are developing rapidly because of their strong potential in many domains of application, such as remote sensing, astronomy, physics, museum, cosmetics, medicine, high-accuracy colour printing, computer graphics, etc. Multispectral images provide information about a number of spectral bands, from three bands per pixel for colour images up to thousands of bands for hyperspectral images. Hyperspectral image acquisition systems remain complex, expensive and difficult to manage due to the huge amount of data to be stored and processed. For high-end multimedia applications we have considered a more affordable approach based on digital imaging techniques in which an electronically tunable filter is used with a CCD camera. In previous publications, 4 we have discussed mostly theoretical aspects of the multispectral image capture process, validated by simulations. In this paper the theoretical models and simulations of the previous research are validated in practice. We describe the experimental setup of a multispectral image acquisition system consisting of a Peltier cooled 2-bit monochrome CCD camera and a liquid crystal tunable filter (LCTF) in which the spectral transmittance can be controlled electronically. To convert the camera output signals to device-independent data, two main approaches are proposed and evaluated. One consists in applying an extended version of the colorimetric scanner characterization method proposed previously 5 to convert from K different color channels (K > 3) to a 3-dimensional colour space such as CIEXYZ or CIELAB. Another method is based on a modified spectral model of the acquisition system. The parameters of the model are estimated through a spectral characterisation technique. By inverting the model using a principal-components approach, we estimate the spectral reflectance of each pixel of the imaged surface. In Section 2 we describe the setup and initial calibration of our image acquisition equipment. In Section 3 we perform a spectral characterisation of the image acquisition system. In Section 4 we examine how colorimetric and spectrophotometric information can be determined from the camera responses. Reprint from SPIE Proc. Vol. 3963, pp JYH is now with Conexant Systems, Inc., Redmond, Washington, USA. For further information please send an to jon.hardeberg@conexant.com, schmitt@tsi.enst.fr,orbrettel@tsi.enst.fr 47

2 2. EXPERIMENTAL SETUP AND CALIBRATION The main components of our multispectral acquisition system are described in this section: the CCD camera, the tunable filter, and the illumination. Then, in Section 2.2 we describe our initial calibration. 2.. Equipment The camera we used in our experiments is a SensiCam Super-VGA monochrome CCD camera 6 from PCO Computer Optics GmbH. It has a resolution of pixels, a dynamic range of 2 bit, exposure times from ms to s, and it operates at a 2.5 MHz readout frequency. The CCD is grade (no defective pixels), and it is cooled to 2 ffi C to reduce noise to a minimum. The camera is controlled from a PC via a PCI-board. It is delivered with a Software Developers Kit (SDK) which has enabled us to develop efficient image acquisition software corresponding to our needs. 7 By grouping several pixels within rows or columns with a technique called binning, the sensitivity can be increased while reducing the resolution proportionally. Although having a rather limited spatial resolution compared to the Kodak Eikonix 42 line-scan CCD camera we have been using in previous experiments,8,9 the SensiCam has the enormous advantage of being several orders of magnitude faster. This is of great importance since several acquisitions must be made for each scene in order to acquire a multispectral image. In our earlier experiments done with the Eikonix camera, this process was prohibitively slow. If a higher resolution is required, for example for fine-art paintings, we can either upgrade to a higher resolution camera, once all the algorithms have been developed and tested, or apply mosaicing techniques. 8,9 Coupled with the camera, we use a Liquid Crystal Tunable Filter (LCTF), the VariSpec from Cambridge Research & Instrumentation (CRI), Inc. This system is made of two units which provide us with two different set-ups, one for narrow-band filters, the other for wide-band filters. We will not go into details on the physics behind the functionnality of such filters here, but only mention that it consists of several consecutive layers of Lyot-type bi-refringent filters,,7 each layer containing linear parallel polarisers sandwiching a liquid crystal retarder element. Each layer is operating in a higher order than the previous ones, thus being able to select narrower bandpass characteristics of varying peak wavelengths. The peak wavelength can be controlled electronically from an external controller unit, or from a computer via a RS-232 interface, in the range [4 nm, 72 nm]. The average Full-Width-at-Half-Maximum (FWHM) bandwidths are approximatively 5 nm and 3 nm for the narrow-band and wide-band set-ups, respectively. Compared to another type of tunable filters, the Acusto-Optical Tunable Filters (AOTF), the LCTF technology offers a reasonably wide field of view (±7 ffi from the normal axis) but, nevertheless, the limitation in field-of-view is a parameter that has to be treated with care for imaging applications. The spectral transmittances of the filter when varying the peak wavelength in nm steps from 4 to 72 nm was measured with the Ocean Optics Model SD spectrometer (see Figure ). Several interesting conclusions can be drawn from these Spectral transmittance, φ(λ) Spectral transmittance, φ(λ) Wavelength, λ [nm] Wavelength, λ [nm] Figure. Spectral transmittances of the narrow-band (left) and wide-band (right) set-ups of the LTCF filter when varying the peak wavelength in nm steps from 4 to 72 nm. 48

3 transmittance spectra: The transmittances have Gaussian-like shapes, except for the wide-band set-ups at peak wavelengths > 65 nm. The filter spectral transmittances are cut at the red end of the spectrum, indicating the presence of an infrared filter. The FWHM is not constant; for the wide-band set-up, it varies from 5 to 8 nm. For the wide-band set-up and peak wavelengths» 44 nm, there are unwanted secondary peaks at long wavelengths. We conclude that, even if the filter characteristics do not completely fullfill the manufacturer s promises, it as a very valuable tool for multispectral imaging. The importance of the illumination in image acquisition systems is often neglected. Factors that need to be taken into consideration include geometry, power, spectral properties, and stability. For our experiment we have used one 2V tungsten halogen lamp connected to a stabilised power supply. We used no diffuse reflectors to make sure that the spectral properties were spatially constant, and we also made sure there were no unwanted reflections by covering surrounding items with a black cloth Illumination and dark current compensation For our analysis we need to make sure that the camera response is linear with regards to the energy of the incident light. 3,9 We obtain this by correcting for the camera s dark current. Furthermore, we correct for the uneven distribution of the lighting. The first step of our calibration is to measure the dark noise of the camera. To do so, we acquire a set of images where no light was entering the objective, i.e. with the lens cap on in a dark room, using varying integration times. We found that the dark noise was approximately e d =6± 5 (on a scale of digital counts on 2 bit, giving values from to 2 2 = 495). This number shows almost no variation with the integration time. A slight augmentation was found only for integration times of several seconds, and we thus assume that the black noise is constant and independent of the acquisition parameters. In the second step, we acquire an image W (i; j) of a uniform diffuse surface. This provides us information about the spatial distribution of the illuminant. An image I(i; j) of the Macbeth ColorChecker Color Rendition Chart 2 is then acquired, and we calculate a normalised image I n (i; j) as follows: I n (i; j) =k I I(i; j) e d W (i; j) e d ; () the normalisation factor k I being chosen so that the pixel values of the normalized image are limited to a given maximal value. The images W (i; j) and I(i; j) are encoded as 2 bits per pixel, and the normalised image I n (i; j) as 6 bits per pixel and saved to a file for further use. Note that the normalisation factor k I is image-dependent. Having done this we verify the linearity of the image I n (i; j) by extracting the mean pixel values of the grey patches of the Macbeth chart and comparing these to the reflectance factors of the patches. We observed that the data could be fitted reasonably well to a straight line, and we concluded that no further linearisation was needed for this camera. 3. SPECTRAL CHARACTERISATION OF THE IMAGE ACQUISITION SYSTEM The main components involved in the image acquisition process are depicted in Figure 2. We denote the spectral radiance of the illuminant by l R ( ), the spectral reflectance of the object surface imaged in a pixel by r( ), the spectral transmittance of the optical systems in front of the detector array by o( ), the spectral transmittance of an optical colour filter by ffi k ( ), andthe spectral sensitivity of the CCD array by a( ). In this section we first consider an unfiltered monochrome camera and will omit the spectral transmittance ffi k ( ) in the calculations. Supposing a linear optoelectronic transfer function of the acquisition system, the camera response c to an image pixel is then equal to Z max c = l R ( )r( )o( )a( ) d = r( )!( ) d (2) min min where!( ) = l R ( )o( )a( ) denotes the system unknowns. The assumption of system linearity is based on the fact that the CCD sensor is inherently a linear device. However, for real acquisition systems this assumption may not hold, due for example to electronic amplification non-linearities or stray light in the camera. Then, appropriate nonlinear corrections may be necessary. By uniformly sampling the spectra at N wavelength intervals, we can rewrite Equation (2) as a scalar product in matrix notation, c = r t!; where! =[!( )!( 2 ) :::!( N )] t and r =[r( ) r( 2 ) :::r( N )] t. Z max 49

4 Illumination l ( ) R λ o( λ ) φ ( λ) k a( λ) Observed object r( λ) Sensor Optical path Colour filter c k Camera response Figure 2. Schematic view of the image acquisition process. Direct spectrophotometric measurements with monochromatic light require expensive equipment. We have opted 3,9 for an indirect approach where the vector! describing the system unknowns is estimated from the camera responses c p to a set of P samples with known reflectances r p. Denoting the sampled spectral reflectances of all the patches as the matrix R =[r r 2 :::r P ], the camera response c P =[c c 2 :::c P ] t to these P samples is then given by c P = R t!: (3) Several methods have been proposed for the estimation ~! of the camera characteristics!. The simple system inversion by: ~! =(RR t ) Rc P =(R t ) c P ; (4) where (R t ) denotes the Moore-Penrose pseudo-inverse of R t, yields very large errors in the presence of noise. Instead, the Principal Eigenvector method should be used, the noise sensitivity of the system inversion being reduced by only taking into account the singular vectors corresponding to the r most significant singular values. We denote this method PE(r). A Singular Value Decomposition (SVD) is then applied to the (P N ) matrix R t of the spectral reflectances of the observed patches: R t = UWV t ; where U and V are (P P ) and (N N ) unitary matrices repectively, W is a (P N ) matrix with general diagonal entry the singular values w i, i =:::R, and zeros elsewhere, R being the rank of R t. U and V being unitary matrices, it can easily be verified that (R t ) = VW U t,wherew has a general diagonal entry equal to w i, i =:::R, and zeros elsewhere. It is well known that the singular values of a matrix of spectral reflectances such as R t are strongly decreasing, and by consequence that reflectance spectra can be described accurately by a small number of parameters. By taking into account only the first r<rsingular values in the system inversion, the spectral sensitivity may thus be estimated by : ~! = VW (r) U t c P ; (5) where W (r) has now a general diagonal entry equal to w i, i =:::r, (r <R). 3.. Mono-channel spectral sensitivity estimation We applied the PE(r) method to the experimental data, that is, to the normalised mean pixel values I n (i; j) of the patches of the Macbeth chart. After estimating the spectral sensitivity ~! we can then simulate the camera response ~c P = R t ~!, and compare these values to the experimentally observed camera responses c P. We show in Figure 3 the results of the PE method with the number of Principal Eigenvectors, r, varying from to 6. For each value of the parameter r, the estimated sensitivity is shown together with a comparison of the experimentally observed camera responses and those predicted using the model with this sensitivity. We report the RMS ratio kc P ~c P k=kc P k between the observed and predicted values, and observe that the RMS ratio decreases with increasing r. However, it is clear that the estimate becomes poor when r 6. As will be justified later in this section, we choose the estimate obtained with r =5for further analysis. 5

5 Estimated sensitivity for PE= Estimated sensitivity for PE= 2 Estimated sensitivity for PE= Normalized sensitivity Normalized sensitivity Normalized sensitivity Acquisition simulation for PE= giving RMS error. Observed.9.8 Acquisition simulation for PE=2 giving RMS error.79 Observed.9.8 Acquisition simulation for PE=3 giving RMS error.54 Observed Camera response Camera response Camera response Normalized sensitivity MacBeth patch # Estimated sensitivity for PE= Acquisition simulation for PE=4 giving RMS error.42 Observed Normalized sensitivity MacBeth patch # Estimated sensitivity for PE= Acquisition simulation for PE=5 giving RMS error.42 Observed Normalized sensitivity MacBeth patch # Estimated sensitivity for PE= Acquisition simulation for PE=6 giving RMS error.39 Observed Camera response Camera response Camera response MacBeth patch # MacBeth patch # MacBeth patch # Figure 3. Spectral sensitivity estimation of the image acquisition system consisting of a PCO SVGA SensiCam CCD camera and a tungsten halogen illuminant, using the PE method with r =:::6 Principal Eigenvectors and the Macbeth colour chart. The estimated sensitivity obtained with r =5is chosen Multi-channel spectral sensitivity estimation As a basis for further analysis we have performed an acquisition of a 7-channel multispectral image of the Macbeth chart, varying the peak wavelength of the tunable filter in 2 nm steps from 4 nm to 72 nm. Nine channels of this multispectral image are shown in Figure 4. By selecting subsets of these images, we can simulate multispectral image acquisition with different numbers of channels. A tungsten halogen lamp driven by 4.A/.4V, a CCD binning of (H2, V2) giving a resolution of pixels, and an aperture of f/2.8 was used. The integration times were chosen individually for each of the channels so as to yield a maximum digital signal without causing signal clipping. Then we corrected these images for the illuminant and the dark current as described in Section Model evaluation We will now establish a linear model for the multispectral image acquisition, taking into account the integration times and the normalisation factors. The vector c K =[c c 2 :::c K ] t representing the response to all K filters (after normalisation) may be 5

6 Figure 4. Nine channels of a multispectral image of the Macbeth colour checker using the PCO SensiCam CCD camera and the LCT filter with varying peak wavelengths. described as c K = A t r; (6) where is the known matrix of filter transmittances multiplied by the estimated spectral sensitivity, i.e. the matrix element of is kn = ffi k ( n )!( n ),» k» K,» n» N. The matrix A consists of the weights a kk on the diagonal, and zeros elsewhere: a kk = ffk Ik t k (7) The common normalisation factor ff is introduced in the model to be able to work with relative measurements of the spectral sensitivity, spectral power distribution of the illuminant, etc. It is determined by minimising the RMS camera response estimation error using the model. Using this model, we can estimate the camera response to the Macbeth patches for each of the 7 filters, and we compare the estimates to the observed camera responses. We perform this simulation using six different estimations of the spectral sensitivity, obtained by a PE(r) estimation with the number of PE s r varying from to 6, cf. Figure 3. The results of this simulation are shown in Figure 5. By examining the overall RMS ratio corresponding to the estimation errors for different choices of r, as defined in Section 3., we see that r =3; 4; 5 gives reasonably small estimation errors, with a minimum of.37 for r =3. However, we know that an IR cut-off filter is present in the optical path, and we can thus exclude r =3since it has an important sensitivity in the red end of the spectrum, cf. Figure 3. We confirm therefore the choice of r =5as the optimal parameter for the spectral sensitivity estimation, cf. Section Modified acquisition model Having chosen PE(5) as the spectral sensitivity we perform a further comparison of the observed and estimated camera responses for the 24 Macbeth patches using the 7 different filters, as shown in Figure 6. We see that the differences between the 52

7 RMS camera response estimation errors PE=, mean RMS=.295 PE=2, mean RMS=.34 PE=3, mean RMS=.37 PE=4, mean RMS=.77 PE=5, mean RMS=.6 PE=6, mean RMS= Peak wavelength Figure 5. RMS camera response estimation errors over the 24 Macbeth patches using the 7 different filters. We have also included in the figure the mean RMS of these over the 7 camera channels, and we note that r = 3; 4; 5 might be good solutions. Note the relationship between high errors for given wavelengths, and obvious errors in acquisition system sensitivity estimations (Figure 3), especially for r =6. observed and estimated camera responses are relatively large, especially for the 42 nm filter. These results are not satisfactory. We see from the figure that the errors are not randomly distributed. For a given filter there is often a tendency to either over- or under-estimation. We propose thus to modify the normalisation matrix A in the model of Equation 6 to allow for independent normalisation of each channel, that is, we redefine a kk as compared to Equation 7, as a kk = ff k k Ik t k ; (8) and we choose the normalisation factors ff k such that the camera response estimation errors for each channel are minimised. By using this modified model, we reduce the mean RMS camera estimation error by more than a factor 2, from.6 to.77 (see Figure 7). The use of separate normalisation factors ff k for each channel is mainly justified from the fact that we have limited confidence in the spectral sensitivity estimation. 4. RECOVERING COLORIMETRIC AND SPECTROPHOTOMETRIC IMAGE DATA We now examine how colorimetric and spectrophotometric information can be determined from the camera responses. 4.. Model-based spectral reconstruction Given the camera responses c K = A t r (cf. Equation 6) for a given surface, our goal is to estimate the spectral reflectance of the surface by using a reconstruction matrix Q as follows: ~r = Qc K. In our simulations, 2,3,9 two methods have been evaluated, the simple pseudo-inverse solution, Q = (A t ),anda method that exploits statistical spectral information of the imaged objects, Q = RR t A(A t RR t A). The pseudo-inverse method was abandoned due to extremely poor performance in the simulations in the presence of noise. It became rapidly clear from our experiments also that the unmodified Q method did not give satisfactory results. This is due to the relatively important deficiences of our model in estimating the camera output, as seen for example in Figure 6. The problem of estimating the spectral reflectance given the camera outputs and spectral sensitivities is very much similar to the problem of estimating the spectral sensitivity given the camera outputs and spectral reflectances. Both are inverse problems, and noise is present in both systems. We propose thus to apply a Principal Eigenvector (PE) approach for the estimation of spectral reflectances similar to the one described in Section 3 for the spectral sensitivity estimation. 53

8 4nm, RMS= nm, RMS=.54 44nm, RMS= nm, RMS=.3 48nm, RMS=..5 5nm, RMS= nm, RMS= nm, RMS= nm, RMS=..5 58nm, RMS= nm, RMS= nm, RMS= nm, RMS=..5 66nm, RMS= nm, RMS= nm, RMS=.3 72nm, RMS=.2 Observed.5 Figure 6. Observed and predicted camera responses for the 24 Macbeth patches using the 7 different filters and the spectral sensitivity estimation PE(5). The overall mean RMS camera estimation ratio is.6. 4nm, RMS=.57 42nm, RMS=. 44nm, RMS= nm, RMS= nm, RMS= nm, RMS=..5 52nm, RMS=..5 54nm, RMS= nm, RMS= nm, RMS= nm, RMS= nm, RMS= nm, RMS= nm, RMS= nm, RMS= nm, RMS=.43 72nm, RMS=.55 Observed.5 Figure 7. Observed and predicted camera responses for the 24 Macbeth patches using the 7 different filters and the spectral sensitivity estimation PE(5), and seperate ff k for each channel. The overall mean RMS camera estimation ratio is

9 Four cases were defined, using 3, 6, 9, and 7 channels, defined by the following filter sets: K =3: f46, 56, 66g nm, K =6: f4, 46, 52, 58, 64, 7g nm, K =9: f4, 44, :::, 68, 72g nm, and K =7: f4, 42, :::, 7, 72g nm. For each case, we evaluate the mean RMS spectral reconstruction error, while varying the parameter r. Weevaluatethis model-based estimation using the two variants of the acquisition model, with a global normalisation factor ff or with individual ones ff k for each channel, as described previously. The results for the first variant of the model are shown in Figure 8(a). We see that due to the high level of noise only r =3Principal Eigenvectors can be used in the reconstruction process, and thus there is no improvement of the results (see Figure 8(b)) by using more than three filters. When too many Principal Eigenvectors are taken into account in the system inversion, the noise severely deteriorates the estimation results. For the modified model, the results are significantly better, as expected. The mean RMS spectral reconstruction errors for the four filter sets varying the parameter r is showed in Figure 8(c). For the 9 and 7 filter sets, a minimal mean RMS spectral reconstruction error of.24 is attained with the PE(5) estimation method. We achieve thus a much better spectral reconstuction (see Figure 8(d)). Mean RMS spectral reconstruction error filters.5 6 filters 9 filters 7 filters Number of Principal Eigenvectors Spectral reflectance r(λ) Macbeth Blue Macbeth Green Macbeth Red Reconstructions (a) Mean RMS spectral reconstruction errors varying r using the first model (b) Example of spectral reconstruction using three filters and PE(3) reconstruction Mean RMS spectral reconstruction error filters 6 filters 9 filters 7 filters Spectral reflectance r(λ) Macbeth Blue Macbeth Green Macbeth Red Reconstructions Number of Principal Eigenvectors (c) Mean RMS spectral reconstruction errors varying r using the modified model (d) Example of spectral reconstruction using the set of nine filters and PE(5) reconstruction Figure 8. Spectral reconstruction of the Macbeth patches from the SensiCam camera responses with different filter sets using a Principal Eigenvector approach PE(r). The modified model with ff k normalisation factors used in the lower figures allows for reasonably good spectral reconstruction quality. 55

10 To gain more insight in how the PE(r) spectral reconstruction method succeeds in estimating reflectance spectra, we show in Figure 9 the spectral reflectance estimations of three of the Macbeth patches, for the set of 9 filters, varying r from to 9. We see that for PE() and PE(2) the dimensionality of the solution is too low, for PE(3) the reconstructions start to resemble the original spectra, for r =4to 7 they are quite good, but for 8 to 9 the estimations are slightly worse. In Table we resume the estimation results for different filter sets, and different values for r. We report the mean and maximal RMS difference between the original and reconstructed spectra. The differences are also expressed colorimetrically, in CIEXYZ and CIELAB colour spaces (illuminant A) by applying standardised formulae. An additional filter set marked 3, having peak wavelengths of 44, 56, and 6 nm, was chosen in order to be closer to the XYZ colour matching functions. Compared to the original 3-filter set the spectral errors are larger, while the colorimetrical errors are smaller, as would be expected. We see that generally, using more filters gives smaller reconstruction errors. However, this is true only up to a certain number of filters; there is no significant improvement by using 7 instead of 9 filters. Results better than a mean E ab error of 3 are not obtained. The reason for this relatively poor performance when using many channels is mainly the fact that our spectral model of the image acquisition system does not predict the camera output values as precisely as we had hoped. The prediction of the spectral reflectances by model inversion then becomes somewhat hazardous. We will therefore in the next section evaluate an alternative way of using the camera output values..8 PE().8 PE(2).8 PE(3) PE(4) PE(7) PE(5) PE(8) PE(6) PE(9) Figure 9. Spectral reconstruction of three of the Macbeth patches from the Sensicam camera responses using the nine-filter set, and the PE(r) reconstruction method, with r varying from to 9. 56

11 #of Recon. Spectral RMS XY Z E filters method Mean Max Mean Max Mean Max 3 PE(3) PE(3) PE(3) PE(5) PE(5) PE(9) PE(5) PE(9) Table. Mean and maximal errors expressed in spectral reflectance space, XYZ space and CIELAB space, for different filter sets and different spectral reconstruction methods. The filter set marked 3 was chosen in order to be closer to the XYZ colour matching functions Direct colorimetric regression The idea here is to consider the acquisition system parameters as a completely unknown system, a black box, and simply try to recover directly the XYZ values from the camera output by regression. We selected heuristically different subsets of the 7 channels to evaluate the method for different numbers of filters. Two sets of three and four filters were chosen with regards to the CIEXYZ colour matching functions, while sets of 6, 7, 9, 5 and 7 filters were chosen to have equidistant peak wavelengths. The CIEXYZ tristimulus values were estimated by linear regression, and we report the mean Euclidean distance in XYZ space, XY Z, as well as the mean and maximal E ab reconstruction errors taken under CIEilluminant A in Table 2. # Wavelengths Method XY Z E E max Linear to XYZ Linear to XYZ Linear to XYZ Linear to XYZ Linear to XYZ Linear to XYZ Linear to XYZ rd order to CIELAB.63.9 Table 2. Resulting colorimetric reconstruction errors using regression methods and different numbers of filters. These results are quite as expected, e.g. when comparing our result using 6 filters ( E =4:4), with Abrardo et al., 3 who attains a mean E error of 2.9 by linear regression from 6 camera channels to XYZ space using a subset of 2 patches of an AGFA IT8.7/3 colour chart. It is however worth noting that the maximal error is greater using four filters than when using three. This may seem surprising. However, such effects are well-known when optimising a RMS error. A minimal RMS error does not necessarily imply a minimal maximal error. For the set of three filters, we also applied a non-linear method, 5,9,4 in which 3rd order 3D polynomial regression is applied on the cubic root of the camera output (see the last line of Table 2). This method with three filters outperforms the linear regression with seventeen filters! It is however worth noting that the Macbeth chart is perhaps not well suited for this method, since the 2 coefficients of the polynomial is optimised using only 24 patches. If we compare the results obtained by colorimetric linear regression (Table 2) with those obtained by model-based spectral reconstruction (Table ) for the same filter sets and the preferred (PE(5)) reconstruction methods, we can draw some interesting conclusions. For the sets of three (marked 3 ) and six filters, the results are nearly equivalent. For nine filters the colorimetric methods are slightly better, while for the set of seventeen filters the colorimetric regression gives much smaller residual errors. 57

12 5. CONCLUSION The multispectral image acquisition experiment presented here has been enriching in several ways. First of all, it has reminded us that simulations and reality are two very different things. We have seen that the noise involved in the image acquisition process was much larger than the quantisation noise used in our previous simulations. This has led us i) to propose a modified image acquisition model with separate normalisation factors for each channel; ii) to propose a new method using a Principal Eigenvector approach for the estimation of a spectral reflectance given the camera responses through several filters; and iii) to consider using simpler regression techniques to obtain colorimetric information from the camera responses. Note however that by doing the latter, we only obtain information about the colour of a surface under a given illuminant, not about its spectral reflectance as with the model-based spectral reconstruction method. We conclude that the multispectral image acquisition system we have assembled presents several strong interests. The computer-controlled CCD camera and LCTF tunable filters are easy to use, and the colorimetric and spectrophotometric quality is quite good. REFERENCES. H. Maître, F. Schmitt, J.-P. Crettez, Y. Wu, and J. Y. Hardeberg, Spectrophotometric image analysis of fine art paintings, in Proceedings of IS&T and SID s 4th Color Imaging Conference: Color Science, Systems and Applications, pp. 5 53, (Scottsdale, Arizona), Nov J. Y. Hardeberg, H. Brettel, and F. Schmitt, Spectral characterisation of electronic cameras, in Electronic Imaging: Processing, Printing, and Publishing in Color, vol. 349 of SPIE Proceedings, pp. 9, May J. Y. Hardeberg, F. Schmitt, H. Brettel, J.-P. Crettez, and H. Maître, Multispectral image acquisition and simulation of illuminant changes, in Colour Imaging: Vision and Technology, L. W. MacDonald and R. Luo, eds., pp , John Wiley & Sons, Ltd., F. Schmitt, H. Brettel, and J. Y. Hardeberg, Multispectral imaging development at ENST, (Chiba, Japan), Oct J. Y. Hardeberg, F. Schmitt, I. Tastl, H. Brettel, and J.-P. Crettez, Color management for color facsimile, in Proceedings of IS&T and SID s 4th Color Imaging Conference: Color Science, Systems and Applications, pp. 8 3, (Scottsdale, Arizona), Nov Also in R. Buckley, ed., Recent Progress in Color Management and Communications, IS&T, pages , PCO Computer Optics, Sensicam: Specifications and typical values. See Products/Specs/dsht_sc.htm, H. Brettel, J. Y. Hardeberg, and F. Schmitt, Multispectral image capture across the Web, in Proceedings of IS&T and SID s 7th Color Imaging Conference: Color Science, Systems and Applications, (Scottsdale, Arizona), Nov F. Schmitt, High quality digital color images, in Proceedings of the 5th International Conference on High Technology: Imaging Science and Technology, Evolution and Promise, pp , (Chiba, Japan), Sept J. Y. Hardeberg, Acquisition and reproduction of colour images: colorimetric and multispectral approaches. Ph.D dissertation, École Nationale Supérieure des Télécommunications,Paris, France, Cambridge Research & Instrumentation, Varispec Tunable Imaging Filter. pdfs/varispec2.pdf, B. Lyot, Un monochromateur à grand champ utilisant les interférences en lumière polarisée, Compt. Rend. 97, p. 593, C. S. McCamy, H. Marcus, and J. G. Davidson, A color rendition chart, Journal of Applied Photographic Engineering 2, pp , A. Abrardo, V. Cappellini, M. Cappellini, and A. Mecocci, Art-works colour calibration using the VASARI scanner, in Proceedings of IS&T and SID s 4th Color Imaging Conference: Color Science, Systems and Applications, pp , (Scottsdale, Arizona), Nov J. Y. Hardeberg, Desktop scanning to srgb, in IS&T and SPIE s Device Independent Color, Color Hardcopy and Graphic Arts V, (San Jose, CA), Jan