Automatic White Balance Algorithms a New Methodology for Objective Evaluation

Automatic White Balance Algorithms a New Methodology for Objective Evaluation Georgi Zapryanov Technical University of Sofia, Bulgaria gszap@tu-sofia.bg Abstract: Automatic white balance (AWB) is defined as correcting an image that was taken under an unknown light source, so that it appears to be taken under a canonical (often white) light source. The main problem with the automatic white balance algorithms is the evaluation of the images obtained after their implementation. This article examines a new methodology, allowing the objective evaluation of algorithms for automatic white balance. For comparison purposes, test images are used, taken at different settings of white balance of digital still camera, various scenes with different color temperatures, and under calibrated and uncalibrated illumination environments. Keywords: Automatic White Balance, Digital cameras, ColorChecker cards. 1. INTRODUCTION The main problem with the Automatic White Balance (AWB) algorithms is to objectively evaluate the quality of the produced images. Most of the published in the literature algorithms are based on different types of objective assessments, except the subjective (visual) evaluation criteria. One possible approach for objective evaluation of AWB algorithms is described in [7]: a series of images with correct white balance (reference) undergoes further processing with so-called "fading matrices" to obtain one which has an incorrect balance. The test algorithms for processing the newly obtained images are applied. The corrected images are compared with the originals using metrics based on CIELAB color space. In [1] Ciurea and Funt propose database of approximately 11,000 images of many indoor and outdoor scenes, captured with a Sony VX-2000 digital video camera. Such a large database have been created using a novel set-up of a digital video camera with a neutral gray sphere attached to it, so that the sphere always appears in the camera field of view. The scene illuminant in each image is measured in terms of the RGB values of the pixels on the sphere. Similar approach and same image dataset are used by Cohen in [3]. In [4] two different test image datasets are used: 1) created by the authors synthetic images. Each image has been transformed as if it is illuminated with different light sources. Testing algorithms have been applied to every transformed image, and then the SNR (Signal to Noise Ratio) is calculated; 2) a test image dataset proposed in [1]. To measure how close the estimated illuminant resembles the true color of the light source, the angular error ε is used: ε = cos 1 ( e ref e est e ref e est ), where e ref e est is the dot product of the two vectors representing the true color of the light source e ref and the estimated color of the light source e est, and e ref e est indi- 113

cates the Euclidean norm d euc. For RGB color channels, Euclidean norm is equal to d euc = (r est r ref ) 2 + (g est g ref ) 2 + (b est b ref ) 2. While test image dataset proposed by Ciurea and Funt include a large number of images, it has some significant drawbacks: 1) the record is on minidv cassettes with lower resolution - 240 360 pixels; 2) the time interval of the recording is too short; 3) as a result the correlation between the images is too large and from all of the 11346 images about 600 uncorrelated with each other can be extracted; 4) as the gray area is located too close to the camera, its illumination does not always coincide with that of the main scene; 5) no RAW versions of the images are available. A new AWB algorithm has been proposed in [9] and tested over 50 images taken under different lighting conditions. Some of the images have been taken by placing the ColorChecker Classic card in the captured scene [10]. YCC color space is used for objective assessment. The lower six achromatic squares on the card are used for assessing the results. The objective quality evaluation has been made by averaging the chromaticity of these squares Ave chr = (C 2 b + C 2 r ) 1/2. The smaller value indicates a better white balance. In [2] a new method of implementation of an AWB algorithm is proposed that uses images of the same ColorChecker Classic card taken under 7 different light sources as objective criteria. The best results are assed using the average chromaticity Ave chr of the 20th square of the card as base. In [6] the Euclidean distance E ab in CIELAB color space between the images with the correct white balance and the resulting images after the implementation of AWB algorithms is used as objective evaluation metrics. The same metrics is used by Song et al. in [8]. Again, a ColorChecker card has been used, but it is unclear how the reference images have been obtained. In [5] Kao et al. also use this card. For an objective metrics the average chromaticity ( a ) 2 + ( b ) 2 for channels a b in CIELAB color space is used. The overview of methods for objective evaluation of the AWB algorithms shows that no unified methodology for this assessment exists. Therefore, the aim of the paper is to propose a new methodology, appropriate for the evaluation of any AWB algorithms. The paper is structured as follows: In Section II the proposed methodology is evaluated based on the ColorChecker cards. An experimental setup and the obtained results are reported in Section III. Some concluding remarks are given in Section IV. 2. METHODOLOGY FOR EVALUATION OF AWB ALGORITHMS In this paper proposes an objective evaluation methodology, based on the existing assessment methods of the image quality of white balance algorithms, described in the previous section. It consists of the following four steps: Based on the existing methods for assessing the algorithms for AWB described in the previous section, in this paper is proposed a methodology for objective assessment consisting of the following basic steps: 1) creating a test images dataset; 2) creating a dataset of reference images from the test images available; 3) applying the examined algorithms to the image dataset; 4) comparing the resulting image quality with that of the reference images. The proposed methodology has been tested with an image dataset of two types of images: 1) indoor (interior) scenes, taken under controlled (calibrated) and uncalibrated lighting conditions; 2) outdoor (exteriors) scene, taken under uncontrolled lighting conditions. Two types of color reference cards Classic or Digital SG [10] are present in the 114

scenes. Figure 1 illustrates the procedure for setting the test and the reference images. Furthermore, eight AWB algorithms, has been implemented. Seven of them are based on the most frequently used white balance theories and one proposed algorithm that is a modified version of the investigated algorithms. Descriptions of the explored AWB algorithms can be found in [11]. To avoid the gamma nonlinearity and the image quality lost when JPG format is used for camera image recording, the RAW image files are converted to TIFF file format using the Adobe Camera Raw (ACR) software. The AWB algorithms under investigation are applied to the original images TIF orig. Whenever it is possible to ensure constant illumination conditions (an interior scene), two types of reference images are created a first one, using the very basic ability of the ACR convertor for white balancing and the second by preset white balance setting, using a white balance reference card. The assessment of the AWB algorithms is based on the comparison of an average values (R est, G est и B est ) with the average values (R ref, G ref и B ref ) for a part of the ColorChecker card squares in both captured and reference images in RGB color system ( R = R est R ref, G = G est G ref, B = B est B ref ). A criterion of the best algorithm is the closeness of reference values. Creation of test and reference images Case 1: Indoor (interior) scenes А. Calibrated light conditions (color light box) 1. Capture the scene in both JPG (used for control functions) and a RAW formats using camera AWB. 2. Saving the unprocessed RAW image obtained by camera AWB, on TIF file format (with Adobe Camera Raw, ACR). Obtaining the original image (TIF orig ) with automatic adjustment of the white balance of the camera during shooting. В. Non calibrated light conditions В1. Variable light conditions 1. Step А1. 2. Double-saving unprocessed RAW image obtained by camera AWB, on TIF format (with ACR): а) original image (TIF orig ) with automatic adjustment of the white balance of the camera during shooting; b) reference image (TIF ref ) white balance is realized by ACR and the middle of the three most bright gray square of ColorChecker Classic card. В2. Constant light conditions 1. Taking a photo of reference white card. A card ColorChecker White Balance is shooting, to define preset white balance of the camera. To minimize the error in its identification, card is placed in the center of the scene perpendicular to the light source. 2. Double shooting the scene: а) with camera AWB (JPG1 and RAW1); б) with pre-defined in section B2.1 camera white balance (preset) (JPG2 and RAW2). 3. Saving the native RAW files in TIF format (by ACR): RAW1 in original image (TIF orig ) and reference image 1 (TIF1 ref ), according step В1.2; RAW2 in reference image 2 (TIF2 ref ). Случай 2: Outdoor (exterior) scenes in analogy to case В1. Figure 1: Methodology for creation of test and reference images. 3. EXPERIMENTAL SETUP AND RESULTS The experimental studies have been made with test images dataset, recorded with a digital still camera Olympus E-P2. All images are taken at the maximum resolution maintained by the camera (4032 3024 pixels) in Adobe RGB color space. The test scenes are captured in different seasons and under different light conditions (Fig. 2): daylight, fluorescent light (FL), combination of these two, shadow, tungsten, two types energy saving lamps (CLF compact fluorescent lamps and LED light emitting di- 115

odes), under candles and etc. Over 100 images are in JPG and RAW file formats and therein is situated one of the two cards ColorChecker Classic or Digital SG (Fig. 2 (a) and (d)), while the rest of images are in JPG file format only. (a) sunset (b) snow, clouds (c) tungsten light (d) LED lamps Fig. 2: Part of the test images taken under different lighting conditions. To verify whether the reference values obtained using the White Balance card and the Adobe Camera Raw are close to each other under various light sources, experiments with interior scenes (under constant lighting conditions) have been made. For this purpose, two types of light sources with continuous spectrum of light emitted incandescent lamps and candles are used. In the first case, the same scene is illuminated with two types of incandescent lamps (type mat and type opal ) with unknown color temperatures. The second scene was illuminated by candles. All images were taken on a tripod with the same sensitivity settings and camera aperture. (a) original, matt lamp (b) with WB card (c) with ACR (d) WB card, candles Fig. 3: Results after AWB for indoor scenes, illuminated with incandescent lamps and candles Tab. 1: RGB reference values obtained by using ColorChecker Classic card (by WB card and ACR), with candles and two types of incandescent lamps. Gray Opal lamp Matt lamp Candles G Opal lamp Matt lamp Candles patches (GP) WBC ACR WBC ACR WBC ACR P WBC ACR WBC ACR WBC ACR R 194 190 198 195 185 177 100 96 104 101 84 78 G 192 188 197 195 181 176 100 96 104 101 83 79 B 190 186 197 193 176 175 98 94 104 99 83 81 R 168 163 173 170 155 147 68 65 68 66 57 52 G 168 163 173 170 152 146 68 65 69 67 57 54 B 168 162 177 171 149 146 68 65 71 67 61 58 R 138 133 144 140 119 112 38 36 34 33 37 34 G 138 133 144 141 117 112 38 36 35 33 38 36 B 138 132 147 141 115 114 40 38 38 36 42 41 Legend: WBC values obtained by using White Balance Card; ACR values obtained after correction with Adobe Camera Raw. 116

Both captured scenes are shown in Fig. 3. For the first scene are shown: the original image, illuminated by matt incandescent lamp (Fig. 3(a)) and the images, obtained after the correction with a card WB and by ACR (Fig. 3(b) and (c)). Visually compare the two corrected images, it can be seen that there are no visible color differences. The correction of the second scene (Fig. 3(d)) is made by the WB card. The average values of the lower six square of the card ColorChecker are shown in Table 1. It can be seen that in all three cases, each of these six squares is in grayscale. There is little increase of the received with a card WB values than those with ACR. The main reason is that the presets made for a white balance of the digital camera are more complicated because of the scene specific (for example, the presence of lampshade). Candles cannot also provide exactly the same light conditions in two consecutive frames. However, the results illustrate the applicability of the two approaches for reference images obtaining. The Figure 4 shows the images after applying the studied algorithms for a scene, where the yellow color dominates. As seen, the leftmost two images are too blue while those on the right side, it seems more correct. As the scene has been captured under natural light conditions, the correction of the unprocessed RAW data is made only by the Adobe Camera Raw. The difference between the reference and the experimentally evaluated values for six of the squares of the card ColorChecker is given in Table 2. (a) GWT (б) SDWDW (в) SDLWGW2 (г) SDL Fig. 4: Results after AWB at outdoor scene with a predominant yellow color Tab. 2: RGB values (reference and evaluating) obtained by using ColorChecker Classic card at outdoor scene with a predominant yellow color before and after application of algorithms for AWB Patches 1 2 3 4 5 6 7 1 2 3 4 5 6 7 R 152 169-37 -10-14 -47-12 197 210-41 -7-12 -54-9 G А 128 127 1 5 1-10 3 E 208 210-1 4-1 -21 1 B 137 118 31 14 13 102 8 226 207 43 6 5 48-4 R 121 139-31 -9-12 -39-12 163 182-39 -11-15 -50-12 G B 48 44 4 5 4 0 4 F 180 182-1 3-1 -17 1 B 59 49 58 8 7 45 5 199 182 72 11 8 73 1 R 191 207-40 -7-12 -53-9 130 147-34 -12-15 -43-13 G C 180 177 3 7 3-7 5 G 146 147-1 3-1 -12 0 B 81 65 84 14 13 66 10 169 148 107 11 10 107 4 Legend: 1 original image values; 2 values obtained after correction with ACR; 3 7 ( R, G, B): 3 GWT; 4 SDLWGW1; 5 SDLWGW2; 6 SDWGW; 7 SDL. A E F G B C D The results obtained show that after the use of ACR for the preparation of reference data, the three lowermost squares (E G) of these cards are in grayscale again. This is 117

not true for the values of the same squares in the original image. For images with predominant color (in this case yellow), the algorithms associated with the gray world theory, give the worst results in theory, since the averages color channels do not meet the theory itself. Experimental data confirm the theoretical predictions and the visual assessment in both algorithms (GWT and SDWGW), based on this theory only, the results are worse. These algorithms not only compensate the reduced blue component in the image, but further increase it (columns 3 and 6 from Table 2). Best results are obtained using further the illumination for white balance (algorithms SDLWGW1 and 2; SDL), which allow for proper recovery of red and blue components. 4. CONCLUSION In this paper a new methodology for objectively evaluation of automatic white balance algorithms in digital cameras based on standard reference cards ColorChecker: Classic, Digital SG and White Balance is proposed. The proposed methodology showed that with its help can be tested arbitrary algorithms for automatic white balance. The obtained in all test scenes objective evaluations coincide fully with the subjective evaluation of the same scenes. 5. REFERENCES [1] Ciurea, F., and Funt, B., A Large Image Database for Color Constancy Research, IS&T/SID Eleventh Color Imaging Conference, 2003, pp. 160-164. [2] Chiu, L., Fuh, C., Calibration-Based Auto White Balance Method for Digital Still Camera, Journal of Information Science and Engineering 26, 713-723 (2010). [3] Cohen, N., A Color Balancing Algorithm for Cameras, EE368 Digital Image Processing, Spring 2011, pp. 1-11. [4] Gijsenij, A., and Gevers, T., Color Constancy Using Natural Image Statistics and Scene Semantics, IEEE transactions on Pattern analysis and machine intelligence, Vol. 33, No. 4, April 2011, pp. 687-698. [5] Као et al., Color Reproduction for Digital Imaging Systems, ISCAS 2006, pp. 4599-4602. [6] Lam, E.Y. (2005), Combining gray world and Retinex theory for automatic white balance in digital photography, in Proceedings of the International Symposium on Consumer Electronics, Macau, China, pp. 134 139. [7] Nikitenko, D., Michael Wirth and Kataline Trude, White-balancing algorithms in colour photograph restoration, IEEE International Conference on Systems, Man and Cybernetics, 2007, pp. 1037-1042. [8] Song, H., Yin, G., Jiang, T., Auto White Balance Based on the Similarity of Chromaticity Histograms, Journal of Computational Information Systems, pp. 2557-2564, 2013. [9] Weng, C., Chen, H., Fuh, C., (2005), A novel automatic white balance method for digital still cameras, in Proceedings of the IEEE International Symposium on Circuits and Systems, Japan, vol. 4, pp. 3801 3804. [10] X-Rite: http://xritephoto.com/ph_product_overview.aspx?catid=28. [11] Zapryanov, G., Ivanova, D., Nikolova, I., Automatic White Balance Algorithms for Digital Still Cameras a Comparative Study, Journal of Information Technologies and Control, 1, 2012, pp. 16-22. 118