Comp Computational Photography Spatially Varying White Balance. Megha Pandey. Sept. 16, 2008

Transcription

1 Comp Computational Photography Spatially Varying White Balance Megha Pandey Sept. 16, 2008

2 Color Constancy Color Constancy interpretation of material colors independent of surrounding illumination.

3 Color Constancy Color Constancy interpretation of material colors independent of surrounding illumination.

4 Digital Images under Varying Illumination Cameras can not adapt to varying illumination as humans do images have a color cast depending on the light source.

5 Cameras can not adapt to varying illumination as humans do images have a color cast depending on the light source.

6 Color Temperature Color temperature of a light source is the temperature of an ideal black body radiator at which the color of the color of the light source and the black body are identical.

7 Incandescent Light Orange Color Cast

8 Moonlight Blue Color Cast

9 Fluorescent Light : Green Color Cast

10 Color Balance Color Balance adjusting the color components to eliminate color casts. Chromatic Adaptation : estimation of representation of object as it would appear under a different light source than the one in which it was recorded. White Balance aims to render neutral colors correctly to emulate the property of color constancy

11 Color Balance adjusting the color components to eliminate color casts. White Balance aims to render neutral casts correctly to render visually pleasing images. white balanced image

12 White Balance Tools Digital Cameras

13 Auto White Balance

14 Gray Cards White Balance Caps

15 Take a picture of a neutral object (white or gray) Deduce the weight of each channel If the object is recoded as R w, G w, B w use weights 1/R w, 1/G w, 1/B w

16 Auto WB Custom WB

17 Color Correction Filters

18 Mixed Lighting

19 Light Filters Gel Filters Light Filters

20 White Balance under Mixed Lighting Barnard [1997] adaptation of gamut-based color constancy technique, Assumes smooth illumination Kawakami [2005] outdoor scenes with hard shadows, illuminants restricted to black-body radiators

21 Lischinski [2006] user scribbles, correct localized color casts

22 Ebner [2004] local color shifts, Gray World Assumption

23 Local Color Shift

24 Light Mixture Estimation for Spatially Varying White Balance Eugene Hsu Tom Mertens Sylvain Paris Shai Avidan Fredo Durand (Several slides from Eugene Hsu)

25 Algorithm Overview Recovers the dominant material colors and uses them to estimate the relative proportion of the two light colors at each of the pixels. Input image illuminated by two light types

26 Voting scheme to recover dominant material colors in the scene.

27 Estimate light mixture at reliable pixels and interpolate missing values.

28 Estimated light mixture is used to achieve spatially varying white balance.

29 Assumptions Two light sources specified by the user Interaction of light can be described using RGB channels only Surfaces are Lambertian and non-fluorescent - which implies that the image color is the product of illumination and reflectance. Color bleeding due to indirect illumination can be ignored

30 Image Formation Model Observed pixel color is material color multiplied by scaled light color.

31 White Balance Proper white balance is achieved by inverting the effect of the light source color.

32 Proper white balance is achieved by inverting the effect of the light source color.

33 Image model with two light sources

34 Proper white balance can be achieved if the relative proportion of the two light sources is known.

35 Solving for α is under-constrained since the actual material colors are not given.

36 Material Color Estimation Assume scene is dominated by a small set of material colors, hence reflectance spectra is sparse.

37 Material Color Estimation Assume scene is dominated by a small set of material colors, hence reflectance spectra is sparse.

38 Material Color Estimation Assume scene is dominated by a small set of material colors, hence reflectance spectra is sparse. Scene viewed in white light

39 Material Color Estimation Assume scene is dominated by a small set of material colors, hence reflectance spectra is sparse. Scene viewed in mixed light

40 Sample material colors and find the one that accounts for the observed color of most pixels.

41 Given a candidate material color

42 a pixel votes for a material color only if the observed color can be explained by a combination of given light sources.

43 If this expression holds, we say that the pixel votes for the material color.

44 48%

45 48 % 16 %

46 Light mixture estimation for reliable pixels

47 Mixture Interpolation Assume L 1B and L 2B are 1, divide out the blue channels. This looks exactly like Image Matting.

48 Interpolation is performed using Matting Laplacian [Levin et al. 2006]

49 Scene shot with multiple exposures so that ground truth is available.

50 Constraint the marked points and interpolate the rest

51 Smooth interpolation is pretty bad.

52 Edge-aware interpolation doesn t work satisfactorily either.

53 Matting Laplacian gives much better result.

54

55

56

57 Experiments Synthetic Data Input

58 Output

59 Input Output Ground Truth Comparison

60

61 Experiments Real Data Input Alpha Map Output

62 Input

63 Output

64 Input LME Local Color Shift

65 LME Local Color Shift

66 Scene Relighting Separate the two lighting contributions from the white Balanced image

67 The observed scene is a blend of two images as seen by either of the light sources in proportions α and 1- α. Multiply the white balanced image by α for the first contribution

68 Multiply the white balanced image by α for the first Contribution and by 1- α for the second contribution

69

70

71

72 We can choose new lights and add desired effects.

73 Input

74 Output

75

76 Discussion Works best for raw image data Better results for indoor scenes Handles specularities and inter-reflections Material colors should exhibit enough color variation for the voting to work. Accurate specification of light sources is required. Scalability Issues