Spatial Color Algorithms Milano Retinex and NASA Retinex. Alessandro Rizzi Dept. of Computer Science University of Milan

Transcription

1 Spatial Color Algorithms Milano Retinex and NASA Retinex Alessandro Rizzi Dept. of Computer Science University of Milan

2 Outline Milano Retinex differences with McCann Retinex sampling problem and locality various versions Land Designator differences with McCann and Mi-Retinex NASA Retinex variants of this family

3

4 Spatial Color Algorithm Common Structure First phase: pixel computation according to the visual (spatial) image characteristics Second phase:scaling onto the available quantization Independent computations on the 3 chromatic channels

5

6 Sequential products (Land and McCann 1971) Random paths Sequential products serching the global/local neighborhood to compute the ratios

7 The Retinex algorithm j I k I k+1!, = #. log i, j x+ 1 rl m, s x" path I x I i " $! = 1, if log I x +1 # > threshold I $ x % 0 else

8 The Retinex algorithm j I k I k+1 i

9 Mi-Retinex j I k I k+1 i

10 Mi-Retinex R i l, m, s N! = k = 1 r N i, jk l, m, s

11 j I k I k+1 i j I k I k+1 i

12 Searching the best paths: how to realize locality

13 Effects of paths

14 Brownian paths Cortical receptive field distribution [S. Zeki] Image and Vision Computing, 2000

15 Noise vs number of paths

16 Noise vs number of paths (enlargements)

17 LUT Retinex

18 LUT Retinex

19 Retinex MLV Machine Graphics and Vision, 2002

20 Retinex MLV

21 Noise comparison BRW LUT MLV (enlargements)

22 Alternative paths

23 not MI-Retinex JOSA A 2011

24 local / global

25 Termite Retinex Reconsider path exploration using swarm intelligence in search of a local reference white. Termites are also known as white ants hence the name Termite Retinex.

26 Ant Colony System Heuristic method developed by Marco Dorigo in 1997 inspired by the behavior of real ants in finding short paths between food sources and their nest. Three ideas from natural ant behavior are transferred to the artificial ant colony: The preference for paths with a high pheromone level. The higher rate of growth of the amount of pheromone on shorter paths. The trail mediated communication among ants.

27 Termite Retinex Pixels are considered as cities. A termite move on one of 8 neighboring pixels. Preference for a brighter pixel, using the bilateral distance, referred to as closeness (c): where and are the distance in coordinates and in intensity values respectively.

28 Termite Retinex Preference for paths with a low poison level in order to explore different areas of the image. The poison is the inverse of the pheromone of Dorigo

29 Termite Retinex An artificial termite k in pixel r chooses the pixel s to move to among those which do not belong to its working memory M k by applying the following probabilistic formula: where is the amount of poison on pixel u, is the closeness between pixels r and u and, α and β are parameters that allow the user to balance poison and closeness.

30 alpha and beta Test with 8 images and 20 observers 3 configurations: A. α=0.1 and β=0.9 B. α=0.9 and β=0.1 C. α=0.5 and β=0.5 Unitary quantity for poison to add. A B C 50 steps paths

31 Brownian Montagna Fynlayson Montagna and Finlayson, Constrained pseudo-brownian motion and its application to image enhancement, JOSA A (2011). Termites

32 Termites and steps original t=10, s=10 t=100, s=10 t=200, s=50 t=500, s=100 t=500, s=500

33 More results Original K=500 Ns=100 K=500 Ns=200 K=500 Ns=400

34

35 From MI- Re+nex to RSR Lot of redundancy in a path è use a «spray»

36 Reset j I k I k+1 i j I k I k+1 i

37 local white ref

38 RSR for each pixel and for each chroma+c channel NO THRESHOLD IEEE TIP, 2007

39 Point sampling + max selec+on + averaging n points (a spray) from neighborhood N different sprays

40 Spray genera+on

41 Tuning the sprays

42 Tuning number of sprays and points number of sprays number of points per spray Cumilative pixel difference between successive filtering

43 Probabilis+c formaliza+on of RSR Reference white value based on the percen+le values of the pixel popula+on Two versions of the algorithm: one with global behavior one with local behavior,

44 MI- Re+nex

45 No- Threshold MI- Re+nex W(i) = reference- white level for target i

46 RSR for low number N of sprays it is noisy All sampling processes are intrinsecally noisy

47 QBRIX ra+onale Passing from the sample average to the mean of the corresponding sampling distribu+on

48 QBRIX sampling Extract the value of interest instead of sampling, directly from the popula/on Global QBRIX: take the image histogram Local QBRIX: take the image histogram with pixels weighted according to the distance from the target The reference white level turns out to be a specific high quan+le of the popula+on

49 Playing with high quan+les

50 Color dominant removal

51 original RSR Global- QBRIX

52 Local- QBRIX Spa+ally non- homogeneous RSR Sampling Obtained by weigh+ng the entries based on the distance form the target E.g.

53

54

55 Local- QBRIX Noisless correc+on

56 ACE (Automatic Color Equalization)

57 ACE structure Spatial pixel computation Quantization scaling Pattern Recognition Letters, 2003

58 ACE structure Spatial pixel computation Quantization scaling Pattern Recognition Letters, 2003

59 1st phase Pixel recomputation For each channel C

60 1st phase Pixel recomputation For each channel C

61 1st phase Pixel recomputation For each channel C

62 1st phase Pixel recomputation For each channel C

63 1st phase Pixel recomputation For each channel C

64 1st phase Pixel recomputation For each channel C

65 Contrast tuning Linear Saturation Signum

66 Saturation Signum

67 Tested distances Original Euclidean Manhattan Euclidean 2 Manhattan 2

68 ACE structure Spatial pixel computation Quantization scaling

69 ACE structure Spatial pixel computation Quantization scaling

70 ACE: 2nd phase Dynamic tone reproduction scaling WP/GW scaling: Estimated white White Patch Estimated gray Gray World

71 ACE: 2nd phase Dynamic tone reproduction scaling WP/GW scaling: Estimated white White Patch Estimated gray Gray World R Histogram

72 ACE: 2nd phase Dynamic tone reproduction scaling WP/GW scaling: Estimated white White Patch Estimated gray Gray World R Histogram O Histogram

73 ACE: 2nd phase Dynamic tone reproduction scaling WP/GW scaling: Estimated white White Patch Estimated gray Gray World R Histogram O Histogram medium gray point R max

74 ACE: 2nd phase Dynamic tone reproduction scaling WP/GW scaling: Estimated white White Patch Estimated gray Gray World R Histogram O Histogram medium gray point R max

75 ACE: 2nd phase Dynamic tone reproduction scaling WP/GW scaling: Estimated white White Patch Estimated gray Gray World R Histogram O Histogram medium gray point R max

76 Some results Original ACE Original ACE

77

78

79 Local behaviour Original ACE

80 Local behaviour Original (255,255,255) ACE

81 Local behaviour Original ACE

82 Local behaviour Original ACE RGB differences ACE-Original around 128 gray

83 Data driven dequantization

84 Local Linear LUT

85 subsampling Local Linear LUT

86 Local Linear LUT subsampling ACE

87 Local Linear LUT subsampling ACE

88 Local Linear LUT subsampling ACE

89 RACE Retinex + ACE

90 RACE

91 RACE

92 RACE

93 STRESS

94 local black ref local white ref

95 local white ref local black ref 77

96 STRESS [ Spatial Temporal Retinex-like Envelope with Stochastic Sampling]

97

98

99

100

101 STRESS Tone Rendering

102 Original Color to Gray RGB average STRESS

103

104

105

106 The designator (1983)

107 Content of the paper Mondian experiment (van Doesburg) Dynamic version Static version Horn version Locality term designator Experiment with Hubel

108 Dynamic version

109 Static version NO Reset

110 Dynamic vs Static

111 The term Designator

112 Locality

113 Experiment with Hubel

114 A summarizing statement

115 The designator (1986)

116 Definition of Designator

117

118 Locality again

119 NASA Retinex

120 NASA Retinex

121

122 SSR vs MSR

123 NASA Retinex SSR MSR

124 Basic characteristics Filter shape does not depend on the image -----> need of CR

125

126

127

128

129 From WP to GW

130

131

132 Follow up

133 Follow up

134

135 NASA Retinex with integrated surround

136

137

138

139 SCA filtering: searching for locality

140 Changing the radius - locality top row: RSR, bo_om: STRESS Original diag diag/4 diag/8

141 Changing the number of itera+on Original

142 Changing the number of itera+on Samples=10 Itera+on=1 (RSR - STRESS)

143 Changing the number of itera+on Samples=10 Itera+on=10 (RSR - STRESS)

144 Changing the number of itera+on Samples=10 Itera+on=100 (RSR - STRESS)

145 Changing the number of itera+on Samples=10 Itera+on=500 (RSR - STRESS)

146 Changing the number of samples Itera+on=10 Samples=5 (RSR - STRESS)

147 Changing the number of samples Itera+on=10 Samples=50 (RSR - STRESS)

148 Changing the number of samples Itera+on=10 Samples=500 (RSR - STRESS)

149 YACCD2 Yet Another Color Constancy Database v2 2 objects (2D and 3D)

150 LDR: 5 light sources YACCD2

151 YACCD2 HDR: 3 light sources (built with 7 different exposures)

152 YACCD2 Original LDR (fluo cold, halogen, fluo yellow, blue led)

153 YACCD2 RSR, 20 itera+ons, 20 samples

154 YACCD2 STRESS, 20 itera+ons, 20 samples

155 ACE, Slope 5 YACCD2

156 YACCD2 RACE (slope 64, spray per pixel: 20, pixel per spray:200)

157 YACCD2 Termite (Termi+:10 passo: 100 alfa:0.1 beta:0.9 poisson:1 tabù:1 thread:8)

158 YACCD2 QBRIX density: 100, percen+le: 97

159 YACCD2 HDR generated by 7 mul+ple exposures (here +3,0,- 3) Fluo cold Use the SCAs as tone renderer

160 YACCD2 HDR generated by 7 mul+ple exposures (here +3,0,- 3) Fluo warm

161 YACCD2 HDR generated by 7 mul+ple exposures (here +3,0,- 3) Halogen

162 YACCD2 First row: RSR: 30 itera+ons, 20 samples Bo_om: STRESS: 30 itera+ons, 20 samples Fluo cold fluo warm halogen

163 Original Natural images

164 Natural images RSR, itera+on: 200, samples: 20

165 Natural images RSR, itera+on: 20, samples: 200

166 Natural images STRESS, itera+on: 200, samples: 20

167 Natural images STRESS, itera+on: 20, samples: 200

168 Natural images Termite:10 passo: 100 alfa:0.1 beta:0.9 poisson:1 tabù:1 thread:8

169 Natural images Termi+:100 passo: 100 alfa:0.1 beta:0.9 poisson:1 tabù:1 thread:8

170 ACE, slope:5 Natural images

171 ACE, slope:80 Natural images

172 Natural images RACE (slope 64, spray per pixel: 20, pixel per spray:200)

173 Natural images QBRIX, density: 100, percen+le: 97

174 Original Natural images

175 Natural images RSR, itera+on: 200, samples: 20

176 Natural images RSR, itera+on: 20, samples: 200

177 Natural images STRESS, itera+on: 200, samples: 20

178 Natural images STRESS, itera+on: 20, samples: 200

179 Natural images Termite:10 passo: 100 alfa:0.1 beta:0.9 poisson:1 tabù:1 thread:8

180 ACE, slope:5 Natural images

181 ACE, slope:80 Natural images

182 Natural images RACE (slope 64, spray per pixel: 20, pixel per spray:200)

183 Natural images QBRIX, density: 100, percen+le: 97

184 SCA and movies Used with success in movies restora+on ACE

185 SCA and movies Used with success in movies restora+on RSR

186 SCA and movies Used with success in movies restora+on STRESS

187 Characteristics

188 Image dynamic adjustment

189 Color Constancy Original Filtered No constraint and no a priori information required

190 Color Constancy Original (255,255,255) Filtered No constraint and no a priori information required

191 Color constancy by spatial comparison

192 Color constancy by spatial comparison equal different

193 Unwanted color removal Alternative solution: apply SCA only on Lightness channel

194 Local filtering effect Original Filtered

195 Local filtering effect Original Filtered RGB differences filtered-original around 128 gray

196 Contrast correction global global Original Filtered Original Filtered local local

197 ACE local contrast enhancement original image levels stretching global contrast enhancement local contrast enhancement with ACE

198 Shadow removal and gradients Orig Filt O-F Rearrange contrast locally Do not separate illuminant and reflectance

199 Tone remapping (LC) ALL Filtered Original ACE Original GW required

200 Spatial dequantization WP WP + GW

201 Applications

202 Application: image DB Matching? uncalibrated SCA Prefiltering Matching!

203 Prefiltering for computer vision or medical imaging

204 Prefiltering for CV

205 Visble-IR fusion

206 original image Vis+NIR with ACE processing

207 Interfaces visual assessment

208

209 User preferences originale ACE 20 0 luminosità colori leggibilità preferenza globale

210 Natural Pref Tech Total Natural Pref Tech Total Original ACE Filtered N/A Original ACE Filtered N/A User preferences on prints B/W Colour Carinna Parraman, Alessandro Rizzi, Searching User Preferences in Printing: A Proposal for an Automatic Solution, SPb 06 Printing Technology, St. Petersburg, June 2006

211 HDR images rendering ~8000 cd/m 2 ~1000 cd/m 2 L=205 L=105 [Gatta] 8:1 2:1

212 Automatic digital movie restoration

213 Automatic digital movie restoration

214 Automatic digital movie restoration

215 Another example Original image After filtering Hue histogram original image Hue histogram result image

216 Thank you