A Revised Underwater Image Formation Model

Transcription

1 A Revised Underwater Image Formation Model erya Akkaynak Tali Treibitz University of Haifa Abstrat The urrent underwater image formation model desends from atmospheri dehazing equations where attenuation is a weak funtion of wavelength. We reently showed that this model introdues signifiant errors and dependenies in the estimation of the diret transmission signal beause underwater, light attenuates in a wavelengthdependent manner. Here, we show that the baksattered signal derived from the urrent model also suffers from dependenies that were previously unaounted for. In doing so, we use oeanographi measurements to derive the physially valid spae of baksatter, and further show that the wideband oeffiients that govern baksatter are different than those that govern diret transmission, even though the urrent model treats them to be the same. We propose a revised equation for underwater image formation that takes these differenes into aount, and validate it through in situ experiments underwater. This revised model might explain frequent instabilities of urrent underwater olor reonstrution models, and alls for the development of new methods.. Introdution Researhers aiming to olor orret underwater images are frequently faed with unstable results: available methods are either not robust, are too sensitive, or only work for short objet ranges. This is usually explained by the hallenge in the orretion, e.g., images having low SNR, severe distortions and loss of olor, et. Here we suggest that there is a more fundamental reason to these instabilities than merely low quality images, and show that they stem from using an inaurate image formation model. Current underwater olor orretion methods [5, 8, 9, 2, 3, 26, 27, 3, 36, 40] rely on a ommonly used image formation model first derived for haze [3, 4, 4, 7, 39]. However, light propagation in the oean differs from that in the atmosphere in major ways, whih renders this model inadequate when applied to underwater images. In pure air, attenuation (sum of absorption and sattering) is only due to gas moleules and is an inverse funtion Figure. a) On the surfae of the earth where most photographs are taken, air is mixed with aerosols whih inrease insatter, making attenuation independent of wavelength. Cloud, fog, and haze oeffiients are in situ measurements []; pure air and aerosol are theoretial values [32]. b) In the oean, attenuation is a strong funtion of wavelength. ) Oeanographer Nils G. Jerlov [2] defined 5 open (I-III) and 5 oastal oean (-9C) lasses based on the diffuse downwelling attenuation oeffiients K d he measured globally. Here we show oeffiients of beam absorption a, sattering b, attenuation, and K d for Jerlov s water types [38]. In ontrast to the atmosphere, absorption in the oean is generally not negligible ompared to sattering [22]; e.g., types I (open Paifi), IA (eastern Mediterranean), I (open Atlanti) are absorption dominated. of the fourth power of wavelength [24, 32, 25] (Fig. a). Near the surfae of the earth, however, air is mixed with solid and liquid partiles (aerosols) that reate the states we all louds, dust, haze, smoke, smog, mist, fog and vog (fog from volani ash). In the presene of aerosols, whose mean diameters an be up to ten times larger than the inident light wavelength, multiple sattering effets beome signifiant and wavelength dependeny dereases [32]. In haze and fog, sattering beomes non-seletive and attenuation

2 in the visible part of the eletromagneti spetrum beomes pratially independent of wavelength [7, 29, 30, 32] (Fig. a). In ontrast to the atmosphere, natural bodies of water exhibit extreme differenes in their optial properties (Fig. b). Some lakes are as lear as distilled water [5], some are pink [6], and others hange olor several times a year transitioning between white, blue, green, red, brown, and blak [34]. In the oean, oastal harbors are often brown and murky while offshore waters are blue and lear [, 2]. Thus, a major differene between light propagation in the atmosphere and in natural waters is wavelength dependent attenuation. Additionally, absorption in the atmosphere is generally negligible [22], but in the oean its magnitude an be omparable to, and sometimes larger than sattering [28]. In fat, the wavelength dependeny of attenuation in the oean almost entirely omes from absorption (Fig. ). The ratio of sattering to attenuation oeffiients affet the baksattered signal, whih is the main ause of degradation in underwater images [35, 37]. Thus, whether the water body is absorption dominated or sattering dominated must be taken into aount when modeling or estimating relevant olor orretion parameters. These differenes suggest that strong wavelength dependent attenuation may render the atmosphere-derived image formation model inadequate when applied to underwater senes. Indeed we reently showed [] that the diret signal obtained from the urrent underwater imaging model yields wideband attenuation oeffiients (the projetion of the physial attenuation oeffiient onto the RG domain) that depend not only on inherent properties of the water body, but also on the amera sensor, sene radiane, and imaging range. In most works, these dependenies are not taken into onsideration. For example, in [5, 2, 27] wideband attenuation oeffiients were estimated by piking values at single wavelengths from physial attenuation urves (like those in Fig. b), ignoring the sensor response of the amera used for imaging, and all the other dependenies we showed. In one ase a sum on the disrete bands of the sensor response was used but only with one water type [26]. In [6, 7] the full wavelength-dependent model was used to model underwater photographs, but was not used for image orretion. Here, we expand our work in [] and show that the urrent underwater image formation model atually introdues more dependenies that were ignored until now. Speifially, we show that the wideband oeffiient for baksatter strongly depends on the veiling light (i.e., time of day and water depth), and whether the water body is absorption or sattering dominated. Most importantly, we show that the wideband attenuation oeffiients for the diret signal and the baksattered signal are different, despite the Figure 2. Ambient light attenuates exponentially with depth d reahing an objet (here a giant uttlefish), and attenuates further between the objet and the sensor along the diretion, for a distane of z. This signal is further degraded by light sattered by partiles from all diretions towards the sensor. fat that the urrent model assumes them to be the same. Through real-world experiments, we demonstrate the need for revising the image formation model. We perform an extensive error analysis omparing the urrent model to the revised one, and suggest how relevant oeffiients an best be piked when all inputs to the revised model are not available. Our findings an explain the diffiulties enountered thus far with image orretion using the urrent model. 2. Light Transport in Sattering Media 2.. rief Oeanographi akground In the oean, inherent and apparent optial properties (IOPs and AOPs) govern light propagation (see Table for variables and abbreviations used throughout the paper). The IOPs are only a funtion of the water onstituents. These are beam absorption a( ), beam sattering b( ), and beam attenuation ( ) oeffiients, where a( )+b( ) = ( ). The AOPs depend on external fators suh as the ambient light field [28], but are easier to measure than IOPs. Here the only AOP we use is K d ( ), the diffuse attenuation oeffiient of the downwelling light, whih is the extintion experiened by light penetrating a water olumn vertially. The magnitude of K d only weakly depends on ambient light, so it has been used to haraterize the optial qualities of water bodies, suh as Jerlov s water types [2] (Fig. b,). Sine both and K d are attenuation oeffiients, whih do we use in underwater omputer vision? The answer is both, and their differene is important. While governs the radiant power lost from a ollimated beam of photons, K d is defined in terms of the derease of the ambient downwelling irradiane with depth, due to all photons heading in the downward diretion in a diffuse (or, unollimated) manner [28]. Generally, is 2-5 times greater than K d [23], and both ontribute to the effetive attenuation aptured in a photograph, as we disuss in the next setion.

3 Variable esription wavelength a( ) beam absorption oeffiient b( ) beam sattering oeffiient ( ) beam attenuation oeffiient: a( )+b( ) K d ( ) diffuse downwelling attenuation oeffiient E(z, ) irradiane L(z, ) radiane Y luminane S ( ) sensor spetral response ( ) refletane ( ) veiling light olor hannels R,G, wideband attenuation oeffiient wideband veiling light I RG image with attenuated signal J RG image with unattenuated signal d depth (vertial range) z range along LOS diretion in 3-spae baksattered light diret transmitted light F forward sattered light AOP apparent optial properties IOP inherent optial properties LOS line of sight RTE radiane transfer equation VSF volume sattering funtion Table. Variables and abbreviations used in the paper The Radiane Transfer Equation In Fig. 2, we are interested in quantifying the radiane leaving the skin of the uttlefish and reahing the point where the diver hovers. This senario, and light propagation in other sattering media (suh as air) is governed by the radiane transfer equation (RTE) [0], whih aounts for the time-dependent, three-dimensional behavior of radiane. In its simpler and more ompat lassial anonial form, the RTE for a homogeneous, soure free water body (i.e. no inelasti sattering or emission) is given by [28, 33]: ( )z + L(d; ; )=L 0(d 0; ; )e L (d; ; )e K d( )z os h i e [ ( ) K d( )os ]z, ( ) K d ( )os where L 0 (d 0 ; ; ) is the total radiane leaving the objet; L(d; ; ) is the total radiane reahing the diver; is a diretion in 3 spae, is wavelength, is the angle measured from nadir diretion (positive looking down as onventional in oeanography), and z is the geometri distane along. L is the path funtion desribing radiane gained along the diretion from photons arriving from all diretions [28]. () When viewing diretion is horizontal ( = 90 ;d = d 0 ), Eq. beomes a funtion of only one attenuation oeffiient ( ) and simplifies to: ( L(d 0; ; )=L 0(d 0; ; )e )z L (d; ; ) ( + ( e )z ) ( ) (2) When 6= 90, however, we are reminded of one of the reasons why olor reonstrution of underwater images is hallenging: depending on the viewing angle, the magnitude of effetive attenuation oeffiient in a given sene hanges! It an range from [ ( ) K d ( )] when looking up, improving visibility, to [ ( )+K d ( )] when looking down, signifiantly reduing it. The first term in both Eqs. & 2 is the objet radiane resulting from photons traveling diretly from the objet to the observer, and the seond term is the path radiane, whih aounts for the photons reahing the observer from all diretions. From here on, we will refer to them as for diret signal, and for baksattered signal. Then, Eq. an be written as: L = +. (3) Note that in Eq., we omitted the in-sattering term (also alled forward sattering, F ). This term would have represented the light that was refleted from the objet away from the line of sight (LOS), but through re-sattering, got realigned at small angles along the LOS. Authors in [35] showed quantitatively that F, and it does not ontribute signifiantly to the degradation of an image aksattered Signal, We investigated the diret signal in Eq. in []. Here, we fous on baksatter [35], also alled path radiane [33]. Partiles in the medium satter the light inident on them in many other diretions, ating as soures of light. aksatter is the signal formed by these photons reahing the observer arrying no information regarding the sene that is being viewed. In Eq., the path radiane L is the radiane gained along a diretion owing to sattering into that diretion from photons traveling in all other diretions 0 [28]. The probability of a photon traveling in a given diretion after hitting a partile is determined by the volume sattering funtion (VSF). The VSF (a fundamental IOP from whih all other sattering oeffiients are derived), hanges based on the type and onentration of partiulate matter in the water body, is diffiult to measure, and has only generally been quantified for a lear lake and a turbid oastal harbor [28]. The integral of the VSF aross all diretions yields the total sattering oeffiient b( ), whih is the main parameter governing baksatter [7, 30, 35, 4], and is readily available for Jerlov s water types [38]. Consider again an infinitesimally small disk of thikness dz, that is not on the LOS (upper disk in Fig. 2). The ra-

4 diane dl sattered from this disk in all diretions is given by [7, 20, 28, 33]: dl(z, )=b( )E(d, )dz, (4) where E(d, ) is ambient light at depth d, whih is also the radiane inident on the disk in this ase. Along the LOS, at a distane z away, the reeived radiane based on eer s Law of exponential deay beomes [, 28, 33]: d(z, )=dl(z, )e ( )z. (5) Substituting Eq. 4 in Eq. 5 and integrating with respet to z from z =0to z 2 = z gives us the baksattered signal as a funtion of wavelength : (z, )= b( )E(d, ) ( ) e ( )z. (6) When z is seleted to be large enough, we an obtain the value of baksatter at infinity, also termed veiling light. Thus as z!: ( )= b( )E(d, ) ( ) Then, the total signal T at the observer is: T = E(d, )e ( )z + ( )( e 2.4. Working in Camera Spae. (7) ( )z ). (8) Now, we assume the observer has a amera with spetral response S ( ) where = R,G, represents olor hannels. The signal in Eq. 8 is integrated to obtain the intensity of the image formed at the sensor at a horizontal distane z away from the objet: I = apple apple S ( ) ( )E(d, )e S ( ) ( )( ( )z d + e ( )z )d, where ( ) is the refletane spetrum of the objet, apple is a salar governing image exposure and amera pixel geometry [9], and and 2 define the bounds of integration over the eletromagneti spetrum. At depth d, the unattenuated image J is: J = apple The veiling light = apple (9) S ( ) ( )E(d, )d. (0) as aptured by the same sensor is: S ( ) b E d. () 2.5. The Current Underwater Imaging Model The urrent underwater imaging model for ambient illumination assumes amera response to be delta funtions (S ( )=S ( )), or alternatively, attenuation to vary negligibly with wavelength. Aordingly, Eq. 9 is simplified to [4, 8, 2, 7, 26, 30, 35]: I = J e z + ( e z ), (2) similarly to the atmospheri dehazing equation. Here the wideband (R,G,) attenuation oeffiients. 3. The Need for a Revised Model 3.. ependenies of Attenuation Coeffiients are We showed in [] that the wideband attenuation oeffiients estimated from the diret signal approximated by Eq. 2 have impliit dependenies on sensor response S ( ), imaging range z, sene refletane ( ), and irradiane E( ). From this point on, we label this oeffiient to indiate that it has been derived from the diret transmission () term. From Eq. 2, it is given as []: apple (z) =ln z. (3) (z + z) Evaluating the first term in Eq. 0 with z z 2 = z + z, we obtain []: 2 =ln6 4 S ( ) ( )E( )e S ( ) ( )E( )e ( )z d ( )(z+ z) d = z, and 3, 7 5 z. (4) Now we examine the baksattered signal. Following Eq. 9, the baksatter at the sensor at a distane z is: = apple S ( ) ( ) e ( )z d, (5) We equate this exat equation to the baksatter term of the urrent underwater imaging model given by Eq. 2: (z) = ( e z ). (6) Thus, the wideband baksatter oeffiient from Eq. 2 is: (z) = ln z, (7) Substituting Eq. 5 into Eq. 7 yields = ln R 2 S ( ) ( )( e ( )z )d R 2 ( )S ( )d! z. (8)

5 Figure 3. a) Physially valid spae of baksatter (Eq. 5) for a Nikon 90. lak x s denote the veiling light for a given water type. b) Veiling light distane (when baksatter is saturated), varies an order of magnitude aross absorption dominated (I-I) versus sattering dominated water types (II,III,-9C). Colored pathes are RG renderings of the expeted hue at 2m depth when baksatter is saturated. Water properties are used from [38]. We an see that depends on the sensor response S ( ), range z, and the veiling light, whih depends on the sattering and attenuation oeffiients b( ) and ( ), and ambient light E( ) (Eq. ). Comparing Eq. 7 with the diret signal effetive attenuation I (z) (z) = ln z, (9) reveals that the effetive wideband oeffiients from the two terms of Eq. 2 are theoretially different although they are urrently treated the same. J 3.2. Physially Valid Spae of Next, we derive the physially valid spae of analogous to that of in []. We use the spetral response of a Nikon 90 and assume CIE 65 at the surfae. We use Eq. 5 to alulate, and Eq. 7 to extrat for z values ranging from m to the veiling light distane (i.e. when baksatter saturates). Values of b( ) were taken from [38]. Fig. 3a shows for eah water type at 2m depth, where eah x denotes the veiling light alulated using Eq.. Note that the distane at whih veiling light is reahed an range from 0s to 00s of meters depending on the attenuation oeffiient of the water (Fig. 3b). Fig. 4a shows that hanges very little with z for a given water type (filled irles). This allows us to use the mean value as a representation of eah water type. We fit two lines to these means in 3-spae: one for lear water where attenuation is dominated by absorption (Fig. ; I-I), and one for water types where sattering is more dominant (Fig. ; II, III, -9C). These lines denote the lous of in 3-spae (Fig. 4b). The magnitude of veiling light is diretly proportional to the ambient light (i.e., depth), whih in turn, auses the lous of to move (Fig. 4,d). Finally, we note that while the lous of was shown to depend on amera sensor response, that of is less sensitive to it. This is likely beause the bakattered signal is formed independent of the refletane of objets in a sene, and depends, most strongly, on ambient light, whih attenuates rapidly with depth. Fig. 4e shows the lous for 74 ameras (Arriflex, Canon, Casio, Fuji, Hasselblad, Kodak, Leia, Manta, Nikon, Nokia, Olympus, Panasoni, Pentax, Phase One, Point Gray, Sony, Sigma) for absorption-dominated water types; amera details and loi for sattering-dominated water types are given in Supplementary Material Vs. Fig. 4f&g ompare and for the same sene in the same water. We simulated a diver at 5m, photographing two Mabeth harts plaed horizontally at &0m from him. We alulated the RG values of these harts using Eq. 9 with the response of a Nikon 90, assuming CIE 65 at the surfae for lear oeani water (Jerlov I, Fig. 4f), and a murky oastal harbor (Jerlov III, Fig. 4g). Then, using Eqs. 7&9, we extrated the wideband attenuation oeffiients. As we showed in [], depends on refletane, and ends up having a different value for eah path of the Mabeth hart (olored squares). The value of also depends on the distane between the amera and the harts, so the harts luster in different parts of the spae. In ontrast, there is one value per sene (blak x s), and it varies very little by distane. Yet, the value of for both water types is different than the values for both harts. 4. A Revised Image Formation Model In light of our findings in Ses. 3. & 3.2, we propose the following revised underwater image formation model: I = J e (v) z + e (v ) z. (20) Here, the vetors v and v represent the oeffiient dependenies v = {z,, E, S, } and v = {E,S,b, }. To parallel Eq. 2, we also only onsidered the horizontal imaging ase, but extension to other diretions as given in Eq. is straightforward. 5. Validation Via Real-world Experiments 5.. aksatter Estimation From Photographs The image signal I in Eq. 9 is exatly equal to baksatter for a perfet blak objet ( =0). While efforts have been made to manufature a surfae lose to a perfet blak [8], these materials are not ommerially available in small quantities. Thus, we used a olor hart and leveraged the fat that the gray pathes reflet light uniformly at different perentages in a linear image [2]. We fit a line to the RG intensities orresponding to eah path using their known luminanes (hromatiity oordinate Y ). The value of this line at Y =0, whih would have been the luminane of perfet blak (Fig. 5a), is the for that olor hannel.

6 Figure 4. Physially valid spae of. a) We alulated at a vertial depth of m, for horizontal distanes from m up to the veiling light distane. The resulting varied only slightly with horizontal distane. b) We averaged the for all horizontal distanes within eah water type, and fit two lines in 3-spae that desribe its lous in absorption (blak) and sattering (red) dominated water types. The oeffiient of determination, R2, shows an exellent linear fit for both. & d): The lous of is sensitive to ambient light, whih affets ( ). Lines are plotted aording to vertial depth, whih attenuates the ambient illumination. e) For most ommon ameras the lous of is weakly sensitive to sensor response. Here we show the lous for 74 ameras, for absorption dominated water types only, at 0 m depth, and 65 as surfae light. See Supplementary Material for amera details and other water types. f) The behavior of (blak x s) vs. (olored squares) in lear and g) murky water. As predited by Eqs. 4 & 8, and are different. Moreover, there is a single value for the entire sene, but a value for every olor (and distane) in the sene. Figure 5. a) Calulating from gray pathes, here Y = 3, 9, 30, 35, 58, 90% for Mabeth hart. The ordinates of eah line (X s) denote baksatter. b) Simulation of the expeted behavior of based on Eq. 8. ) Underwater experiment validating the baksatter estimation method in Se. 5.. d) extrated for eah hart in ), showing the expeted exponential inrease with distane. Fig. 5b shows simulated in water type II (equally dominated by absorption and sattering) using Eq. 9, along a horizontal transet of 0m, at a depth of 2m. We used the response of a Sony NEX5N and assumed surfae light was CIE 65. Fig. 5&d show the validation of this method in the oastal part of the Red Sea (type I or II []). For this experiment, we used a Sony RX00. ue to the differenes in sensor response and ambient onditions, magnitudes in Fig. 5b&d are not diretly omparable, but both onvey the expeted exponential behavior of with z Validation of lous We onduted underwater experiments in the Red Sea and the Mediterranean (Fig. 6). For eah experiment, we laid out 5 olors harts at horizontal distanes of,3,5,7 and 9m from the amera. In the Red Sea, we tested a Nikon 80 and a Sony RX00 simultaneously (Fig. 6a), taking 5-0 photographs of senes at 2&6m depth. In the Mediterranean, we photographed senes at 6&0m only with a Nikon 80. All images were taken in raw format, in manual mode, keeping the settings onsistent for eah amera for a given sene. From eah photo, we alulated as desribed in Se. 5., and extrated using Eq. 7. We did not have the Nikon 80 response, so used the mean response of all Nikons from Fig. 4e to draw the loi in Fig. 6b, and used CIE 65 for surfae light. The loi for the Red Sea 2&6m senes (Fig. 6b) were different as predited. The extrated from both Nikon (green dots) and Sony photos (red dots) aligned losely with the mean Nikon lous; eah dot represents the average oeffiient ( 2m = [0.32, 0.22, 0.8] for 2m, and 6m = [0.29, 0.25, 0.2]) of five olor harts in eah photo. The lous of is expeted to move based on sensor []. We did not have Sony RX00 response, so in Fig. 6 we show lous for mean Nikon, for water type II, for distanes varying from -9m (mathing our experiments). Eah filled irle represents for a given olor path at a given distane in the sene. As with, experimental data validate the expeted lous of, and as predited, and do not have the same values.

7 Figure 6. Real-world experiments in the Red Sea and the Mediterranean. a) We laid out 5 GK waterproof olor harts at depths of 2, 6, and 0m at horizontal distanes of,3,5,7 and 9m from the amera. In the Red Sea, we used a Nikon 80 and a Sony RX00 simultaneously. In the the Mediterranean, we only used the Nikon 80. In all ases loi were drawn for type II, aording to the mean Nikon response of ameras in Fig. 4e. b) Estimated. As our simulations predited, varies more strongly with water type and depth, than with sensor response, and oeffiients from both ameras fall on the lous. ) Estimated. As with, the oeffiients extrated from underwater experiments fall losely on the predited lous. More importantly, despite the fat that the urrent imaging model treats them to be the same, and are different. 6. Error Analysis: Current Vs. Revised Model While the model in Eq. 20 is the most aurate, we realize it is diffiult to obtain all its parameters given their dependenies. Therefore, we performed an extensive error analysis to quantify the effet of every part in the revised model. We tested seven senarios of olor reonstrution using the urrent (Eq. 2) and revised (Eq. 20) models with different parameters: S. Current model with =. S2. Current model with derived from the hart at z =m (i.e., = (z = m)). S3. Current model with = (z = 5m). S4. Revised model with the orret and (z = m). S5. Revised model with the orret and (z = 5m). S6. Revised model with the orret and (z), where (z) is obtained from the average of (z, ) values for all olor pathes for eah hart at a given z. S7. Revised model with the orret and (z, ), where (z, ) is the oeffiient alulated for eah olor path at eah range. We synthesized a sene using Eq. 9 where a diver at 2m depth is photographing 5 olor harts horizontally in front of him at z =, 3, 5, 7, 9m. For all simulations we used b and of Jerlov II, S of a Nikon 90, took E(0, ) to be CIE 65, and the refletanes of a Mabeth hart. The ground truth orresponding to this simulation are shown in Fig. 7a. Error between the unattenuated olor J (Eq. 0) and the reonstruted one Ĵ (found either inverting Eq. 2 or 20) is measured as the dissimilarity between them in RG spae (Fig. 7b): os = J Ĵ/( J Ĵ ). (2) 6.. Corretions Using The Current Model For all pathes, errors resulting from using the urrent image formation model using only are onsistent aross range z (S, blak urves in Fig. 7b), exept for olored pathes where they slightly inrease with z. In ontrast, using the urrent model with extrated from only one of the harts at z =m, averaged for all olors, yields errors that are small for z =mbut signifiantly inrease with z (S2). This is beause the magnitude of (z = m) is larger than at other z, and it also yields an inorret baksatter alulation. However when the same orretion is arried out using only (z = 5m) (S3), the overall errors are redued. This is beause (z = 5m) for every olor hannel is muh loser to the mean in the sene, and also loser in magnitude to, resulting in an aeptable baksatter removal. Thus, when using the urrent model errors are redued when alibration of oeffiients takes plaes not at proximity to the amera. Thus, if one is only able to estimate a single value, hoosing oeffiients from a mean z might prevent extremely high or low values in the orreted image. Note that this suggests that estimating oeffiients from olor harts that are lose to the amera, as is often done, is a bad habit. Yet, S3 still yields visible errors in the hues of the reonstruted olors, espeially for large z Corretions Using The Revised Model If baksatter is removed using the orret, but the olors are reonstruted using the extrated only from the first hart (S4), errors remain almost idential to S2, implying that has a more prominent role in olor reon- strution than. The only notieable improvement is in the reonstrution of blak, but the errors still inrease with

8 Figure 7. a) Ground truth values for (dashed line) and (lines olored aording to the 24 pathes). The value of almost does not hange with z, while dereases logarithmially. b) Color reonstrution error for the seven ases listed in Se. 6 using Eq. 2, and ) RG visualization of errors. Eah hart is white balaned using the white path of the ground truth image. z, likely due to the overompensation from alulated at z =m. The overompensation due to alulated at distanes lose to the amera is relevant for all water types (see Supplementary Material). In S5, we extrat from the middle hart plaed at z = 5m (yellow urves in Fig. 7b). This auses a drop in error when ompared to S4 beause as Fig. 7a shows, dereases logarithmially with inreasing distane in a sene. Thus, (z = m) greatly overompensates the olors in the rest of the sene, whereas (z = 5m) is lose to the average in the entire sene. When is expressed as a funtion of z, the error magnitudes derease signifiantly (S6). In this ase, we alulated one value alulated per hart (i.e., at the z or- responding to eah hart), whih is averaged over all refletanes. For this senario, the highest errors arise for the olors that ontain red. This is beause for those ol- ors when z > 5m the magnitude of (z) beomes less than that of (z = 5m), and (z = 5m) estimates the orresponding oeffiient better. In ontrast, for olors ontaining blue, when z>5m, the opposite happens: (z) value beomes loser to the orresponding oeffiients and the errors derease when ompared to S5. Yet, even with the higher errors assoiated with olors ontaining red, errors resulting from S6 are almost as low as errors resulting from S7 - a nearly perfet reonstrution using (z, ) (magenta urves in Fig. 7b). In real life, having for every z and for a sene is diffiult, but an be done, for example, using struture-frommotion to group pathes orresponding to the same objets at different known distanes [8]. Then our loi alulation and simulation might be used to estimate the value per distane per objet. Even if this sheme is not possible, our insights demonstrate the importane of hoosing the best distane to estimate from. 7. isussion We demonstrated through theoretial analysis and realworld experiments that the ommonly used underwater image formation model yields errors that were not aounted for thus far. We showed that the oeffiient assoiated with baksatter varies with sensor, ambient illumination, and water type; and most importantly, it is different than the oeffiient assoiated with the diret signal. These, together with dependenies we showed in [], might explain many inauraies and instabilities in urrent algorithms. Our revised model will lead to the development of methods that will better orret omplex underwater senes. Aknowledgements This work was supported by the The Leona M. and Harry. Helmsley Charitable Trust, the Maurie Hatter Foundation, and Ministry of Siene, Tehnology and Spae grant #3 2487, the Tehnion Ollendorff Minerva Center for Vision and Image Sienes, the University of Haifa institutional postdotoral program. We thank Tom Shlesinger, eborah Levy and Matan Yuval for help with experiments, and the Interuniversity Institute of Marine Sienes in Eilat for logistial support. Additionally, we thank Ella Treibitz for the invaluable support we reeived from her during various phases of this manusript. Referenes []. Akkaynak, T. Treibitz, T. Shlesinger, R. Tamir, Y. Loya, and. Iluz. What is the spae of attenuation oeffiients in underwater omputer vision? In Pro. IEEE CVPR, , 3, 4, 5, 6, 8

9 [2]. Akkaynak, T. Treibitz,. Xiao, U. A. Gürkan, J. J. Allen, U. emiri, and R. T. Hanlon. Use of ommerial off-theshelf digital ameras for sientifi data aquisition and senespeifi olor alibration. JOSA A, 3(2):32 32, [3] Y. ahat and M. Irani. lind dehazing using internal path reurrene. In Pro. IEEE ICCP, 206. [4]. erman, T. Treibitz, and S. Avidan. Non-loal image dehazing. In Pro. IEEE CVPR, 206., 4 [5]. erman, T. Treibitz, and S. Avidan. iving into hazelines: Color restoration of underwater images. In Pro. ritish Mahine Vision Conferene (MVC), 207., 2 [6] H. lasinski and J. Farrell. A three parameter underwater image formation model. Eletroni Imaging, 206(8): 8, [7] H. lasinski and J. E. Farrell. Simulation of underwater imaging systems. In Imaging Systems and Appliations, pages IT3A 3. Optial Soiety of Ameria, [8] M. ryson, M. Johnson-Roberson, O. Pizarro, and S.. Williams. True olor orretion of autonomous underwater vehile imagery. J. of Field Robotis, 205., 4, 8 [9] N. Carlevaris-iano, A. Mohan, and R. M. Eustie. Initial results in underwater single image dehazing. In Pro. IEEE/MTS Oeans, 200. [0] S. Chandrasekhar. Radiative transfer. Courier Corporation, [] C. C. Chen. Attenuation of eletromagneti radiation by haze, fog, louds, and rain. Tehnial report, RAN CORP SANTA MONICA CA, 975. [2] J. Y. Chiang and Y.-C. Chen. Underwater image enhanement by wavelength ompensation and dehazing. IEEE Trans. Image Proessing, 2(4): , 202., 2, 4 [3] P. rews, E. Nasimento, F. Moraes, S. otelho, and M. Campos. Transmission estimation in underwater single images. In Pro. IEEE ICCV Underwater Vision Workshop, pages , 203. [4] R. Fattal. ehazing using olor-lines. ACM Trans. on Graphis (TOG), 34():3, 204. [5] M. P. Gall, R. J. avies-colley, and R. A. Merrilees. Exeptional visual larity and optial purity in a sub-alpine lake. Limnology and Oeanography, 58(2):443 45, [6] U. T. Hammer. Saline lake eosystems of the world, volume 59. Springer Siene & usiness Media, [7] K. He, J. Sun, and X. Tang. Single image haze removal using dark hannel prior. Trans. IEEE PAMI, 33(2): , 20., 2, 3, 4 [8] M. K. Hedayati, M. Javaherirahim,. Mozooni, R. Abdelaziz, A. Tavassolizadeh, V. S. K. Chakravadhanula, V. Zaporojthenko, T. Strunkus, F. Faupel, and M. Elbahri. esign of a perfet blak absorber at visible frequenies using plasmoni metamaterials. Advaned Materials, 23(45): , [9] C. P. Huynh and A. Robles-Kelly. Comparative olorimetri simulation and evaluation of digital ameras using spetrosopy data. In igital Image Computing Tehniques and Appliations, 9th iennial Conferene of the Australian Pattern Reognition Soiety on, pages IEEE, [20] J. S. Jaffe. Computer modeling and the design of optimal underwater imaging systems. IEEE J. Oeani Eng., 5(2):0, [2] N. Jerlov. Irradiane Optial Classifiation. Elsevier, 968., 2 [22] A. Kokhanovsky. Optial properties of terrestrial louds. Earth-Siene Reviews, 64(3):89 24, 2004., 2 [23] Z. Lee, S. Shang, C. Hu, K. u, A. Weidemann, W. Hou, J. Lin, and G. Lin. Sehi disk depth: A new theory and mehanisti model for underwater visibility. Remote sensing of environment, 69:39 49, [24] A. Levis, Y. Y. Shehner, and A.. avis. Multiplesattering mirophysis tomography. In Pro. IEEE CVPR, 207. [25] A. Levis, Y. Y. Shehner, and R. Talmon. Statistial tomography of mirosopi life. In Pro. IEEE CVPR, 208. [26] H. Lu, Y. Li, S. Nakashima, and S. Serikawa. Turbidity underwater image restoration using spetral properties and light ompensation. IEICE trans. on Information and Systems, 99():29 227, 206., 2, 4 [27] H. Lu, Y. Li, L. Zhang, and S. Serikawa. Contrast enhanement for images in turbid water. JOSA A, 32(5): , 205., 2 [28] C.. Mobley. Light and water: radiative transfer in natural waters. Aademi press, , 3, 4 [29] S. G. Narasimhan and S. K. Nayar. Chromati framework for vision in bad weather. In Pro. IEEE CVPR, volume, pages IEEE, [30] S. G. Narasimhan and S. K. Nayar. Vision and the atmosphere. Int. J. Computer Vision, 48(3): , , 3, 4 [3] Y.-T. Peng, X. Zhao, and P. C. Cosman. Single underwater image enhanement using depth estimation based on blurriness. In Pro. IEEE ICIP, pages , 205. [32] U. Platt, K. Pfeilstiker, and M. Vollmer. Radiation and Optis in the Atmosphere, pages Springer New York, New York, NY, 2007., 2 [33] R. Preisendorfer. Hydrologi optis. volume i. introdution , , 4 [34]. Rouwet,. W. Christenson, F. Tassi, and J. Vandemeulebrouk. Volani Lakes. Springer, [35] Y. Y. Shehner and N. Karpel. Clear underwater vision. In Pro. IEEE CVPR, , 3, 4 [36] Y. Y. Shehner and N. Karpel. Reovery of underwater visibility and struture by polarization analysis. IEEE J. Oeani Engineering, 30(3): , [37] M. Sheinin and Y. Y. Shehner. The next best underwater view. In Pro. IEEE CVPR, pages , [38] M. G. Solonenko and C.. Mobley. Inherent optial properties of jerlov water types. Applied optis, 54(7): , 205., 3, 5 [39] R. Tan. Visibility in bad weather from a single image. In Pro. IEEE CVPR, [40] H.-Y. Yang, P.-Y. Chen, C.-C. Huang, Y.-Z. Zhuang, and Y.-H. Shiau. Low omplexity underwater image enhanement based on dark hannel prior. In Pro. IEEE Int. Conf. on Innovations in io-inspired Computing and Appliations, pages 7 20, 20.

10 [4] X. Zhao, T. Jin, and S. Qu. eriving inherent optial properties from bakground olor and underwater image enhanement. Oean Engineering, 94:63 72,