Limitations of the Oriented Difference of Gaussian Filter in Special Cases of Brightness Perception Illusions

Short Report Limitations of the Oriented Difference of Gaussian Filter in Special Cases of Brightness Perception Illusions Perception 2016, Vol. 45(3) 328 336! The Author(s) 2015 Reprints and permissions: sagepub.co.uk/journalspermissions.nav DOI: 10.1177/0301006615602621 pec.sagepub.com Ashish Bakshi Machine Intelligence Unit, Indian Statistical Institute, Kolkata Sourya Roy and Arijit Mallick Department of Instrumentation and Electronics Engineering, Jadavpur University, Kolkata Kuntal Ghosh Machine Intelligence Unit, Indian Statistical Institute, Kolkata Center for Soft Computing Research, Indian Statistical Institute, Kolkata Abstract The Oriented Difference of Gaussian (ODOG) filter of Blakeslee and McCourt has been successfully employed to explain several brightness perception illusions which include illusions of both brightness contrast type, for example, Simultaneous Brightness Contrast and Grating Induction and the brightness-assimilation type, for example, the White effect and the shifted White effect. Here, we demonstrate some limitations of the ODOG filter in predicting perceived brightness by comparing the ODOG responses to various stimuli (generated by varying two parameters, namely, test patch length and spatial frequency) with experimental observations of the same. Keywords brightness illusion, ODOG, spatial filtering, length scale Introduction Spatial filters are primarily used for image processing and computer vision tasks. They can also be used to develop models of visual illusion phenomena in general and brightness illusions in particular. The illusion of Simultaneous Brightness Contrast (SBC) is well known. In this illusion, a gray patch of a given intensity in a white surrounding appears darker than another gray patch of the same intensity in a black surrounding. In other words, the brightness of a test patch in Corresponding author: Ashish Bakshi, Indian Statistical Institute, Kolkata, India. Email: ashishbakshi@outlook.com

Bakshi et al. 329 the input stimulus is affected by the surrounding luminance at the edges of the bounded region. Thus, according to this phenomenon, the perceived intensity strongly depends on the surrounding intensity as well. Hence, we perceive a spatial variation in brightness for the same gray patch in different intensity surroundings. This process can also explain the Grating Induction illusion. In the White effect (Figure 1(a)), two gray test patches of equal intensity are placed over a square wave grating of alternating black and white stripes. One of the gray patches lies on top of a white bar and is flanked by black bars on both its left and right sides, while the other is on a black bar flanked by white bars on either side. In this case, the perceived luminance of the gray test patch does not depend on the perimeter intensity. It hence indicates a mechanism which contradicts the above mentioned explanation of the SBC. One explanation includes, pattern specific inhibition (White, 1979), where it is assumed that cortical filters receiving inputs from adjacent retinal locations tend to inhibit one another and might produce the effect. Another explanation concerns the T-Junction effect (Todorovic, 1997; Zaidi, Spehar, & Shy, 1997). A T-Junction is nothing but the meeting point of regions of three different intensities. The stem of the T belongs to a region known as a collinear region where the black and gray or white and gray junctions meet. The top of the stem is known as the flanking region. According to this explanation, for brightness illusions having T junctions, intensity of a given gray patch primarily depends on the adjacent intensity of the collinear region. At the collinear region, brightness contrast occurs as in SBC. To explain these phenomena, spatial filtering models of brightness perception were introduced. Earliest of these efforts include the Difference of Gaussian (DOG) model (Blakeslee & McCourt, 1997). The DOG filter successfully accounted for the SBC and Grating Induction phenomena but completely fails to explain the White effect. The DOG model was later developed into Oriented Difference of Gaussian (ODOG) model (Blakeslee & McCourt, 1999, 2004), introduced by Blakeslee and McCourt, and some modified versions of Figure 1. (a) The normal White illusion at a moderate frequency and (b) The shifted White illusion at high frequency.

330 Perception 45(3) ODOG also followed after that, such as, Locally normalized ODOG (LODOG) and Frequency specific Locally normalized ODOG (FLODOG) models of Robinson, Hammon, and de Sa (2007). The ODOG model has been extremely successful in explaining a large number of brightness illusions. In brief, the spatial filtering implementation of the ODOG model consists of a set of six, 2- D anisotropic filters. Each anisotropic filter, otherwise identical, is oriented along a different direction. The directional orientation of each filter is separated from its previous filter by a fixed angular rotation. Each of these anisotropic filters itself consists of a linear sum of an array of seven anisotropic DOG filters whose space constants are separated by octaveintervals (Georgeson & Sullivan, 1975). The outputs of each of these six anisotropic filters are then nonlinearly combined to produce the final output. This final step of nonlinear combination is called contrast normalization and involves dividing each output by its root mean square value. Hence a sum total of 42 filters give the final resultant response. While the ODOG model is very successful in explaining a large number of illusions, there still are some special cases where they fail to provide us the correct outcome. In this article, we try to explore the range of validity of the ODOG filter using the Matlab implementation of the ODOG model that has been used in the work of Robinson, Hammon, and de Sa (2007), where they compare the performances of the ODOG model with their own LODOG and FLODOG models. We have tried to highlight the limitations and failures of this forefront mathematical model of brightness perception in use today. As we have tried to study the most popular algorithm in use, extreme parametric variation should not be an exception. Parametric variations include varying spatial frequency and length of patches to extreme limits. We compare the ODOG output with responses from test subject having healthy and normal vision. Experiment In this section, we describe the experimental procedure used to obtain the experimental observations from two test subjects. Two types of stimuli were taken for test observation, the White effect (Figure 1(a)), and the shifted White effect (Figure 1(b)). These test stimuli belong to the group of illusions known as brightness assimilation illusions. As it is well known that most brightness illusions are well explained by the ODOG model (Blakeslee & McCourt, 1997, 1999, 2004), we have only considered those illusions which highlight the failures of this model. Various stimuli were generated by varying the vertical length of the gray patch (patch height) and frequency (half of inverse bar width) of the black and white bars. The gray patches had a patch intensity of 52.5 cd/m 2. These stimuli were then fed into the ODOG filter as input, and the response of the ODOG filter was noted. As already mentioned above, we have used Alan Robinson s Matlab implementation of the ODOG filter (Robinson, Hammon, & de Sa, 2007) for this comparison. The same sets of stimuli were then presented to our subjects for intensity matching experiments. The ODOG outputs were then compared with the experimental observations for each patch height or frequency value. Similar experiments have also been performed by McCourt and Blakeslee (1994). Experimental Setup For a given stimulus, the two test subjects were told to match the perceived gray-patch brightness by choosing from a separate palette of grayscale intensities varying from black to white in 26 steps (Figure 2), that was displayed beside the stimulus on the same display

Bakshi et al. 331 Figure 2. Grayscale palette used for brightness matching along with their luminance levels in candela per square meter. screen. The final intensity output is based on mean analysis of five readings obtained from each of the test subjects. Stimuli were generated on a microcomputer system. The stimuli were displayed on a high resolution (1280 pixels 1024 pixels, 43.18 cm diagonal, 5:4 aspect ratio, 33.72 cm width 26.97 cm height) LCD monitor. The stimu\li were generated as 30.5 30.5 images (viewed from a distance of 48.3 cm) on the display monitor. The monitor had been linearized with the help of a photometer. Subjects stationed their head at a distance of 48.3 cm from the display monitor with their general line of vision toward the stimulus, and readings were taken after having a constant gaze at the stimulus. This distance of 48.3 cm was chosen to comply with Alan Robinson et al. s requirement of 0.03125 ( ¼ 2 5 ) degrees per pixel, as mentioned in their implementation of the ODOG filter (Robinson, Hammon, & de Sa, 2007). Both subjects AB and AM have normal vision. Subjects were given a rest time between the consecutive illusion readings in order to avoid any afterimage of the last stimuli displayed in the screen. Observations and Results In this section, we report the results of our tests on the ODOG filter and compare them with experimental observations. This section consists of two parts, the first deals with White s illusion and the second deals with the shifted White effect. All the input stimuli shown in this section have been scaled down to fit within the available space. In all the graphs that follow, the term Gray patch on coaxial white bar refers to the gray patch which has white regions on its upper and lower boundaries, whereas the term Gray patch on coaxial black bar refers to the gray patch which has black regions bordering its upper and lower sides. The positions of the sample stimuli, Figures 3(a), 4(a), 4(c), and 4(e), are marked in the graphs. The experimental data points represent the mean value of all the readings, and the error bars represent the standard error of those readings. White s Illusion Figure 3(a) shows a White s illusion with a large patch height of 9.35 (scaled down for illustration). Its ODOG output profile, shown in Figure 3(b), however predicts that the gray patch on the left in between the white strips should be brighter than its counterpart. But in reality, the gray patch on the left, as can be readily seen in Figure 3(a), appears much darker than the right one, and the effect is very prominent and unambiguous. Figure 3(d) shows a plot of the experimental observations of the apparent brightness of the two gray patches as seen by the subjects, with respect to the patch height. It clearly shows that the gray patch on top of the black bar (i.e., the gray patch on the right in Figure 3(a)) always looks brighter than the other gray patch at all patch lengths. This is contradicted by Figure 3(c) which is a plot of the ODOG-predicted brightness of both the gray patches with respect to

332 Perception 45(3) Figure 3. (a) Input stimulus of White s illusion with patch height of 7.9 cm (9.35 at 48.3 cm) and frequency 0.47 cycles/cm (0.40 cycles/degree at 48.3 cm; scaled down to fit in available space). (b) Output intensity graph from ODOG for the stimulus shown in Figure 3(a) incorrectly predicts that the left gray patch should be brighter than the right gray patch. (c) Intensity versus patch height characteristic, as predicted by ODOG at a constant frequency of 0.40 cycles/degree. It can be seen that there is an inversion in intensity characteristic as the patch height is increased. The point of inversion in intensity is called Threshold Inversion Point (TIP; 7.48, here). (d) Experimental subject observation of intensity versus patch height at a constant frequency of 0.40 cycles/degree. (e) ODOG-predicted curves of intensity versus frequency at a fixed patch height of 9.35. (f) Experimentally observed frequency characteristic at a fixed patch height of 9.35.

Bakshi et al. 333 Figure 4. (a) Shifted White input stimulus having a patch height of 5.3 cm (6.24 at 48.3 cm) and frequency 1.9 cycles/cm (1.6 cycles/degree at 48.3 cm; scaled down to fit within available space). (b) ODOG-predicted output intensity profile of Figure 4(a) incorrectly predicts that gray patch on the left should appear darker than the gray patch on the right. (c) Shifted White input stimulus having a patch height of 5.3 cm (6.24 at 48.3 cm) and frequency 0.74 cycles/cm (0.57 cycles/degree at 48.3 cm; scaled down to fit within available space). At this frequency, our experiments show that both gray patches should appear equally bright on average, albeit with some variance between observers. (d) ODOG-predicted output intensity profile of Figure 4(c) does not corroborate with our experimental results, in this case. (e) Shifted White input stimulus having a patch height of 5.3 cm (6.24 at 48.3 cm) and frequency 0.19 cycles/cm (0.16 cycles/degree at 48.3 cm; scaled down to fit within available space). Observe that the illusory effect has been reversed compared with Figure 4(a). (f) ODOG-predicted output intensity profile correctly predicts that gray patch on the right should appear brighter than the gray patch on the left. (g) Predicted intensity versus patch height curves as obtained from ODOG filter at a frequency of 1.6 cycles/degree. (h) Experimentally observed intensity versus patch height curves at a frequency of 1.6 cycles/degree. (i) ODOG-predicted intensity versus frequency curves for a constant patch height of 6.24. (j) Experimentally observed frequency characteristics for a constant patch height of 6.24 show an inversion point at 0.57 cycles/degree.

334 Perception 45(3) Figure 4. Continued. patch height. In Figure 3(c), the two curves intersect, that is, there is a specific patch height beyond which the ODOG-predicted brightness gets inverted. This point is called the Threshold Inversion Point (TIP) from here on. The TIP is present in the ODOG output of the White effect for a very wide range of spatial frequencies. This is shown in Figure 3(e), for a patch height value of 9.35, that is, beyond the TIP. In Figure 3(e), the brightness curve for the gray patch on coaxial black bar is below the brightness curve for the gray patch on coaxial white bar. Figure 3(f) shows the corresponding experimental curves. It shows that the brightness curve for the coaxial black bar should be above the brightness curve for the gray patch on coaxial white bar, unlike the results in Figure 3(e). The Shifted White Effect The shifted White illusion, as shown in Figure 4(a), is a modified White s illusion where the portion of the grating containing to the gray patches are shifted horizontally by a distance of one bar width with respect to their upper and lower regions. As in the case of White s illusion above, the graph in Figure 4(g) shows the ODOG-predicted brightness variation with respect to patch height. An inversion in characteristic can be seen, in this case, at 4.8 patch height. We define this patch height as the TIP. This inversion phenomenon predicted by ODOG is not seen in our experimental observations (Figure 4(h)). At a patch height of 6.24 (beyond TIP of 4.8 ), the experimentally observed intensity versus spatial frequency curve is shown in Figure 4(j). The corresponding ODOG-predicted curve is shown in Figure 4(i). While the experimental observations show an inversion point in

Bakshi et al. 335 Figure 4(j), at 0.57 cycles/degree, no such inversion is seen in Figure 4(i). Thus, for the shifted White effect at long patch lengths, ODOG fails to predict the experimentally observed inversion point with respect to frequency. Summary The ODOG filter has been demonstrated by Blakeslee and McCourt (1999, 2004) to successfully explain the White and shifted White illusions. But Blakeslee and McCourt have only shown this for moderate values of gray-patch height. Through our studies, we have found that the ODOG filter fails beyond a moderate range of gray-patch height values, for both the White and shifted White illusions. From the experimental observations as mentioned above (Figures 3(d) and 4(h)), it is evident that, at a fixed spatial frequency, changing the vertical length of the gray patch should not cause significant change in its brightness. However, as shown in Figures 3(c) and 4(g), beyond a certain value of patch height, the brightness of the gray patches within the bars is predicted to be inverted by the ODOG filter, contrary to our experimental observations. Further, in case of the shifted White illusion, there is an experimentally observed inversion effect with respect to changing frequency (Figure 4(i)), which, for moderate patch heights, the ODOG filter correctly predicts, as Blakeslee and McCourt had already previously demonstrated (Blakeslee & McCourt, 2004). But they had not demonstrated this for large patch heights. For large patch heights, we have found that ODOG predicts no such inversion with changing frequency. Conclusion The ODOG filter is known to successfully predict the nature of a wide array of illusions. However, these experimental results imply that the model fails to explain certain simple parametric variations of the most common brightness perception illusions. Investigating the failure points of ODOG can help in the evaluation of newer models that aspire to explain brightness illusions. Acknowledgements The authors are very grateful to Alan Robinson for providing his Matlab implementation of the ODOG filter that has been used in the work of Robinson, Hammon, and de Sa (2007). Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Ashish Bakshi would like to thank the Council of Scientific and Industrial Research (CSIR), India, for providing him financial support which enabled him to carry out the present work.

336 Perception 45(3) References Blakeslee, B., & McCourt, M. E. (1997). Similar mechanisms underlie simultaneous brightness contrast and grating induction. Vision Research, 37, 2849 2869. Blakeslee, B., & McCourt, M. E. (1999). A multiscale spatial Eltering account of the White effect, simultaneous brightness contrast and grating induction. Vision Research, 39, 4361 4377. Blakeslee, B., & McCourt, M. E. (2004). A unified theory of brightness contrast and assimilation incorporating oriented multiscale spatial filtering and contrast normalization. Vision Research, 44, 2483 2503. Georgeson, M. A., & Sullivan, G. D. (1975). Contrast constancy: Deblurring in human vision by spatial frequency channels. Journal of Physiology (London), 252, 627 656. McCourt, M. E., & Blakeslee, B. (1994). A contrast matching analysis of grating induction and suprathreshold contrast perception. Journal of the Optical Society of American A, 11, 14 24. Robinson, A., Hammon, P., & de Sa, V. (2007). Explaining brightness illusions using spatial filtering and local response normalization. Vision Research, 47, 1631 1644. Todorovic, D. (1997). Lightness and junctions. Perception, 26, 379 395. White, M. (1979). A new effect of pattern on perceived lightness. Perception, 8, 413 416. Zaidi, Q., Spehar, B., & Shy, M. (1997). Induced effects of back-ground and foregrounds in threedimensional conegurations: The role of T-junctions. Perception, 26, 395 408.