IMAGE PROCESSIG (RRY25) USE OF FT I IMAGE PROCESSIG Optics- originofimperfectionsinimagingsystems(limited resolution/blurring related to 2D FTs)- need to understand using Continuous FT. Sampling -Capturecontinuousimageontoasetofdiscrete pixel values, can arrange to do without loss off informationn if yquist sampled, requires understanding FTs. IMAGIG, SAMPLIG -2DDISCRETEFOURIERTRASFORMPART-I Filtering -Havingnowcapturedadigitalimagecando Discrete-FT (DFT) to Fourier domain and then keep only low spatial frequencies (hence smooth image) or just high spatial frequencies (to sharpen images). Fast Convolution - If we take sampled image to Fourier domain, multiply it by a function and inverse FT we achieve a fast convolution - using the DFT can do the operations we described in previous lectures for smoothing/sharpending/edge detection etc much faster than doing eveything in image domain. 2 USE OF FT I IMAGE PROCESSIG (cont) Image Restoration -Canremoveopticdistortionsdescribed above under optics by DFT, filter multiplication and inverse DFT (see future lectures on image restoration) Image Compression -DFTtoFourierdomain,deleteand don t transmit high spatial frequencies that are not visibleto eye - can use but other transforsm (i.e. cosine and wavelet) better. EXAMPLE OF COTIUOUS FT - Diffraction limited telescope/camera imaging Consider a telescope or camera. Looking at a distant object at a single wavelength. Consider a point in the image, say the a star or point on a person s belt buckle, s(x, y) =δ(x x o.y y o ). From optics the electric field E(u,v) at the aperture is related to the continuous Fourier Transform of the source amplitude as a function of angular coordinates, that is E(u, v) =FT(s(x, y)). Object APERTURE A(u,v) CCD LES s(x,y) E(u,v) E (u,v) s(x,y) Continuous FT Continuous FT Figure : After being blocked by the aperture function A(u, v) = over a circular aperture and A(u, v) = elsewhere,then. E (u, v) =E(u, v)a(u, v) The lens does another FT such that the electric field at the CCD is the forward Fourier transform of E (u, v) so FT(E (u, v)) = s( x, y) FT(A(u, v)) 3 4
[where we use the fact that two forward Fourier transforms inverts the image FT(FT(s(x,y)) = s(-x,-y)] In fact the CCD detects the incident power falling, (square of the electric field) so the effective convolving function (Point Spread Function of PSF) is FT(A(u, v)) 2. All points on the image are independant in their electric field properties so it can be shown that when viewing a complicated image s(x,y) the image formed at the CCD is s(-x.-y) * FT(A(u, v)) 2 The final image formed by the lens is thus the true image convolved with a Point Spread Function (PSF). For a circular aperture this is an Airy function or Modified Bessel fnction squared. A cluster of stars as imaged by a telescope is then not seen as a set of points. Instead each point is a little disk surrounded by weak ripples. The disk gets smaller as the lens gets bigger and the resolving power of the telescope or camera gets larger. LES AS SPATIAL LOW PASS FILTER Consider camera viewing corrugated roof functions of different spatial frequency (cycles/radian). For low spatial frequency (or large angular spacing θ beween peaks) light just pass through the lens and produces a ripple on the CCD of wavelength L θ on the CCD (where lens focal length is L). But there is a maximum spatial frequency (minimum θ min ) that passes through, for this spatial frequency the FT of the input image (Electric field at aperture) lies outside the range passed by the lens. This minimum θ min = λ/d, where d is lens diameter and λ is wavelength, corresponding to a maximum spatial freq of /θ = d/λ. Hence the minumum spacing between ripple peaks on CCD is then x min = Lλ/d. The convolution in the image domain by FT(A(u, v)) 2 is equivalent to multiplying the FT of the image by the autocorrelation of the aperture A(u,v) with itself - see autocorrelation theoroem in last lecture. The radius within which this function in non-zero is equal to the lens diameter. Hence aaperture/lensactsasalowpassspatialfrequencyfilter, passing spatial freq smaller than d/λ. 5 6 YQUIST SAMPLIG After passing through the lens, the image that falls on the CCD is a bandlimited function, i.e contains a maximum spatial frequency. If we can have a CCD spacing which is x min /2(yquist sampling) then we can recover all information in the continuous image. Figure 2: To see this consider the sampled image and it FT. If f(x, y) is the bandlimited imaging falling on the CCD then f(x, y) III x (x, y) isthesampledimage,wherethesecondfunction is the bed of nails function introduced in the last lecture. The FT of this sampled image is (using the convolution theorem) F (u, v) III / x (u, v). This is shown in the bottom figure on the next page. Figure 3: Top is the bed-of-nails sampling function (top), with δ functions separated by x. Thespectrumofthesampled image is shown at the bottom, it is the FT of the unsampled image F (u, v) copiedmultipletimesonagridof seperation / x. If light passed through an ellipical shaped lens before falling on the CCD, the FT of the image before sampling (F(u,v)) is only non-zero within the grey ellipses. In the special case of a circular lens the ellipse becomes circles 7 8
of radius d/(lλ) whichcanbeenclosedwithinsquares of size 2W u =2W v =2d/(Lλ). If this is smaller than the grid spacing / x as shown here there is no overlap between the circles. This is the yquist sampling criteria that x <Lλ/2d = x min /2 If the yquist criteria holds and there is no overlap between copies of F (u, v) thenwecanrecoverf (u, v) from the FT of the sampled image by multiplying by a top hat of width 2W u =2W v. Once we know F (u, v) we can inverse FT to get f(x, y) -andhencecanrecoverthe original pre-sampled image from the sampled data. If the sampling is less dense than yquist in the FT of the sampled image the grid has smaller seperation and the FTs will overlap, we get aliased frequencies, and we cannot recover the original unsampled image. 2D DISCRETE FOURIER TRASFORMS So far we have discussed the Continuous Fourier Transforms. Important for understanding optics of telecopes/cameras etc. Course deals mostly with the Discrete Fourier Transform (DFT), calculated by computer on a discretely sampled image. If this image is bandlimited and yquist sampled then DFT of image will be very close to continous image FT. Can use DFT on sampled image to filter, remove optical distortions, compress etc etc. For such x sampled pixel arrays the 2D Discrete Fourier Transform (DFT) and Inverse DFT are defined to be (Gonzalez and Wood definition). F (k, l) = m= n= f(m, n)exp( 2πi (km + ln)) f(m, n) = k= l= F (k, l)exp(+ 2πi (km + ln)) Where normally m,n,k,l all are assumed to run from to -. otethe definition used by MATLAB has instead for the forward transform the factor unity in front of the summation and for the inverse transform the factor / 2.Also in MATLAB indices run for to, not to -. 9 Some similarities and differences to continuous FT. -DFTusesasampledversionf(m, n) ofthecontinuous image f(x, y) -Usessumsnotintegrals -Limitsareand-not± -DFThasascalefactor(/) O-CETRED AD CETRED DFTs Using the normal definition of DFT, we get the largest amplitudes in the corners. We can obtain a plot which is more consistent with continuous Dourier transforms if we centre the DFT (i.e. implemented with the fftshift command in MATLAB). Evaluate in two steps, first a D transform of each row, then D transforms of each column of result. The DFT can be evaluated with a fast transform which takes of order 2 2 Log 2 operations for an x image. The DFT gives an aaccurate version of the continuous FT of the image falling on the CCD provided that this is a bandlimited image that has been yquist sampled. If not true the fact that the DFT works on a sampled image gives rise to aliased spatial frequencies in the FT. The DFT can be evaluated at any value of the spatial frequencies k,l. The uncentered DFT plots k,l for to -, we can instead evaluate and plot from -/2 to /2 -. This result uses the periodicity property of the DFT i.e. that F (k + j,l + k) =F (k, l), where j,k are integers. 2
Given that F (k, l) = f(m, n)exp( 2πi (km + ln)) Object f(m,n) n - m k on-centred DFT F(k,L) l - - what is the DFT at spatial frequencies k+h, l+j where handjareintegers? Extended DFT Calculated at all k,l F (k + h, l + j) = f(m, n)exp( 2πihm)exp( 2πijn)exp( 2πi (km+nl)) /2 h,m,j,m are all integers so exp( 2πihm) = and exp( 2πijn) = -/2 l /2- k /2- Hence F (k + h, l + j) =F (k, l)) The centred DFT is the contents of the blue square. Can be considered the DFT calculated in the range k=-/2 to /2-; l=-/2 to /2- Figure 4: 3 4 DFTs OF TYPICAL IMAGES Input Image In general transforms give a complex number at each sampled spatial frequency, k, l. Canplotrealandimaginary parts of transform or more commonly the amplitude, A(k, l) andphase,φ(k, l). Amplitudes normally has a very large range, often therefore plot Log(+ A(k,l)). It is the phase that contains most of the information about the position of edges in the image. The amplitude tells us mainly how sharp these edges are (see last lecture). Log Amp of Centred DFT Phase of Centred DFT many images have spikes along the k,l axes. These can be explained in terms of the approximations used in doing DFTs. Other sources of spikes are regions of sharp edges or narrow rectangles within the image (e.g. camera legs in the cameraman image). Figure 5: The log amplitude and phase of the centred DFT of the cameraman image 5 6
RELATIOSHIP BETWEE COTIUOUS FT AD DFT y f(x,y)dx y Can consider implementing DFT via continuous FTs. First take the sampled version of image f(x,y), then repeat this periodically an infinite number of times, then do a continuous FT. The result in the centre is the DFT of f(x,y) (see Following figures). x Mathematically form III x, y (x, y)) [III δx,δy (x, y)) f(x, y)] 2D Continuous Fourier Transform v F(o,v) v D Continuous FT Then take continuous FT which gives (after applyimg convolution theoreom twice) III / x,/ y (u, v)) [III /δx,/δy (u, v)) F (u, v) u Figure 6: How to implement 2D DFT via the 2D continuous FT. Illustrates relationship between the two types of transform. Take original x sampled image (tire), repeat an infinite number of times then take 2D continuous transform. The x DFT of the tire is found within the central square. Even if there are no sharp edges within the input image, there can be sharp disconti- 7 8 nuities between the top and bottom (or left and right sides) of the repeated image, as shown by the slice at the top right. When we take FT these discontinuities can give strong vertical (and horizontal) spikes along the u and v axes. EDGE EFFECTS/ORIGI OF SPIKES Can get spikes in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge. But also get large vertical spike when there is a large difference in brightness between the top and bottom of picture and get a large horizontal spike when there is adifferencebetweenleftandright. If we consider the DFT in terms of doing a continuous FT of a sampled repeated version of the input image we can understand the spikes in terms of requiring high spatial frequencies to represent discontinuities. One reason why the DFT is not optimum for image compression, power at high spatial frequencies. JPEG for instance uses the Cosine transform, which avoids this problem. Edge effects much more significant on images than in D signal processing (where there also is an effect if the starting and ending samples are different). Reason is much shorter length of each row/column compared to length of D signal (often 496 samples or longer). 9 2