Image Processing. What is an image? קורס גרפיקה ממוחשבת 2008 סמסטר ב' Converting to digital form. Sampling and Reconstruction.

Transcription

1 Amplitude 5/1/008 What is an image? An image is a discrete array of samples representing a continuous D function קורס גרפיקה ממוחשבת 008 סמסטר ב' Continuous function Discrete samples 1 חלק מהשקפים מעובדים משקפים של פרדו דוראנד, טומס פנקהאוסר ודניאל כהן-אור Sampling and ion Converting to digital form Convert continuous sensed data into digital form Sampling ion Quantization Sampling 3 Sampling Theory Sampling and ion How many samples are required to represent a given signal without loss of information? What signals can be reconstructed without loss for a given sampling rate? Figure 19.9 FvDFH 1

2 5/1/008 Spectral Analysis So our image (function f(x,y)) describes how the signal changes over time (x and y axes) Aliasing occurs when we use too few samples (what is enough?) What happens when we use too few samples? Aliasing Aliasing The more an image changes, the more we need to sample it. How do we measure how fast a signal changes? Frequencies 8 Figure FvDFH Fourier Joseph Fourier discovered in 18 that Any periodic function can be expressed as the sum of sines and/or cosines if different frequencies (Fourier Series) Even functions that are not periodic can be expressed as the integral of sines and/or cosines (Fourier ) Initial application was in heat diffusion Spatial domain: Function: f(x) ing: convolution Spectral Analysis Frequency domain: Function: F(u) ing: multiplication Any signal can be written as a sum of periodic functions. 10 Fourier transform: F( u) Fourier (1D) f ( x) e Inverse Fourier transform: ixu dx Fourier (1D) f ( x) F( u) e iux du Figure.6 Wolberg

3 5/1/008 Pixel operations Add random noise Add luminance Add contrast Add saturation ing Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph Sampling Theorem A signal can be reconstructed from its samples, if the original signal has no frequencies above 1/ the sampling frequency - Shannon The minimum sampling rate for bandlimited function is called Nyquist rate A signal is bandlimited if its highest frequency is bounded. The frequency is called the bandwidth. Adjusting Contrast Adjusting Brightness Compute mean luminance L for all pixels luminance = 0.30*r *g *b Scale deviation from L for each pixel component Must clamp to range (e.g., 0 to 1) Simply scale pixel components Must clamp to range (e.g., 0 to 1) L More Contrast Brighter Linear ing (Spatial Domain) Convolution Each output pixel is a linear combination of input pixels in neighborhood with weights prescribed by a filter = Pixel operations Add random noise Add luminance Add contrast Add saturation ing Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph 18 3

4 5/1/008 More on blur (lowpass filters) We can either take a uniform kernel (mean filter) Adjust Blurriness Convolve with a filter whose entries sum to one Each pixel becomes a weighted average of its neighbors Or a Gaussian kernel A Gaussian kernel tends to provide gentler smoothing and preserve edges better What do you think happens in the frequency domain? Blur 1 16 = Sharpen Sum detected edges with original image Edge Detection Convolve with a filter that finds differences between neighbor pixels Sharpened Detect edges = = Non-linear filtering Any operation on a neighborhood around each pixel For example: Selecting the median value of the neighborhood 3x3 5x5 Emboss Convolve with a filter that highlights gradients in particular directions 7x7 11x11 15x15 4 Embossed =

5 5/1/008 Quantization Reduce intensity resolution Frame buffers have limited number of bits per pixel Physical devices have limited dynamic range n=0.5 Pixel operations Add random noise Add luminance Add contrast Add saturation ing Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph 6 Uniform Quantization Images with decreasing bits per pixel: P(x,y) = round(i(x,y)) Uniform Quantization I(x,y) 8 bits 4 bits bits 1 bit 8 P(x,y) bits per pixel 7 Distribute errors among pixels Exploit spatial integration in our eye greater range of perceptible intensities Dithering Reducing effects of Quantization Dithering Random dither Ordered dither Error diffusion dither Halftoning Classical halftoning (8 bits) Uniform Quantization Floyd-Steinberg Dither 9 5

6 P(x,y) P(x,y) 5/1/008 Random Dither Randomize quantization errors Errors appear as noise Random Dither (8 bits) Uniform Quantization Random Dither I(x,y) I(x,y) P(x, y) = trunc(i(x, y) + noise(x,y) + 0.5) 1 bit D Bayer s ordered dither matrices 4Dn D (1,1) U Dn 4Dn D (,1) U n n D Ordered Dither 4D 4D n n D (1,) U D (,) U n n For each pixel (x,y) oldpixel = I(x,y) +D(x mod n,y mod n) P(x,y)= find_closest_color(oldpixel) Ordered Dither Pseudo-random quantization errors Matrix stores pattern of threshholds 3 1 D 0 Basic idea: organize successive integers such that the average distance between two successive numbers in the map is as large as possible Ordered Dither Ordered Dither An example Palette consists of 8 red tones, 8 green tones and their combinations (64 colors) image had colors (8 bits) Random Dither Ordered Dither Undithered Dithered 35 6

7 5/1/008 Floyd-Steinberg Algorithm for (x = 0; x < width; x++) { for (y = 0; y < height; y++) { P(x,y) = trunc(i(x,y) + 0.5) e = I(x,y) - P(x,y) I(x,y+1) += a*e; I(x+1,y-1) += b*e; I(x+1,y) += g*e; I(x+1,y+1) += d *e; } } Error Diffusion Dither Spread quantization error over neighbor pixels Error dispersed to pixels right and below b g a b g d 1.0 a d Figure 14.4 from H&B More examples Threshold Random Bayer Error Diffusion Dither Floyd-Steinberg Jarvice, Judice & Ninke Stucki Burkes (8 bits) Random Dither Ordered Dither Floyd-Steinberg Dither 40 Classical Halftoning Use dots of varying size to represent intensities Area of dots proportional to intensity in image Reducing effects of Quantization Dithering Random dither Ordered dither Error diffusion dither Halftoning Classical halftoning I(x,y) P(x,y) 41 7

8 5/1/008 Halftone patterns Classical Halftoning Use cluster of pixels to represent intensity Trade spatial resolution for intensity resolution Newspaper Image Figure from H&B From New York Times, 9/1/99 Pixel operations Add random noise Add luminance Add contrast Add saturation ing Blur Detect edges Sharpen Emboss Median Quantization Uniform Quantization Floyd-Steinberg dither Warping Scale Rotate Warps Combining Composite Morph Halftone patterns How many intensities in a n x n cluster? Figure from H&B Image Warping Image Warping Issues How do we specify where every pixel goes? (mapping) How do we compute colors at destination pixels? (resampling) Move pixels of image Warp Warp Source image Destination image Source image Destination image 8

9 5/1/008 Image Warping Example Image warping requires resampling of image Image Scaling (x,y ) = (sx*x, sy*y); I(x,y ) =? Resampling Aliasing (again) In general: Artifacts due to under-sampling or poor reconstruction Specifically, in graphics: Spatial aliasing Temporal aliasing BACK TO SAMPLING Under-sampling Figure FvDFH 51 Spatial Aliasing Spatial Aliasing Artifacts due to limited spatial resolution Artifacts due to limited spatial resolution Jaggies 9

10 5/1/008 Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering Temporal Aliasing Artifacts due to limited temporal resolution Strobing Flickering ed function Antialiasing at higher rate Not always possible Doesn t always solve problem Pre-filter to form bandlimited signal Form bandlimited function (low-pass filter) Trades aliasing for blurring 10

11 5/1/008 ed function ed function Discrete s Continuous Function ed function ed function ed Function ed Function ed function ed function Discrete samples Bandlimited Function 11

12 Convolution 5/1/008 Frequency domain Spatial domain Ideal Bandlimiting sin x Sinc( x) x Figure 4.5 Wolberg ed function Triangle Convolution with triangle filter Input Output Figure.4 Wolberg Finite low-pass filters Point sampling (bad) Triangle filter Gaussian filter Practical ed function Gaussian Convolution with Gaussian filter AND BACK TO WARPING Input Output 7 Figure.4 Wolberg 1

13 5/1/008 Image Resampling Compute weighted sum of pixel neighborhood Output is weighted average Image Resampling What if we are resampling a D image? dst(u,v)=0; for(ix=u-w;ix<=u+w;ix++) for(iy=v-w;iy<=v+w;iy++) d=dist between (ix,iy) and (u,v) dst(u,v) += k(ix,iy) * src(ix,iy) (u,v) W (u,v) d (ix,iy) Image Resampling For isotropic Triangle and Gaussian filters, k(ix,iy) is a function of d and w Image Resampling For isotropic Triangle and Gaussian filters, k(ix,iy) is a function of d and w (u,v) W (u,v) W d d (ix,iy) (ix,iy) Gaussian ing Triangle ing (width <= 1) Kernel is a Guassian function (u,v) d w 3 Bilinearly interpolate four closest pixels a = linear interpolation of src(u 1,v ) and src(u,v ) b = linear interpolation of src(u 1,v 1 ) and src(u,v 1 ) dst(x,y) = linear interpolation of a and b a (u 1,v ) (u,v ) (u,v) (ix,iy) (u 1,v 1 ) b (u,v 1 ) 13

14 5/1/008 Point sampling Simple but causes aliasing How do we resample? Triangle and Gaussian Algorithm as we saw earlier Float resample(src,u,v,w) { int iu = round(u); int iv = round(v); return src(iu,iv); } 80 Scale (src, dst, sx, sy): w max(1/sx,1/sy); for (int ix = 0; ix < xmax; ix++) { for (int iy = 0; iy < ymax; iy++) { float u = ix / sx; float v = iy / sy; dst(ix,iy) = resample(src,u,v,k,w); } } v (u,v) y u Scale 0.5 Image Scale (x,y) x Image Warping (in General) Alternative (forward) Image Warping (in General) Reverse Mapping 8 81 That s it for today Next time? Finishing corners on image processing ations and Projections Rendering 83 14