Multimedia Systems Giorgio Leonardi A.A Lectures 14-16: Raster images processing and filters

Transcription

1 Multimedia Systems Giorgio Leonardi A.A Lectures 14-16: Raster images processing and filters

2 Outline (of the following lectures) Light and color processing/correction Convolution filters: blurring, sharpening, digital noise reduction Image transform: size enlargement/reduction Images compositing/fading

3 Image processing An image as a two-dimensional signal, which can be seen as a matrix of pixels These pixels can be processed in many ways. In these lectures we will study: Point processing: modify pixels independently Filtering: modify based on neighborhood Compositing: combine several images

4 Point processing: Light and color processing and correction

5 Color processing Many of the tools offered by a photo editing software allow us to work on colors. These tools are usually used to analyze color distribution and to correct the defects that affect a photographic image.

6 Color processing The processing algorithms consider color image pixels one at a time

7 Color processing And apply a formula to each pixel. This formula, for each color in input, produces a new color output.

8 Color processing The repetitiveness' of the procedure makes these algorithms highly parallelizable.

9 Color processing Parallelization can be performed assigning different pixels processing to different CPU threads. Furthermore, All modern video cards are equipped with a GPU containing several independent processing unit (e.g. 128). The use of the GPU to perform this processing allows the speed-up remarkably, allowing its use even in real-time applications.

10 Color processing We will focus on the following techniques: Levels of Brightness Color curves Brightness / contrast Intensity of colors Color Balance

11 Levels of Brightness One of the most common mistakes made in photography is to take a picture with the wrong exposure. The following techniques allows to detect the levels of exposure (brightness) and to correct them (at least partially) to avoid these defects.

12 Pixels for each color Levels of Brightness The histogram in the levels of brightness, indicates how many pixels in an image are characterized by a certain brightness. For example, this is a histogram of a B/W image: Black (color= 0) Brightness scale White (color= 255)

13 Levels of Brightness The histogram of an underexposed picture (too dark), is compressed to the left: most of the pixels have dark colors; few or no pixels have light colors.

14 Levels of Brightness The histogram of an overexposed picture (too light), is flattened on the right: most of the pixels have light colors; few or no pixels have dark colors.

15 Levels of Brightness In a low-contrast image, the values of the histogram are concentrated in the middle of the graph: many mid-tones pixels respect to darker and lighter pixels.

16 Levels of Brightness Pictures with proper exposition and contrast have well-balances histograms, with pixels distributed over the entire histogram.

17 Levels of Brightness For true color pictures, we can create three different histograms: one for R, one for G and one for B. These three histograms can be combined to obtain the overall color intensity (RGB) histogram Intensity RGB= 0.333*R *G *B

18 Levels of Brightness The color histogram is constructed in a very simple way, just by counting the number of pixels with a given brightness. for x = 1.. Image.w { for y = 1.. Image.h { R = Image.getRed(x,y) red[r]++ G = Image.getGreen(x,y) green[g]++ B = Image.getBlue(x,y) blue[b]++ intensity= round(0.333*r *G *B) RGB[intensity]++ } }

19 Brightness levels correction The correction of the intensity levels is done by selecting a minimum value v m and a maximum value v M.

20 Brightness levels correction All pixels with brightness levels lower than the minimum v m will be displayed with color white. All pixels with brightness levels higher than the maximum v M will be displayed with white color. The intensity levels of the other pixels will be equally scaled between the two extremes.

21 Brightness levels correction The new values for R, G and B for each pixel are thus calculated as: R = max (0, min(255, 255*(R v m )/(v M v m ))) G = max (0, min(255, 255*(G v m )/(v M v m ))) B = max (0, min(255, 255*(B v m )/(v M v m ))) The operations: max (0, min (255, c)) ensure that the output value is between 0 and 255, and are needed in all processes

22 Brightness levels correction for x = 1.. Image.w { for y = 1.. Image.h { R = 255*(Image.getRed(x, y) vm)/(vm vm) if (R < 0) then R= 0 else if (R > 255) then R= 255 Image.setRed(x, y, round(r)) G = 255*(Image.getGreen(x, y) vm)/(vm vm) if (G < 0) then G= 0 else if (G > 255) then G= 255 Image.setGreen(x, y, round(g)) } } B = 255*(Image.getBlue(x, y) vm)/(vm vm) if (B < 0) then B= 0 else if (B > 255) then B= 255 Image.setBlue(x, y, round(b))

23 Brightness levels correction Readaptation of histograms: rebuild using the previous algorithm, or adapt following these rules: In the new histogram H, the value H [0] must be H[v m ] In the new histogram H, the value H [255] must be H[v M ] The values between H[v m ] and H[v M ] must be placed proportionally Proportion: for each index I between v m and v M, its new index for H is: x= (I v m )/(v M v m ) Histogram H Histogram H

24 Brightness levels correction The following pseudo-code will re-adapt the intensity levels of the R, G, B and overall histograms, using the proportion introduced before. for I = vm.. vm { x= round ((I vm)/(vm vm)) * 255 red [x]= red[i] green [x]= green[i] blue [x]= blue[i] } RGB [x]= RGB[I]

25 Brightness levels correction v m v M

26 Brightness levels correction Correction can be applied to all the channels, or just one or two channels, to correct exposure of single color components In this case, R and G are correctly exposed, but B is underexposed, so correction can be applied on B

27 Color gamma: Color curves Gamma curve is a remapping of image gamma (proportional to tonality levels). It is specified as a function from input level to output level, used as a way to emphasize colors or other elements in a picture. It is possible to remap color gamma, using pre-defined or customized curves, on single channels or on the overall tonality

28 Color gamma: Color curves In a gamma curve, the horizontal axis specifies the input color, while the vertical axis specifies the output color:

29 Color gamma correction Gamma correction is simply performed by applying the gamma function curve to each pixel of the image:

30 Color gamma correction Gamma correction is simply performed by applying the gamma function curve to each pixel of the image: public double g (int colorvalue) { tonality function} for x = 1.. Image.w { for y = 1.. Image.h { R = g(image.getred(x, y)) Image.setRed(x, y, round (R)) G = g(image.getgreen(x, y)) Image.setGreen(x, y, roung (G)) B = g(image.getblue(x, y)) Image.setBlue(x, y, round (B)) } }

31 Color gamma correction The more complex formulas for gamma curves are calculated using parametric beizer curves (out of our scope) Beizer curves are defined between[0, 1] and return values between [0, 1], therefore our function g must be defined as: public double g (int colorvalue) { x= colorvalue/255 y= beizercurve(x) return y*255 }

32 Color tonality correction Even without beizer, we could define simple (and useful) curves: 1. No change: y= x, x=[0, 1]

33 Color tonality correction Even without beizer, we could define simple (and useful) curves: 2. Darken image colors: y= x^p, p > 1 and x=[0, 1]

34 Color tonality correction Even without beizer, we could define simple (and useful) curves: 3. Lighten image colors: y= x^p, p < 1 and x=[0, 1]

35 Color tonality correction Even without beizer, we could define simple (and useful) curves: 4. Negative: y= -x, x= [0, 1]

36 Color tonality correction Using beizer curves, more effects can be applied: 5. High contrast

37 Brightness and contrast Other common mistakes you can find in photographic images are related to brightness and contrast Even if they can be corrected using curves, simpler and effective formulas exist to process these two important features

38 Brightness and contrast Brightness is usually implemented by a parameter l that varies between -1 and 1: If l <0, the image is darkened (colors are moved in the direction of black). The following formula is used when l <0:

39 Example Original image Brightness (l= -0.6) Gamma curve

40 Brightness and contrast Brightness is usually implemented by a parameter l that varies between -1 and 1: If l > 0, the image is lightened (colors are moved in the direction of white). The following formula is used when l > 0, and it is a bit more complicated, because we want an interpolation between the current color level and the maximum value (white): R = R R l G = G G l B = B B l

41 Example Original image Contrast (l= 0.6) Gamma curve

42 Brightness and contrast The contrast k= [-1, 1] is obtained moving the color towards or away the medium value of the color to be moved: d = tan k + 1 π 4 R = R 255 G = G 255 B = B d d d 255 The coefficient d, depending from a tangent function, adds a non-linear effect on the contrast, making it easier to control.

43 Color balance The color captured by a camera, and put in the photograph, strongly depends on the light source used during shooting. The color balance eliminates unnatural colors, or allows you to create special effects, enhancing some color shade.

44 Color balance For example, the picture on the left is dominated by a strong presence of red. Removing part of the red gives a more balanced picture

45 Color balance Basically, to remove part of a color, you need to add more of his opposite:

46 Color balance Usually, photo editing tools allow you to balance the three primary colors through 3 sliders that move between a primary and its complement.

47 Color balance In principle, the color balance is applied simply by multiplying the primary colors for different coefficients.

48 Color balance For a better control of color modifications, we can use a weight w(x) which weights the intensity of the color in input, wrt on whether we want to act on the lights, the halftones or shadows. w x = for shadows 255 x x 1 2 for mid tones for lights x

49 Color balance Color balance is then calculated using the user-specified value g and the weight function to specify if we are acting on the shades, on the half-tones or on the lights. w(x) returns a value from 0 to 1, therefore It must be multiplied for 255 again R = R g r w R G = G g g w G B = B g b w B

50 Questions for you 1. How can we convert a color image into a grayscale one (from 24 to 8 bits)? 2. How can we define a function for thresholding effect? (and for n-level thresholding effect?)

51 Image filtering

52 Image filtering Point processing is very useful for color-based operations, but it is limited to accomplish tasks like sharpening or smoothing These tasks involve calculations on groups of pixels, since it is impossible, for example, to blur a pixel without using information about its neighborhood A common way to perform these operations in a non-trivial and naïve way is to carry out filtering through convolution

53 Convolution Many image processing operations can be modeled as a linear system (LSI = Linear Space Invariant system): Where f(x, y) is the pixel matrix in input; g(x, y) is the (constant invariant) transformation function, and h(x, y) is the output matrix after applying g(x,y) to f(x,y)

54 Convolution For such a system, the output h(x, y) is the convolution of f(x, y) with the impulse response g(x, y) Convolution is denoted by the operator and (for continuous signals) is defined as:

55 Convolution Since our images are discrete in space and in color, the integrals become sum, to be applied only in the dimensions of our transform function g(x, y) N 2 M 2 h x, y = f x + k, y + k g[k + N 2, l + N 2 ] k= N 2 l= M 2 The transform function g is our filter (also called kernel): an N*M matrix of integer numbers (usually, N= M= 3 or N= M= 5 on the basis of the strenght of the effect to be applied)

56 Convolution After applying the convolution, the final value must be «normalized» dividing h[i, j] for the sum of all the values in g: f x, y = N 1 i=0 1 M 1 g[i, j] j=0 N 2 M 2 f x + k, y + k g[k + N 2, l + N 2 ] k= N l= M 2 2

57 Convolution An example kernel is shown here: A kernel is a (usually) small matrix of numbers that is used in image convolutions. Differently sized kernels containing different patterns of numbers produce different results under convolution (different effects on the image). The size of a kernel is arbitrary but 3x3 is often used. Sizes are odd values. Larger kernels usually amplify its effect on the image, but is has more impact on complexity

58 Convolution example Let s apply our example kernel to an image: f[x, y] = g[i, j] = We must: Put the kernel g over the image, with its central value on the pixel to be processed Calculate the convolution h[1, 1] and divide this value for the sum of the values in g (which is 5) Store the calculated video in a second raster image f [x, y], which will contain the processed image

59 Convolution example f 1, 1 = 1 5 f x, y = 2 i=0 1 2 j=0 N 1 i=0 1 g[i, j] M 1 g[i, j] j=0 1 1 k= 1 l= 1 N 2 M 2 k= N l= M 2 2 f x + k, y + k g[k + N 2, l + N 2 ] f 1 + k, 1 + k g[k + 1, l + 1] = 0, = 233 =

60 Edge pixels What to be done with edge pixels? For example, use one of the following alternatives: 1. Wrap the image 2. Ignore edge pixels and only compute for those pixels with all neighbors 3. Duplicate edge pixels so the pixel at (2,n) (where n would be non-positive will have a value of (2,1)

61 Types of filters Uniform weight filters All values in the kernel has the same value Non-uniform weight filters Values in the kernel are different and usually follow a distribution function For example, gaussian blur is obtained using a gaussian filter Median filters The new pixel is the median value of the pixels under the mask The best choice for removing digital noise

62 Linear low pass filter: blur The simplest convolution kernel or filter is of size 3x3 with equal weights and is represented as follows: g[i, j] = This filter produces a simple average (or arithmetic mean) of the 9 nearest neighbors of each pixel in the image. It is one of a class of what are known as low pass filters. they remove short wavelengths in the image, which correspond to abrupt changes in image intensity, i.e. edges. Thus we get blurring.

63 Example g[i, j] = 3 4 f 2, 2 = 2 i=0 1 2 j=0 g[i, j] 1 1 k= 1 l= 1 f 2 + k, 2 + k g[k + 2, l + 2] = = 180

64 Example

65 High pass filter: edges High pass filter passes only short wavelengths, i.e. where there are abrupt changes in intensity and removes long wavelengths, where the image intensity is varying slowly. When applied to an image, it shows only the edges of the image.

66 High pass filter: edges High pass filtering of an image can be achieved by the application of a low pass filter to the image and subsequently subtraction of the low pass filtered result from the image. Shortly, the kernel Is obtained by subtracting the low pass filter to the Identity kernel (the one leaving the original image unchanged): h = i - l

67 High pass filter: edges l = h = i l = - =

68 Example

69 Parametrized HPF: Sharpen To sharpen the image, rather than just extract edges, we must blend (or mix) the original image with the high pass filtered image, to obtain a sort of band pass filter. The filter: s = (1-f)*i + f*h allows to mix a fraction (1-f) of identity (original image = i) and the remaining fraction (f) of high pass filtered image (h) The parameter f is a fraction between 0 and 1. When f=0 s=i and when f=1 s=h. A typical choice for sharpening is to use something in between, say, f=0.5.

70 Parametrized HPF: Sharpen h = = high pass kernel s = 1 f f = f 9 f 9 f 9 9 f 9 f 9 f f 9 f 9 f 9

71 Example f 9 f 9 f 9 f 9 9 f 9 f 9 f 9 f 9 f 9

72 Empowering the filters These filters can appropriately treat the most of the images with small/medium defects to be corrected (out-of focus, shake, ecc..). Some images show defects for which the application of 3X3 kernels is not enough to generate an appreciable result: Too much blur, image too big for the 3X3 kernel to be effective, ecc.. We need to apply stronger filters. How can our filter make a stronger impact?

73 Empowering impact of the filters We can not change the values in the kernels, since they are defined by default, but we can enlarge the size of the kernels Enlarging the size of the kernel allows the low pass filter to blur more, since each pixel is the mean value of the 25 pixels around, not only 9 Our strategy is to calculate new versions of the kernel starting from a 5X5 low pass filter

74 5X5 low pass filter The 5X5 kernel for low pass filter is defined as: h = = high pass kernel

75 5X5 identity filter The 5X5 kernel for identity filter is defined as: i = = identity kernel

76 5X5 high pass filter The 5X5 kernel for identity filter is defined as: h = = high pass kernel

77 5X5 sharpening filter How can the sharpening filter calculated? Start from the formula: s = (1-f)*i + f*h

78 3X3 vs 5X5 filters Original image 3X3 blur («blur» filter in Photoshop) 5X5 blur («blur more» in Photoshop) 3X3 sharpen («sharpen» in Photoshop) 5X5 sharpen («sharpen more» in Ph.)

79 Gaussian filter: smoothing Gaussian filters are mainly used for blurring images in a more precise way than a linear filter can do Gaussian filters are a class of linear smoothing filters with the weights chosen according to the shape of a Gaussian function. The gaussian function assigns greater importance to the central value, weighting it more than the other pixels around

80 Gaussian filter: smoothing Gaussian fiters have a kernel which is calculated automatically with the following formula: g x, y = e x2 +y2 2σ 2 Where σ is a parameter chosen by the filter designer

81 Gaussian smoothing Advantages of Gaussian filtering: rotationally symmetric (for large filters) filter weights decrease monotonically from central peak, giving most weight to central pixels Simple and intuitive relationship between size of σ and the smoothing.

82 Gaussian kernel Applying the formula: g x, y = e x2 +y2 2σ 2 we can automatically generate the values for the gaussian kernel of dimension N x N, which has indexes from -N/2 to N/2 For example, a 7X7 kernel, with σ = 2 has values: [x, y] g x, y =

83 Gaussian kernel If we desire the filter weights to be integer values for ease in computations, we can take the value at one of the corners in the array, and choose k such that this value becomes 1: [x, y] g x, y = 0.11*k = 1 k= 1/0.11 = 91

84 Gaussian kernel Now, by multiplying the rest of the weights by k, we obtain: [x, y] g x, y = This is the 7X7 kernel we can use for gaussian blur, obtained with σ = 2

85 Linear vs Gaussian Original image blur 3X3 blur («blur» filter in Photoshop) 5X5 blur («blur more» in Photoshop) Gaussian blur with σ = 1 Gaussian blur with σ = 2

86 Median filter: noise removal Digital noise in pictures is due to the electronical interferences between the image sensors in the moment where a picture is taken by a digital camera Digital noise shows itself as spots (pixels) whose colors seems random compared to their immidiate neighbors

87 Digital noise

88 Median filter: noise removal We can remove digital noise by applying blur filters. Gaussian blur offers more control than linear filters, because the the amount of blurring depends on value of σ, instead of depending only on the size of the kernel However, in many cases too many details are destroyed and the image seems clean from noise, but too blurry to be appreciated

89 Median filter Instead of performing multiplications over the kernel, the median filter extracts the median value among all the values chosen by the mask. Common dimension for median filters is 3X3:

90 Noise removal comparison Original image Linear blur 3X3 Median filter 3X3 Gaussian blur with σ = 1

91 Image resizing

92 Image resizing Image resizing is the operation transforming an image from a starting size/resolution, to a different size/resolution Resizing an image is necessary for many operations: Matching the printer s resolution to print a photographic image Reducing the resolution to maintain the size of an image on our monitor Reducing the size to publish our pictures on a social network

93 Image resizing For example, we would want to change the resolution of a photograph, to match the printing resolution of our printer. E.g. the starting image has size of 600X450 pixels, with resolution of 150 ppi image size of (600/150 X 450/150) = 4 X 3 inches Our printer prints at 300 dpi. To have the same size (4 X 3 inches), we need to rescale the original image to (300*4 X 300*3) = 1200 X 900 pixels. Thus, we need to enlarge the image.

94 Image resizing Another example: we want to change the resolution of the same photograph, to watch it with the same size on our monitor. Our starting image has size of 600X450 pixels, with resolution of 150 ppi image size of (600/150 X 450/150) = 4 X 3 inches Our monitor displays at 72 dpi. To have the same size (4 X 3 inches), we need to rescale the original image to (72*4 X 72*3) = 288 X 216 pixels. Thus, we need to reduce the size.

95 Image resizing Each operation of enlargement or reduction causes a loss of quality. The problem is due to the fact that you lose the 1:1 correspondence between the pixels in the source image, and those in the destination.

96 Image resizing Since there is no direct correspondence between the pixels, the colors of the pixels in the resized image must be "calculated" with an algorithm, resulting in an inevitable loss of quality.

97 Image resizing The ideal solution requires a match between the domain of the input and the range of the mapping function. This is achieved by converting the discrete image samples into a continuous surface, i.e. by image reconstruction. Once the input is reconstructed, it can be resampled at any position.

98 Ideal resampling Ideal image resampling is achieved through four basic steps: (1) reconstruction, (2) warping, (3) prefiltering, and (4) sampling

99 Image resampling For performance reasons, ideal resampling is very difficult to achieve, but many strategies have been defined to obtain results which match the desired quality with respect to the computational issues. Enlarging or reducing the image size (number of pixels), is performed with these two operations: Image down-sampling (resample at a lower rate) Image up-sampling (resample at a higher rate)

100 Up-sampling In the case of magnification, you will have 'a greater number of pixels than the original image.

101 Up-sampling In this case, the image is interpreted as a signal.

102 Up-sampling Up-sampling algorithm aims at reconstructing the original signal, and sample it again at a higher frequency.

103 Up-sampling This is why the techniques we are going to study are called resampling (up-sampling)

104 Up-sampling There are several ways to reconstruct the original signal to up-sample it. Among them, the most popular are: Nearest Neighbor Bilinear filtering Bicubic filtering

105 Nearest Neighbor We want to magnify our original picture f(x, y) by a factor M, we must find the position of each x and y in the new, larger, image. For each x, y of the original image: x = M*x x = x / M y = M*y y = y / M Where x, y are the new coordinates in the larger image. Since M > 1, there will be space between the new pixels. We must find a way to assign a color to these «empty» new pixels

106 Nearest neighbor The easiest case is for M=2: (0,0) (7,0) (0,0) (14,0) f(x, y) = f (x, y ) = (0,7) (7,7) Green samples are retained in the interpolated image; Orange samples are estimated from surrounding green samples. (0,14) (14,14)

107 Nearest Neighbor (x, y) (x+1, y) f(x, y) a b (i, j) (i, j) = (x /M, y /M) a= i x /M b= j y /M (x, y+1) (x+1, y+1) f (x,y ) (the resized image) takes the value of the sample at the topleft of (i= x /M, j= y /M) in f(x,y) (the original image): f [x, y ] = f[(int)(x /M), (int)(y /M)]= f[floor(x /M), floor(y /M)] Also known as pixel replication: each original pixel is replaced by MxM pixels of the sample value Equivalent to using the sample-and-hold interpolation filter.

108 Example In nearest neighbor, missing pixels are obtained by replication of the original ones: f(x, y) = f'(x, y ) = Each value in f is replicated M x M times in f

109 Exercise Given the following image: f(x, y) f(x, y) = Resample the image with nearest neighbor, with repllication factor M= 1.5

110 Nearest neighbor Nearest neighbor is the simplest algorithm to resize an image It is very easy to calculate Preserve sharp edges No support for smoothing and anti-aliasing A better support is provided by interpolation

111 Bilinear interpolation Bilinear interpolation (also called bilinear filtering), is performed as a double linear interpolation between the four (2x2) pixels around the position where the pixel is reconstructed. (x, y) (x+1, y) f(x, y) a b (i, j) (1-a) (i, j) = (x /M, y /M) x= x /M y= y /M (1-b) a= i x /M b= j y /M (x, y+1) (x+1, y+1) Interpolation is calculated as the weighted median of the four pixels around, weighted by their horizontal and vertical proximity to (i= x /M, j= y /M)

112 Bilinear interpolation (x, y) (x+1, y) f(x, y) a b (i, j) (1-b) (1-a) (i, j) = (x /M, y /M) x= x /M y= y /M a= i x /M b= j y /M (x, y+1) (x+1, y+1) Interpolation is calculated as the weighted median of the four pixels around, weighted by their horizontal and vertical proximity (not distance) to (i= x /M, j= y /M) f [x, y ]= (1-a)(1-b)f[x, y] + (a)(1-b)f[x, y + 1] + (1-a)(b)f[x + 1, y] + (a)(b)f[x + 1, y + 1] Note: with this method, there is no replication of the original values. All the points in the new image are recalculated/resampled!

113 Example f(x, y) = f'(x, y ) = We calculate the value of: f [0, 1] and M= 2. x= 0 y = ½ = 0 a= 0 b= ½ ½ = 0.5 f [x, y ]= (1-a)(1-b)f[x, y] + (a)(1-b)f[x, y + 1] + (1-a)(b)f[x + 1, y] + (a)(b)f[x + 1, y + 1] f [0, 1]= (1)(0.5)f[0, 0] + (0)(0.5)f[0, 1] + (1)(0.5)f[1, 0] + (0)(0.5)f[1, 1] = 0.5* * = = 105.5

114 Exercise Given the following image: f(x, y) f(x, y) = Resample the image with bilinear interpolation, with repllication factor M= 1.5

115 Bilinear interpolation Bilinear interpolation offers a very good tradeoff between complexity and quality of the resampled image.

116 Bicubic interpolation Bicubic interpolation uses a matrix of the 4X4 pixels near the point to be interpolated, and performs a cubic interpolation of the surface over these 16 points. Cubic interpolation is the third-grade polynomial, interpolating the known points with a piecewise spline passing through them. Bicubic means that this cubic interpolation is performed in both the horizontal and the vertical direction.

117 Bicubic Interpolation Obtaining the bicubic interpolation formula goes beyond our goals, but: It is considered the «godzilla» of resampling, due to its high computational complexity But it is the algorithm offering the best quality with negligible defects

118 Bicubic interpolation (x-1, y-1) (x, y-1) (x+1, y-1) (x+2, y-1) (i, j) = (x /M, y /M) (x-1, y) (x, y) (x+1, y) (x+2, y) x= x /M y= y /M f(x, y) a b (i, j) (1-a) a= i x /M b= j y /M (x-1, y+1) (x, y+1) (1-b) (x+1, y+1) (x+2, y+1) (x-1, y+2) (x, y+2) (x+1, y+2) (x+2, y+2) Cubic interpolation formula can be seen as the weighted median of the 16 points Weights are calculated using 4 different interpolation polynomials, which depend on the row or the column chosen. Variable a is used for horizontal interpolation and variable b for vertical interpolation

119 f(x, y) Bicubic interpolation (x-1, y-1) (x, y-1) (x+1, y-1) (x+2, y-1) (x-1, y) (x-1, y+1) (x, y) (x+1, y) (x, y+1) a b (i, j) (1-b) (1-a) (x+1, y+1) (x+2, y) (x+2, y+1) (i, j) = ( x /M, y /M ) a= i x /M b= j y /M (x-1, y+2) (x, y+2) (x+1, y+2) (x+2, y+2) 2 2 f x, y = f x + k, y + k θ k a θ l b k= 1 l= 1 Where θ k a and θ l b are the interpolating polynomial chosen on the basis of the row l or the column k

120 Bicubic interpolation 2 2 f x, y = f x + k, y + k θ k a θ l b k= 1 l= 1 Where θ k a and θ l b are the interpolating polynomial chosen on the basis of the row l or the column k

121 Bicubic interpolation Bicubic interpolation offers a the best quality of the resampled image, at the price of a heavy calculation.

122 Up-sampling visual performance

123 Image downsampling To reduce the image size, the same techniques can be applied, but specifying a value of M < 1 Nearest neighbor simply copies part of the original values in the output images (bypassing/deleting some row/column) It preserves sharp edges It is very efficient It probably introduces artefacts or glitches Bilinear filtering mediates the 4 points around the one to be shrinked Offers a good quality It is computationally good It will blur the image, losing detail Icubic interpolation calculates the interpolating polynomial surface of the 16 pixels around the one to be shrinked Offers the best quality Does not blur as much as bilinear filtering It is computationally very heavy (probably too much for downsampling)