DIGITAL IMAGE PROCESSING

Transcription

1 DIGITAL IMAGE PROCESSING Lecture 1 Introduction Tammy Riklin Raviv Electrical and Computer Engineering Ben-Gurion University of the Negev

2 2 Introduction to Digital Image Processing Lecturer: Dr. Tammy Riklin Raviv Teaching assistants: Assaf Arbelle and Boris Kodner No.: Time: Wednesday 14:00-17:00 Location: Building 28 Room 204 Prerequisites: Digital Signal Processing Introduction to Stochastic Processes Course website:

3 3 Course Objectives The primary objective of this course is to provide the students the necessary computational tools to: l Understand the main principles of image processing and computer vision. l Be familiar with different classical and commonly used algorithms and understand their mathematical foundation l Implement (Matlab) and test commonly used image analysis algorithms l Develop critical reading of computer vision and digital signal processing and analysis literature

4 4 Course Resources Szeliski, Richard. Computer vision: algorithms and applications. Springer Science & Business Media, Gonzalez, Rafael C., and Richard E. Woods. "Image processing." Digital image processing 2 (2007). Forsyth, David A., and Jean Ponce. "A modern approach." Computer vision: a modern approach (2003). Duda, Richard O., Peter E. Hart, and David G. Stork. Pattern classification. John Wiley & Sons, Bishop, C. "Pattern Recognition and Machine Learning (Information Science and Statistics), 1st edn corr. 2nd printing edn." Springer, New York(2007).

5 5 What should I do in order to succeed in the course? 6 Matlab assignments assignments, each 5% -> 30% Bonus assignments Final project and presentation 70%

6 6 The instructor Tammy Riklin Raviv, Research interests: Signal processing: Biomedical Image Analysis, Computer Vision, Machine Learning Contact info: Telephone: Fax: Office: 212/33 Reception hours: please Personal web page:

7 7 The Final Project A list of possible projects will be distributed around Passover Students can choose a subject out of this list or come up with their own ideas. It is the students responsibility to schedule a meeting with the teaching assistants to discuss the project of their choice. Students can work with a single or two team mates. Students are expected to based their project on a scientific publication, make sure they understand it and are able to implement it using Matlab. All students should present their final project. Date TBD.

8 8 List of topics (Tentative) Overview on digital image processing, Visual Perception What is an image? Sampling, quantizatiotion, Histogram processing Color image processing Edge detection Frequency domain analysis, Fourier transform 2D shape representation: Hough transform, pyramids, quad trees Imaging geometry: Scaling, rotation, camera model, pose estimation

9 List of topics (tentative) Photometry, shape from shading Image segmentation representation and descriptors, SIFTs and Hogs Stereo and Motion Face detection

10 A graphical view on the syllabus

11 The Rest of Today s Class Brief Overview Human Vision and Visual Perception What is an Image?

12 The Rest of Today s Class Brief Overview Human Vision and Visual Perception What is an Image?

13 A brief overview: What is an Image? Brown s CV course

14 A Brief Overview: Image Formation Output Image Camera Sensor Brown s CV course

15 A Brief Overview: Image Formation See: Introduction to Medical Imaging Magnetic Resonance Imaging

16 A Brief Overview: Image Formation Main Focus

17 A Brief Overview: Image Formation Sensor Array CMOS sensor James Hays

18 A Brief Overview: Image Processing Quantization James Hays

19 A Brief Overview: Image Processing Images as 2D Signals Sampling

20 A Brief Overview: Image Processing Images as 2D Signals Sampling

21 Resolution geometric vs. spatial resolution Both images are ~500x500 pixels

22 A Brief Overview: Image Representation Grayscale Digital Image Brightness or intensity x y Danny Alexander

23 A Brief Overview: Light and Color Danny Alexander

24 A Brief Overview: Edge detection Danny Alexander

25 A Brief Overview: Frequency Analysis Fourier domain Image domain

26 A Brief Overview: Frequency Analysis Fourier domain Image domain

27 A Brief Overview: Frequency Analysis Fourier domain Image domain

28 A Brief Overview: Image Segmentation

29 A Brief Overview: Image Segmentation Texture segmentation

30 A Brief Overview: Image Segmentation Prior based segmentation Riklin Raviv et al, IJCV 2007

31 A Brief Overview: Image Segmentation Symmetry based segmentation Riklin Raviv et al, TPAMI 2009

32 A Brief Overview: Feature Detection

33 A Brief Overview: Correspondences

34 A Brief Overview: Correspondences

35 A Brief Overview: Stereo and 3D reconstruction

36 A Brief Overview: Object Detection and Recognition

37 A Brief Overview: Motion Optical Flow

38 A Brief Overview: Tracking Hyun Tae Na Thesis

39 A Brief Overview: Tracking RPE Arbelle s Thesis

40 The Rest of Today s Class Brief Overview Human Vision and Visual Perception What is an Image?

41 Human Vision/ Visual Perception The human eye is a camera Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris What s the sensor? photoreceptor cells (rods and cones) in the retina Slide by Steve Seitz

42 Human Vision: the Retina Cross-section of eye Cross section of retina Ganglion axons Ganglion cell layer Bipolar cell layer Pigmented epithelium Receptor layer

43 Human Vision: Retina up-close Light

44 Human Vision: Retina up-close Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision Two types of light-sensitive receptors Stephen E. Palmer, 2002 James Hays

45 Rod / Cone sensitivity

46 Distribution of Rods and Cones # Receptors/mm2 150, ,000 50, Rods 60 Cones 40 Fovea 20 0 Blind Spot Rods Cones Visual Angle (degrees from fovea) Night Sky: why are there more stars off-center? Averted vision: Stephen E. Palmer, 2002 James Hays

47 Electromagnetic Spectrum Human Luminance Sensitivity Function

48 Human Visual Perception What can we learn from human visual perception?

49 Human Perception Muller-Lyer illusion

50 Human Perception brightness constancy Ted Adelson, people/adelson/checkershadow illusion.html

51 Human Perception Hermann grid illusion Hany Farid,

52 Human Perception pop-out effect (Treisman 1985) Count the red X

53 The Rest of Today s Class Brief Overview Human Vision and Visual Perception What is an Image?

54 What is an Image? This Image is taken from Brown s Computer Vision Course

55 How would it look through the computer s eyes?

56 Why is this an image?

57 Hello Word!

58 Hello Word!

59 Hello Word!

60 Hello World J R G B

61 Let s zoom in

62 Let s Pixel Pixel = Picture Element

63 Let s Pixel

64 Let s count 1 Pixel = 8 bits (UINT 8) = 1 Byte 1,205,592 myfirstimage worth maybe 1000 words but costs much more.

65 What is an Image? An image I is a two dimensional (2D) function that maps the image domain to [0, 255] I :! [0, 255] or (for RGB) I(x) =I(x, y) =~v, v 2 [0, 255] 3 and the value of a single pixel is: I(x) =I(x, y) =v, v 2 [0, 255] I(x) =I(x, y) =(v R,v G,v B ) Gray Level Pixel RGB pixel

66 Is this an image?

67 What s the difference?

68 Building an histogram

69 Can we measure the differences?

70 A bit on information theory (in the context of images) {0, 1} Choose an arbitrary pixel in an image, can you guess its value? Well, we can build an histogram and gamble on the value with the highest frequency

71 A bit on information theory {0, 1} Choose an arbitrary pixel in an image, can you guess its value? Well, we can build an histogram and gamble on the value with the highest frequency

72 A bit on information theory (in the context of images) By normalizing an histogram, one can get the probability for the occurrence of the i-th value. The Shannon entropy (measured in bits) is given by: H = where X i p i p i log 2 (p i ) log 2 (p i ) is the self-information, which is the entropy contribution of an individual pixel.

73 Entropy of an Image What does it mean? Does it mean anything? Entropy = 7.98 Entropy = 6.98

74 Entropy of an Image What does it mean? Does it mean anything? Entropy = 7.98 Entropy = 6.98 But Entropy = 0

75 Entropy of an Image What does it mean? Does it mean anything? Entropy ( ) = Entropy ( ) Obtained by pixel permutation

76 Entropy of an Image What does it mean? Does it mean anything? SmallI1 SmallI1im Obtained by pixel permutation

77 Next-door neighbors You have chosen a pixel and you know its value. What can you say about the value of its next-door neighbor?

78 Next-door neighbors

79 Next-door neighbors

80 Next-door neighbors Random permutation of the red channel of myfirstimage

81 Mutual Information (a little bit more on information theory) The Mutual Information of two random variables is a measure of the variables mutual dependence. The most common unit of measurement of mutual information is the bit.

82 Mutual Information (a little bit more on information theory) The Mutual Information of two random variables is a measure of the variables mutual dependence. I(X; Y )= X y2y X x2x p(x, y) log p(x, y) p(x)p(y) p(x, y) is the joint probability function of X and Y. p(x),p(y) X are the marginal probability distribution functions of and (respectively). Y

83 Mini-assignment #1 (bonus) Read an image (any image) Present one of its RGB channels I1 Permute I1 and present it. Present the histogram of I1. Calculate its entropy Calculate the Mutual Information between I1 pixels and their respective left-neighbors Calculate the Mutual Information between the permutation image s pixels and their respective left-neighbors

84 Is it enough?

85 Is it enough?

86 Is it enough?

87 not yet there

88 Next Class Sampling Quantization Histogram Processing