Digital Image Processing and Machine Vision Fundamentals

Transcription

1 Digital Image Processing and Machine Vision Fundamentals By Dr. Rajeev Srivastava Associate Professor Dept. of Computer Sc. & Engineering, IIT(BHU), Varanasi

2 Overview In early days of computing, data was numerical. Later, textual data became more common. Today, many other forms of data such as voice, music, speech, images, computer graphics, etc. Each of these types of data are signals. Loosely defined, a signal is a function that conveys information. 8/3/2014 2

3 Relationship of Signal Processing to other fields As long as people have tried to send or receive through electronic media : telegraphs, radar, telephones, television etc there has been the realization that these signals may be affected by the system used to acquire, transmit, or process them. Sometimes, these systems are imperfect and introduce noise, distortion, or other artifacts. 8/3/2014 3

4 Understanding the effects these systems have and finding ways to correct them is the fundamental of signal processing. Sometimes, these signals are specific messages that we create and send to someone else (e.g., telegraph, telephone, television, digital networking, etc.). That is, we specifically introduce the information content into the signal and hope to extract it out later. 8/3/2014 4

5 Sometimes, these man-made signals are encoding of natural phenomena (audio signal, acquired image, etc.),but sometimes we can create them from scratch (speech generation, computer generated music, computer graphics). Finally, we can sometimes merge these technologies together by acquiring a natural signal, processing it, and then transmitting it in some fashion. 8/3/2014 5

6 Concerned fields of Signal Processing Digital Communication Compression Speech Synthesis and Recognition Computer Graphics Image Processing Computer Vision 8/3/2014 6

7 What is an image? A representation, likeness, or imitation of an object or thing A vivid or graphic description 8/3/2014 7

8 Why do we need images? Various imaging modalities help us to see invisible objects due to -Opaqueness (e.g., see through human body) -Far distance (e.g., remote sensing) -Small size (e.g., light microscopy) Other signals (e.g., seismic) can also be translated into images to facilitate the analysis Images are important to convey information and support reasoning A picture is worth a thousand words! 8/3/2014 8

9 Fields related to images Computer Graphics Image Processing Computer Vision Input /Output Image Description Image Image Processing Computer Vision Description Computer Graphics AI 8/3/2014 9

10 Computer Graphics Computer Graphics deals with generation of 2D computer images from the descriptions of real time 3D object or data. Computer graphics deals with designing suitable 2D scene images to simulate our 3D world. 8/3/

11 Image Processing Image processing is the manipulation of an image for the purpose of either extracting information from the image or producing an alternative representation of the image. Image processing is a subclass of signal concerned specifically with pictures. processing Improve image quality for human perception and/or computer interpretation. 8/3/

12 Computer Vision Computer vision is related to the construction of the 3D world from the observed 2D images. Computer vision deals with the analysis of image content. Computer graphics pursues the opposite direction in designing suitable 2D scene images to simulate our 3D world. Image processing can be considered as the crucial middle way connecting the vision and graphics fields. 8/3/

13 Computer Vision Computer Vision components: Image Processing Image Analysis Image Understanding Process Input Output Image Processing Image Analysis Image Understanding Image Image Image Image Measurements High-level description 8/3/

14 Related Fields Control Robotics Signal Processing Artificial Intelligence Machine Learning Computer Vision Image Processing Machine Vision Physics Imaging Mathematics NeuroBiology 8/3/

15 DIGITAL IMAGE 8/3/

16 Satellite image Volcano Kamchatka Peninsula, Russia 8/3/

17 Satellite image Volcano in Alaska 8/3/

18 Medical Images: MRI of normal brain 8/3/

19 Medical Images: X-ray knee 8/3/

20 Ultrasound: Five-month Foetus 8/3/

21 Astronomical images 8/3/

22 Categorization by Image Sources Radiation from the Electromagnetic spectrum Acoustic Ultrasonic Electronic (in the form of electron beams used in electron microscopy) Computer (synthetic images used for modelling and visualization) 8/3/

23 Radiation from EM Spectrum 8/3/

24 Electromagnetic Spectrum 8/3/

25 X-ray Imaging

26 Imaging in Ultraviolet Band

27 Imaging in Visible and Infrared Bands

28 Imaging in Microwave Band

29 Imaging in Radio Band

30 Other Imaging Modalities that uses neither of energy bands from EM radiation Acoustic Imaging :Use of sound waves to capture images of an object. Examples: Geographical Explorations, Ultrasound Imaging. Computer generated images 8/3/

31 Acoustic Imaging

32 Ultrasound Imaging

33 Generated Images by Computer

34 Application

35

36

37 8/3/

38

39

40 Image Acquisition An image is captured by a sensor such as a monochrome or color TV camera and digitized. If the output of the camera or sensor is not already in digital form, an analog-todigital converter digitizes it. 8/3/

41 Digital Image Acquisition Process (Ref: R.C. Gonzalez) 8/3/

42 FROM ANALOG TO DIGITAL Imaging systems Sample and quantize Digital storage (disk) Digital computer On-line buffer Display output Object Observe Digitize Store Process Refresh /store Record 8/3/

43 Digital Image Representation Images are 2D signals represented in 2D matrix form. Each element of matrix is known as pixels. Each pixel is associated with some integer/real value that represents intensity or color at that point. 8/3/

44 Types of images Binary Images Gray scale images Indexed color images RGB color images Multispectral images 8/3/

45 Binary Image Each Pixel have either 0 or 1 value. 0 white 1 black 8/3/

46 Intensity (Gray-Level) Image Each pixel is associated with 8-bits of Intensity. A pixel may attain any value in between 0 to /3/

47 Coloured Image 8/3/

48 Indexed Color Images Typically 256 colors (GIF-format) 8/3/

49 RGB Color Images Red, green and blue channels, typically 256 levels each: 2 (3*8) = colors. (e.g. TIF and JPEG formats) 8/3/

50 General Purpose Image Processing System Image Displays Mass Storage Computer Hardcopy Specialized Image Processing Hardware Image Processing Software Image Sensors Problem Domain 8/3/

51 Camera Hardware Required

52 Hardware Required Frame Grabber

53 Image Processing Manipulation of multidimensional signals image (photo) video f f ( x, y) ( x, y, t) CT, MRI f ( x, y, z, t)

54 Image Processing Why it is needed? For: Coding/compression Enhancement, restoration, reconstruction Analysis, detection, recognition, understanding Visualization 8/3/

55 Image Processing Tasks Enhancement Restoration Edge Detection Segmentation Compression Object Description etc. 8/3/

56 Image Enhancement

57 Another example : MRI Power Law Transformation: CR γ (Ref: R.C. Gonzalez) i. A magnetic resonance image of an upper thoracic human spine with a fracture dislocation and spinal cord impingement The picture is predominately dark An expansion of gray levels are desirable needs < 1 ii. Result after power-law transformation with = 0.6, c=1 iii. Transformation with = 0.4 (best result) iv. Transformation with = 0.3 (under acceptable level)

58 Image Restoration Distorted Image Restored Image

59 Types of Distortion in Images: Distortion due to Camera Misfocus Original image Distorted image 8/3/

60 Distortion due to motion Camera lens 8/3/

61 Distortion due to Random Noise Original image Distorted image 8/3/

62 Types of noises in an image 8/3/

63 Applications: Image Inpainting 8/3/

64 Image Inpainting 8/3/

65 Applications: Reduction of Speckle Noise From Ultrasound Images 8/3/

66 Applications: Reduction of Speckle Noise From Remote Sensing SAR Images 8/3/

67 Applications: Restoration and Enhancement of Microscopic Images 8/3/

68 Image Segmentation

69 Image Segmentation 8/3/

70 Applications of Segmentation Extraction of the desired object/constituent from an image Where do we require it? For volumetric analysis in MRI images for early detection of diseases like Alzheimer s, Parkinson s disease and Schizophrenia. Motion tracking : e.g. finding the speed of an aeroplane. Detection or identification of objects in an image in the field of Robotics Automatic car assembly in robotic vision Various Techniques used Edge Detection Active Contour Snake model Registration and Masking in images 8/3/

71 Color Image Processing

72 Compression

73 Image Compression Signal-Processing Based: Encoder f ( x, y) H g( x, y) Compressed Representation Decoder g( x, y) f ˆ ( x, y) 1 H 8/3/

74 Morphological Processing

75 Representation and Description

76 Representation and Description

77 Recognition and Description

78 Knowledge Base

79 Ex: Postal Code Problem Not all the processes are needed

80 Computer Vision System: Framework Data Analysis Conclusion from Analysis Input Image Image Preproce -ssing Feature Extraction Segmentn Feature Extraction Classification and description Image Analysis System Symbolic Reprstn Interpretation and description Image Understanding System 8/3/

81 8/3/

82 Application Areas of Computer Vision Controlling Processes : An industrial robot or an autonomous vehicle Detecting Events: For Visual Surveillance or people counting Organizing Information: For indexing databases of images and image sequences Modeling objects or environments: Industrial Inspection ( Detection in circuit of PCB layout) and Medical Image Analysis Interaction : As the input to a device for computer human interaction 3D- shape modeling 8/3/

83 Digital Image processing and Machine Vision: Applications Areas of applications Gray level image processing Biometrics (Forensic science) Remote sensing Medical Imaging DIP and Machine Vision Copy right protection- Digital water marking RADAR imaging Image Compression Digital holography And many more: object recognition, 3D vision, Robotics, Industrial Inspection etc. 8/3/

84 Other application Areas of Image Processing and Vision Image Registration Optical Coherence Tomography Remote Sensing and Agriculture Astronomy Digital Watermarking Microscopic image processing of biological samples Scientific Visualization 8/3/

85 Applications: Medical Image Registration Initial Condition of MR-PET Registration 8/3/

86 Final Configuration for MR-PET Registration 8/3/

87 Applications of Registration Register 2 MRIs of brain to visualize anatomy and tumor 8/3/

88 Applications of Registration Create at 3-D model for surgical planning and visualization Tumor(green), Vessels(red), Ventricles(blue), Edema (orange) 8/3/

89 Segmentation Applications in Medical Imaging: Semi-Automated Detection of Alzheimer s Disease using segmentation (Active-Contour Model) Coronal view Axial view Saggital view 8/3/

90 Framework for snakes A higher level process or a user initializes any curve close to the object boundary. The snake then starts deforming and moving towards the desired object boundary. In the end it completely shrink-wraps around the object. 8/3/

91 ALZHEIMER S DISEASE (AD) Most Common cause of Dementia Characterized by progressive cognitive deterioration Can be diagnosed accurately only after Biopsy Latest research : AD leads to atrophy of Hippocamus and Corpus Callosum of the Brain Corpus Callosum 8/3/

92 SEGMENTED HIPPOCAMPUS HIPPOCAMPUS IN CORONAL SLICE 8/3/

93 SEGMENTED CORPUS CALLOSUM CORPUS CALLOSUM IN SAGGITAL SLICE 8/3/

94 Digital Watermarking Host Image (Top Left), Watermark (Top right), Visible Watermarked image (Bottom Left), and Invisible watermark (Bottom Right) 8/3/

95 Applications of Computer Vision in Space 8/3/

96 Remote Sensing and Vision in Agriculture 8/3/

97 Remote Sensing and Vision in Agriculture 8/3/

98 References: [BOOK]Digital Image Processing By R.C. Gonzalez