Digital Image Processing and Machine Vision Fundamentals By Dr. Rajeev Srivastava Associate Professor Dept. of Computer Sc. & Engineering, IIT(BHU), Varanasi
Overview In early days of computing, data was numerical. Later, textual data became more common. Today, many other forms of data such as voice, music, speech, images, computer graphics, etc. Each of these types of data are signals. Loosely defined, a signal is a function that conveys information. 8/3/2014 2
Relationship of Signal Processing to other fields As long as people have tried to send or receive through electronic media : telegraphs, radar, telephones, television etc there has been the realization that these signals may be affected by the system used to acquire, transmit, or process them. Sometimes, these systems are imperfect and introduce noise, distortion, or other artifacts. 8/3/2014 3
Understanding the effects these systems have and finding ways to correct them is the fundamental of signal processing. Sometimes, these signals are specific messages that we create and send to someone else (e.g., telegraph, telephone, television, digital networking, etc.). That is, we specifically introduce the information content into the signal and hope to extract it out later. 8/3/2014 4
Sometimes, these man-made signals are encoding of natural phenomena (audio signal, acquired image, etc.),but sometimes we can create them from scratch (speech generation, computer generated music, computer graphics). Finally, we can sometimes merge these technologies together by acquiring a natural signal, processing it, and then transmitting it in some fashion. 8/3/2014 5
Concerned fields of Signal Processing Digital Communication Compression Speech Synthesis and Recognition Computer Graphics Image Processing Computer Vision 8/3/2014 6
What is an image? A representation, likeness, or imitation of an object or thing A vivid or graphic description 8/3/2014 7
Why do we need images? Various imaging modalities help us to see invisible objects due to -Opaqueness (e.g., see through human body) -Far distance (e.g., remote sensing) -Small size (e.g., light microscopy) Other signals (e.g., seismic) can also be translated into images to facilitate the analysis Images are important to convey information and support reasoning A picture is worth a thousand words! 8/3/2014 8
Fields related to images Computer Graphics Image Processing Computer Vision Input /Output Image Description Image Image Processing Computer Vision Description Computer Graphics AI 8/3/2014 9
Computer Graphics Computer Graphics deals with generation of 2D computer images from the descriptions of real time 3D object or data. Computer graphics deals with designing suitable 2D scene images to simulate our 3D world. 8/3/2014 10
Image Processing Image processing is the manipulation of an image for the purpose of either extracting information from the image or producing an alternative representation of the image. Image processing is a subclass of signal concerned specifically with pictures. processing Improve image quality for human perception and/or computer interpretation. 8/3/2014 11
Computer Vision Computer vision is related to the construction of the 3D world from the observed 2D images. Computer vision deals with the analysis of image content. Computer graphics pursues the opposite direction in designing suitable 2D scene images to simulate our 3D world. Image processing can be considered as the crucial middle way connecting the vision and graphics fields. 8/3/2014 12
Computer Vision Computer Vision components: Image Processing Image Analysis Image Understanding Process Input Output Image Processing Image Analysis Image Understanding Image Image Image Image Measurements High-level description 8/3/2014 13
Related Fields Control Robotics Signal Processing Artificial Intelligence Machine Learning Computer Vision Image Processing Machine Vision Physics Imaging Mathematics NeuroBiology 8/3/2014 14
DIGITAL IMAGE 8/3/2014 15
Satellite image Volcano Kamchatka Peninsula, Russia 8/3/2014 16
Satellite image Volcano in Alaska 8/3/2014 17
Medical Images: MRI of normal brain 8/3/2014 18
Medical Images: X-ray knee 8/3/2014 19
Ultrasound: Five-month Foetus 8/3/2014 20
Astronomical images 8/3/2014 21
Categorization by Image Sources Radiation from the Electromagnetic spectrum Acoustic Ultrasonic Electronic (in the form of electron beams used in electron microscopy) Computer (synthetic images used for modelling and visualization) 8/3/2014 22
Radiation from EM Spectrum 8/3/2014 23
Electromagnetic Spectrum 8/3/2014 24
X-ray Imaging
Imaging in Ultraviolet Band
Imaging in Visible and Infrared Bands
Imaging in Microwave Band
Imaging in Radio Band
Other Imaging Modalities that uses neither of energy bands from EM radiation Acoustic Imaging :Use of sound waves to capture images of an object. Examples: Geographical Explorations, Ultrasound Imaging. Computer generated images 8/3/2014 30
Acoustic Imaging
Ultrasound Imaging
Generated Images by Computer
Application
8/3/2014 37
Image Acquisition An image is captured by a sensor such as a monochrome or color TV camera and digitized. If the output of the camera or sensor is not already in digital form, an analog-todigital converter digitizes it. 8/3/2014 40
Digital Image Acquisition Process (Ref: R.C. Gonzalez) 8/3/2014 41
FROM ANALOG TO DIGITAL Imaging systems Sample and quantize Digital storage (disk) Digital computer On-line buffer Display output Object Observe Digitize Store Process Refresh /store Record 8/3/2014 42
Digital Image Representation Images are 2D signals represented in 2D matrix form. Each element of matrix is known as pixels. Each pixel is associated with some integer/real value that represents intensity or color at that point. 8/3/2014 43
Types of images Binary Images Gray scale images Indexed color images RGB color images Multispectral images 8/3/2014 44
Binary Image Each Pixel have either 0 or 1 value. 0 white 1 black 8/3/2014 45
Intensity (Gray-Level) Image Each pixel is associated with 8-bits of Intensity. A pixel may attain any value in between 0 to 255. 8/3/2014 46
Coloured Image 8/3/2014 47
Indexed Color Images Typically 256 colors (GIF-format) 8/3/2014 48
RGB Color Images Red, green and blue channels, typically 256 levels each: 2 (3*8) = 16777216 colors. (e.g. TIF and JPEG formats) 8/3/2014 49
General Purpose Image Processing System Image Displays Mass Storage Computer Hardcopy Specialized Image Processing Hardware Image Processing Software Image Sensors Problem Domain 8/3/2014 50
Camera Hardware Required
Hardware Required Frame Grabber
Image Processing Manipulation of multidimensional signals image (photo) video f f ( x, y) ( x, y, t) CT, MRI f ( x, y, z, t)
Image Processing Why it is needed? For: Coding/compression Enhancement, restoration, reconstruction Analysis, detection, recognition, understanding Visualization 8/3/2014 54
Image Processing Tasks Enhancement Restoration Edge Detection Segmentation Compression Object Description etc. 8/3/2014 55
Image Enhancement
Another example : MRI Power Law Transformation: CR γ (Ref: R.C. Gonzalez) i. A magnetic resonance image of an upper thoracic human spine with a fracture dislocation and spinal cord impingement The picture is predominately dark An expansion of gray levels are desirable needs < 1 ii. Result after power-law transformation with = 0.6, c=1 iii. Transformation with = 0.4 (best result) iv. Transformation with = 0.3 (under acceptable level)
Image Restoration Distorted Image Restored Image
Types of Distortion in Images: Distortion due to Camera Misfocus Original image Distorted image 8/3/2014 59
Distortion due to motion Camera lens 8/3/2014 60
Distortion due to Random Noise Original image Distorted image 8/3/2014 61
Types of noises in an image 8/3/2014 62
Applications: Image Inpainting 8/3/2014 63
Image Inpainting 8/3/2014 64
Applications: Reduction of Speckle Noise From Ultrasound Images 8/3/2014 65
Applications: Reduction of Speckle Noise From Remote Sensing SAR Images 8/3/2014 66
Applications: Restoration and Enhancement of Microscopic Images 8/3/2014 67
Image Segmentation
Image Segmentation 8/3/2014 69
Applications of Segmentation Extraction of the desired object/constituent from an image Where do we require it? For volumetric analysis in MRI images for early detection of diseases like Alzheimer s, Parkinson s disease and Schizophrenia. Motion tracking : e.g. finding the speed of an aeroplane. Detection or identification of objects in an image in the field of Robotics Automatic car assembly in robotic vision Various Techniques used Edge Detection Active Contour Snake model Registration and Masking in images 8/3/2014 70
Color Image Processing
Compression
Image Compression Signal-Processing Based: Encoder f ( x, y) H g( x, y) Compressed Representation Decoder g( x, y) f ˆ ( x, y) 1 H 8/3/2014 73
Morphological Processing
Representation and Description
Representation and Description
Recognition and Description
Knowledge Base
Ex: Postal Code Problem Not all the processes are needed
Computer Vision System: Framework Data Analysis Conclusion from Analysis Input Image Image Preproce -ssing Feature Extraction Segmentn Feature Extraction Classification and description Image Analysis System Symbolic Reprstn Interpretation and description Image Understanding System 8/3/2014 80
8/3/2014 81
Application Areas of Computer Vision Controlling Processes : An industrial robot or an autonomous vehicle Detecting Events: For Visual Surveillance or people counting Organizing Information: For indexing databases of images and image sequences Modeling objects or environments: Industrial Inspection ( Detection in circuit of PCB layout) and Medical Image Analysis Interaction : As the input to a device for computer human interaction 3D- shape modeling 8/3/2014 82
Digital Image processing and Machine Vision: Applications Areas of applications Gray level image processing Biometrics (Forensic science) Remote sensing Medical Imaging DIP and Machine Vision Copy right protection- Digital water marking RADAR imaging Image Compression Digital holography And many more: object recognition, 3D vision, Robotics, Industrial Inspection etc. 8/3/2014 83
Other application Areas of Image Processing and Vision Image Registration Optical Coherence Tomography Remote Sensing and Agriculture Astronomy Digital Watermarking Microscopic image processing of biological samples Scientific Visualization 8/3/2014 84
Applications: Medical Image Registration Initial Condition of MR-PET Registration 8/3/2014 85
Final Configuration for MR-PET Registration 8/3/2014 86
Applications of Registration Register 2 MRIs of brain to visualize anatomy and tumor 8/3/2014 87
Applications of Registration Create at 3-D model for surgical planning and visualization Tumor(green), Vessels(red), Ventricles(blue), Edema (orange) 8/3/2014 88
Segmentation Applications in Medical Imaging: Semi-Automated Detection of Alzheimer s Disease using segmentation (Active-Contour Model) Coronal view Axial view Saggital view 8/3/2014 89
Framework for snakes A higher level process or a user initializes any curve close to the object boundary. The snake then starts deforming and moving towards the desired object boundary. In the end it completely shrink-wraps around the object. 8/3/2014 90
ALZHEIMER S DISEASE (AD) Most Common cause of Dementia Characterized by progressive cognitive deterioration Can be diagnosed accurately only after Biopsy Latest research : AD leads to atrophy of Hippocamus and Corpus Callosum of the Brain Corpus Callosum 8/3/2014 91
SEGMENTED HIPPOCAMPUS HIPPOCAMPUS IN CORONAL SLICE 8/3/2014 92
SEGMENTED CORPUS CALLOSUM CORPUS CALLOSUM IN SAGGITAL SLICE 8/3/2014 93
Digital Watermarking Host Image (Top Left), Watermark (Top right), Visible Watermarked image (Bottom Left), and Invisible watermark (Bottom Right) 8/3/2014 94
Applications of Computer Vision in Space 8/3/2014 95
Remote Sensing and Vision in Agriculture 8/3/2014 96
Remote Sensing and Vision in Agriculture 8/3/2014 97
References: [BOOK]Digital Image Processing By R.C. Gonzalez