Digital Image Fundamentals

Transcription

1 Digital Image Fundamentals Computer Science Department The University of Western Ontario Presenter: Mahmoud El-Sakka CS2124/CS2125: Introduction to Medical Computing Fall 2012 October 31,

2 Objective During the last few lectures, various medical image modalities have been introduced to you, including: X-ray Angiography Fluoroscopy Computed Axial Tomography (CAT), or simply Computed Tomography (CT) Magnetic Resonance Imaging (MRI) Ultrasound Nuclear medicine In this lecture, an attempt will be made to: Explain the common thing among all these modalities The image (to see inside the body without invasion or surgery) Demonstrate the main differences between these modalities Why do we need all these modalities? Explore ways to maximize the benefit from these images 2

3 What is an Image? An image is a way of representing the information in a given scene Describe the information represented in the image What is the dimension of each of these two image? three-dimensional image two-dimensional image something else 3

4 What is an Image Composed of? Generally, an image is composed of discrete units called picture elements (or simply pels, or pixels) Each pixel occupies a small rectangular region of the image and displays one color at a time Pixels are arranged so that they form a two-dimensional array

5 What is an Image Composed of? Images are constructed by adjusting the color of individual pixels How many pixels do we have in this image? How many different images (that have the same size as the picture shown) can be generated?

6 Images can be Color components in Images Color (each pixel displays one multi-color vlaue) Gray (each pixel displays one shade of gray) Binary (each pixel displays one of two colors black/white) 6

7 Color components in Images If we divided each of the main three axes to 256 quantization levels, how many colors will we have in total? 7

8 How Do We See Things? An object is seen/recognized from the visible light reflected from it Without light, we can not see If white light is shone onto a green object, most wavelengths are absorbed, except green light is reflected from the object Do not forget that white light can be decomposed into seven colors Mahmoud R. El-Sakka 8 Colours Absorbed CS 2124/2125: Introduction to Medical Computing

9 Image Acquisition Image acquisition is very similar to how we see things illuminating the scene by an energy source recording the reflected or transmitted energy using sensors 9

10 Image Acquisition A digital sensor can only measure a discrete set of energy levels Quantization is the process of converting a continuous analogue signal into a digital representation of this signal At the end, you get some numbers, not colors 10

11 Image Acquisition Image viewers convert these numbers into color before displaying them A digital image is always an approximation of a real world scene

12 Image Acquisition In the case of humans, the energy source is the visible light, which is a kind of electromagnetic wave There are various electromagnetic waves, for example: Based on the wavelength of the energy source used, you get various medical image modalities For each electromagnetic waves, we need a special sensor to record and quantized reflected energy (humans can not see any electromagnetic waves, other than the visible spectrum) 12

13 Image Acquisition In a vacuum, electromagnetic waves travel with a speed equal to the speed of the light (300,000,000 meters/second) The relation between wavelength, frequency and speed is Speed = Wavelength Frequency High frequencies mean shorter wavelengths more details Less penetration 13

14 Image Acquisition Let us revisit the medical image modalities that we know X-ray Angiography Fluoroscopy Computed Axial Tomography (CAT), or simply Computed Tomography (CT) Magnetic Resonance Imaging (MRI) Ultrasound Nuclear medicine The right modality should be used to visualize what you want to see 14

15 Image Acquisition Mahmoud R. El-Sakka 15 CS 2124/2125: Introduction to Medical Computing

16 Image Size A typical x-ray digital image of size (1 M. Pixel) requires at least 1 MB of storage space To upload this uncompressed image over a 1 Mbits/second modem, it would take at least 8 seconds Now think of an endless number of such images!! Needless to mention uncompressed CT and MRI images!! Indeed there is an urgent need to apply image compression schemes to reduce the size of such images 16

17 What Does Compression Mean? In real life, compression means making things smaller by applying pressure Image compression is not about physically squashing images, but about finding ways to represent it in fewer bytes 17

18 What Does Compression Mean? From this point of view, compression can be defined as a process intended to yield a compact representation of a given object The objective of image compression is to achieve compact digital image representation, with no, or at most minimal, perceived loss of picture quality 18

19 What are Lossless and Lossy Compressions? Image compression schemes can be classified as Lossless schemes Compressing an image and expanding it again produces an image which is identical bit-by-bit to the original image All the information is preserved Lossy schemes Compressing an image and expanding it again produces an image which is close to the original image, i.e., it is not an exact match Some information might be lost 19

20 Data Versus Information Data and information are not synonymous Data is the means by which information is conveyed Various amounts of data may be used to represent the same amount of information Think of data as raw material information as final product 20

21 Data Redundancies Data redundancy is a central issue in digital image compression In digital image compression, three basic data redundancies can be identified and exploited Psychovisual redundancy Encoding redundancy Inter-pixel (a.k.a. spatial) redundancy Image compression is achieved when one, or more, of these redundancies are reduced 21

22 Psychovisual Redundancy 256 grey levels (8 bits per pixel) 128 grey levels (7 bpp) 64 grey levels (6 bpp) 32 grey levels (5 bpp) 16 grey levels (4 bpp) 8 grey levels (3 bpp) 4 grey levels (2 bpp) 2 grey levels (1 bpp) 22

23 The stop sign image Is a gray-scale image Encoding Redundancy Needs bytes (200 KB) to represent its pixel values 23

24 Encoding Redundancy The image has only 4 gray-scale values, which are 0, 67, 189, and 255 Pixel values can be represented as follows: 0 to be represented by (00) 2 67 to be represented by (01) to be represented by (10) to be represented by (11) 2 in this case, only /8 bytes (50 KB) are needed Pixel values can also be represented as follows: 0 to be represented by (000) 2 (8,156 cases) 67 to be represented by (1) 2 (169,320 cases) 189 to be represented by (001) 2 (2,567 cases) 255 to be represented by (01) 2 (24,757 cases) in this case, only 8,156 3/ ,320 1/8 + 2,567 3/8 + 24,757 2/8 bytes (30.64 KB) are needed 24

25 Interpixel Redundancy The difference between adjacent pixels can be used to represent an image by applying this scheme to the stop sign image using the following code: to be represented by (0000) 2 (178 cases) to be represented by (0001) 2 (178 cases) 0 67 to be represented by (0010) 2 (202 cases) 67 0 to be represented by (0011) 2 (202 cases) to be represented by (010) 2 (208 cases) to be represented by (011) 2 (208 cases) , , 67 67, or 0 0 to be represented by (1) 2 (203,623 cases) in this case, only 1 (to encode the first pixel in the image) / / / / / / ,623 1/8 bytes (25.38 KB) are needed 25

26 Image Compression Schemes Image compression schemes can be classified into: Statistical-based compression Huffman encoder Arithmetic encoder Prediction-based compression Differential Pulse Code Modulation (DPCM) Binary Tree Predictive Coding (BTPC) Context-based compression Prediction by Partial Matching (PPM) Two-Dimensional Run Length encoding (2D-RL) Dictionary-based compression Lempel-Ziv encoding (LZ77, LZ78, LZW) Transform-based compression Wavelet Transform (WT) Discrete Cosine Transform (DCT) Burrows Wheeler Transform (BWT) Quantization-based compression Scalar quantization Vector Quantization (VQ) 26

27 Image Compression Schemes Mixture between two or more of the these schemes is also possible Joint Photographic Experts Group, (JPEG and JPEG-2000, JPEG-LS) Joint Bi-level Image Experts Group (JBIG) Set Partitioning In Hierarchical Trees (SPIHT) Context Adaptive Lossless Image Compression (CALIC) Graphic Interchange Format (GIF) Portable Network Graphics (PNG) ZIP compression. The list can go on and on As a representative, only one dictionary-based compression example will be given 27

28 Dictionary-based Compressions How many words do we have in a dictionary? Pocket dictionary: about 25,000 words Full-size dictionary: about 60,000 words If we will give a sequential number for each of these words, Pocket dictionary: we only need 15 bits (0 to 32,767) Full-size dictionary: we only need 16 bits (0 to 64,535) 28

29 Dictionary-based Compressions Assume that an English word consists of 6 letters (on average) Consider having an English dictionary containing half a million words (without their definition) This dictionary is searched for each word need to be encoded If a match is found, this word is encoded by a pointer to that word (needs 20 bits) Otherwise, 1 bit to say that match was found 19 bits as a dictionary index to that word the word is encoded without any compression, i.e., a raw word (needs 50 bits on average), 1 bit to say that match was not found 7 bits to identify the length of the word 6 characters per word (on average) 7 bits per characters 29

30 Dictionary-based Compressions After reading and compressing N words, the size of the compressed file will be (on average) N (20 P + 50 (1-P)) bits, where P is the probability of a match Without compression, we need N 49 bits to encode these N words To achieve compression, the following relation must be hold N (20 P + 50 (1-P)) < N 49 or in other word, P must be greater that 1/30!! 30

31 Compression / Decompression For each compression program, there is a matching decompression program (both programs work hand-in-hand) Images can be compressed once and decompressed many time To view a compressed image, you have to have the correct decompressor Distribution issue 31

32 PACS A picture archiving and communication system (PACS) is a medical imaging technology which provides economical storage and convenient access to images from multiple modalities A PACS consists of four major components: acquisition devises (imaging systems), e.g., X-ray, MRI and CT equipment a secured network for the transmission of patients information, archives for the storage/retrieval of images/reports, and workstations for interpreting and reviewing images Electronic images and reports are transmitted digitally via PACS eliminating the need to manually file retrieve or transport film jackets Non-image data, such as scanned documents, may be incorporated using consumer industry standard formats like PDF The universal format for PACS image is DICOM format (Digital Imaging and Communications in Medicine) 32

33 PACS 33

34 DICOM The Digital Imaging and Communications in Medicine (DICOM) standard was created to aid in the distribution and viewing of medical images A single DICOM file contains both a header (which stores information about the patient's name, the type of scan, image dimensions, etc), as well as all of the image data, which can be compressed using a variety of lossy or lossless compression schemes, including JPEG, JPEG Lossless, JPEG 2000, and Run-length encoding (RLE) DICOM does not store the image data in one file (*.img) and the header data in another file (*.hdr) 34

35 Image Enhancement Image enhancement is the process of making images more useful The reasons for doing this include: Removing noise from images Making images more visually appealing Highlighting important details in images There are various techniques to enhance images I will shed some light on some of these techniques via examples 35

36 Image Enhancement (Example 1) To examine the fine blood vessel structures at the top of a patient s head, an iodine medium is injected into the blood stream before taking the X-ray To enhance this image, an X-ray image for the same area is taken just before injecting the iodine medium (mask image) The mask image is subtracted from the image taken after the injection The contrast of the resulted image is stretched Before injection After injection Difference Contrast stretched 36

37 Image Enhancement (Example 2) A nuclear whole body bone scan is used to detect diseases such as bone infection and tumors The image is difficult to enhance due to The narrow intensity dynamic range The high noise content To enhance such an image, we need to apply several enhancement techniques 37

38 Image Enhancement (Example 2) Get the 2 nd derivative of the image using the Laplacian operator Add them together to get Original+ α 2 nd derivative image (sharpened but too noisy) + = Original image 2 nd derivative (scaled) Original + 2 nd derivative 38

39 Image Enhancement (Example 2) Extract the edges of the Original+ α 2 nd derivative image using the Sobel operator Smooth the Sobel image (averaging) Original + 2 nd derivative Edges using Sobel 39 Smoothed edge image

40 Image Enhancement (Example 2) Multiply the smoothed image by the Original+ α 2 nd derivative image x = Smoothed edge image Original + 2 nd derivative 40 Multiplied image

41 Image Enhancement (Example 2) Add the original image to the generated multiplied image to get a sharpened image with less noise + = Original image Multiplied image 41 sharpened image/less noise

42 Image Enhancement (Example 2) The contrast of the resulted image is stretched sharpened image/less noise 42 Final result

43 Image Enhancement (Example 2) Now, let us compare the original image to the final image Original image 43 Final result

44 Image Understanding To make computers understand an image is not an easy task Do not forget that a digital image is just an array of numbers Can you tell me what is in this image? There is nothing new to you in this picture!! Well, it is.. 44

45 CAD systems Various medical image modalities yield a great deal of information, which radiologists have to analyze and evaluate comprehensively in a short time Computer-aided detection (CADe) and computer-aided diagnosis (CADx) are procedures in medicine that assist doctors in the interpretation of medical images CAD systems seek to Identify and highlight suspicious structures in the image (image processing and computer vision) Analyze, quantify (feature extraction) and classify (or at least give a score to) every detected region (machine learning) At the present stage of the technology, CAD systems can not, and may not, substitute the radiologist, but rather play a supporting role The radiologist is always responsible for the final interpretation (liability issue) 45

46 CAD systems A CAD system may suggested that the person has the disease (positive) These cases will be further investigated to verify the results (double checking) does not have the disease (negative) These cases may not receive further checking If the suggestion is correct, we call it true output not correct we call it false output Hence, we have four counts True positive (TP): Sick people correctly diagnosed as sick (useful) True negative (TN): Healthy people correctly identified as healthy (useful) False positive (FP): Healthy people incorrectly identified as sick (costly) False negative (FN): Sick people incorrectly identified as healthy (dangerous) 46

47 CAD systems Real positive cases TN FN TP FP CAD Positive cases To standardize the TP, TN, FP, and FN values, the following statistical measures are calculated: Sensitivity (also called recall rate or hit rate): the ability to correctly identify real positive cases Type II error = 1 Sensitivity Specificity: the ability to correctly identify real negative cases Type I error = 1 Specificity 47

48 CAD systems Real positive cases TN FN TP FP CAD Positive cases To standardize the TP, TN, FP, and FN values, the following statistical measures are calculated: Precision (also called positive predictive value): The proportion of TP to total CAD positive cases (regardless T or F) Negative predictive value: The proportion of TN to total CAD negative cases (regardless T or F) Accuracy 48

49 CAD systems Real positive cases TN FN TP FP CAD Positive cases Example: Out of 2000 cases, we have TP = 20, FP = 180, FN = 10, TN = 1790 Sensitivity (also called recall rate or hit rate): the ability to correctly identify real positive cases = = 66.67% Type II error = 1 Sensitivity = = 33.33% Specificity: the ability to correctly identify real negative cases = = 90.86% Type I error = 1 Specificity = = 9.14% 49

50 CAD systems Real positive cases TN FN TP FP CAD Positive cases Example: Out of 2000 cases, we have TP = 20, FP = 180, FN = 10, TN = 1790 Precision (also called positive predictive value): The proportion of TP to total CAD positive cases (regardless T or F) = = 10% Negative predictive value: The proportion of TN to total CAD negative cases (regardless T or F) = = 99.44% Accuracy = = 90.5% 50

51

52 52