EEE 508 - Digital Image & Video Processing and Compression http://lina.faculty.asu.edu/eee508/ Introduction Prof. Lina Karam School of Electrical, Computer, & Energy Engineering Arizona State University karam@asu.edu 1
Why Image and Video? Sample image-and video-based applications Entertainment Communications Medical imaging Security Monitoring Visual sensing and control 2
Basic Imaging System z Imaged Scene y x CAMERA Imaging Device DIGITIZER STORAGE PROCESS Sampling + Compression Display, Analysis, Quantization Enhancement, Restoration, Compression for transmission Colored lights from scene are captured into red, green, and blue pixels (picture elements) Scene viewed through color filters that separate the image into 3 color components Digital camera systems contain optics that image light onto sensors typically a CCD array with filters Copyright 2007 2012 by Lina J. Karam 3
Basic Imaging System z Imaged Scene y x CAMERA Imaging Device DIGITIZER STORAGE PROCESS Sampling + Compression Display, Analysis, Quantization Enhancement, Restoration, Compression for transmission z Quality of captured image depends on imaging optics and electronics, color filter characteristics, digitization, and processing Copyright 2007 2012 by Lina J. Karam 4
Basic Imaging System Imaged Scene x CAMERA Imaging Device z y DIGITIZER STORAGE Compression PROCESS Display, Analysis, Enhancement, Restoration, Compression for transmission High-end digital cameras make use of dichroic filters to split the light into, red, green, and blue components Beam splitter is used to split light into three beams that are directed through filters that filter out all but one color for each chip ( dichroic indicates that 2 out of the 3 colors are filtered). Each color component is imaged separately onto an array of sensors: one chip sees red (R), one sees green (G), and one sees blue (B). Three values (R, G, B) captured at each pixel position Copyright 2007 2012 by Lina J. Karam 5
Basic Imaging System z Imaged Scene y x CAMERA Imaging Device DIGITIZER STORAGE PROCESS Sampling + Compression Display, Analysis, Quantization Enhancement, Restoration, Compression for transmission Common digital cameras have a single imaging element (typically one CCD chip) and make use of tiled Color Filter Array (CFA) Light from scene CFA Bayer CFA Each captured pixel is either Green (G), Red (R), or Blue(B). Interpolation is used to recover (R,G,B) values at each pixel. Image Sensor Copyright 2007 2012 by Lina J. Karam 6
Digitization: Sampling and Quantization Imaged Scene y x t CAMERA Imaging Device I(x,y;t) I(n 1,n 2; n 3 ) DIGITIZER STORAGE PROCESS Sampling + Display, Analysis, Quantization Compression Enhancement, Restoration, Compression for transmission Original Imaged Scene : analog (continuous in space and time) I(x,y;t) for video and I(x,y) for still image I: image intensity and color at position (x,y) and at time t Digitized sensed image/video: digital (sampled in space and time, plus discrete amplitudes) I(n 1,n 2 ;n 3 ) for video and I(n 1,n 2 ) for a still image I: image intensity and color at integer sample position (n 1,n 2 ) and integer time index n 3 Copyright 2007 2012 by Lina J. Karam 7
Basic Imaging System z S 3 or z Imaged Scene x S 1 or x t CAMERA Imaging Device S 2 or y I(s 1,s 2,s 3,t) : ANALOG SIGNAL DIGITIZER STORAGE PROCESS Sampling + Compression Display, Analysis, Quantization Enhancement, Restoration, Compression for transmission I : real value or vector of real values (s 1,s 2,s 3,t) : set of real continuous space (time) variables I(n 1,n 2, n 3,n 4 ) : DISCRETE SIGNAL (DIGITAL) I : discrete (quantized) real or integer value (n 1,n 2,n 3,n 4 ) : set of integer indices Copyright 2007 2012 by Lina J. Karam 8
Examples Sampled Black & White Photograph: I(n 1,n 2 ) I (n 1,n 2 ) scalar indicating pixel intensity at location (n 1,n 2 ) For example: I = 0 Black I = 1 White 0 < I < 1 In-between Sampled color video/tv signal I R (n 1, n 2, n 3 ) I R (n 1, n 2, n 3, n 4 ) 2D TV: I G (n 1, n 2, n 3 ) ; 3D TV: I G (n 1, n 2, n 3, n 4 ) I B (n 1, n 2, n 3 ) I B (n 1, n 2, n 3, n 4 ) EEE 508 - Lecture 1 9 9#
Digitization: Sampling and Quantization Video Sampling Temporal sampling affects frame (image) rate and perceived motion quality. 50 to 60 frames per second produce smooth apparent motion 25 (PAL) or 30 (NTSC) frames per second is standard for television pictures; interlacing can be used to improve the appearance of motion Frame rate can be referred to as temporal resolution. Copyright 2007 2012 by Lina J. Karam 10
Digitization: Sampling and Quantization Video Sampling Progressive and Interlaced Sampling Progressive sampling: all lines (rows) in a frame are sampled Interlaced sampling: alternate between sampling the odd rows (odd field) for one frame followed by the sampling the even rows (even field) for next frame Odd or Top Field Even or Bottom Field Copyright 2007 2012 by Lina J. Karam 11
Spatial Resolution A digital image is represented as a rectangular array of picture elements (pixels or pels). 503 pixels 365 pixels 503x365 pixels Total pixels = 183,595 Spatial resolution commonly refers to the number of pixels in the horizontal and vertical directions. Copyright 2007 2012 by Lina J. Karam 12
Spatial Resolution Video Formats based on Resolution 176 352 720 1280 1920 144 288 QCIF CIF, 101 Kpixels 480 576 720 480i SDTV, 345 Kilo pixels SDTV, 415 Kilo pixels High Definition (HDTV) 1 Mega pixels 1080 High Definition (HDTV) 2 Mega pixels Copyright 2007 2012 by Lina J. Karam 13
Spatial Resolution 4CIF: 704x576 CIF: 352x288 Copyright 2007 2012 by Lina J. Karam 14 QCIF: 176x144 SCIF: 128x96
Spatial Resolution Choice of frame resolution depends on application and on available storage and transmission capacity. Perceived resolution refers to the maximum number of line pairs that can be resolved on the display screen or to the smallest details that can be resolved. Depends on viewing distance. Depends on display Display resolution is commonly expressed in pixels per inch. Copyright 2007 2012 by Lina J. Karam 15
Aspect Ratio Aspect ratio is the ratio of the image s width to its height. 720 1280 480 SDTV Video 4:3 1.33:1 Widescreen SDTV HDTV 16:9 1.78:1 720 1920 Full HDTV 1.78:1 1080 Copyright 2007 2012 by Lina J. Karam 16
How do we process images? Exploit visual perception properties Use/Develop image/video processing (computer vision) algorithms Use DSP concepts as tools EEE 508 - Lecture 1 17 17#
How many possible images are there? We represent pixels as amplitude values (gray scale). 256 levels 1 0 128 levels 1 0 64 levels 1 0 32 levels 1 0 How much to sample (quantize) the gray scale? Humans can distinguish in the order of 100 levels of gray. 18# EEE 508 - Lecture 1 18
How many possible images are there? An image has pixels and dimensions, say 200x200 and assume 64 pixel values (64 gray levels). A 1x1 image about 64 images A 1x2 image about (64) 2 images A 200x200 image about (64) 40000 images A large but finite number due to human perceptive properties. 19# EEE 508 - Lecture 1 19