CS 376b Computer Vision 09 / 03 / 2014 Instructor: Michael Eckmann
Today s Topics This is technically a lab/discussion session, but I'll treat it as a lecture today. Introduction to the course layout, assignments, exams, topics etc. What is Computer Vision? Fill out a questionnaire to allow me to learn about the backgrounds of you all which will inform some decisions about the material we will cover. College - CS 376b - Fall 2014
Who is your instructor? I'm Mike Eckmann and this is my eleventh year at Skidmore. Before that I was at Lehigh University in PA. I studied Mathematics and Computer Engineering and Computer Science all at Lehigh University. I was employed as a programmer (systems analyst) for eight years. I have research interests in computer vision. College - CS 376b - Fall 2014
1 credit honors add-on course do not have to be an honors forum student to enroll part of The Alien Invasions Project. Our cluster is called Drones. work with students from Physics, Business, Foreign Languages (Italian) and Computer Science on the topic of "Drones." the work you do over the course of the semester will be largely student driven, with a final collaborative project at the end. HF-200 011 (Alien Invasions: Drones with Michael Eckmann) and it is open for enrollment now. meet from 6:30-7:30 p.m. on Wednesdays, but this time/day may change if amenable to all involved. Students can add the class any time before the add deadline.
1 credit honors add-on course project will involve collaboration with a subset of the students in three other courses (taught by Cathy Hill, Evan halstead and Barbara Garbin.) Each course has it own honors add-on section, and we ll pool these into one larger team. Each of those student cohorts (and professors) will bring different interests, perspectives and proficiencies to the table. College - CS 376b - Fall 2014
Course Information For programming assignments in this course, you will be using the OpenCV (open source computer vision) library. Early in the semester I will give an overview of some of the features of the library as well as sample code. College - CS 376b - Fall 2014
Course Information (cont.) Expect to have 4 programming assignments 1 st one will make sure you understand certain important opencv library concepts in your code 2 nd one will deal with image processing techniques 3 rd and 4 th will probably have to do with computer vision tasks (e.g. edge/feature detection, segmentation, tracking, matching.) College - CS 376b - Fall 2014
Course Information (cont.) In addition to the programming assignments, we will have a few homeworks dealing with some concepts that aren't covered in the programming assignments We will have two exams plus a final exam. APPROXIMATE DATES are exam 1 on or about 10/02/2014 exam 2 on or about 11/06/2014 final exam is 12/16/2014 6pm (set by registrar) We will have weekly lab/discussion sessions for image processing and vision concepts as well as interaction with OpenCV. College - CS 376b - Fall 2014
Computer Vision The goal of Computer Vision, according to one text is to make useful decisions about real physical objects and scenes from sensed images. That is, understanding images. College - CS 376b - Fall 2014
Computer Vision Basic (oversimplified) observations on the relationship between Computer Vision, Computer Graphics and Image Processing. Computer Graphics Given models, generate images Computer Vision Given images, generate models Image Processing Given images, generate (other) images College - CS 376b - Fall 2014
Computer Vision Let me read you a list of some applications of computer vision. (p. 3-5 of A Practical Introduction to Computer Vision with OpenCV, by Kenneth Dawson-Howe, c. 2014, Wiley). College - CS 376b - Fall 2014
Computer Vision Many computer vision tasks require image processing techniques. So, a substantial portion of the course will be spent on learning image processing techniques and how they can be applied to computer vision problems. College - CS 376b - Fall 2014
Computer Vision For example if the computer vision task was to find, in an image, a particular flat object for which you know the basic outline. The task might be solved by reducing noise in the image (image processing) processing the image to find all edges and generate an edge image (image processing) process the edge image to connect the edges into a contour determine if any of the contour shapes match the contour of the shape you're trying to find. College - CS 376b - Fall 2014
Computer Vision Topics Image formation and representation how do we acquire digital images and in what format are they stored Binary Image Analysis what information can we get out of a black and white image (as opposed to a color image or a greyscale image)? what are typical operations on binary images and for what are they used? College - CS 376b - Fall 2014
Computer Vision Topics Image Filtering and Enhancement how do we reduce different kinds of noise how do we enhance an image to show better detail in areas of low contrast use of histograms applying image operators detecting edges etc. Color techniques and information from color images College - CS 376b - Fall 2014
Computer Vision Topics Texture how can we capture texture of an object in an image and what can we do with it Motion different techniques for detecting motion in image sequences (video) and what information can be gained from this Segmentation techniques to segment an image into meaningful areas based on various criteria College - CS 376b - Fall 2014
Computer Vision Topics Matching in 2D how to find correspondences between two images how to use these correspondences to generate meaningful information College - CS 376b - Fall 2014
What is a digital image 2 dimensional grid of pixels Each pixel contains numeric data that describe what color the pixel is (or what brightness the pixel is for grayscale images.) Depending on whether it is a color or grayscale image, each pixel contains either 3 numbers (Red, Green and Blue) or 1 number (brightness).
What is a digital image The smallest unit in an image is a pixel. Each pixel contains one color.
What is a digital image Both images below are in public domain
What is a digital image Let's examine a digital image using gimp (Gnu Image Manipulation Program) that has similar functionality to Photoshop. I'll zoom in and point out RGB values of pixels I'll convert to grayscale and show the brightness values of pixels
RGB Since color images are so prevalent, you should get a feel for what values of R, G and B channels describe what colors. Let's use the applet on this site: https://yuilibrary.com/yui/docs/color/rgb-slider.html Observe what colors you get when all 3 are equal (R=G=B). Let's keep two channels constant and slide the other channel among all values.
RGB Often the size of each of the R, G and B color channels is 1 byte which has a range of 0-255. How many different colors can be made if each channel has 1 byte of space?
RGB Often the size of each of the R, G and B color channels is 1 byte which has a range of 0-255. How many different colors can be made if each channel has 1 byte of space? 256 * 256 * 256 = 16,777,216 One of 16,777,216 different colors can be specified when 1 byte per color channel is used to store the color.
Byte A byte is 8 bits A bit holds one value (a 0 or a 1) Since there are 2 different values for a bit and there are 8 bits in a byte, there are 2^8=256 different values for a byte.
Grayscale Grayscale images hold 1 number per pixel. Each pixel is usually 1 byte. So, one of 256 different brightness values can be stored per pixel. 0 = black 255 = white All numbers in between are varying levels of gray
True black and white (monochrome) In layman's terms a grayscale image is often described as black and white. This is not accurate. How much space would one need per pixel to store a true black and white image?
True black and white (monochrome) In layman's terms a grayscale image is often described as black and white. This is not accurate. How much space would one need per pixel to store a true black and white image? 1 bit (0=black, 1=white) How might one convert a grayscale image to true black and white?
Electromagnetic radiation Physically, color is electromagnetic radiation in the visible light frequency range. College - CS 376 - Fall 2014
Electromagnetic radiation Frequency range of visible light is approx. from: 4.3*10^14 Hz up to 7.9*10^14 Hz College - CS 376 - Fall 2014
Electromagnetic radiation Human eye senses radiation wavelengths between approx. 380-700 nanometers radiation with wavelengths less than visible light Ultraviolet, X-rays radiation with wavelengths greater than visible light Infrared, Microwaves, TV, Radio There are sensors to detect these various non-visible wavelengths College - CS 376b - Fall 2014
Imaging devices x-ray images can penetrate the human body/bones infrared (IR) images (higher temperatures emit higher infrared radiation) often used for night photography film camera digital camera usually a CCD with cells in a rectangular grid or CMOS sensors For digital image processing and computer vision to be done on the images, we need a digital image. Also, if the device does not detect visible light, typically the different values captured by the devices are converted to visible colors. College - CS 376 - Fall 2014
CCD cameras image figure 2.2 in Computer Vision by Shapiro and Stockman discrete cells convert light energy into electrical charges and they integrate the light energy falling on them during the time the shutter is open College - CS 376 - Fall 2014
Video 30 frames (images) per second for NTSC video can be a different rate of images e.g. 15, 60 almost always compressed due to the large size example: a 640 x 480 image with 3 bytes per pixel uncompressed is how big? a 90 minute video w/ 30 frames per second uncompressed with each frame as above is how big? COTS (commercial off the shelf) CCD video cameras are made for other purposes than for computer vision e.g. pixels may not be square (4:3 ratio) computer vision most likely prefers/assumes square pixels College - CS 376 - Fall 2014
Problems in digital images geometric distortion due to an imperfect lens the light beams don't travel on the correct path e.g. barrel distortion for small focal lengths scattering medium that the light travels through is rarely a vacuum, hence the light bends (e.g. air, water, other liquids ) (see Snell's law) blooming each CCD cell is close to their neighbors so, light can leak between neighboring cells CCD variations some cells might not be all equally sensitive due to variations in the manufacturing process College - CS 376 - Fall 2014
Problems in digital images clipping or wrap-around when an analog value is converted to digital, high values may be clipped to the max digital value wrap around is like a modulus operator quantization effects converting a continuous range to a discrete range chromatic distortion lens has the index of refraction (bending) different for different wavelengths (colors) let's look back at the lightbulb picture, what's the effect? College - CS 376 - Fall 2014