Week No. 01 IntroducMon (course: Computer Vision)

Transcription

1 Week No. 01 IntroducMon (course: Computer Vision) e- mail: Department of So9ware Engineering, Mehran UET Jamshoro, Sind, Pakistan

2 Outline Vision, Visual system Color vision, computer vision Image processing steps ApplicaMons

3 Vision Vision state of being able to see Vision is the process of discovering, what is present in the world and where it is. PercepMon Process of becoming aware of something through senses percepmon is the process of acquiring, interpremng, selecmng, and organizing sensory informamon. Image source: Gonzales, R. C., & Woods, R. E. Digital image processing, 1993.

4 Visual System The visual system allows to assimilate informamon from the environment The act of seeing starts when the lens of the EYE focus an image of the outside world onto a light- sensimve membrane in the back of the eye, called the ReMna The remna is actually part of the brain that is isolated to serve as a transducer for the conversion of pa]erns of light into neuronal signals Visual = of vision System: set of connectors/parts/nodes forming as a whole ReMna: Layer at back of eyeball, it has cells & is sensimve to light, it triggers neurons Impulses to brain

5 Visual System

6 Visual System The human eye achieves proper focus of the object by changing the shape of the lens - changing its focal- length The lens gets fla]ened (thinned) to focus the distant objects, and gets thickened to focus the near objects The focal length of the lens varies between 14 mm and 17 mm The distance between the lens and the remna along the visual axis is around 17 mm An inverted image of the object is formed on the fovea region of the remna

7 Visual System

8 Pin hole Camera A simplest imaging device which can map a 3- D scene onto a 2- D image plane. The projecmon from 3- D to 2- D is a perspecmve projecmon. A pinhole camera forms an inverted image of the object, so does the Human eye

9 Color vision Color vision is the capacity of an organism or machine to dismnguish objects based on the wavelengths (or frequencies) of the light they reflect or emit. The nervous system derives color by comparing the responses to light from the several types of cone photoreceptors in the eye. For humans, the visible spectrum ranges approximately from 380 to 750 nm.

10 Color vision A 'red' apple does not emit red light. Rather, it simply absorbs all the frequencies of visible light shining on it except for a group of frequencies that is perceived as red, which are reflected. An apple is perceived to be red, only, because the human Eye can dismnguish between different wavelengths. Three things are needed to see color a light source, a detector (e.g. the Eye) a sample to view.

11 Computer vision Computer vision is the science and technology of machines that see. Computer Vision is the study of analysis of pictures and videos in order to achieve results similar to those as by human.

12 Computer vision Since percepmon can be seen as the extracmon of informamon from sensory signals, computer vision can be seen as the scienmfic invesmgamon of armficial systems for percepmon from images or mulm- dimensional data Computer vision can also be described as a complement of Biological Vision, as computer vision, studies and describes armficial vision system that are implemented in so9ware and/or hardware.

13 Related Disciplines Image processing Computer graphics Pa]ern recognimon ArMficial intelligence Applied mathemamcs Learning

14 Related Disciplines

15 Fields of an IMAGE Input Image Image Description Image Image Image Processing Computer Graphics Pattern Recognition Computer Vision Output Image Image Description Statistics Action

16 Image Sources Images can be categorized according to their source (e.g., visual, X- ray, etc) Principle energy sources for images ElectromagneMc energy spectrum acousmc, ultrasonic Electronic (in the form of electron beams used in electron microscopy) SyntheMc images, used for modeling and visualizamon,are generated by computer

17 Light Types AchromaMc light Light that voids color is called achromatic or monochromatic light The only attribute of such light is its intensity, or amount The term gray level generally is used to describe monochromatic intensity because it ranges from black to grays, and finally to white ChromaMc light Chromatic light spans the electromagnetic energy spectrum from approximately 0.43 to 0.79 µ.m

18 ChromaMc Light ChromaMc light Three basic quantities are used to describe the quality of a chromatic light source: 1) Radiance 2) Luminance 3) Brightness Radiance: The total amount of energy that flows from the light source, and it is usually measured in watts (W) Luminance: measured in lumens (lm), gives a measure of the amount of energy an observer perceives from a light source Brightness: Brightness is a subjective descriptor of light perception that is practically impossible to measure It embodies the achromatic notion of intensity and is one of the key factors in describing color sensation

19 ChromaMc Light Example light emitted from a source operating in the far infrared region of the spectrum could have significant energy (radiance), but an observer would hardly perceive it; its luminance would be almost zero

20 ElectromagneMc Spectrum ElectromagneMc Spectrum: Band of Radiations RadiaMon: Energy that travels and spreads out as it goes Energy of one photon (electron volts) The electromagnetic spectrum arranged according to energy per photon Planck's constant, or h i- e

21 ElectromagneMc Spectrum Visible light: Light that comes from a lamp Radio waves: Light that comes from a radio stamon and are types of electromagnemc radiamon Microwaves Infrared Ultraviolet X- rays Gamma- rays Ho]er, more energemc objects and events create higher energy radiamon than cool objects Only extremely hot objects or parmcles moving at very high velocimes can create high- energy radiamon like X- rays and gamma- rays

22 ElectromagneMc Spectrum Gamma- rays The highest energy, shortest wavelength electromagnetic radiations. Usually, they are thought of as any photons having energies greater than about 100 kev Radioactive materials (some natural and others made by man in things like nuclear power plants) can emit gammarays Big particle accelerators that scientists use to help them understand what matter is made of. The biggest gamma-ray generator of all is the Universe! It makes gamma radiation in all kinds of ways Major uses of imaging based on gamma rays include nuclear medicine and astronomical observations In nuclear medicine, the approach is to inject a patient with a radioactive isotope that emits gamma rays as it decays Images are produced from the emissions collected by gamma ray detectors

23 ElectromagneMc Spectrum X- rays Electromagnetic radiation of very short wavelength and very high-energy; X-rays have shorter wavelengths than ultraviolet light but longer wavelengths than gamma rays Doctors use them to look at bones, Dentist to look at teeth. Hot gases in the Universe also emit X-rays, X-rays are among the oldest sources of EM radiation used for imaging The best known use of X-rays is medical diagnostics, but they are also used extensively in industry and other areas, like astronomy Angiography is another major application in an area called contrast-enhancement radiography. This procedure is used to obtain images (called angiograms) of blood vessels Perhaps the best known of all uses of X-rays in medical imaging is computerized axial tomography (CAT) X-rays, are used to examine circuit boards for flaws in manufacturing, such as missing components or broken traces

24 ElectromagneMc Spectrum Ultraviolet Electromagnetic radiation at wavelengths shorter than the violet end of visible light X-rays, are used to examine circuit boards for flaws in manufacturing, such as missing components or broken traces Sun is a source of ultraviolet (UV) radiation, because it is the UV rays that cause our skin to burn! Stars and other "hot" objects in space emit UV radiation The atmosphere of earth effectively blocks the transmission of most ultraviolet light Applications of ultraviolet "light include lithography, industrial inspection, microscopy, lasers, biological imaging, and astronomical observations Lithography :A method of planographic prinmng from a metal or stone surface

25 ElectromagneMc Spectrum Infrared Electromagnetic radiation at wavelengths longer than the red end of visible light and shorter than the microwaves (roughly between 1 and 100 microns) Almost none of the infrared portion of electromagnetic spectrum can reach the surface of earth Applications include light microscopy, astronomy, remote sensing, industry, and law enforcement

26 ElectromagneMc Spectrum ThemaMc Bands in NASA s LANDSAT Satellite LANDSAT satellite obtains and transmits images of Earth from space for monitoring environmental condimons of the planet.

27 Microwave Electromagnetic radiation which has a longer wavelength (between 1 mm and 30 cm) than visible light Microwaves can be used to study the Universe, communicate with satellites in Earth orbit, and cook popcorn The dominant application of imaging in the microwave band is radar. The unique feature of imaging radar is its ability to collect data over virtually any region at any time, regardless of weather or ambient lighting conditions Some radar waves can penetrate clouds, and under certain conditions can also see through vegetation, ice, and extremely dry sand. In many cases, radar is the only way to explore inaccessible regions of the Earth's surface An imaging radar works like a flash camera in that it provides its own illumination (microwave pulses) to illuminate an area on the ground and take a snapshot image Instead of a camera lens, a radar uses an antenna and digital computer processing to record its images ElectromagneMc Spectrum

28 ElectromagneMc Spectrum Radio waves Electromagnetic radiation which has the lowest frequency, the longest wavelength, and is produced by charged particles moving back and forth The atmosphere of the Earth is transparent to radio waves with wavelengths from a few millimeters to about twenty meters Yes, this is the same kind of energy that radio stations emit into the air for boom box to capture and turn into favorite tunes. But radio waves are also emitted by other things... such as stars and gases in space The major applications of imaging in the radio band are in medicine and astronomy In medicine radio waves are used in magnetic resonance imaging (MRI) This technique places a patient in a powerful magnet and passes radio waves through his or her body in short pulses

29 ElectromagneMc Spectrum

30 Imaging ModaliMes Although imaging in the electromagnetic spectrum is dominant by far, there are a number of other imaging modalities that also are important Other imaging modalities are: acoustic imaging electron microscopy Synthetic (computer-generated) imaging Imaging using "sound" finds application in Geological exploration Industry Medicine Geological applications use sound in the low end of the sound spectrum (hundreds of Hertz) Imaging in other areas use ultrasound (millions of Hertz)

31 Imaging ModaliMes Ultrasound Imaging ultrasound imaging is used routinely in manufacturing The best known applications of this technique are in medicine, especially in obstetrics, where unborn babies are imaged to determine the health of their development Obstetrics: The branch of medicine dealing with childbirth and care of the mother Fractal images: are striking examples of computer generated images

32 Image processing steps 1. Image acquisimon 2. Pre- processing 3. Feature extracmon 4. DetecMon/SegmentaMon

33 1) Image acquisimon A digital image is produced by one or several image sensors. These sensors may include: Light- sensimve cameras Range sensors Tomography devices Radar and ultra- sonic cameras, etc. Depending on the type of sensor, the resulmng image data is an ordinary 2D image, a 3D volume, or an image sequence. The pixel values typically correspond to light intensity in one or several spectral bands

34 1) Image acquisimon Image source: Gonzales, R. C., & Woods, R. E. Digital image processing, 1993.

35 2) Pre- processing Before a computer vision method can be applied to image data in order to extract some specific piece of informamon. It is usually necessary to process the data in order to assure that it samsfies certain assumpmons implied by the method. Examples may include: Re- sampling in order to assure that the image coordinate system is correct Noise reducmon in order to assure that sensor noise does not introduce false informamon Contrast enhancement to assure that relevant informamon can be detected

36 3) Feature extracmon Image features at various levels of complexity are extracted from the image data Typical examples of such features are: Lines, edges and ridges Localized interest points such as corners, blobs or points More complex features may be related to texture, shape or momon Ridge: Edge formed where two sloping sides of roof meet at the top. Blob: Spot of color

37 4) DetecMon/SegmentaMon At some point in the processing a decision is made about which image points or regions of the image are relevant for further processing Examples: SelecMon of a specific set of interest points SegmentaMon of one or mulmple image regions which contain a specific object of interest

38 Vision and Graphics

39 ApplicaMon Areas Law enforcement Nuclear medicine and Defense AutomaMc character recognimon Industrial applicamons (machine vision) Satellite imagery for weather predicmon Solving problems with machine percepmon Enhance the contrast or code the intensity levels into color for easier interpretamon InterpretaMon of X- rays and other Images used in industry, medicine and biological sciences Remote Sensing

40 Digital Image Processing Two principal applicamon areas are: 1. Improvement of pictorial informabon for human interpretabon 2. Processing of image data for storage, transmission, and representabon for autonomous machine percepbon Vision is the most advanced of human senses Images play the most important role in human percepmon Humans are limited to the visual band of the electromagnemc (EM) spectrum Imaging machines cover almost the enmre EM spectrum, ranging from gamma to radio waves

41 Digital Image Processing UlMmate goal of computer vision is to use computers to emulate human vision, including learning and being able to make inferences and take acmons based on visual inputs This area itself is a branch of armficial intelligence (AI) whose objecmve is to emulate human intelligence The area of image analysis (also called image understanding) is in between image processing and computer vision There are no clear- cut boundaries in the conmnuum from image processing at one end to computer vision at the other. However, one useful paradigm is to consider three types of computerized processes

42 Three computerized processes are: Low Level processes Medium Level processes High Level processes Digital Image Processing Low- level processes involve primimve operamons such as image preprocessing to reduce noise, contrast enhancement, and image sharpening. A low- level process is characterized by the fact that both its inputs and outputs are images Mid- level processing on images involves tasks such as segmentamon, descripmon of those objects to reduce them to a form suitable for computer processing, and classificamon (recognimon) of individual objects

43 Digital Image Processing A mid- level process is characterized by the fact that its inputs generally are images, but its outputs are a]ributes extracted from those images (e.g., edges, contours, and the idenmty of individual objects) Finally, higher- level processing involves "making sense" of an ensemble of recognized objects, as in image analysis, and, at the far end of the conmnuum, performing the cognimve funcmons normally associated with vision

44 Digital Image Processing Digital Image Processing Set of operamons performed on digital image Image Enhancement The simplest and most appealing area of digital image processing, it is subjecmve technique in a sense that is based on human subjecmve preferences regarding what consmtutes a "good" enhancement result The idea behind enhancement techniques is to bring out detail that is obscured, or simply to highlight certain features of interest in an image Example: when someone increases the contrast of an image because "it looks be]er.

45 Digital Image Processing Image RestoraMon An area that also deals with improving the appearance of an image it is objective technique in the sense that restoration techniques tend to be based on mathematical or probabilistic models of image degradation Color Image Processing An area that deals with the spectrum of frequencies of an image. It has been gaining in importance because of the significant increase in the use of digital images over the Internet Wavelets Wavelets are the foundation for representing images in various degrees of resolution

46 Digital Image Processing Compression deals with techniques for reducing the storage required to save an image, or the bandwidth required to transmit it Morphological Processing deals with tools for extracting image components that are useful in the representation and description of shape SegmentaMon partitioning an image into its constituent parts or objects In general, autonomous segmentation is one of the most difficult tasks in digital image processing

47 Components of General Purpose Image Processing System

48 Components of General Purpose Image Processing System Image Sensors Two elements are required to acquire digital images. The first is a physical device that is sensitive to the energy radiated by the object intended to image. The second, called a digitizer, is a device for converting the output of the physical sensing device into digital form. Specialized Image Processing hardware usually consists of the digitizer and hardware that performs other primitive operations, such as an arithmetic logic unit (ALU) performing arithmetic and logical operations in parallel on entire images. This type of hardware sometimes is called a front-end subsystem Image Processing So9ware It consists of specialized modules performing specific tasks. A well-designed package includes the capability for the user to write code that, as a minimum, utilizes the specialized modules

49 Components of General Purpose Image Processing System Image Processing So9ware More sophisticated software packages allow the integration of those modules and general-purpose software commands from at least one computer language Mass Storage Mass storage capability is a mandatory in image processing applications An image of size 1024 X 1024 pixels, in which the intensity of each pixel is an 8-bit quantity, requires one megabyte of storage space if the image is not compressed Digital storage for image processing applications falls into three principal categories: 1) short term storage required during processing 2) on-line storage for relatively fast recall 3) archival storage characterized by infrequent access

50 Components of General Purpose Image Processing System Image Displays Image displays in use today are mainly color (preferably flat screen) TV monitors. Monitors are driven by the outputs of image and graphics display cards that are an integral part of the computer system Hardcopy devices Hardcopy devices for recording images include laser printers, film cameras, heat-sensitive devices, inkjet units, and digital units, such as optical and CD-ROM disks Networking Networking is almost a default function in any computer system in use today. Because of the large amount of data inherent in image processing applications, the key consideration in image transmission is bandwidth. In dedicated networks, this typically is not a problem, but communications with remote sites via the Internet are not always as efficient. Fortunately, this situation is improving quickly as a result of optical fiber and other broadband technologies

51 Conclusions UlMmate goal of Computer vision is to emulate human vision Including: Learning, to make inferences and take acmons based on visual input

52