Computer and Machine Vision

Transcription

1 Computer and Machine Vision Lecture Week 4 Part-1 February 5, 2014 Sam Siewert

2 FFMPEG FAQ Read It!! You should know how to Decode Video (recorded from your camera or prerecorded by someone els) You should know how to Encode Video (to turn in with your labs) Sam Siewert 2

3 Outline of Week 4 Practical Methods for Dealing with Camera Streams, Frame by Frame and De-coding/Re-encoding for Analysis (And Lab Reports!!) Deeper Dive on Color, Human Vision Characteristics Wrap-up On Convolution and Transformation Finish Reading Through Chapter 3 in CV (Image Processing and Transforms) Finish Reading in OpenCV through Chapter 6 Start asking Questions about Example Code as We go Introduction to Segmentation and Recognition Problem and Approaches Next Step is Histograms and Thresholds Goal is Recognition Then 3D Sam Siewert 3

4 Ffmpeg (avconv) Notes sudo apt-get install ffmpeg ffmpeg -i movie.mpg ss 30 t 30 movie%d.ppm sec ssiewert@ssiewert-virtualbox:~/a485/media$ ffmpeg -i big_buck_bunny_480p_surround-fix.avi -ss 30 -t 30 bbb%d.ppm ffmpeg version : ubuntu , Copyright (c) the Libav developers built on Apr :02:36 with gcc Input #0, avi, from 'big_buck_bunny_480p_surround-fix.avi': Duration: 00:09:56.45, start: , bitrate: 2957 kb/s Stream #0.0: Video: mpeg4 (Simple Profile), yuv420p, 854x480 [PAR 1:1 DAR 427:240], 24 tbr, 24 tbn, 24 tbc Stream #0.1: Audio: ac3, Hz, 5.1, s16, 448 kb/s Incompatible pixel format 'yuv420p' for codec 'ppm', auto-selecting format 'rgb24' 0x907700] w:854 h:480 pixfmt:yuv420p 0x9054c0] auto-inserting filter 'auto-inserted scaler 0' between the filter 'src' and the filter 'out' 0x905b60] w:854 h:480 fmt:yuv420p -> w:854 h:480 fmt:rgb24 flags:0x4 Output #0, image2, to 'bbb%d.ppm': Metadata: encoder : Lavf Stream #0.0: Video: ppm, rgb24, 854x480 [PAR 1:1 DAR 427:240], q=2-31, 200 kb/s, 90k tbn, 24 tbc Stream mapping: Stream #0.0 -> #0.0 Press ctrl-c to stop encoding... Last message repeated 719 times -0kB time=29.00 bitrate= -0.0kbits/s frame= 720 fps= 38 q=0.0 Lsize= -0kB time=30.00 bitrate= -0.0kbits/s video:864686kb audio:0kb global headers:0kb muxing overhead % ssiewert@ssiewert-virtualbox:~/a485/media$ Sam Siewert 4

5 Now with PPM Frames PPM is Simple, but No Compression Good for CV - Read this! JPEG, PNG are Compressed TIFF is an Alternative, but More Complex Sam Siewert 5

6 Simple Re-encode When Quality is not a Concern, Keep it Simple ffmpeg -f image2 -i bbb%d.ppm bbbtrans.mpg vlc bbbtrans.mpg Sam Siewert 6

7 Quality Encoding is Tricky Use MPEG4 HQ Settings, Encode 480p, AR=4:3 ffmpeg -f image2 -i bbb%d.ppm -maxrate 20000k - bufsize 32M -s 640x480 -vcodec mpeg4 -qscale 1 bbbtranshq.mp4 Sam Siewert 7

8 Deeper Dive on Color and Human Vision Color, 3D Cues, and a Bit of Physiology / Psychology of Vision Sam Siewert 8

9 Reminder Computer Vision has the Goal to Emulate, Understand, Extend and Repair Human Vision Systems Machine Vision has the Goal to Automate a Process Using Instrumentation including Photometers Sam Siewert 9

10 Radiometry vs. Photometry Radiometry the study of light from viewpoint of Physics Energy (joules), Flux (photons/cm 2 ), Power (watts/m) Full Electromagnetic Spectrum including Visible from Far Infrared to Gamma-rays Photometry study of light from perspective of useful, often visible light Sam Siewert 10

11 Machine Vision Color in Brief Physics of Wavelength and Intensity of Electromagnetic Waves Simplified Luminance (Y), Chrominance U=B-Y (blue-luma), V=R-Y (red-luma) Additive Primary Hues (R, G, B) Secondary Hues (Combined R+G, R+B, G+B) CMY Complete subset of Colors is Gamut Chromaticity is independent of the luminance Additive model Measured in Visible for Classic Photometry (and Infrared) Visible Photometer (detectors sensitive to intensity in λ bands) Luminous flux compared to standard source (lumen) visible to eye Number of photons or flux in photons/cm2 Spectrograph (frequency separation from incident wave) Radiometry or Radiant flux total power in watts Subtractive model Physiology of What We Can See Tristimulus L ( nm) S ( nm) M ( nm) of Photoreceptors What We See in Various Lighting Conditions Different and Perhaps Not Identical to the Photometer Perhaps Not Identical to Display in Lighting Conditions Encoding from Measurements to Digital Samples Per Pixel, by color band, in an array Sam Siewert 11

12 Displayed Color Transformation Demo Adapting Rec 709 srgb Color Model to Different Lighting Conditions Cinematic, Ambient Fluorescent or Tungsten, Full-spectrum sunlight Based on Primaries, Tristimulus, Lighting, We See a different Gamut Rec 709 Color Space And D65 White Point Plot of Colors Visible to Eye CRT Gamut and CIE Chromaticity Diagram Chromaticity Diagram L ( nm) S ( nm) M ( nm Sam Siewert 12

13 Demo of Transform Work with eecolor for RT Color Transform at 1080p Gamut Boost for Lighting Conditions and Display Characteristics Sam Siewert 13

14 Color and Object Recognition Demo (Revisited) Object Recognition and Tracking Using Color in Real-Time Use Color Models or Signatures for Known Objects General Color Perception and Recognition Computer Vision Specific Color Signature Recognition Machine Vision Controlled Lighting, Apriori algorithm, not tracking primary colors, but rather centroids of objects with a color signature ECEN 4623/5623 University of Colorado Boulder Sam Siewert 14