Goal: Label Skin Pixels in an Image. Their Application. Background/Previous Work. Understanding Skin Albedo. Measuring Spectral Albedo of Skin

Transcription

1 Goal: Label Skin Pixels in an Image Statistical Color Models with Application to Skin Detection M. J. Jones and J. M. Rehg Int. J. of Computer Vision, 46(1):81-96, Jan 2002 Applications: Person finding/tracking Gesture recognition Filter Adult Content Their Application Background/Previous Work Skin is well-modeled by a dichromatic reflectance model. transparent medium (dermis) pigmentations (hemaglobin, melanin) specular reflection (oil on skin) Dichromatic reflectance model Measuring Spectral Albedo of Skin Understanding Skin Albedo Increase in melanin yields darker skin, masking the absorbtion band pattern of the hemaglobin.

2 Analytic Model Generate different skin albedos by using observed curve for caucasian, and calculate the reduction in reflectance due to an increase in melanin (a substance that has a known absorbtion) Problem: Color Variation Apparent color varies due to lighting color and and camera spectral response. Simpler approximation: I 1 ( ) ~ s I 2 ( ) ; wavelength Sample from Oulu Physics-Based Face Database Illuminant SPD could calibrate, if you knew the light source Camera Spectral Response could calibrate, if you knew the camera too Blackbody sources (for theoretical calculations) Artificial light sources SONY DXC-755P 3CCD (manufacturer can supply this) Jones and Rehg, 2002 Statistical Color Models with Application to Skin Detection, M. J. Jones and J. M. Rehg, Int. J. of Computer Vision, 46(1):81-96, Jan 2002 General Idea: Drop the physics. Learn from examples instead. Learn distributions of skin and nonskin color Histograms; Mixture of Gaussians Bayesian classification of skin pixels Approach:Learning from Examples First, have some poor grad student hand label thousands of images P(rgb skin) = number of times rgb seen for a skin pixel total number of skin pixels seen P(rgb not skin) = number of times rgb seen for a non-skin pixel total number of non-skin pixels seen These statistics stored in two 32x32x32 RGB histograms R Skin histogram G B Non-Skin histogram R G B

3 Learned Distributions Likelihood Ratio Label a pixel skin if P(rgb skin) P(rgb not skin) > (cost of false positive) P( seeing not skin) (cost of false negative) P( seeing skin) Sample Pixel Classifications Jones and Rehg Model A compact description is provided by converting the histogram-based model into a Gaussian Mixture model. Jones and Rehg Mixture Model Jones and Rehg Mixture Model

4 Sample Use: Adult Image Classification Adult Image Classification Based on Five Features: Percentage of pixels detected as skin. Average probability of the skin pixels. Size in pixels of the largest connected component of skin. Number of connected components of skin. Percentage of colors with no entries in the skin and non-skin histograms Combining Color and Text Trying on My Own Examples Example of False Positives

5 Likelihood ratio is important model both the object AND not-object distributions to avoid thresholds on raw probabilities. Likelihood ratio is important model both the object AND not-object distributions to avoid thresholds on raw probabilities. EM and MoG models used to encode compact descriptions of color histograms.