Using Color Histograms to Recognize People in Real Time Visual Surveillance

Transcription

1 Usig Color Histograms to Recogize People i Real Time Visual Surveillace DANIEL WOJTASZEK, ROBERT LAGANIERE S.I.T.E. Uiversity of Ottawa, Ottawa, Otario CANADA daielw@site.uottawa.ca, lagaier@site.uottawa.ca Abstract: - This paper presets a surveillace system that detects ad recogizes people i idoor scees. To distiguish betwee differet people, color histograms o a perceptually uiform color space are used. The goal here is to associate a sequece showig a perso leavig a room with the previously recorded sequece showig that same perso eterig. Keywords: - Recogitio Color Histogram Surveillace People Trackig 1 Itroductio I may surveillace applicatios, recogizig people i the scee is desired. The most uique visual features oe ca extract from a image of a perso are facial features. People recogitio, therefore, usually proceeds by extractig some facial features ad fidig a match from amog the features of faces already stored i memory. A limitatio to this method is if someoe is i the scee but his face is ot visible from the camera(s). Before a system ca recogize a perso, it must detect ad locate them i the image. A commo method of perso detectio i a sequece of images is comparig the curret image with a referece or backgroud image. Ay pixels i the curret image that differ accordig to some criteria from the backgroud image are labeled as foregroud pixels. The foregroud pixels are the aalyzed ad the locatio of a perso is determied usig some criteria such as shapes of clusters of foregroud pixels. A few approaches to backgroud modelig, foregroud segmetatio ad perso locatio are preseted i [1], [2] ad [3]. Aother method that uses a patter recogitio techique was proposed i [4]. This method ivolves creatig a hidde Markov model of some features extracted from sample images of a perso before the system is brought o lie. Whe the system is o lie, the Viterbi algorithm is used to segmet the image ito foregroud ad backgroud regios. Oce a perso is located i a image the ext step is to determie who this perso is if visual features of this perso have previously bee acquired. There are two steps to recogizig a perso: extractig the proper features ad comparig these features to determie if they come from the same perso or ot. Orwell, et al. [5] combie the foregroud pixels which are idetified as part of a sigle perso ito a vector which is the used as the feature. They the determie if the features extracted from differet images of people match usig statistical methods such as χ 2 probability fuctio. The eviromet that a surveillace system is desiged to moitor greatly iflueces the methods used. For the purposes of visual surveillace the most importat evirometal factors are the lightig ad the expected activity i the scee beig observed. Oe situatio where visual surveillace is useful is whe we would like to moitor a passage iside a buildig. This passage could be a doorway, a hallway or ay other area that people use to go from oe area to aother. The applicatio preseted i this paper moitors the doorway to our laboratory usig a sigle color camera. More specifically, we wat to record ay activity i this scee, to keep cout of how may people are i the laboratory at ay give time ad to keep a record of the activity that occurs i the laboratory durig a certai period of time. The scee uder observatio shows the etrace portio of the lab ad also icludes a view of a door givig access to a adjacet room.

2 2 People Detectio The first step i people detectio is to form the silhouettes of foregroud objects i the image. The foregroud regios of a image are extracted by a pixel wise compariso betwee this image ad a backgroud image. Ay pixels that differ by more tha a threshold are labeled as foregroud (white) pixels i a biary image. All other pixels are labeled as backgroud (black). Each pixel of the backgroud image is modeled usig a Gaussia distributio. The threshold for each pixel is three times the stadard deviatio for that pixel. Ideed observig a idoor scee with good lightig coditios ad with movig objects (people) of relatively importat size does ot ecessitate the use of a more complex backgroud model such as the model preseted i [6]. Media filterig is used ext to remove ay small groups of foregroud pixels that most likely are ot a part of ay object of iterest. Fially, closig morphological operatios are used to improve the shape of each silhouette. Figure 1 shows the silhouettes of two people formed usig this method. Oce the silhouettes have bee formed they must be aalyzed to see if ay of them fit the shape of a perso. To do this, local vertical peaks o the boudary of each silhouette are located usig Quasi- Topological Codes i a similar fashio to what was doe i [3]. From each local peak foud, the silhouette boudary is scaed i the left ad right directios recordig the curvature usig a square four elemet widow. p1 p4 p2 p3 (a) (b) Fig. 1 Example of silhouette formatio. (a) A sample image. (b) The silhouette formed usig backgroud compariso. the local maximum poit. The fial criteria for determiig if each putative head is i fact a perso s head is that it must have a body uder it. To determie this, a regio is defied for each putative head which is bouded horizotally by the extreme horizotal coordiates of the boudary of the head foud above, o top by the vertical coordiate of the maximum poit ad o the bottom by the vertical coordiate of the top plus the width of the head multiplied by some costat. b = m y + k * w Where m y is the vertical coordiate of the maximum poit, k is the expected ratio of a perso s height to the width of his head, ad w is the width of the head. I this applicatio we limit the height to the legth from the top of the head to the begiig of the legs. The possible head is cosidered above a body if this defied regio of the biary image has a high eough percetage of pixels labeled as foregroud. Figure 2 shows two examples of this regio idicated by the black rectagles i the image. c = p1 + 2p2 + 4p3 + 8p4 p1; p2; p3; p4 = 0 or 1 The value of c determies what the curvature of the boudary at a certai poit is. If tracig the silhouette boudary startig from the maximum poit i the left ad right directios yields a covex curve (dowward curve) ad the a vertical drop, the this boudary is likely to be the boudary of a head. Next, the width of the putative head is determied by recordig the horizotal coordiate of the pixel o the boudary of the head which is furthest left of the local maximum poit ad the pixel o the boudary of the head which is furthest right of Fig. 2 A sample image showig two detected persos ad the regios of iterest.

3 3 Feature Extractio I order to moitor the activities that occur i the room uder observatio, we have to be able to recogize a perso that passes i frot of the camera. More specifically our goal is to associate a sequece showig a perso leavig the room with the previously recorded sequece showig that same perso eterig the room. To distiguish betwee differet people we decided to use color iformatio. This will work well if the observed people wear a good variety of clothig ad if we expect that they will ot chage their clothig oce iside the room. Also sice this system moitors a area where people are passig through ad therefore are see from differet agles, we choose ot to use ski or hair colors. These attributes are more likely to be similar betwee differet people ad are more difficult to reliably extract from differet agles. Moreover clothig color iformatio is geerally radially ivariat. To extract color iformatio, a three dimesioal histogram is used. We choose to use a histogram because the histograms of the colors extracted from people wearig differet colored clothig are differet whereas histograms of the same perso take at differet times are more similar to each other. Two dimesios describe the chromiace, q u ad q v, defied by the CIE Uiform Chromaticity Scale ad the third dimesio describes the lumiace defied by the y compoet of the CIE XYZ color space. Note that oly pixels which are labeled as foregroud ad do ot represet a part of the perso s head will be added to the histogram. y = R G B x = R G B z = R G B q 4x u ' = x + 15y + 3 z q 9y v ' = x + 15y + 3 z Where R, G, ad B are tristimulus values from the Rec 709 Primaries. The chromiace defied above are perceptually uiform which meas that two colors which are at a fixed Euclidea distace from each other o the (q u, q v ) plae will have the same relative perceptual differece to a huma o matter where these colors are located o the (q u, q v ) plae. This characteristic allows oe to decide umerically which pair of colors look more alike give several colors. The domai of each dimesio of the histogram is chose to be the rage of the correspodig color compoet. The umber of bis ad the bi boudaries are chose off lie ad are fixed. These parameters are fixed to make accumulatig color iformatio from several images of the same perso i the image sequece ito a sigle histogram more efficiet. The umber of bis, i other words the coarseess of quatizatio, for each dimesio is chose depedig o how much computatioal power is available. The higher the umber of bis, the better the recogitio results ad the more computatioal power required. Of course whe the umber of bis is fairly large, icreasig this umber by ay sigificat amout will ot sigificatly improve the recogitio results. The bi boudaries are chose to be uiformly spread across the domai of each dimesio. Aother importat factor i extractig the color of a perso s clothig is what regio i the image to extract the color from. This regio should most likely be void of ay ski or hair. A simple regio which satisfies this criterio is a rectagular regio below the perso s eck. A example of this regio is idicated by the black squares i figure 2 excludig the area cotaiig the head ad eck of the perso. Choosig to extract the color from this regio greatly reduces the possibility of colors from other people i the image beig mixed with the colors extracted from the perso of iterest due to overlap of body parts. The area of the head ad eck of a perso is approximated by a rectagular regio of width equal to that of the head ad height proportioal to the width. Whe a perso passes through the scee they are tracked ad the color iformatio discussed above is accumulated from every image of the perso as they pass through the scee ito the histogram. The histogram is the ormalized so that each bi ow cotais the percetage of the total umber of pixels accumulated i the histogram. Trackig is accomplished by usig color histograms to compare a perso detected i a image to ay perso detected i recet previous images to determie correspodeces. So if the same perso is detected i

4 several cosecutive or early cosecutive images, the iformatio about this perso extracted from these images is labelled as belogig oly to this perso. Figure 3 shows the path of the head of a perso who is leavig the lab as was determied by trackig this perso. 4 People Matchig To compare two histograms a measure of dissimilarity is computed usig the Earth Mover s Distace, EMD, preseted i [7]. For this measure, oe of the histograms, called the source histogram, is like several piles of earth o a field where each pile represets a bi i the histogram, the mass of earth i each pile represets the value of the histogram at the correspodig bi ad the field represets the domai of each dimesio of the histogram. The other histogram, called the destiatio histogram is like several holes i a field where a hole represets a bi i the histogram, the volume of each hole represets the value of the histogram at the correspodig bi ad the field is the same as the field described for source histogram. The EMD is the miimum eergy required to fill the holes i the field with the earth from the piles i the field. Whe several sequeces of people passig through the scee are compared to a ew image sequece i this way, the two sequeces which yield the lowest value of the EMD are cosidered the most likely to be the same perso. 5 Results To measure the effectiveess of this matchig techique we proceeded as follows: 1. Image sequeces of twety differet people are captured. Two sequeces are captured for each perso: oe of the perso eterig the lab ad oe of the perso leavig the lab. 2. out of the twety people are radomly selected ad these people are assumed to be i the lab. 3. The matchig techique is used to compare a perso s leavig sequece to each eterig sequece of the people i the lab before this perso left. If the miimum value of the EMD is geerated from comparig two sequeces of the same perso the Fig. 3 A image showig the path take by a perso. recogitio is successful. Else the recogitio is usuccessful. 4. Step 3 is repeated usig a differet perso s leavig sequece util each of the people has bee the leavig perso. 5. Steps 2 to 4 are repeated esurig that a differet set of people are chose each time util each of the twety people have bee icluded i at least oe of these sets. 6. Steps 2 to 5 are repeated for each value of ragig from two to twety. Recogitio rate for each value of, R, is the calculated. R = N / N s t s N : The umber times step 3 is performed for the same value of ad results i a successful recogitio. t N : The total umber of times step 3 is performed for the same value of ad results i either a successful recogitio or ot. Figure 4 shows a graph of recogitio rate, R versus the umber of people i the lab,. 6 Coclusio A method to detect ad recogize people i idoor scees has bee preseted. To detect people, the use of silhouette shape cues was used. To recogize people, color histograms were used o a perceptually uiform color space. These methods were successful

5 at least eighty percet of the time for recogizig people who reeter the scee. Refereces: [1] M. Lee, Detectig People i Cluttered Idoor Scees, Proceedigs of CVPR, 2000, pp [2] I. Haritaoglu, D. Harwood, L. Davis, Hydra: Multiple People Detectio ad Trackig Usig Silhouettes, Proceedigs of Secod IEEE Iteratioal Workshop o Visual Surveillace, 1999, pp [3] J. Heikkila, O. Silve, A Real-Time System for Moitorig of Cyclists ad Pedestrias, Proceedigs of Secod IEEE Iteratioal Workshop o Visual Surveillace, 1999, pp [4] G. Rigoll, B. Witerstei, S. Muller, Robust Perso Trackig i Real Scearios with No- Statioary Backgroud Usig a Statistical Computer Visio Approach, Proceedigs of Secod IEEE Iteratioal Workshop o Visual Surveillace, 1999, pp [5] J. Orwell, P. Remagio, G.A. Joes, Multicamera Colour Trackig, Proceedigs of Secod IEEE Iteratioal Workshop o Visual Surveillace, 1999, pp [6] M. Harville, G. Gordo, J. Woodfill, Foregroud Segmetatio Usig Adaptive Mixture Models i Color ad Depth, Proceedigs of IEEE Workshop o Detectio of Evets I Video, 2001, pp [7] Y. Ruber, C. Tomasi, L. J. Guibas, The Earth Mover s Distace as a Metric for Image Retrieval. Techical Report STAN-CS-TN-98-86, Departemet of Computer Sciece, Staford Uiversity, Fig. 4 Graph showig recogitio rate vs. umber of people i the sample set.