A Real-time Computer Vision System for Measuring Trac. Parameters. David Beymer, Philip McLauchlan, Benn Coifman, and Jitendra Malik

Transcription

1 A Real-ime Compuer Vision Sysem for Measuring Trac Parameers David Beymer, Philip McLauchlan, Benn Coifman, and Jiendra Malik Dep. of Elecrical Engineering and Compuer Sciences Universiy of California Berkeley, California, fbeymer, pm, zephyr, Absrac For he problem of racking vehicles on freeways using machine vision, exising sysems work well in free-owing rac. Trac engineers, however, are more ineresed in monioring freeways when here is congesion, and curren sysems break down for congesed rac due o he problem of parial occlusion. We are developing a feaurebased racking approach for he ask of racking vehicles under congesion. Insead of racking enire vehicles, vehicle sub-feaures are racked o make he sysem robus o parial occlusion. In order o group ogeher sub-feaures ha come from he same vehicle, he consrain of common moion is used. In his paper we describe he sysem, a real-ime implemenaion using a nework of DSP chips, and experimens of he sysem on approximaely 44 lane hours of video daa. 1 Inroducion Trac managemen and informaion sysems mus rely on a sysem of sensors for esimaing rac parameers in real-ime. Currenly, he dominan echnology for his purpose is ha of magneic loop deecors, which are buried underneah highways o coun vehicles passing over hem. Video monioring sysems promise a number of advanages. Firs, a much larger se of rac parameers can be esimaed in addiion o vehicle couns and speeds. These include vehicle classicaions, link ravel imes, lane changes, rapid acceleraions or deceleraions, queue lenghs a urban inersecions, ec. Second, cameras are less disrupive and less cosly o insall han loop deecors, which require digging up he road surface. For some years, our group has been developing a prooype vision-based rac surveillance sysem [11, 12]. The core idea is o have video cameras mouned on poles or oher all srucures looking down a he rac scene. Video is capured, digiized, and processed by onsie compuers, and hen ransmied in summary form o a Transporaion Managemen Cener (TMC) for collaion and compuaion of muli-sie saisics such as link ravel imes. Processing occurs in hree sages: 1. Segmenaion of he scene ino individual vehicles and racking each individual vehicle o rene and updae is posiion and velociy in 3D world coordinaes, unil i leaves he racking zone. This research was suppored by FHWA hrough a conrac moniored by JPL, and by Calrans/PATH (MOU 152 and MOU 214). 2. Processing he rack daa o compue local rac parameers including vehicle couns per lane, average speeds, lane change frequencies, ec. These parameers, ogeher wih rack informaion (ime samp, vehicle ype, color, shape, posiion), are communicaed o he TMC a regular inervals. 3. A he TMC, local rac parameers from each sie are collaed and displayed as desired, and/or used in conrolling signals, message displays, and oher rac conrol devices. Compuers a he TMC also process he rack informaion from neighboring camera sies o compue long-disance parameers such as link imes and origin{desinaion couns. In his paper, we focus on he rs wo sages, he vehicle segmenaion and racking sage and he compuaion of rac parameers from he racking daa. 2 Tracking Approach Tracking moving objecs in video sreams has been a popular opic in he eld of compuer vision in he las few years; earlier conribuions o he areas of muli{arge racking and daa associaion were made by conrol and aerospace engineers. Our applicaion enails several sringen requiremens for a proposed scheme: 1. Auomaic segmenaion of a vehicle from he background and oher vehicles so ha here can be a unique rack associaed wih each vehicle. 2. Deal wih variey of vehicles { moorcycles, passenger cars, buses, consrucion equipmen, rucks, ec. 3. Deal wih range of rac condiions { ligh midday rac, rush-hour congesion, varying speeds in differen lanes. 4. Deal wih variey of lighing condiions { day, evening, nigh, sunny, overcas, rainy days. 5. Real-ime operaion of he sysem. Even hough a number of commercial sysems for rac monioring have been inroduced recenly, many of hese crieria sill canno be me. In a recen evaluaion of a group of hese commercial sysems [4], problems were repored wih congesion, long shadows linking ogeher vehicles, and he ransiion beween nigh and day. In he compuer vision lieraure, he dieren racking approaches for video daa can be classied as follows.

2 2.1 3D Model based racking Three-dimensional model-based vehicle racking sysems have previously been invesigaed by several research groups, he mos prominen being he groups a Karlsruhe [1] and a he Universiy of Reading[1, 15]. The emphasis is on recovering rajecories and models wih high accuracy for a small number of vehicles. The mos serious weakness of his approach is he reliance on deailed geomeric objec models. I is unrealisic o expec o be able o have deailed models for all vehicles ha could be found on he roadway. 2.2 Region based racking The idea here is o idenify a conneced region in he image { a \blob" { associaed wih each vehicle and hen rack i over ime using a cross-correlaion measure. Iniializaion of he process is mos easily done by he background subracion echnique. A Kalman ler-based adapive background model[8, 9] allows he background esimae o evolve as he weaher and ime of day aec lighing condiions. Foreground objecs (vehicles) are deeced by subracing he incoming image from he curren background esimae, looking for pixels where his dierence image is above some hreshold and hen nding conneced componens. This approach works fairly well in free-owing rac. However, under congesed rac condiions, vehicles parially occlude one anoher insead of being spaially isolaed, which makes he ask of segmening individual vehicles dicul. Such vehicles will become grouped ogeher as one large blob in he foreground image. 2.3 Acive conour based racking A dual o he region based approach is racking based on acive conour models, or snakes. The idea is o have a represenaion of he bounding conour of he objec and keep dynamically updaing i. The previous sysem for vehicle racking developed in our group [11, 12] was based on his approach. The advanage of having a conour based represenaion insead of a region based represenaion is reduced compuaional complexiy. However, he inabiliy o segmen vehicles ha are parially occluded remains. If one could iniialize a separae conour for each vehicle, hen one could rack even in he presence of parial occlusion[11]. However, iniializaion is he dicul par of he problem! 2.4 Feaure based racking Finally, ye anoher approach o racking abandons he idea of racking objecs as a whole bu insead racks sub-feaures such as disinguishable poins or lines on he objec. The advanage of his approach is ha even in he presence of parial occlusion, some of he sub-feaures of he moving objec remain visible. The echnology of racking poins and line feaures in a Kalman lering formalism is well developed in he compuer vision communiy. Since a vehicle could have muliple sub-feaures, he new problem hen is ha of grouping { wha se of feaures belong o he same objec. 3 Moion-Based Grouping The grouping of vehicle sub-feaures will be based on a common moion consrain, a concep known o Gesal y x Figure 1: A projecive ransform H, or homography, is used o map from image coordinaes (x; y) o world coordinaes (X; Y ). psychologiss as common fae. Poin feaures ha are seen as moving rigidly ogeher will be grouped ogeher ino a single vehicle. Bu since here are many vehicles in raf- c scenes, here is also an imporan segmenaion aspec o he problem. One does no wan o link ogeher subfeaures from neighboring vehicles. The grouping process mus be sensiive enough o pick up a moion ha disinguishes a vehicle from is neighbors, a moion such as a sligh acceleraion or lane drif. To make he grouping sysem robus enough o segmen dieren vehicles, he spaial informaion guiding he grouper will be inegraed over a period of ime, uilizing as many image frames as possible. Only he subfeaures ha are racked from a deecion region a he boom of he image o an exi region near he op will be allowed o paricipae in he nal grouping. Thus, in order o fool he grouper, wo vehicles would have o have idenical moions during he enire ime hey were being racked. In congesed rac, vehicles are consanly changing heir velociy o adjus o nearby rac, hus giving he grouper he informaion i needs o perform he segmenaion. In free-owing rac, vehicles may be more likely o mainain consan spaial headways over ime, hus making he grouping consrain less useful. Bu in his scenario, here is more space beween vehicles, so a spaial proximiy cue is added o aid he grouping/segmenaion process. Since mos road surfaces are a, he grouper explois an assumpion ha vehicle moion will be parallel o he road plane. To describe he road plane, he user simply species four or more line or poin correspondences beween he image road and a separae \world" road plane, as shown in Fig. 1. Based on his o-line sep, he sysem can compue a projecive ransform, or homography, beween he image coordinaes (x; y) and world coordinaes (X; Y ). By wriing poins in homogeneous coordinaes, his is a simple linear ransform " X Y 1 # / H " x y 1 The scaling of H is arbirary, so H(3; 3) is ofen chosen o be 1. The grouper considers sub-feaure poins in pairs. Tha is, he basic grouper compuaion is wheher or no o # H : Y X

3 sabilize deec rack car NTSC subfeaures racks group vehicles vehicle classes classify raffic parameers flow, speed, ec o TMC Figure 3: Example corner feaures locaed by he sysem. Figure 2: Block diagram of our vehicle racking and grouping sysem. group ogeher he 2D poin feaures p a () and p b (). The dependence on ime is wrien o emphasize ha he grouper is working wih sub-feaure racks, and hence has access o he ime hisory of poins. The 3D coordinaes of hese poins in he real world will be wrien in upper case P a () and P b (). Consider he simple case where P a and P b are a he same disance o he camera (e.g. boh on he back face of a ruck). In his scenario, he grouper only needs o look a a simple funcion of he displacemen vecor p a ()? p b (). Since P a and P b are boh a he same disance from he camera d, p a () and p b () are boh scaled by he same scale facor 1=d. Thus, for poins on he same vehicle, p a ()? p b () will be consan over ime if we can simply compensae for he 1=d scaling. Forunaely, he homography can be used for his compensaion. Given a poin (x; y) in he image, we can esimae he scale facor s ha ransforms he region around ha poin o world coordinaes. The dierence vecor p a ()? p b () can hen simply be scaled by s. We have also considered he more general case where P a and P b are no a he same disance from he camera. Space consideraions in hese proceedings preven a discussion of his case; please see [3] for he deails. 4 Algorihm 4.1 O-line camera deniion Before running he racking and grouping sysem, he user species some camera-specic parameers o-line. These parameers include: 1. line correspondences for he homography (Fig. 1), 2. a deecion region near he image boom and an exi region near he image op, and 3. a ducial poin for camera sabilizaion. 4.2 On-line racking and grouping A block diagram for our vehicle racking and grouping sysem is shown in Fig. 2. Firs, he raw camera video is sabilized by racking a manually chosen ducial poin o subpixel accuracy. Nex, he sabilized video is sen o a deecion module, which locaes corner feaures in a deecion zone near he boom of he image. These corner feaures are hen racked over ime in he racking module. Nex, sub-feaure racks are grouped ino vehicle hypoheses in he grouping module. Finally, rac parameers such as ow rae, average speed, and average spaial headway are compued from he vehicle racks. In he fuure, we inend o add a vehicle classicaion module ha will idenify vehicles as auomobiles, moorcycles, rucks, buses, ec. In his secion, we describe he deecion, racking, and grouping modules Feaure Deecion and Tracking Vehicle sub-feaures are deeced and racked in order o be insensiive o parial occlusion. Even if par of he vehicle is obscured due o congesed rac, some of he vehicle's sub-feaures should sill remain visible. Corner feaures are he chosen sub-feaures since hey can be reliably racked. Our corner deecor compues he windowed second momen marix by averaging in a spaial window he 2x2 marix, riri T, where ri is he image gradien [6]. are declared where he numerical rank of his marix is 2 (smaller eigenvalue above hreshold). Fig. 3 shows some example corner feaures deeced by he sysem. When a corner sub-feaure is deeced, a small 9x9 emplae of he grey level image is exraced and used for correlaion in he racking module. Also, while here are some undesirable corners presen near he vehicle boundaries and background, hese corners will be pruned away by he feaure ess employed by he racker. The racking module racks corner sub-feaures from he deecion region a he boom of he image o he exi region near he op. To address he problem of noisy measuremens, we employ Kalman lering[7] o provide mos likely esimaes of he sae of a vehicle sub-feaure based on accumulaed observaions. In our sysem, he sae vecor conains sub-feaure posiions and velociies (X; Y; X; _ Y _ ) in he world coordinae sysem; vehicle acceleraion is capured in he sysem dynamics noise process. The measuremen process in he Kalman ler is based on normalized correlaion. A each ime frame, he Kalman ler predics where o search for each corner feaure. This predicion is mapped back o he image plane, and hen he emplae exraced when he corner was originally deeced is correlaed in a window around he predicion. The emplae is scaled down over ime o reec he fac ha vehicles are geing smaller as hey move down he road surface. We can use he posiion in world coordinaes o predic he proper scale of he emplae. Once we have locaed he correlaion peak, his measuremen is mapped back ono he road plane. Finally, he sandard Kalman ler equaions for updaing he sae and error variance are employed. Two ess are used o eliminae bad sub-feaure racks: 1. Kalman ler innovaions. The disance beween he Kalman ler predicion and he curren measuremen is compued and he rack is rejeced if he

4 Figure 4: Example racks of corner feaures. Figure 5: Example groups of corner feaures. disance is above a hreshold. 2. Imprecise measuremen es. If he correlaion values form a broad, undened peak around he correlaion maximum, hen he measuremen process is probably no localizing he sub-feaure wihin he needed precision. To measure he peak's curvaure, we compue he number of pixels in he correlaion peak ha are wihin a cerain fracion of he peak. The rack is rejeced if he coun is over a hreshold. Fig. 4 shows he ime evoluion of some example racks, ploed as posiion over ime. The image shown is he frame when he corners were originally deeced Grouping The purpose of he grouping module is o group ogeher sub-feaures ha come from he same vehicle. The cenral cue used by he grouper { common moion { was described already in secion 3. In his secion, we discuss he deails of how he common moion consrain is applied o he sub-feaure racks. The grouper organizes is ask by consrucing a graph over ime. The verices are sub-feaure racks, edges are grouping relaionships beween racks, and conneced componens correspond o vehicle hypoheses. When a new sub-feaure is deeced and is added o he grouping graph, i is iniially conneced o all neighboring racks wihin a cerain radius in he image plane. The aiude of he grouper is ha nearby racks are compaible unil hey prove oherwise hrough relaive moion. For all pairs of racks p a () and p b () joined by an edge, he grouper keeps rack of he relaive displacemen d() = p a ()? p b () as scaled by he deph-compensaing facor compued from he homography. Upon each ime frame, anoher d value is compued for each edge, and he edge is broken if eiher max max d x ()? min d x () > x hreshold, or (1) d y ()? min d y () > y hreshold: This breaks he link beween wo racks if here is enough relaive moion beween he wo. In he normal evoluion of he graph, vehicles are overgrouped near he deecion region since he graph is liberally conneced a rs. Bu as vehicles move down he road, hey are segmened as hey perform a disinguishing moion such as lane drif or an acceleraion. When he las rack of a conneced componen eners he exi region, a new vehicle hypohesis is generaed and he componen is removed from he grouping graph. Fig. 5 shows he nal groups compued for he vehicles in he racking region (which is he middle par of he image). Corner feaures are indicaed by circles, and here is an edge drawn beween grouped corners. How are he grouping hresholds in equaion (1) deermined? Consider how he median vehicle size changes as a funcion of he grouping hreshold (Fig. 6). Here, we assume ha he same hreshold is used for x and y, and vehicle size is measured as he maximum disance beween any wo poins in he group. Empirically, one noices ha he plo of median vehicle size versus hreshold exhibis wo linear regimes: 1. Oversegmenaion. Below opimum hreshold. Vehicle size increases rapidly as one raises he hreshold, as correc groups are sill being consruced ou of vehicle fragmens. 2. Overgrouping. Above opimum hreshold. This par of he graph has a lower slope, as i is harder o group ogeher dieren vehicles han i is o group a single vehicle's sub-feaures. Given his relaionship, our goal is o deec he breakpoin beween he wo regimes. In an o-line sep, we sample he graph by running he grouper a dieren hresholds and compuing he median vehicle size. Nex, wo line segmens are o he graph by minimizing he sum of squared error, which locaes he breakpoin. We performed his procedure for all 7 video sequences in secion 6.2. The hresholds compued led o vehicle recogniion raes ha were very close o he opimum hresholds (opimum hresholds were compued via exhausive search). In he wors case, he compued hresholds led o a decline of only 3.6% in he recogniion rae. 5 Real-ime Sysem We have implemened he racker on a nework of 13 Texas Insrumens C4 DSPs, conneced ogeher as shown in Fig. 7. The compuaionally heavy operaions in he racking algorihm { convoluion in he feaure deecor and correlaion in feaure racker { are placed on he C4 nework, while he grouper is run on he hos PC. Running he grouper on he PC is necessiaed by memory

5 vehicle size oversegmen overgroup opimum hreshold grouping hreshold Figure 6: The plo of vehicle size versus grouping hreshold exhibis wo regimes: an oversegmenaion regime and overgrouping regime. Hos PC rack NTSC Frame- grabber updaes C4 nework Track Conroller images & corners Display Figure 7: The C4 nework used for feaure racking, consising of an NCSC frame-grabber C44 module (C44s are cu-down C4s wih only four communicaion links); a quad C44 module (four processors) for corner feaure deecion; a large memory module (8M RAM) for mainaining he sae of curren feaure racks (rack conroller); six fas SRAM modules for feaure racking; and a VGA graphics module for display. requiremens. The grouper needs o sore rack rajecories, which would quickly exhaus he limied memory available on he C4 modules. Bu keeping he grouper on he PC is also benecial from a load balancing perspecive, as he PC is a 15MHz Penium and hus equivalen o 3 o 4 C4s. The processors are arranged in wo loops, each of which is operaed as a pipeline feeding back o is source. These wo pipelines are conrolled by he frame grabber and he rack conroller; hey compue he corner feaures and he rack updaes respecively. Four C44 processors are assigned o corner deecion, each processing one quarer of he user-dened deecion region. The corners are fed back o he frame-grabber, which passes hem along wih he original image o he rack conroller. A simple eciency gain is achieved by sending he image rs, since he rack conroller can hen updae he exising racks while he corners are compued. The job of he rack conroller is o mainain he sae of he complee lis of curren racks. I does his by receiving updaes for exising racks from is pipeline of six C4s, and creaing new racks a posiions indicaed by he corner deecor. The racker C4s each updae one sixh of he racks. Since rack updaes are fairly homogeneous asks, his achieves good load balancing. The performance of he racker is 7.5Hz in uncongesed rac, dropping o 2Hz in congesed rac, where many more racks are in progress a any given ime. This reducion in speed does no of iself lead o a reducion in performance of he racker, since vehicle speeds in congesed rac are reduced, and so he requiremen for racking Sequence Descripion Lengh N G Highway 55 heavy congesion 2: Florin Rd 1 free ow & congesion 2: Mac Rd 1 free ow & congesion 1: Florin Rd 2 nigh 1: San Jose urban inersecion 1: Mac Rd 2 free ow & congesion 3: Florin Rd 3 free ow & congesion 3: Table 1: Video sequences for laboraory esing. Lengh is in min:sec, N is he number of acual vehicles (couned by a human) and G is he number of repored vehicle groups. rae is naurally reduced. 6 Resuls Our racking and grouping sysem has gone hrough wo major phases of esing. Firs, we esed a sofware-only, o-line version of he sysem in erms of is abiliy o deec vehicles. This esing gave us a \microscopic" view of he sysem, allowing us o analyze errors such as false deecions, false negaives, and overgroupings. Second, he real-ime sysem was esed on a subsanial amoun of daa { 44 lane hours worh { o see if he sysem could accuraely measure he aggregae parameers of ow, velociy, vehicle densiy, and average spacing. 6.1 O-line esing of vehicle deecion In order o analyze he behavior of he sysem a he vehicle level, we esed he sysem's vehicle deecion rae for a se of videoapes covering a range of scene condiions: congesion, free-ow, nigh, and an urban inersecion (see Table 1). Since we waned o measure errors such as vehicle oversegmenaion and overgrouping, vehicle ground ruh was manually dened for each sequence. For a paricular vehicle, ground ruh is a binary mask oulining he vehicle in one or wo frames. The number of ground ruhs is denoed as N in Table 1, and he number of repored groups is G. Table 2 shows he performance of our sysem using auomaically compued grouping hresholds, as well as he disribuion of errors. A separae auomaic evaluaion program compares he vehicle ground ruhs agains he groups repored by he racker/grouper and allies he following evens: 1. True mach. A one-o-one maching beween a ground ruh and a group. 2. False negaive. An unmached ground ruh. 3. Oversegmenaion. A ground ruh ha maches more han one group. 4. False posiive. An unmached group. 5. Overgrouping. A group ha maches more han one ground ruh. In analyzing he resuls, i should be said ha he Highway 55 sequence is a dicul one because of a poor camera posiion and a number of large rucks ha someimes compleely occlude auomobiles. In erms of rading o he dieren error condiions, we have noiced ha oversegmenaion and overgrouping can be raded o by adjusing he grouping hresholds.

6 rue false over- false over- Sequence mach neg. seg pos. group Highway % 18.5% 6.7% 4.9%.4% Florin Rd % 1.6% 6.9% 1.9% 1.5% Mac Rd % 1.5% 1.5% 2.8% 1.4% Florin Rd % 6.9% 3.4% 2.%.% San Jose 85.3% 2.9% 5.9%.% 2.9% Mac Rd 2 8.3% 6.% 1.4% 2.3% 1.5% Florin Rd % 2.2% 1.1%.3% 1.8% Table 2: Performance of he racking/grouping sysem on he o-line es sequences. When compuing raes, he rs hree columns divide he number of rue maches, false negaives, ec., by N; he nal wo columns divide by G. As he rs hree sequences have long shadows, he experimenal resuls show ha he sysem can handle shadows { shadow sub-feaures end o be unsable over ime, especially in congesion. 6.2 On-line esing of rac parameers Our second phase of esing evaluaed he on-line sysem's abiliy o esimae aggregae rac parameers. The parameers ypically used by rac engineers o monior he freeways include: 1. Flow. Number of vehicles per hour. 2. Velociy. Average vehicle velociy. 3. Densiy. Number of vehicles per uni disance. 4. Headway. Average spacing beween vehicles. These parameers are compued separaely for each lane of rac and are averaged over a period of ime (aken o be 5 minues in our experimens). Also, i should be apparen ha hese are no independen variables; we use he mehodology from Edie[5] o compue hese parameers from he vehicle rack daa. Ground ruh is provided from inducive loop daa ha was colleced concurrenly wih he video daa. Each lane of rac has wo loops separaed by 2 fee, giving us an eecive speed rap for measuring velociy. Our sysem was esed on approximaely 44 lane hours of video from he Florin Road inerchange along Highway 99 in Sacrameno, Calif. (see Fig. 9 for an example sho). The daa includes all observed operaing condiions: day, nigh, wiligh, long shadows and rain; congesion and free ow. Lane 1, on he lef, is carpool (HOV) lane and exhibied lile if any congesion. Lane 3, on he righ, exhibied some degree of congesion for approximaely 2% of he ime. Finally, he loops in lane 2 were bad so i was excluded from he nal analysis. The video daa was divided ino 5 minue aggregaion periods, yielding 514 samples for he rac parameers. Overall, here were roughly 4, vehicles in he nal video daa se. The vehicle rack daa from he real-ime sysem can hen be compared wih he loop daa over he 2 foo region of overlap beween he racks and loop daa. Fig. 8 shows scaer plos of he ow and velociy esimaes provided by he loop and vision daa, and Table 3 summarizes he error disribuion for velociy, ow, densiy, and headway. As one would expec from a feaure based racker, he measured velociy is very accurae. Even if he racker overgroups or oversegmens vehicles, he erroneous blobs measured q (veh/hr) measured v (mi/h) minue avg flow; 44 hours of daa ground ruh q (veh/hr) 5 minue avg velociy; 44 hours of daa ground ruh v (mi/h) Figure 8: Scaer plos comparing ow q and velociy v o ground ruh for he 44 lane hours of daa used o es he real-ime sysem. sill move a he prevailing speed. The errors in ow, densiy and spacing are due o missed or over couned vehicles. Ofen, an error of wo or hree vehicles in one sample can be very signican. For example, one missed vehicle in a ve minue sample a 1, veh/hr resuls in a 2% error. A he mean ow for he daa, 91 veh/hr, he error per missed vehicle is slighly higher, a 2.2%. Anoher way o examine esimaed rac parameers is as a ime series. To demonsrae he performance of our sysem during a dramaic change in lighing condiions from nigh o day, in Fig. 1 we show ow q and velociy v for a wo hour srech of coninuous video. The video sars a nigh (5:3 AM, see Fig. 9, lef), progresses % error % vel % ow % dens % headway less han samples samples samples samples 2.5% 86% 18% 19% 19% 5% 95% 31% 33% 34% 1% 1% 6% 59% 6% 15% 1% 79% 79% 81% 2% 1% 91% 9% 89% 25% 1% 96% 96% 94% Table 3: Error disribuion for velociy, ow, densiy, and headway.

7 I68 shock wave A (dashes=ground ruh, solid=racker) Figure 9: Two images from he sar and end of a wo hour run of he real-ime sysem. dis (m) 3 6 flo_17r, 4/1/96, 5 minue velociy 2 1 q (veh/h) v (mi/h) 4 2 racker, lane 1 racker, lane 3 ground ruh ime (hr) flo_17r, 4/1/96, 5 minue flow ime (hr) Figure 1: Flow and velociy as a funcion of ime. The sequence begins a 5:3 AM (nigh lighing condiions) and nishes a dayime. hrough sunrise and long shadows, and ends wih day (7:3 AM, see Fig. 9, righ). In he plo of ow and velociy, here are 48 samples of 5 minue periods and roughly 4,6 vehicles. Noe ha he morning rush hour peak sars during he sequence and approximaely 3 minues of daa from lane hree are under ligh congesion, and hus, frequen occlusions. In addiion, since he primary design goal in developing our sysem was o deal wih congesion, we close he resuls secion wih an example of a \shockwave". Fig. 11 plos vehicle racks as he disance along he lane as a funcion of ime. In his case, ground ruh was enered manually a a number of poins along each vehicle's rajecory. In he regions of he graph where he slope goes o zero, one noices ha vehicles coninue o be racked even when rac has come o a complee sop. 7 Summary We have presened a vehicle deecion and racking sysem ha is designed o operae in congesed rac. Insead of racking enire vehicles, vehicle sub-feaures are racked, which makes he sysem less sensiive o he problem of parial occlusion. In order o group sub-feaures ha come from he same vehicle, he consrain of common moion over rajecory lifeimes is used. A real-ime version of he sysem has been implemened using a nework of C4 DSP chips conneced o a hos PC. The sysem has been esed on approximaely 44 lane hours of daa and has demonsraed good performance no only in congesed rac, bu also on free-owing, nighime, and urban inersecion rac ime (s) Figure 11: Vehicle racks for he real-ime sysem during a shockwave. When he slope is zero, he vehicle is compleely sopped. References [1] K.D. Baker and G.D. Sullivan. Performance assessmen of modelbased racking. In Proc. of he IEEE Workshop on Applicaions of Compuer Vision, pp , Palm Springs, CA, [2] Yaakov Bar-Shalom and Thomas E. Formann. Tracking and Daa Associaion. Academic Press, New York, [3] David Beymer and Jiendra Malik. Tracking vehicles in congesed rac. In SPIE Vol. 292, Transporaion Sensors and Conrols: Collision Avoidance, Trac Managemen, and ITS, pages 8{18, Boson, MA, November [4] A. Chaziioanou, S. Hockaday, L. Ponce, S. Kaighn and C. Saley. Video Image Processing Sysems Applicaions in Transporaion, Phase II. Final Repor, Calif. Poly. Sae Univ., San Luis Obispo, Calif., Dec. 3, [5] L.C. Edie. Discussion of rac sream measuremens and deniions. In Proc. Second Inernaional Symposium on he Theory of Trac Flow, OECD, pages 139{154, Paris, France, [6] W. Forsner and E. Gulch. A fas operaor for deecion and precise locaion of disinc poins, corners, and ceners of circular feaures. In Proc. of he Inercommission Conf. on Fas Processing of Phoogrammeric Daa, pages 281{35, [7] Arhur Gelb, edior. Applied Opimal Esimaion. The MIT Press, Cambridge, MA., [8] Klaus-Peer Karmann and Achim von Brand. Moving objec recogniion using an adapive background memory. In V Cappellini, edior, Time-Varying Image Processing and Moving Objec Recogniion, 2. Elsevier, Amserdam, The Neherlands, 199. [9] Michael Kilger. A shadow handler in a video-based real-ime rac monioring sysem. In IEEE Workshop on Applicaions of Compuer Vision, pages 16{166, Palm Springs, CA, [1] D. Koller, K. Daniilidis, and H.-H. Nagel. Model-based Objec Tracking in Monocular Image Sequences of Road Trac Scenes. Inernaional Journal of Compuer Vision, 1: ,1993. [11] D. Koller, J. Weber, and J. Malik. Robus muliple car racking wih occlusion reasoning. In ECCV, pp. 189{196, Sockholm, Sweden, May 2-6, [12] D. Koller, J. Weber, T. Huang, J. Malik, G. Ogasawara, B. Rao, and S. Russell. Towards robus auomaic rac scene analysis in real-ime. In ICPR, Israel, November [13] H.A. Mallo, H.H. Bulho, J.J. Lile, and S. Bohrer. Inverse perspecive mapping simplies opical ow compuaion and obsacle deecion. Biological Cyberneics, 64(3):177{185, [14] P. Michalopoulos. Vehicle deecion video hrough image processing: he Auoscope sysem. IEEE Trans. on Vehicular Technology, 4:21-29, [15] G.D.Sullivan. Visual inerpreaion of known objecs in consrained scenes. In Phil. Trans. Roy. Soc (B), 337: , 1992.