Automated Multi-Camera Surveillance Algorithms and Practice

Automated Multi-Camera Surveillance Algorithms and Practice

The International Series in Video Computing Series Editor: Mubarak Shah, Ph.D University of Central Florida Orlando, Florida

Automated Multi-Camera Surveillance Algorithms and Practice Omar Javed ObjectVideo Inc. Reston, Virginia, USA Mubarak Shah University of Central Florida Orlando, Florida, US 123

Omar Javed ObjectVideo Inc. Reston, VA 20191, USA Mubarak Shah University of Central Florida Orlando, FL 32816, USA ISSN: 1571-5205 ISBN: 978-0-387-78880-7 e-isbn: 978-0-387-78881-4 DOI: 10.1007/978-0-387-78881-4 Library of Congress Control Number: 2008930788 2008 Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper springer.com

To Captain Michael Holloway, Orlando Police Department (retired), who introduced us to video surveillance and provided enthusiastic support. Omar Javed & Mubarak Shah To my late father Mahmud Javed, who was always there for me; to my mother Dure-Shahwar who taught me the meaning of unconditional love, and to my wife Laila, who constantly brings joy to my life. Omar Javed

Contents 1 AUTOMATED VIDEO SURVEILLANCE... 1 1.1 Introduction... 1 1.2 Automated Systems for Video Surveillance.... 2 1.3 Automated Surveillance System Tasks and Related Technical Challenges.... 4 1.3.1 Object Detection and Categorization... 4 1.3.2 Tracking... 4 1.3.3 Tracking Across Cameras... 5 1.3.4 General Challenges... 6 1.4 Introduction to the Proposed Video Understanding Algorithms for Surveillance... 6 1.5 Book Organization...... 9 2 IDENTIFYING REGIONS OF INTEREST IN IMAGE SEQUENCES 11 2.1 Introduction... 11 2.2 General Problems in Background Subtraction...... 12 2.3 Related Work...... 13 2.3.1 Background Subtraction using Color as a Feature...... 14 2.3.2 Background Subtraction using Multiple Features... 16 2.3.3 Finite State Space Based Background Subtraction..... 17 2.4 Proposed Approach for Background Subtraction..... 17 2.4.1 Assumptions..... 18 2.4.2 Pixel Level Processing...... 18 2.4.3 Region Level Processing.... 21 2.4.4 Frame-Level Processing.... 22 2.5 Results... 22 2.6 Discussion.... 24 3 OBJECT DETECTION AND CATEGORIZATION... 29 3.1 Introduction... 29 3.2 Problems in Object Categorization...... 30 vii

viii Contents 3.3 Related Work...... 30 3.3.1 Periodicity Based Categorization...... 30 3.3.2 Object Categorization using Supervised Classifiers.... 31 3.3.3 Object Categorization using Weakly Supervised Classifiers. 32 3.4 Overview of the proposed categorization approach... 33 3.5 Feature Selection and Base Classifiers... 34 3.6 The Co-Training Framework... 36 3.6.1 Online Learning... 37 3.7 Co-Training Ability Measurement... 39 3.8 Results... 39 3.9 Concluding Remarks..... 42 4 OBJECT TRACKING IN A SINGLE CAMERA... 45 4.1 Introduction... 45 4.2 Related Work...... 45 4.2.1 Feature Point Tracking Methods... 46 4.2.2 Region Tracking Methods... 47 4.2.3 Methods to Track People.... 48 4.3 Problems in Tracking 2D silhouettes of People..... 49 4.3.1 Occlusion... 50 4.3.2 Entries and Exits...... 50 4.4 Proposed Approach for Tracking.... 50 4.4.1 Assumptions..... 50 4.4.2 Object Tracker... 51 4.5 Results... 52 4.6 Discussion.... 54 5 TRACKING IN MULTIPLE CAMERAS WITH DISJOINT VIEWS 59 5.1 Problem Overview and Key Challenges... 59 5.2 Related Work...... 61 5.2.1 Multi-Camera Tracking Methods Requiring Overlapping Views:... 61 5.2.2 Multi-Camera Tracking Methods for Non-Overlapping Views:... 62 5.3 Formulation of the Multi-Camera Tracking Problem..... 64 5.4 Learning Inter-Camera Space-Time Probabilities... 66 5.5 Estimating Change in Appearances across Cameras...... 67 5.5.1 The Space of Brightness Transfer Functions...... 68 5.5.2 Estimation of Inter-Camera BTFs and their Subspace... 71 5.5.3 Computing Object Color Similarity Across Cameras Using the BTF Subspace..... 72 5.6 Establishing Correspondences...... 72 5.7 Results... 74 5.8 Conclusions... 81

Contents ix 6 KNIGHT: SURVEILLANCE SYSTEM DEPLOYMENT... 85 6.1 Introduction... 85 6.2 Deploying Surveillance Systems: Ethical Considerations...... 85 6.3 Knight in Action... 86 6.4 Conclusion.... 89 7 CONCLUDING REMARKS... 91 7.1 What s Next?...... 91 7.1.1 Tracking Crowds...... 91 7.1.2 Understanding Complex Human Interaction & Activities... 92 7.2 The Properties of a Good Surveillance System and How Knight Measures Up...... 92 References.... 95 Index...103