Automated Multi-Camera Surveillance Algorithms and Practice

Transcription

1 Automated Multi-Camera Surveillance Algorithms and Practice

2 The International Series in Video Computing Series Editor: Mubarak Shah, Ph.D University of Central Florida Orlando, Florida

3 Automated Multi-Camera Surveillance Algorithms and Practice Omar Javed ObjectVideo Inc. Reston, Virginia, USA Mubarak Shah University of Central Florida Orlando, Florida, US 123

4 Omar Javed ObjectVideo Inc. Reston, VA 20191, USA Mubarak Shah University of Central Florida Orlando, FL 32816, USA ISSN: ISBN: e-isbn: DOI: / Library of Congress Control Number: Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper springer.com

5 To Captain Michael Holloway, Orlando Police Department (retired), who introduced us to video surveillance and provided enthusiastic support. Omar Javed & Mubarak Shah To my late father Mahmud Javed, who was always there for me; to my mother Dure-Shahwar who taught me the meaning of unconditional love, and to my wife Laila, who constantly brings joy to my life. Omar Javed

6 Contents 1 AUTOMATED VIDEO SURVEILLANCE Introduction Automated Systems for Video Surveillance Automated Surveillance System Tasks and Related Technical Challenges Object Detection and Categorization Tracking Tracking Across Cameras General Challenges Introduction to the Proposed Video Understanding Algorithms for Surveillance Book Organization IDENTIFYING REGIONS OF INTEREST IN IMAGE SEQUENCES Introduction General Problems in Background Subtraction Related Work Background Subtraction using Color as a Feature Background Subtraction using Multiple Features Finite State Space Based Background Subtraction Proposed Approach for Background Subtraction Assumptions Pixel Level Processing Region Level Processing Frame-Level Processing Results Discussion OBJECT DETECTION AND CATEGORIZATION Introduction Problems in Object Categorization vii

7 viii Contents 3.3 Related Work Periodicity Based Categorization Object Categorization using Supervised Classifiers Object Categorization using Weakly Supervised Classifiers Overview of the proposed categorization approach Feature Selection and Base Classifiers The Co-Training Framework Online Learning Co-Training Ability Measurement Results Concluding Remarks OBJECT TRACKING IN A SINGLE CAMERA Introduction Related Work Feature Point Tracking Methods Region Tracking Methods Methods to Track People Problems in Tracking 2D silhouettes of People Occlusion Entries and Exits Proposed Approach for Tracking Assumptions Object Tracker Results Discussion TRACKING IN MULTIPLE CAMERAS WITH DISJOINT VIEWS Problem Overview and Key Challenges Related Work Multi-Camera Tracking Methods Requiring Overlapping Views: Multi-Camera Tracking Methods for Non-Overlapping Views: Formulation of the Multi-Camera Tracking Problem Learning Inter-Camera Space-Time Probabilities Estimating Change in Appearances across Cameras The Space of Brightness Transfer Functions Estimation of Inter-Camera BTFs and their Subspace Computing Object Color Similarity Across Cameras Using the BTF Subspace Establishing Correspondences Results Conclusions... 81

8 Contents ix 6 KNIGHT: SURVEILLANCE SYSTEM DEPLOYMENT Introduction Deploying Surveillance Systems: Ethical Considerations Knight in Action Conclusion CONCLUDING REMARKS What s Next? Tracking Crowds Understanding Complex Human Interaction & Activities The Properties of a Good Surveillance System and How Knight Measures Up References Index...103