MULTIMEDIA VIDEO-BASED SURVEILLANCE SYSTEMS Requirements, Issues and Solutions
THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE
MULTIMEDIA VIDEO-BASED SURVEILLANCE SYSTEMS Requirements, Issues and Solutions edited by Gian Luca Foresti University of Udine Petri Mähönen VTT, Technical Research Center of Finland and the University of Oulu Carlo S. Regazzoni University of Genova Springer Science+Business Media, LLC
Library of Congress Cataloging-in-Publication Multimedia video-based surveillance systems : requirements, issues and solutions / edited by Gian Luca Foresti, Petri Mähönen, Carlo S. Regazzoni. p. cm. (Kluwer international series in engineering and computer science ; SECS 573) Includes bibliographical references and index. ISBN 978-1-4613-6943-1 ISBN 978-1-4615-4327-5 (ebook) DOI 10.1007/978-1-4615-4327-5 1. Multimedia systems. 2. Electronic security systems. 3. Closed-circuit television. I. Foresti, Gian Luca, 1965- II. Mähönen, Petri, 1963- III. Regazzoni, Carlo S. IV. Series. QA76.575.M85245 2000 621.389*28-dc21 00-058397 Copyright 2000 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers, New York in 2000 Softcover reprint of the hardcover 1st edition All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise, without the prior written permission of the publisher, Springer Science+Business Media, LLC Printed on acid-free paper.
CONTENTS PREFACE... ix ACKNOWLEDGEMENTS... xiii 1. EMERGING REQUIREMENTS... 1 1.1 FORENSIC VIDEO INVESTIGATION: THE SURVEILLANCE VIDEOTAPE AS EVIDENCE IN COURT... 3 Z. Geradts and J. Bijhold 1.2 RAIL WAY STATION SURVEILLANCE: THE ITALIAN CASE... 13 N. Ronetti and C. Dambra ].3 REQUIREMENTS FOR VISUAL PERCEPTION OF AUTOMOTIVE ENVIRONMENTS... 21 M. Bertozzi, A. Broggi and A. Fascioli 2. DIGITAL SENSORS AND ADAPTIVE LOW LEVEL PROCESSING... 33 2.1 CMOS IMAGE SENSING FOR SURVEILLANCE APPLICATIONS AND OBJECT TRACKING... 35 J E. Santos, A. S. Teuner, M Hillebrand, N. Stevanovic, S.-B. Park and B. J. Hosticka 2.2 DSP-ORIENTED LOW LEVEL PROCESSING FOR ADAPTIVE TUNING OF VIDEO SURVEILLANCE IMAGES... 46 T. Schardt and R. Hiinermann
Vi Contents 2.3 IMAGE ANALYSIS FOR ADVANCED VIDEO SURVEILLANCE... 57 A. Cavallaro and F. Ziliani 3. DETECTION, LEARNING AND RECOGNITION... 69 3.1 DYNAMIC SHAPE DETECTION FOR MULTIPLE CAMERA SYSTEMS... 71 L. Marcenaro and C.S. Regazzoni 3.2 LEARNING AND CLASSIFICATION OF SUSPICIOUS EVENTS FOR ADVANCED VISUAL-BASED SURVEILLANCE... 84 G.L. F oresti and F. Roli 3.3 A NEW SOLUTION PHILOSOPHY FOR COMPLEX PATTERN RECOGNITION PROBLEMS: APPLICATION TO ADVANCED VIDEO-SURVEILLANCE... 94 R. Mattone, A. Glaeser, and B. Bumann 4. DISTRIBUTED INTELLIGENT SYSTEMS... 105 4.1 AN AGENT SOCIETY FOR SCENE INTERPRETATION... 108 P.Remagnino, i.orwell, D.Greenhill, G.A.Jones and L.Marchesotti 4.2 DYT A: AN INTELLIGENT SYSTEM FOR DYNAMIC TARGET ANALySIS... 118 Y.L. Murphey, H. Lu, R. Karlsen, G. Gerhart and T. Meitzler
Contents vii 4.3 SINGLE CAMERA MULTIPLEXING FOR MUL TI-TARGET TRACKlNG... 130 C. Benabdelkader, P. Burlina, and L. Davis 4.4 NETWORK MANAGEMENT WITHIN AN ARCHITECTURE FOR DISTRffiUTED HIERARCHIAL DIGITAL SURVEILLANCE SYSTEMS... 143 F. Soldatini, P. MiihOnen, M. Saaranen, and C.S. Regazzoni 5. COMMUNICATIONS AND MULTIMEDIA TRANSMISSION.159 5.1 SCALABLE H.324 VIDEO-BASED SURVEILLANCE SYSTEM... 162 M. Raggio, I. Barbieri, A. Cabitto and L. Corte-Real 5.2 BROADBAND MULTIMEDIA TRANSMISSION FOR SURVEILLANCE APPLICATIONS... 173 P. Miihonen and M. Saaranen 5.3 DIGITAL WATERMARKlNGFOR THE AUTHENTICATION OF A VS VIDEO SEQUENCES... 186 M. Bami, F. Bartolini, V. Cappellini and A. Piva 5.4 ACTUAL HIGH-SPEED MODEM SOLUTIONS FOR MULTIMEDIA TRANSMISSION IN REMOTE CABLE-BASED VIDEO-SURVEILLANCE SYSTEMS... 197 C. Sacchi, G. Gera And C.S. Regazzoni
viii Contents 6. PERFORMANCE EVALUATION AND APPLICATIONS... 207 6.1 MINIMAX BASED REGULATION OF CHANGE DETECTION THRESHOLD IN VIDEO-SURVEilLANCE SYSTEMS... 210 F. Oberti, F. Granelli and C.S. Regazzoni 6.2 PERFORMANCE ANALYSIS OF MUL TI- SENSOR BASED REAL-TIME PEOPLE DETECTION AND TRACKING SySTEM... 224 M. Greiffenhagen and V.Ramesh 6.3 USE OF DIFFERENT TIME SCALE REFERENCES FOR MEASURING SEVERAL CROWD SITUATIONS... 238 D. Aubert and F. Guichard 6.4 VEHICLE DETECTION FROM MULTIPLE RADAR IMAGES IN AN ADVANCED SYSTEM FOR DRIVING ASSISTANCE... 248 B. Pani, P. Scala, R. Bolla and F. Davoli 6.5 LICENSE-PLATE RECOGNITION FOR RESTRICTED-ACCESS AREA CONTROL... 260 G. Adorni, F. Bergenti, S. Cagnoni and M. Mordonini 6.6 DYNAMIC VISION FOR LICENSE PLATE RECOGNITION... 272 G. Garibotto, P. Castello and E.Del Ninno INDEX... 285
Preface Multimedia Surveillance Systems are an emerging application field requiring multidisciplinary expertise spanning from Signal and Image Processing, to Communications and Computer Vision. The development of complex surveillance systems is capturing interest of both research and industrial world as there are strong requirements coming from the society in the direction of increasing safety and security in different applications. Transports are the application field from which most pressure comes for innovative research solutions: improving monitoring capabilities on large transport environments is the topic of more interest. Surveillance services of improved capabilities are also more and more requested as a consequence of the increased demand of secure feeling in all-days life coming from people. Finding a trade-off between people security necessity and privacy issues is becoming a worldwide spread political issue. In order to satisfy a potentially so large market, strong research innovations are required that allow system designers to take advantage of multiple architectural solutions and processing tools developed by researchers. The assessment of widely accepted performance evaluation metrics is also an important step to allow surveillance engineers and endusers to select in a straightforward way architectural choices for different applications. This book aims at addressing main current issues in Multimedia Surveillance by providing a selected list of contributions proposed by research laboratories intensively working in the surveillance field in different parts of the world.
x Preface Multimedia Surveillance Systems, MSSs, can be defined as distributed multimedia computer systems based on multisensory input data and provided by scene understanding capabilities oriented at either attention focusing of human operators or to automatic alarm generation. With respect to other multimedia systems devoted to different applications, such as video conferencing systems, teleworking applications or broadcasting, MSSs present some differences and peculiarities that make it necessary separate ad-hoc investigation. From a processing standpoint scene understanding requirements are more severe in MSSs than in current multimedia systems due to high variability and poor structure of monitored scenes in different surveillance applications. Such variability has several consequences in required processing tools. From one point of view, it makes it necessary the use of more sophisticated image processing algorithms for rough signal preprocessing and filtering. On the other hand, highly variable scene conditions imply the necessity of selecting robust scene description and pattern recognition methods. Automatic learning capabilities are an emerging issue in surveillance systems, as the capability of automatically developing models of scenes to be recognized as potentially dangerous events from a training set of presented examples will be a key issue for improving end-user acceptance of MSSs. Real-time and low-cost are two of the most important constraining factors for individuating processing choices acceptable for MSSs. Multisensory nature of surveillance information constitutes another important direction of improvement that is related to improved processing capabilities. Multisensor systems can take advantage from processing either the same type of information acquired from different spatial locations or information acquired by sensors of different type (e.g. tv cameras, microphones, etc.) on the same monitored area. Appropriate processing techniques and new sensors providing in real time information related to different scene characteristics can help both to enlarge the size of monitored environments and to improve performances of alarm detection in areas monitored by more sensors. Communications aspects of MSSs are the second critical key issue to be addressed by research in video-based surveillance. MSSs imply the distributed acquisition of information that must be usually transmitted to some remote control-center. Communications requirements of MSSs are different with respect to those of other multimedia systems as some characteristics of the surveillance problem are very peculiar. For example, the directionality of information flow and related asymmetric bandwidth requirements, lower for the downlink (from control center to observation
Preface xi station) than for the uplink (from sensor to control-center) are typical of MSSs. This situation is opposite to many traditional services such as broadcasting and commonly assessed interactive services (e.g. residential access to Internet services and video-on-demand). Another critical aspect is security of transmission. In many applications surveillance data must be transmitted over open networks with multi-user access characteristics. Information protection on such networks is a critical issue for granting privacy conditions to surveillance services. On the other hand identities of surveillance data can be very important for effective lise for law enforcement purposes. Watermarking and data-hiding techniques for secure sensor identity assessment related to legal aspects becomes therefore an "hot" signal processing research topic that can be critical for a widespread diffusion of surveillance techniques. Strategies for the design at the architectural level of MSSs and the definition of performance evaluation criteria for the evaluation and comparison of MSSs behavior are fundamental aspects to be addressed by current research. The major choice that one has to perform when selecting an architecture for a MSS is between a centralized and a distributed system. Centralized MSSs are usually straightforward improvements of first generation surveillance systems where attention focusing and automatic pattern recognition capabilities have been improved at the control center level. The impressive impulse to availability of bandwidth of these last years makes it possible the development of effective MSS centralized systems, characterized by a large number of sensors in direct communication with remote control centers. Nevertheless, the parallel improvement of low-cost processing capabilities and the simultaneous explosion of bandwidth demand and cost suggest the research of solutions capable to increase the distribution level of intelligence within a MSS. The definition of trade-off points on the basis of available bandwidth and processing resources is an open point of current research, together with the assessment of related optimality criteria taking into account also quality of service concepts. This latter aspect implies the necessity of defining performance evaluation criteria taking into account end-user requirements. Standardized measurements and metrics for quantitatively indicating system performances with respect to end-user goals are the middle term goal of this research line, aiming at widespread diffusion and measurable competition between surveillance products. Another emerging aspect coming from a distributed architectural choice is represented by the necessity of developing distributed reasoning and control tools effective for the coordination of multisensorial data aiming at a consistent scene interpretation at different scales. Distributed uncertainty
XII Preface representation and management represent a further problem to be considered. Solutions to some of the above problems are presented in this book, together with an indication of research lines that require further improvement steps. Examples of specific applications provide an insight in possible solutions in different application fields. The book is structured into six chapters, each of them organized and introduced by one of the co-editors. Chapter one considers emerging requirements coming from the society and transport fields. Legal aspects of video based surveillance and problems in railway and automotive fields have been inspected to summarize some existing solutions and needs as expressed by operators in the field. Chapter two is focused on digital signal processing aspects by considering both new digital sensors for real time acquisition of intermediate level surveillance data and low-level image processing algorithms for improving robustness of early stages of MSSs. Chapter three deals with event detection methods and with related learning approaches: this chapter presents important steps towards automatic generalization of event modeling and their application in MSSs. Chapter four deals with problems related with knowledge representation in MSSs and with its distribution among different nodes of a network: reasoning and control aspects are considered together with the use of agents and of intelligent network management strategies on distributed networks. Communications aspects of MSSs are considered in Chapter five, where wireless and wired multimedia networks and new generation communications tools are analyzed as basic structure for MSSs. Digital watermarking and data-hiding techniques are also introduced as basic tools for improving legal use of surveillance systems. Finally chapter six deals with problems associated with the definition of performance evaluation criteria for MSSs and with the presentation of a selected number of applications in the transport field. Gian Luca Foresti Petri Mahonen Carlo S. Regazzoni
Acknowledgements The editors wish to thank first Prof. Vito Roberto for his help in the preparation of the ICIAP 99 Special Session on Video Based Surveillance that has been an important step towards the preparation of this book. A special thank goes to the authors of the contributions for high quality of their respective work and for their active and timely cooperation. This book has been partially funded under a grant by the Italian Ministry of University and Scientific and Technological Research (MURST). A special thank goes to Franco Oberti for his invaluable work of reformatting and finalizing the book. Moreover, we wish to thank a number of graduate students, research fellows and cooperators working at the Dept. Of Biophysical and Electronic Eng. of the Univ. of Genova, Claudio Sacchi, Lucio Marcenaro, Gianluca Gera, Ilaria Spotomo for their precious help in preparing this book. We express also appreciation to Dr. Alex Greene and Mrs. Patricia Lincoln from Kluwer Academic Publishers for their precious editorial support.