Knowledge-Based Vision-Guided Robots

Knowledge-Based Vision-Guided Robots

Studies in Fuzziness and Soft Computing Editor-in-chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw, Poland E-mail: kacprzyk@ibspan.waw.pl http://www.springer.de/cgi-bin/search_book.pl?series = 2941 Further volumes of this series can be found at our homepage. Vol. 83. S. Barro and R. Marin (Eds.) Fuzzy Logic in Medicine. ISBN 3-7908-1429-6 Vol. 84. L. C. Jain and J. Kacprzyk (Eds.) New Learning Paradigms in Soft Computing, ISBN 3-7908-1436-9 Vol. 85. D. Rutkowska Neuro-Fuzzy Architectures and Hybrid Learning. ISBN 3-7908-1438-5 Vol. 86. M. B. Gorzalczany Computational Intelligence Systems and Applications. ISBN 3-7908-1439-3 Vol. 87. C. Bertoluzza, M. A. Gil and D. A. Ralescu (Eds.) Statistical Modeling, Analysis and Management of' Fuzzy Data, ISBN 3-7908-1440-7 Vol. 88. R. P. Srivastava and T. J. Mock (Eds.) Belief Functions in Business Decisions, ISBN 3-7908-1451-2 Vol. 89. B. Bouchon-Meunier, J. Gutierrez-Rios, L. Magdalena and R.R. Yager (Eds.) Technologies for Constructing Intelligent Systems I, ISBN 3-7908-1454-7 Vol. 90. B. Bouchon-Meunier, J, Gutierrez-Rios, L. Magdalena and R.R. Yager (Eds.) Technologies for Constructing Intelligent Systems 2, ISBN 3-7908-1455-5 Vol. 91. J.J. Buckley, E. Eslami and T. Feuring Fuzzy Mathematics in Economics and EnRineering, ISBN 3-7908-1456-3 Vol. 92. P. P. Angelov Evolving Rule-Based Models, ISBN 3-7908-1457-1 Vol. 93. V. V. Cross and T. A. Sudkamp Similarity and Compatibility in Fuzzy Set Theory, ISBN 3-7908-1458-X Vol. 94. M. MacCrimmon and P. Tillers (Eds.) The Dynamics of Judicial Proof', ISBN 3-7908-1459-8 Vol. 95. T. Y. Lin. Y. Y. Yao and L. A. Zadeh (Eds.) Data Mining, Rough Sets and Granular Computing, ISBN 3-7908-1461-X Vol. 96. M. Schmitt, H.-N. Teodorescu, A. Jain, A. Jain, S. Jain and L. C. Jain (Eds.) Computational Intelligence Processing in Medical Diagnosis, ISBN 3-7908-1463-6 Vol. 97. T. Calvo, G. Mayor and R. Mesiar (Eds.) Aggregation Operators, ISBN 3-7908-1468-7 Vol. 98. L. C. Jain, Z. Chen and N. Ichalkaranje (Eds.) Intelligent Agents and Their Applications, ISBN 3-7908-1469-5 Vol. 99. C. Huang and Y. Shi Towards Efficient Fuzzy Information Processing, ISBN 3-7908-1475-X Vol. 100. S.-H. Chen (Ed.) Evolutionary Computation in Economics and Finance, ISBN 3-7908-1476-8 Vol. 101. S. J. Ovaska and L. M. Sztandera (Eds.) Soft Computing in Industrial Electronics, ISBN 3-7908-1477-6 Vol. 102. B. Liu Theory and Praxis of Uncertain Programming, ISBN 3-7908-1490-3

Nick Barnes Zhi-Qiang Liu Knowledge-Based Vision-Guided Robots With 99 Figures and 3 Tables Physica-Verlag A Springer-Verlag Company

Dr. Nick Barnes University of Melbourne Department of Computer Science and Software Engineering 3010 Victoria Australia nmb@cs.mu.oz.au Professor Zhi-Qiang Liu City University of Hong-Kong School of Creative Media Tat Chee Ave., Kowloon Hong Kong P. R. China zliu@cs.mu.oz.au ISSN 1434-9922 ISBN 978-3-662-00312-1 Library of Congress Cataloging-in-Publication Data applied for Die Deutsche Bibliothek - CIP-Einheitsaufnahme Barnes, Nick: Knowledge based vision guided robots: with 3 tables / Nick Barnes; Zhi-Qiang Liu. - Heidelberg; New York: Physica-VerI., (Studies in fuzziness and soft computing; Vol. 103) ISBN 978-3-662-00312-1 ISBN 978-3-7908-1780-5 (ebook) DOI 10.1007/978-3-7908-1780-5 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Physica-Verlag. Violations are liable for prosecution under the German Copyright Law. Physica-Verlag Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH Physica-Verlag Heidelberg Softcover reprint of the hardcover 1 st edition The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Hardcover Design: Erich Kirchner, Heidelberg SPIN 10876877 88/2202-5 4 3 2 I 0 - Printed on acid-free paper

Acknow ledgements The authors would like to thank members of technical services of the department, particularly, John Horvath, Andrew Peel, Thomas Weichert, and David Hornsby, for keeping robots and cameras going, despite our best efforts to burn, crash, and destroy them in anyway possible. Nick Barnes would also like to thank Andrew Howard, Sandy Dance, and Les Kitchen for the challenging research discussions. Thank-you to my family, Betty, Barry, Lisa and Dale for supporting me in this strange enterprise called research. Also, my other family John and Wendy for support, encouragement and Sunday dinners. Finally to Nina, the greatest person whom I know. Her inspiration is a great source of strength to me. Her support of everything that I do makes challenges seem easy. Her company makes problems seem trivial. Her smile makes everything else seem unimportant. Her writing and editing skills have improved this book beyond recognition. Without her help this book would never be completed.

... if there were machines which bore a resemblance to our body and imitated our actions as far as it was morally possible to do so, we should always have two very certain tests by which to recognise that, for all that, they were not real men. R. Descarte. circa 1628. [95]1 I possessed the capacity of bestowing animation, yet to prepare a frame for the reception of it, with all its intricacies of fibres, muscles, and veins, still remained a work of inconceivable difficulty and labour. Frankenstein. Mary Shelley, 1818. [201]... to manufacture artificial workers is the same thing as to manufacture motors. The process must be of the simplest, and the product of the best from a practical point of view. R. U. R. (Rossum's Universal Robots). (English translation) Karel Capek, 1923. [39] lsee conclusion full text.

Contents 1 Introduction 1 1.1 Background.. 2 1.1.1 A vision-guided approach 2 1.1.2 Computer vision and vision-guided mobile robots 3 1.1.3 Applying high-level computer vision to guide mobile robots. 4 1.2 Aims of the Research Presented in this Book: A Problem in Robot Vision. 4 1.3 The Approach of this Book 5 1.4 About the Chapters 7 2 Related Systems and Ideas 9 2.1 Basic computer vision approaches. 9 2.1.1 Frame-based computer vision 10 2.1.2 Active vision 10 2.2 Vision-Guided Mobile Robot Systems 11 2.2.1 Mobile robot subsystems and concepts 12 2.2.2 Mobile robot object recognition. 18 2.2.3 Maps and path planning. 19 2.2.4 Temporal sequencing for complex tasks 20 2.2.5 Vision-guided mobile robot systems 20 2.2.6 Reactive navigation 22 2.2.7 Model-based vision systems for mobile robots 22 2.2.8 Knowledge-based mobile robotic systems. 23 2.2.9 Vision-guided mobile robots using stereo. 24 2.2.10 Active perception systems for mobile robots. 25 2.2.11 Application of vision-guided mobile robots. 28 2.3 Computer Vision for Mobile Robots 31

x Contents 2.3.1 Traditional model-based vision 3D object recognition.... 2.3.2 Shape-from-shading 2.3.3 Pose determination. 2.4 Conclusion.... 32 35 43 43 3 Embodied Vision For Mobile Robots! 45 3.1 Introduction... 46 3.1.1 Embodiment.......... 46 3.1.2 Phenomena and noumena... 47 3.2 The Classical Computer Vision Paradigm 47 3.2.1 Non-classical computer vision... 48 3.3 Problems with Classical Computer Vision 49 3.4 Applying Embodied Concepts in Human Vision. 51 3.4.1 Models play an analogous role in computer vision 52 3.5 Embodiment of Vision-guided Robots... 53 3.5.1 Embodiment, task and environment 54 3.5.2 The role of the task... 55 3.5.3 The role of the environment.. 55 3.6 Embodiment for Vision-guided Robots 55 3.6.1 Physical embodiment... 56 3.6.2 Embodiment in a task... 3.6.3 Embodiment in an environment. 3.7 Conclusion.... 4 Object Recognition Mobile Robot Guidance 63 4.1 Introduction... 63 4.2 System Perspective... 65 4.3 Object Recognition... 65 4.3.1 Canonical-views 65 4.3.2 Match verification 73 4.3.3 Edge matching.. 74 4.3.4 Edge-based features for ground-based robots 74 4.3.5 View prediction............. 77 4.4 Determining Object Pose and Distance.... 78 4.4.1 Active determination of the sign of (J 82 4.4.2 Error analysis. 4.5 Conclusion.... 57 58 59 82 85

Contents XI 5 Edge Segmentation and Matching 5.1 Edge Extraction.... 5.1.1 Edge extraction.... 5.1.2 On the choice of window size and quantisation of p and e....... 99 5.2 Edge Matching....... 101 5.2.1 Evaluating matches 101 5.2.2 5.2.3 5.2.4 5.2.5 5.2.6 5.2.7 Spatial elimination. Edge coverage.... Position estimation consistency Geometric verification.. Quadratic edge extraction Further active processing 103 103 105 105 107 107 6 Knowledge Based Shape from Shading 109 6.1 Introduction... 110 6.1.1 Motivation and system perspective 111 6.1.2 Assumptions............ 114 6.1.3 Knowledge-based representation of objects. 115 6.2 Using Object Model Knowledge for Shape-From-Shading 115 6.3 A New Boundary Condition for Shape-From-Shading. 116 6.4 Knowledge-based Implementation.. 120 6.4.1 Knowledge / frame topology 120 6.4.2 Fact knowledge...... 121 6.4.3 Procedural knowledge... 123 6.4.4 Shape processing rulebase. 124 6.5 Experimental Method and Results 6.5.1 Synthetic images. 6.5.2 Real images... 6.5.3 Domain knowledge 6.6 Conclusion.... 87 87 89 124 124 130 131 133 7 Supporting Navigation Components 135 7.1 Model-based Path Planning........... 135 7.1.1 Path planning and obstacle avoidance. 142 7.2 Odometry and Obstacle Avoidance Subsystem 142 7.2.1 Obstacle avoidance strategies 143 7.2.2 Coordinate transforms.... 145

XII Contents 8 Fuzzy Control for Active Perceptual Docking 147 8.1 Introduction... 147 8.1.1 Fuzzy control.............. 148 8.1.2 Fuzzy control for mobile robot control 150 8.1.3 TSK fuzzy model............ 151 8.1.4 Visual motion-based approaches to mobile robots and the docking problem.. 153 8.2 Direction Control for Robot Docking... 154 8.2.1 The log-polar camera... 154 8.2.2 Docking for a ground-based robot 155 8.2.3 Noise in the input parameter 157 8.3 A Fuzzy Control Scheme. 158 8.4 Results... 159 8.5 Conclusion... 161 9 System Results and Case Studies 165 9.1 Evaluation of Components. 165 9.1.1 Experimental setup.... 166 9.1.2 View matching...... 166 9.1.3 Pose determination - power supply 168 9.1.4 Pose determination - model vehicle. 180 9.2 Case Studies... 183 9.2.1 Moving around the corner of an object. 183 9.2.2 Distinguishing a particular object among similar objects........... 186 9.2.3 Docking... 186 9.2.4 Object circumnavigation. 188 9.2.5 Obstacle avoidance. 190 9.3 Conclusion 191 10 Conclusion 203 10.1 Limitations of the Research Presented and Future Work 204 10.2 Extended quotation from Descartes... 207 Index 231