Studies in Computational Intelligence Volume 748 Series editor Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland e-mail: kacprzyk@ibspan.waw.pl
The series Studies in Computational Intelligence (SCI) publishes new developments and advances in the various areas of computational intelligence quickly and with a high quality. The intent is to cover the theory, applications, and design methods of computational intelligence, as embedded in the fields of engineering, computer science, physics and life sciences, as well as the methodologies behind them. The series contains monographs, lecture notes and edited volumes in computational intelligence spanning the areas of neural networks, connectionist systems, genetic algorithms, evolutionary computation, artificial intelligence, cellular automata, self-organizing systems, soft computing, fuzzy systems, and hybrid intelligent systems. Of particular value to both the contributors and the readership are the short publication timeframe and the worldwide distribution, which enable both wide and rapid dissemination of research output. More information about this series at http://www.springer.com/series/7092
Subhasis Chaudhuri Amit Bhardwaj Kinesthetic Perception A Machine Learning Approach 123
Subhasis Chaudhuri Department of Electrical Engineering Indian Institute of Technology Bombay Mumbai, Maharashtra India Amit Bhardwaj Department of Electrical Engineering Indian Institute of Technology Bombay Mumbai, Maharashtra India ISSN 1860-949X ISSN 1860-9503 (electronic) Studies in Computational Intelligence ISBN 978-981-10-6691-7 ISBN 978-981-10-6692-4 (ebook) https://doi.org/10.1007/978-981-10-6692-4 Library of Congress Control Number: 2017954488 Springer Nature Singapore Pte Ltd. 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore
To Syomantak and Ushasi My beloved family Subhasis Chaudhuri Amit Bhardwaj
Preface Haptics as an area of research has picked up a great momentum in the last two decades. The primary reason for such a proliferation of research is due to gradual and continuous development in mechatronics, making such devices available to the scientific community. A few such systems are now available for gaming and medical purposes. We expect a wider acceptance of such developments in practice as more and more such devices come into the market. However, all such devices currently work as stand-alone boxes and interoperability among these devices would be the key component in future developments. A proper standardization effort is required to provide this interoperability. One of the major applications foreseen for haptics is in teleoperation. Being able to perceive the forces at the teleoperator end will provide a great boost in improving the performance of the operator. Researchers have been working on various aspects of designing a good teleoperation system that involves design of manipulators, kinesthetic and tactile sensing, data communication, delay compensating controllers, and immersion into the virtual workspace. The data communication module requires meeting quite a restrictive quality of service guarantee. This requirement is very severe for delay-sensitive haptic data. The currently available Internet is often unable to meet the demand. With the introduction of tactile Internet, we expect this constraint to gradually ease out in future. Notwithstanding haptic data communication shall continue to be a major issue in any teleoperation system over a shared network. Thus an appropriate haptic compression engine is required to be a part of such a teleoperation system. How does one compress the haptic data without affecting the immersion of the operator into the virtual world? This brings in the question of the effect of data compression on haptic perception. Any perception-aware data compression technique utilizes the fact that a small change in the stimulus is often non-perceivable. Thus, when the temporal variation in the haptic signal is relatively very small, these data samples need not be transmitted. Several researchers have proposed different adaptive sampling strategies for haptic data and have demonstrated that a substantial reduction in data rate can be achieved. However, all these techniques require the determination of the perceptual threshold which is always dependent on vii
viii Preface the perceptual abilities of an individual. The primary motivation for this monograph is to design a methodology to estimate the subject-specific perceptual threshold. However, instead of considering both kinesthetic and tactile perceptions, the studies are limited to kinesthetic perception only. The psychophysics of human perception is a classical area of research and has a firm foundation on methodical study of determining the perceptual threshold. However, such studies are quite limited as regards to analyzing kinesthetic perception since it is only very recently that such mechatronic devices are available which can exert a given amount of force with a reasonably good accuracy. In parallel, there has been a substantial growth in research in the seemingly unrelated area of machine learning that offers a number of excellent data-driven tools to arrive at a decision or to estimate certain quantities without one having to define a functional or parametric relationship. Although a functional relationship may offer to estimate the unknown quantity very efficiently, it may suffer from the assumptions, including those on the distribution of the measurements, when inappropriate. However, the use of machine learning techniques requires generation of a large number of ground-truthed data. In this monograph, we demonstrate how the recently developed machine learning techniques can be used to determine the perceptual thresholds. Thus, the purpose of the monograph is to provide an engineering perspective on how some of the traditional problems in classical psychophysics can be solved. Quite naturally, we were required to generate a huge corpus of human response data for various types of kinesthetic stimuli. The book is addressed to a fairly broad audience. It is meant for graduate students studying the subjects of haptics, system science, and virtual reality. It may also serve as a reference book for scientists working in the area of human perception. For the benefit of such scientists, we plan to make all collected data available for further research. Needless to say, the monograph will be of great use to the practitioners developing various types of teleoperation systems. A basic familiarity of the readers with machine learning would help in better understanding of the book. We shall be very happy to receive comments and suggestions from the readers. Mumbai, India July 2017 Subhasis Chaudhuri Amit Bhardwaj
Acknowledgements The first author is indebted to Prof. Eckehard Steinbach at the Technical University of Munich (TUM), Germany for introducing him to the fascinating research area of haptics during several of his visits to TUM. Both the authors are thankful to Dr. Onkar Dabeer for many insightful discussions at the initial phases of research on the current topic and for his contributions in developing the contents of Chap. 3 of this monograph. Thanks are also due to Prof. Abhishek Gupta of Mechanical Engineering Department at IIT Bombay for his comments and suggestions. We are also grateful to Prof. Debraj Chakraborty and Prof. V. Rajbabu of Electrical Engineering Department at IIT Bombay for their constructive comments. We offer our most sincere gratitude to a large number of volunteers who happily complied with our requests in participating as subjects and spent hours and hours of their valuable time to help us generate a large amount of labeled data. Without their help, we could not have dared to take up this study. A few figures in the monograph have appeared in some of our publications elsewhere. We are thankful to ACM, Springer, and IEEE for allowing us to reuse the figures. We are thankful to various sources of funding: JC Bose Fellowship, National Programme on Perception Engineering, Indian Digital Heritage Project, Alexander von Humboldt Fellowship, and Bharti Centre for Communication. Finally, our acknowledgment is not complete unless we thank our family members for their constant support and encouragements. Mumbai, India July 2017 Subhasis Chaudhuri Amit Bhardwaj ix
Contents 1 Introduction... 1 1.1 Basics of Haptics... 1 1.1.1 Various Research Areas in Haptics... 2 1.1.2 Possible Applications... 5 1.2 Kinesthetic Perception... 7 1.3 Perception: Aware Engineering Design... 8 1.4 Organization of the Book... 10 References... 12 2 Perceptual Deadzone... 17 2.1 Haptic Data Compression... 17 2.2 Perceptual Deadzone for Multidimensional Signals... 21 2.3 Effect of Rate of Change of Kinesthetic Stimuli... 24 References... 26 3 Predictive Sampler Design for Haptic Signals... 29 3.1 Introduction... 29 3.2 Experimental Setup... 30 3.2.1 Device Setup... 30 3.2.2 Signal Characteristics... 31 3.2.3 Lag in User Response... 31 3.2.4 Collected Data... 32 3.3 Classification of Haptic Response... 33 3.3.1 Performance Metric... 33 3.3.2 Weber Classifier... 34 3.3.3 Level Crossing Classifier... 37 3.3.4 Classifiers Based on Decision Tree and Random Forests... 40 3.3.5 Effect of Temporal Spacing... 46 3.3.6 Significance Test for Classifiers... 47 xi
xii Contents 3.4 Applications in Adaptive Sampling... 48 References... 52 4 Deadzone Analysis of 2-D Kinesthetic Perception... 55 4.1 Introduction... 55 4.2 Experimental Setup... 57 4.2.1 Signal Characteristics and User Response... 57 4.2.2 Data Statistics... 57 4.3 Determination of Perceptual Deadzone... 59 4.3.1 The Weber Classifier... 60 4.3.2 Level Crossing Classifier... 61 4.3.3 Elliptical Deadzone... 62 4.3.4 Oriented Elliptical Deadzone... 64 References... 68 5 Effect of Rate of Change of Stimulus... 69 5.1 Introduction... 69 5.2 Design of Experiment... 71 5.2.1 Kinesthetic Force Stimulus... 71 5.2.2 Data Collection... 72 5.3 System Correction... 73 5.4 Estimation of Decision Boundary... 76 5.4.1 Parametric Decision Boundary... 76 5.4.2 Nonparametric Decision Boundary... 80 5.5 Analysis of Results... 83 References... 87 6 Temporal Resolvability of Stimulus... 89 6.1 Introduction... 89 6.1.1 Motivation for the Study... 89 6.1.2 Related Work... 91 6.1.3 Our Approach... 92 6.2 Experimental Setup... 92 6.2.1 Signal Characteristics... 92 6.2.2 Data Collection... 94 6.3 Estimation of Temporal Resolution... 94 6.4 Effect of Fatigue... 96 6.5 Application in Data Communication... 98 References... 99 7 Task Dependence of Perceptual Deadzone... 101 7.1 Introduction... 101 7.1.1 Objective of the Study... 102 7.1.2 Prior Work... 103 7.1.3 Our Approach... 103
Contents xiii 7.2 Design of Experiment... 103 7.2.1 Kinesthetic Force Stimulus... 104 7.2.2 Data Statistics... 107 7.3 Estimation of Perceptual Deadzones... 107 References... 115 8 Sequential Effect on Kinesthetic Perception... 117 8.1 Introduction... 117 8.2 Sequential Effect.... 118 8.3 Quantification of Sequential Effect... 119 8.3.1 Logistic Regression... 119 8.3.2 Description of the Regression Model... 121 8.4 Analysis of Effect on Comparative Task... 123 8.5 Analysis of Effect on Discriminative Task... 126 References... 129 9 Conclusions... 131 Index... 135
About the Authors Prof. Subhasis Chaudhuri received his B.Tech. Degree in Electronics and Electrical Communication Engineering from the Indian Institute of Technology Kharagpur, Kharagpur in 1985. He received his M.Sc. and Ph.D. degrees, both in Electrical Engineering, from the University of Calgary, Canada, and the University of California, San Diego, respectively. He joined the Department of Electrical Engineering at the Indian Institute of Technology Bombay, Bombay in 1990 as Assistant Professor and is currently serving as the KN Bajaj Chair Professor. He has also served as the Head of the Department, the Dean (International Relations), and a Deputy Director. He has also served as a Visiting Professor at the University of Erlangen-Nuremberg, the Technical University of Munich and the University of Paris XI. He is a Fellow of the science and engineering Academies in India. He is a recipient of the Dr. Vikram Sarabhai Research Award (2001), the Swarnajayanti Fellowship (2003), the S.S. Bhatnagar Prize in engineering sciences (2004), and the J.C. Bose National Fellowship (2008). He is a coauthor of the books Depth from Defocus: A Real Aperture Imaging Approach, Motion-Free Super-Resolution, and Blind Image Deconvolution: Methods and Convergence, all published by Springer, New York (NY). He is currently an associate editor of the journal International Journal of Computer Vision. His primary areas of research include image processing and computational haptics. Amit Bhardwaj received his B.Tech. and M.E. degrees in Electronics and Communication Engineering from the YMCA Institute of Engineering, Faridabad, Haryana, and the Delhi College of Engineering, Delhi, in 2009 and 2011, respectively. He has recently completed his Ph.D. in Electrical Engineering at the Indian Institute of Technology Bombay, Bombay, and is currently an Alexander von Humboldt Fellow at the Technical University of Munich. His current research areas include signal processing, haptics, kinesthetic perception, haptic data communication, and applications of machine learning. xv