Università degli Studi di Napoli Federico II

Transcription

1 Università degli Studi di Napoli Federico II FACOLTÀ DI INGEGNERIA DIPARTIMENTO DI INFORMATICA E SISTEMISTICA TESI DI LAUREA MAGISTRALE IN INGEGNERIA MECCANICA PER LA PROGETTAZIONE E LA PRODUZIONE Vision-based Control of an Industrial Robot Arm RELATORE: CHIAR.MO PROF. ING. BRUNO SICILIANO CORRELATORI: CHIAR.MO PROF. ING. OLE RAVN DOTT.SSA HAIYAN WU CANDIDATO: WALTER TIZZANO matr. M ANNO ACCADEMICO

2 Chapter 1 Introduction On either side there stood gold and silver mastiffs which Vulcan, with his consummate skill, had fashioned expressly to keep watch over the palace of king Alcinous; so they were immortal and could never grow old. Homer, Odyssey In this introductory chapter we will first see a brief historical excursus, that gives us the background and the motivation behind the project this report is about; it follows a description of the project itself and an overview of the structure of the report. 1.1 Background Mankind had started being fascinated by robotics 1 thousands of years ago: already in Greek mythology (precisely in Iliad and also in the successively written Odyssey, both attributed to Homer) the god of fire and volcanoes Hephaestus created golden mechanical servants [2] to protect the palace of king Alcinous. Over the following centuries, an impressive number of mechanical devices that could be considered the ancestors of modern robots have been built. Some noteworthy examples of these devices are the digesting duck 2 (figure 1.1(a)) or the tea-serving doll (figure 1.1(b)). In the last decades, technology gave to the human kind the possibility to dramatically improve these devices and science fictional writers were speculating more and more on how complex and futuristic these devices could become in the future, because 1 The term robot derives from the slavic word robota (forced labor), and it became popular after having been coined by the playwright Karel Čapek in his play R.U.R. (acronym for Rossum s Universal Robots), which premiered in 1921 [1]. 2 An automaton created by Jacques de Vaucanson in 1739 consisting in a mechanical duck capable of eating grain and digesting it [3]. 1

3 2 CHAPTER 1. INTRODUCTION (a) The automaton Digesting Duck (b) A mechanical tea-serving doll Figure 1.1: Famous automata some things previously considered myth or fiction had actually already came to pass. On the other hand, modern robots are considerably different from what it was expected by these writers in the past: most robots are used in industries for highly repetitive and burdensome tasks, while anthropomorphic and domestic robots are mostly object of research and not something actually commercially available (there are some remarkable exceptions, like the several hundred thousands vacuum cleaning robots -the most famous being IROBOTS Roomba, figure 1.2- working in this exact moment in residential houses). The success of industrial robots in factories is easy to understand: it is not conceptually hard to design and build a robot much stronger than the human body (in terms of maximum weight liftable, velocity and resistance to fatigue), and in these environments to adapt the surroundings to the robot is usually possible [4]; nevertheless, actions which require a basic degree of flexibility, awareness of the surrounding environment and dexterity (all of these took for granted by humans) are devilishly complex for a robot and this can explain why we do not have yet anthropomorphous robot assistants helping us around. This can be justified considering the remarkable complexity of the human brain and the human senses which grants us the capability to solve problems, think new strategies and adapt to the surroundings in an

4 1.2. RESEARCH PROBLEM DESCRIPTION 3 Figure 1.2: IROBOTS Roomba astonishingly efficient way. To create a robot which is able to be effective in an unknown environment (which cannot be contrived at reasonable costs to fit the robot) and to execute tasks requiring a high dexterity, a big computational power and highly performing sensors are both needed. This two requirements are fulfilled more and more overtime: we can now buy for few cents an amount of computational power that costed several thousands dollars back in the seventies and a similar pattern can be identified in most the components a robot needs to work properly [5]. This evolution in the components is granting us the possibility to make commercially convenient to design and produce on the one hand robots which still work in the industries but are more flexible and less task specific than their ancestors, and on the other hand robots that can safely interact with humans in an unstructured environment. The sense of vision plays an important role in our capability of interacting with our environment and to perform dexterous tasks and the object of this project is to design and build a vision system allowing a robot arm to achieve some results in this area, i.e., bouncing an unconstrained ball. It is worthy to highlight how such a vision system requires a computational power that was not widely available only a couple of decades ago. 1.2 Research problem description The object of this project is to design and build a stereoscopic vision system capable of tracking a fast-moving object and predicting its future positions,

5 4 CHAPTER 1. INTRODUCTION enabling a UR5 UNIVERSAL ROBOTS arm (figure 1.2) to interact with it. A colour-based object detection has been used and the position in 3D coordinates has been estimated with two different strategies, compared in this report: one is a geometrical triangulation, the other one is a neural network trained for this purpose; a curve fitting algorithm predicted the future position of the tracked object and a paddle moved by the robot arm is oriented and positioned according to a strategy meant to stabilize the bouncing. The robot motion has been performed with different strategies, one of them involving the need of the inverse kinematics of the arm, which was calculated using a neural network. 1.3 Structure of the report This report is structured as follows: in Chapter 2 some related work will be summarized; Chapter 3 consists in a brief reference to some noteworthy theory used to carry on the tests and to write the report itself; Chapter 4 describes the equipment used; in Chapter 5 there is a detailed description of the algorithm written to carry out the task (object tracking and prediction, motion of the robot); Chapter 6 provides the reader with information about the procedures used to calibrate the cameras and to train the neural networks used to calculate the inverse kinematics and solve the stereoscopic problem, and presents also a Figure 1.3: UR5 UNIVERSAL ROBOTS Arm

6 1.3. STRUCTURE OF THE REPORT 5 comparison between these strategies; Chapter 7 presents the results obtained and the issues encountered; Chapter 8 summarizes the conclusion we had came to after the work behind this project; some appendices, finally, are included; they consist in additional results and theory that the interested reader can make reference to if he wants more detail. A comprehensive bibliography is included also, for the reader s convenience. A CD is bundled with the present report; its content is described in appendix D.

7 Chapter 2 Related work If I have seen further it is by standing on ye shoulders of Giants Isaac Newton In the introductory chapter we have seen, among the other things, the background behind this project, that provides the motivation for it. But before starting a new project, it is also important to explore the related literature, to have an idea of the state of the art in the key parts of the project itself, that might inspire different approaches to the ones that could have been followed without such a research. Since this project is made up by several parts, an exploration of the related work for each part has been performed. In the following sections the results of this study are reported. 2.1 Robot Vision One of the most important senses in a lot of different biological species (especially in the most evolute ones) is the vision and this is true also in many robots, that use it for different reasons; the capability of a robot to cope with an unstructured environment can take significant advantage of a vision system: visual information can be used directly used to perform closed-loop position control of the robot [4] and usually for this purpose multiple cameras are used. A vision system can be defined as a remote sensor gathering information from a portion of the environment in a contactless fashion [6]. The first challenge is to extract meaningful information from the pictures; this process is called image segmentation and it consists in partitioning the image into multiple segments (e.g., isolating objects or boundaries). This can be done in many different ways and a method used in several applications (and in this project, as we will see in 5.1) is colour segmentation [7]. Noise and errors due to several factors (e.g., lens distortion) can lead to unacceptable inaccuracy of position and orientation estimation; when a 7

8 8 CHAPTER 2. RELATED WORK sequence of images is available, the accuracy can be enhanced using the extended Kalman filter. Siciliano et al. [8] proposed an algorithm based on Kalman filter for the position and orientation estimation in real time of moving objects using multiple cameras; one of the drawbacks of this method is the big computational power required, which could be a problem in real time applications but this is hardly an issue with the fast growing capabilities of microprocessors, which are significantly reducing the computational time required to run the algorithm. 2.2 Neural networks used in robotics Neural networks are the artificial counterpart of their remarkable analogous in biology; since animals have noteworthy abilities in a very wide range of different tasks, it has been natural for researchers to try to use these networks in robotics, for several different purposes. Neural networks can be a fundamental part of the vision system of a robot, in many different ways [9]; e.g., in computer stereo-vision, as we will see in 3.2, the knowledge of some parameters relative to the cameras (described in 6.1) is needed to perform the triangulation 1 ; several methods exist to compute these parameters, one of them being the usage of a neural network trained for this goal [10]; Ruichek and Postaire [11] proposed to solve the correspondence problem 2 using a neural network; a neural network can also be used to solve the stereoscopic problem (see Do [12] and Xing et al. [13]), as we will see in 6.2. To drive a robot in joint space, the knowledge of its inverse kinematics is needed and, when the robot is relatively complex (even a robot arm can be, if it has a sufficient number of degrees of freedom), this problem can be difficult, since it involves equations which are strongly non-linear and which often do not have an unique or analytical solution [14]; in this project to solve this problem was also necessary and it was decided to follow an approach similar to the one proposed by Tejomurtula and Kak [15], consisting in training a feedforward neural network for this purpose. Another crucial issue in robotics is the trajectory control, and this problem can become really complex for non-trivial robots, since it involves the knowledge of the inverse-dynamics model of the arm, and control strategies that can be tricky to develop; Miyamoto et al. [16] proposed for this purpose the usage of a feedback neural network, and the result was that, once the network learned how to control some simple and slow movements, it was then able to generalize this knowledge and perform more complex and fast motions. 1 The operation consisting in converting stereo-pair images from the two cameras in 3D world coordinates. 2 This is a problem present in stereoscopic vision, it consists in matching features extracted from two images that are projections of the same entity in the 3D world.

9 2.3. ROBOTS USED FOR DEXTEROUS TASKS Robots used for dexterous tasks The idea of performing dexterous tasks (e.g., play table tennis) with a robot has fascinated researchers all over the world, as a challenge both in machine vision and control fields [17]. These tasks often require a robust and high-speed vision system and fast and precise control of robot arm (e.g., Li proposed a high speed vision system consisting in two cameras working at high frame rate tracking a ball and predicting its future positions[18]; Acosta et al. proposed similar system, a ping-pong robot player, that uses instead a 25 Hz pair of cameras [19]). A typical example of such a dexterous task is the system consisting in a bouncing ball over a vibrating paddle; it is a very simple system with a very complex dynamical behavior and for this reason it has been studied by many authors (e.g., Holmes [20] and Tufillaro et al. [21]). Several approaches have been used over time to bounce an unconstrained ball: e.g., some authors [22] used a memory-based learning approach to juggle with the ball; others [23] used a blind juggler to bounce an unconstrained ball (they could avoid using cameras or other sensors by shaping the paddle conveniently, precisely they tried to minimize the H 2 norm 3 by giving to the vibrating surface a curvature they calculate in their report). A method similar to the one described in the previous lines will be used in this project, but with important differences, biggest of them being the usage of cameras. The same authors created later an improved version of their juggler (figure 2.1), able to juggle the ball back and forth along a horizontal distance of about 1 m and a vertical apex of 1.1 m [25]. 2.4 Summary In this chapter we saw some work related with this project, which inspired the author in exploring new possibilities in the research for the solution of problems encountered over its development. In the next chapter instead, some theoretical basics essential for this project will be pointed out, for a better understanding of what comes afterwards. 3 The standard H 2 norm of a system can be considered as the RMS response of the output signal when the input is a unit variance white noise [24].

10 10 CHAPTER 2. RELATED WORK Figure 2.1: The pendulum juggler [25]