An Integrated HMM-Based Intelligent Robotic Assembly System H.Y.K. Lau, K.L. Mak and M.C.C. Ngan Department of Industrial & Manufacturing Systems Engineering The University of Hong Kong, Pokfulam Road, Hong Kong Abstract Soft computing has aroused vast interests among the robotics community in which intelligent control for robot is targeted. Examples of soft computing including neural networks, genetic algorithms and fuzzy logic are widely applied in the field of robotic assembly, which is widely recognized as a complicated task in robotics. In order to achieve an intelligent robot, three issues should be considered in conjunction, namely, hardware design and robot model (fundamental), control algorithm (intermediate) and system integration (advanced). Hardware design involves motor, actuator and sensor designs as well as the design of an efficient computational architecture. Algorithmic implementation requires feedback control algorithms such as position, force and hybrid control to assist a robot to accomplish a task as accurate as possible. Finally, system integration, consisting of architecture and modeling, aims at integrating hardware, robot models and control algorithm designs efficiently and effectively to accomplish specific tasks intelligently and successfully. The paper starts with an overview of the intrinsic interrelationships between hardware design and robot modeling, control algorithm, and system integration for building an intelligent robotic system. A brief outline of the hardware design and robot model is given whereas control algorithms and system integration will be focused and discussed in some depth in this paper. It is noted that hardware design and robot model are mostly viewed as the first step of intelligent system. The principal theme of this paper is to introduce the use of hidden Markov models (HMMs) to act as state recognizers, which is the key component of the intelligent robotic assembly system. The state information is eventually sent to the control strategy generator (CSG), another major component of the proposed system, in which soft computing techniques are proposed as decision-makers to determine appropriate motion strategies based on these state information. The paper concludes with the proposal of a hierarchical intelligent framework that integrates the HMM-based state recognizer and control strategy generator. This framework is structured such that it forms a generic robotic system that has the remarkable feature of easily extensible to any other types of robot. Key-Words: Intelligent Robotic System, Force Control, Hidden Markov Models, Soft Computing 1 Introduction Industrial robots have been widely applied in automation, medical surgery or disable services. These sectors require highly intelligent robotic systems to deal with dynamically changing environments. It is difficult to define the term of intelligence, however, it is easy to understand that intelligence should consist of perception, knowledge representation, reasoning and understanding [1]. This paper focuses on the application of industrial robots in automated assembly, which is recognized as a complicated task that is not fully solved in the past two decades. Recent developments in robotic assembly are extensively investigated. However, progress is limited in automated assembly due to the complexity working cells. In addition, robotic manipulators have to interact with manipulated objects or environments, that increases the complexity of robot control. In general, intelligent robot control consists of three issues, namely, hardware design (including motor, actuator and sensor designs) and robot model (including kinematics and dynamics modeling), control algorithm implementation, and system integration [2, 3, 7]. In the past, robotic systems are task-specific. The task-specific characteristic contradicts with the flexibility of robots. Thus, it is time-consuming to redesign a robotic system to achieve specifications. Therefore, one of goals of this paper is to develop an intelligent system for robotic assembly, which can be extended in generalized autonomous systems that require few modifications. The unifying framework hereby discussed is not intended to give a standard architecture for robotic assembly, instead, it is to outline a systematic scheme of functional blocks required for intelligent control systems.
In the remainder of this paper, several issues are concentrated. Section 2 surveys the researches of hardware designs including motors, actuators and sensors. Further, the importance of robot model and three classes of control, namely, model-based, AIbased and probability-based approaches, are then discussed. In addition, the issue of system integration is also addressed. In Section 3, a set of criteria to achieve intelligent control is outlined. Further, a proposed intelligent framework for automated assembly that is based on a number of key criteria. A general formulation of hidden Markov models (HMMs) is introduced in Section 4. Section 5 focuses on the design of a HMM-based state recognizer and control strategy generator (CSG). These two components are major features in the proposed intelligent robotic system. Section 6 outlines the on-going development of the intelligent automated assembly system. Section 7 gives a conclusion. 2 Literature Review Three recognized aspects, namely, hardware design and robot model, control algorithm implementation, and system integration, are tightly coupled in the context of intelligent control. Hardware design involves actuator, motor and sensor designs, as well as the design of an efficient computational architecture for robotic controller. Robot model involves mapping of robotic behaviours to actuators and motors. Control algorithm implementation requires feedback control such as position, force and hybrid control. Finally, system integration, consisting of system architecture and system modeling, aims at integrating the hardware, robot models and control algorithms efficiently and effectively to accomplish the specific tasks successfully. 2.1 Robotic Hardware and Robot Model The considerations of robotic hardware and robot model are layered in the fundamental level in the generic intelligent framework. On one hand, hardware design and development is the first step in the construction of robot. Robotic hardware such as actuators, motors and joints determines the accuracy of tasks accomplished by a robot. Precise hardware components enhance the accuracy of control. On the other hand, in order to execute desired behaviours, robot models such as kinematics and dynamics modeling, are required to control the robot. In order to enhance the efficiency computing robot model and data processing such as sensed information acquired by industrial sensors, different computer architectures for controlling a robot are extensively researched [4]. These issues are considered in the fundamental development for an intelligent robot. 2.2 Control Algorithms A robot task is basically divided into two categories, namely, non-contact and contact. The non-contact scenario is simply position-controlled. The contact scenario complicates the control issue. Accurate robot model and environment model (which describes spatial relationship between a manipulated object and the environment) are not easily acquired. Therefore, it is possible to employ suitable sensing technique to relax such constraints. As such, a welldeveloped feedback control system is required. With regard to the entire robotic system, control algorithms integrated with sensory information form a closed-loop system which is used to monitor the accuracy of task accomplishments of robotic tasks. In general, three approaches, namely, mathematical theory (model-based), probability theory and artificial intelligence, are deployed in robotic control. Model-based control algorithms are more related to mathematical modeling of relationships between actual and sensed information. Hybrid position/force control and compliance control are instances of model-based control algorithms. For increasing intelligence of robots, another advanced class, i.e., artificial intelligence (AI), is deployed in the last two decades. In retrospect, AI considers the representation of knowledge and reasoning. Recently, algorithmic developments such as neural networks (NNs), genetic algorithms (GAs) and fuzzy logic are categorised in the topic of AI [13]. In contrast to AI, probability-based control involves the uses of probability and statistical theories to model dynamic environment, rather than using symbolic expressions etc. At present, this class of control attracted much interest [5, 6]. The intelligent framework proposed in this paper deployed a wellknown stochastic model from speech recognition, hidden Markov models (HMMs), which are used to continuously recognize spatial state information during robotic assembly. 2.3 System Integration System architecture and system modeling constitute the study of system integration. In retrospect, different robotics communities have pursued intelligent robots independently. As a result, control systems are application-specific and task-oriented. No unified architecture and modeling methodology are yet present. Common architectures include
hierarchical design [6, 7], where each control elements is arranged in a hierarchical manner; modular design [8] where elementary components are designed with specialised blocks; and behavioural design [9] where each control element represents a corresponding behaviour. Another issue concerning about system integration is the modeling technique. The approach of robotic modeling concerned here is about the system modeling of robotic controller and framework in accordance with software aspect. At present, objectoriented model and discrete-event system models are widely used. Hardware Designs Motor, Actuator, Sensor Hardware and Robot Model Robot Model Kinematics, Dynamics, Trajectory Position, Force, Hybrid Intelligent Robot Control Algorithms Filtering and Sensing Architecture Design Figure 1: Scheme for intelligent robot. System Integration Modeling A scheme for intelligent robot is outlined in Figure 1. 3 A Generic Intelligent Framework As mentioned, robotic assembly is recognized as a complex task that is not ideally solved in robotics. Incorporation of expert system, knowledge-based system, fuzzy logic control, neural network and genetic algorithms have been explored to assist robotic controllers to accomplish assigned operations. In this section, the principal factors for intelligent robot are discussed. The section begins with a discussion on criteria for intelligent control. Then, a proposed intelligent framework is outlined. The framework presented is not to give a constrained architecture for intelligent robot. Instead, the viewpoint taken here is intended to give a framework consisting of intelligent components. 3.1 Criteria for Intelligent Control Numerous researches have focused on intelligent control. It is not completely defined in the literature that intelligence is hard to define [1]. However, it is easy to perceive that intelligence should include perception, knowledge representation and learning. In short, the intelligent framework/system should posses the following characteristics: 1. It should have correct robot models. Fast computation of (inverse) kinematics and (inverse) dynamics are prerequisites for intelligent control. In addition to modeling, precise hardware, robotic sensors are generally required as the foundation of intelligent robot. 2. It should be adaptable to deal with unstructured or uncertain environments. 3. It should respond to external stimuli in real-time mode. 4. The error recovery capability should be provided to minimise deviation of actual paths and reduce participation of robotic programming. 5. The system should provide an emulator that can simulate virtual environment and results of robotic programming. A graphical user interface (GUI) is particular important in space exploration and tele-robotics. More details of these discussions can be found in [1, 4, 17]. 3.2 A Proposed Framework In the past, designs of robotic control systems are heuristic and application-specific. These designs have limitations and will only work under a number of assumptions. This section aims at providing a unified framework to include elementary components of intelligent control so as to achieve the criteria (except Point 5) enlisted in Section 3.1. The result will be a framework that is flexible for system reconfiguration and representing better unified system architecture. A remarkable feature of this design is its hierarchical information flow, where the modules within the framework are structure hierarchically where a low-level module has to report information to high-level module [6]. FREE SPACE MOTION CONTROL STRATEGY CONTROLLER TRAJECTORY PLANNER ROBOT CONTROLLER CONSTRAINED MOTION RECOGNIZED STATE F/T SIGNALS STATE RECOGNIZER SPECTRAL PREPROCESSING ACTUATORS AND SENSORS Advanced Level Intermediate Level Fundamental Level Intelligent Component Figure 2: The generalized hierarchical framework for robotic system. A generalized framework for an intelligent robotic system is depicted in Figure 2 in which the system is
layered in a hierarchical framework. Three levels are layered. The lower level is relatively close to hardware design such as motors and sensors, and robot modeling such kinematics and dynamics. Control algorithms are covered in the intermediate level in which position control, force control and hybrid control are used to perform compliant motion under contact scenario. In addition, as sensing techniques are deployed in control, filtering technique is generally considered, which is aimed at reducing the noise level of sensed signals. The highest level, or the planner level, consists of a state recognizer, which is used for analyse the sensed signals to obtain corresponding robotic states, and a control strategy generator (CSG), which is used for choosing appropriate control strategy such as position control and force control that is based on the state information provided by the state recognizer. These two features are the principal components that can enhance the intelligence of a robot. 4 Formulation of HMMs HMM is evolved from a Markov chain. The Markov chain can be interpreted as a chain of states. A state is analogous to an outcome of an experiment and each state in a Markov chain is linked to other states. The links between these states are weighted by probabilistic values. In contrast to a Markov chain, an HMM is a doubly stochastic process in which one observable process is governed by a particular probability density function (PDF), and the other hidden stochastic process is a Markov chain [15]. The philosophy of HMM is the investigation of observation symbol (o i ) distribution, rather than directly observing hidden Markov chain (q i ). According to Figure 3, it is seen that each observation symbol is generated at a corresponding state. Classification of HMMs depends on the types of PDF. If the distribution of observations is discrete, the HMMs are termed discrete hidden Markov models. The HMMs with continuous PDF is called continuous hidden Markov models. Before applying HMMs to real world applications, three fundamental problems namely, evaluation, state sequence optimization and parameter optimization, have to be solved. Evaluation is concerned about the problem of evaluating the probability of the observation sequences generated by a given model, i.e. the computation of Pr(O λ), where λ is the given model. It is significant to pattern recognition applications as one of the major problems is to find the corresponding HMM given an observation symbol sequence. The forward-backward algorithm is commonly used to solve this probability computation [15]. State Sequence Optimization is to find the hidden state sequence with given observation symbols. The most common and popular algorithm to solve this problem is the Viterbi algorithm (VA), proposed by Viterbi [15]. The VA is a recursive optimal algorithm for estimating the state sequence of a discrete-time finite-state Markov process observed in memoryless noise. However, it is not a necessary step to develop an HMM-based system. In addition, VA can also be used to compute Pr(O λ) [16]. Parameter Optimization is an optimization process for optimizing the model parameters so as to best describe how a given observation sequence comes about. There are several re-estimation algorithms to address the problem and one of the commonly used algorithms is the Baum-Welch re-estimation algorithm [15]. ot 2 ot 1 ot ot + 1 + ot 2 q t 2 q t 1 q t q t+ 1 q t+ 2 Figure 3: A graphical interpretation of basic hidden Markov model. 5 HMM-based State Recognizer and Control Strategy Generator 5.1 HMM-Based State Recognizer State recognizer is a key component in the proposed hierarchical framework as it can provide accurate state information to the robotic controller [11, 12]. Based on the experience of applying HMMs to speech recognition, it is found that HMMs are appropriate for modeling stochastic processes and their temporal processing capability are ideal for modeling dynamic processes. As such, an HMM is considered an appropriate mathematical model for modeling assembly state. When comparing HMMs with other learning algorithms such as neural networks (NNs) which have been widely used in robotic assembly, HMM has two advantages over NNs. One of these is the temporal process capability of HMM and the other is the provision of a compact and tractable mechanism for handling temporal information [16]. Furthermore, the training process of HMMs is easier in terms of the
availability of parameter optimization algorithms such as the Baum-Welch re-estimation algorithm. As a result, the structure of HMMs is compact and converges rapidly in the learning process. In contrast, most of the NNs algorithms are static in structure, and becomes infeasible when the system concerned is dynamic and non-stationary. Another characteristic of HMMs is the ability to represent state information. Each state information or contact event in an assembly operation can be incorporated into an HMM, while it is impossible to achieve the same capability if NNs are used. For example, the number of states and the topology of HMMs can often be selected based on the knowledge of the process. From the perspective of modeling flexibility, an assembly operation consists of a sequence of subtasks or events, each HMM represents a subtask and the sequence of subtasks can be modeled by combining these HMMs. As a combination of HMMs is still a stochastic process, this provides an efficient way to represent hierarchical skills. IDENTIFIED ASSEMBLY STATE THAT ARE FED TO CSG RECOGNIZE STATE SELECT HMM WITH MAX PROBABILITY STATE RECOGNIZER HMM for Force Pattern #1 HMM for Force Pattern #2... HMM for Force Pattern #n A SEQUENCE OF A SSEMBLY SUBTASKS Figure 4: An HMM-based state recognizer. 5.2 Control Strategy Generator In general, an assembly operation depicted in Figure 5, consists of three phases, namely, gross motion, approach motion and fine motion [18]. As it is almost impossible to develop a unique motion strategy for each different motion phase, control strategy is therefore needed to generate the identified states so that appropriate motion can be generated. In general, gross motion can be controlled by a position-controlled algorithm in which position signal is fed back to robotic controller to form a closed-loop system. The closed-loop feedback system is designed for the continuously monitoring of spatial relationship with reference to the destination. As interactions between the endeffector and operating environment are not presented in this stage, force signals are not required in the feedback loop. For the approach motion phase, the controller is required to acquire motion planning from a free motion state. This falls into a planning level in which controller has to decide when to switch the control strategy from free motion strategy to fine motion strategy. The last and the most complex is control of contact scenairo, which requires fine motion strategy that force/torque sensor signals are acquired and used in the control loop. Strategies such as compliance control, impedance control, hybrid control and contact sensing techniques are some of the commonly used fine motion control strategies. Therefore, the objective of the control strategy generator is to generate a reference motion strategy for the robot and the strategy is based on a particular input state observed from the state recognizer. The strategy generator facilitates the selections of gross motion and fine motion control schemes that are based on the current state information. For instances, when end-effector holding a peg is far away from a hole, it is recommended to use gross motion control strategies such as a positioncontrolled algorithm. On the other hand, if a peg makes a contact with the hole, it is recommended to use fine motion control such as force-controlled algorithm to control the motion of the robot. The main advantage of this approach for generating appropriate strategies is to shorten the assembly required time for a particular assembly operation. Another merit is to reduce impact forces while transition from a non-contact state to a contact state. Gripper Workpiece Hole GROSS APPROACH FINE Figure 5: Classes of motion in assembly operation. By adopting a scientific approach instead of a symbolic approach, it is appropriate to use soft computing techniques such as fuzzy logic or neural networks to infer the control strategy [13]. Fuzzy logic is adopted by the framework as an initial approach. The inferencing rule introduced hereby is to combined current state information up to n-step motions. It is advantageous that multiple state information captured are used to infer the next motion strategy. 6. Conclusions Task-oriented or system-specific frameworks for implementing an intelligent robot are extensively discussed among the community. However, a
thorough discussion of generic framework which can be extended to a generalized robotic system is seldom given. In addition, the consideration of hardware and robot model is not included in the available robotic framework. In contrast, this paper conveys the idea of a tight integration of hardware and robot modeling for an intelligent robot that is a pre-requisite for the realization of a truly human-like robot. As intelligent robot, it should possess the ability to learn and reason the information it captured, the proposed framework is an attempt to introduce a unified intelligent framework for robotic assembly. To achieving learning and reasoning capability, two intelligent components, namely, the state recognizer and the motion strategy generator, are integrated to the proposed framework. In this paper, the feasibility of using HMMs in robotic assembly is discussed and an HMM-based state recognizer is proposed. Its adaptive learning capability allows the noisy information captured from a F/T sensor in a dynamic environment to be used. The HMM-based state recognizer acts as a process monitor and provides state information to the motion strategy generator. It is expected that when this intelligent framework is completely implemented and tested with an industrial robotic system, it can greatly enhance the learning and reasoning capability of the robot in performing assembly operations in a dynamic environment. Reference [1] Brady, M., 'Artificial intelligence and Robotics', Artificial Intelligence, Vol. 26 (1985), pp. 79-121. [2] Brady, M., ed., Robotics science, The MIT Press, 1989. [3] Brady, M., Hollerbach, J. M., Johnson, T. L., Lozano-Perez, T. and Mason, M. T., eds., Robot motion: planning and control, The MIT Press, 1982. [4] Graham, J. H., 'Special computer architecture for robotics: tutorial and survey', IEEE Transactions on Robotics and Automation, Vol. 5, Issue 5 (1989), pp. 543-554. [5] Shafer, G. and Pearl, J., eds, Uncertain reasoning, Morgan Kaufamnn, 1990. [6] Valavanis, K. P. and Saridis, G. N., Intelligent robotic systems: theory, decision and applications, Kluwer Academic Publishers, 1992. [7] Vukobratovic, M., Introduction to Robotics, Springer-Verlag, 1989. [8] Fryer, J. A., McKee, G. T. and Schenker, P. S., 'Configuring robots from modules: an object oriented approach', Proceedings of the 1997 IEEE International Conference on Robotics and Automation (1997), pp. 907-912. [9] Brooks, R. A. 'A robust layered control system for a mobile robot', A. I. Memo 864, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1985. [10] Hannaford, B. and Lee, P., 'Hidden Markov model analysis of force/torque information in telemanipulation'. International Journal of Robotics Research, Vol. 10, No. 5 (1991), pp. 528 539. [11] Hovland, G. E. And McCarragher, B. J., 'Hidden Markov Models as a process monitor in robotic assembly'. International Journal of Robotics Research, Vol. 17, No. 2 (1998), pp. 153 168. [12] Hovland, G. E., Control of sensory perception for discrete event systems, Ph.D. Thesis, The Australian National University, Australia, 1997. [13] Jain, L. C. and Fukuda, T., eds., Soft computing for intelligent robotic systems, Physica-Verlag, 1998. [14] Lee, M. H. and Rowland, J. J., eds., Intelligent assembly systems, Word Scientific, 1995. [15] Rabiner, L. and Juang, B. H., Fundamentals of speech recognition, Prentice Hall, 1993. [16] Tebelskis, J., Speech recognition using neural networks, Ph.D. Thesis, The Robotics Institute, Carnegie Mellon University, USA, 1995. [17] Trevelyan, J., 'Simplifying robotics - a challenge for research', Robotics and Autonomous Systems, Vol. 21, Issue 3 (1997), pp. 207-220. [18] Whitney, D. E., 'A survey of manipulation and assembly: development of the field and open research issues'. Robotics Science, Michael Brady (eds), The MIT Press, 1989.