1. Aims of Soft Computing

Transcription

1 1. Aims of Soft Computing 1.1. Soft Computing (SC) as Key Methodology for Designing of Intelligent Systems Artificial intelligence as a science has been existing for about 40 years now. The main problem of this science is replication of human reasoning processes and behavior with the aid of computers and other artificial devices as well as construction of machines simulating decision making by humans in imprecise and uncertain environments. In most cases these various areas, where precise models, methods, and algorithms for solving problems characterized by uncertainty are not available, are attributed to the field of artificial intelligence. Methods of artificial intelligence are based on two characteristic features: 1. Use of information in symbolic form i.e. letters, words, phrases, signs, figures; 2. Search with the aid of symbolic logic. When processing symbolic information, the computer converts the words and phrases to the form of binary digits. Then the computer recognizes or compares the sequences of such symbols (converted to digital form). The classics of artificial intelligence stated that the ability of computers to manipulate symbols as easily as numbers, compare sequences of symbols, and then, depending on the results of comparison perform subsequent operations, would allow realization of the functions typical for the human mind, i.e. functions of deductive logical reasoning, in machines. It may seem that the potential abilities of a computer in creation of artificial intelligence based on the symbolic information processing are unlimited. Despite the significant success of artificial intelligence (in the classical sense) in developing a wide range of systems for solving problems, automatically proving theorems, recognizing patterns as well as in constructing expert systems and natural language understanding systems, the expectations have not been achieved to a full degree. Traditional artificial intelligence is not capable of solving problems which require the use of common sense, and it does not accept procedures, which are similar to human abilities of understanding and reasoning. Traditional artificial intelligence has not succeeded in solving problems for intelligent robotics, computer vision, recognition of speech and hand-written graphics, machine translation, learning through experience and many other important real-world 1

2 Soft Computing and Its Applications problems. The above problems, as well as many others have intrinsic imprecision and uncertainty that cannot be neglected. As Prof. L.Zadeh noted, the traditional artificial intelligence could achieve more success in pursuing its goals if it did not limit itself to processing symbolic information only and using the first order logic. All traditional artificial intelligence systems have been implemented using the Hard Computing technology, which restricts considerably the abilities of those systems. Moreover, the traditional artificial intelligence, due to the features shown above does not accept the numerical methods, which are important for accounting for uncertainty and imprecision. Due to the above limitations, the MIQ (Machine Intelligence Quotient) for traditional artificial intelligence systems is not sufficiently high. There is a strong need to increase MIQ for intelligent systems. Soft Computing methodology implies cooperative activity rather than autonomous one for such new computing paradigms as fuzzy logic, neural networks, evolutionary computation and others. This approach allows solving many important realworld problems, which were impossible to solve using traditional artificial intelligence methods [1-4,8,15]. The combination of such intelligent paradigms (used as computing techniques) as Fuzzy Logic (FL), Neural Networks (NN), Probabilistic Reasoning (PR), Genetic Algorithms (GA), and Chaos Theory (ChT) dealing with pervasive imprecision and uncertainty of the real-world problems is named Soft Computing (SC). Unlike traditional Hard Computing (HC), SC can tolerate imprecision, uncertainty and partial truth without loss of performance and effectiveness for the end use. In no more than a decade we will see re-orientation of Artificial Intelligence towards Soft Computing from the traditional Hard Computing. L.Zadeh noted that, unlike the traditional Hard Computing, Soft Computing aims at accommodation with the pervasive imprecision of the real world. The guiding principle of Soft Computing is: exploit the tolerance for imprecision, uncertainty, and partial truth to achieve tractability, robustness and low solution cost [5]. We can easily come to the conclusion that precision has a cost (unfortunately, this obvious principle often is neglected). Therefore, in order to solve a problem with an acceptable cost we need to aim at a decision with only the necessary degree of precision, not exceeding the requirements. The impressive examples of the aforesaid are problems of landing a helicopter or parking a car. Let's consider the second case. One can park a car without taking any distance and angle measurements, because the final position of the car is not specified clearly. If it is specified, though, then the measurements are necessary, say, in the range of fractions of millimeter or several seconds of arc. This will require many hours of manoeuvres and measurements from the devices for solving the problem. Moreover, the cost of decision will increase exponentially as the precision increases. Soft Computing technology is of great importance for data compression, especially in HDTV, audio recording, speech recognition, image understanding, and related fields. Actually, soft-computing-based concepts and techniques have

3 Aims of Soft Computing 3 already been playing an essential role in the conception, design and manufacturing of high MIQ products and systems. As noted by Zadeh, the perfect model of SC is human brain Strtucture and Constituents of Soft Computing As it was mentioned above, all traditional artificial intelligence systems, including expert systems, widely used in various areas of human activity, have been implemented on the base of Hard Computing, often using computers. However, this base obviously limits the effectiveness and, generally, the possibility of creating artificial intelligence systems for different purposes. Currently, the significant increase can be noticed in the number of applied artificial intelligence systems based not on numerical (symbolic) computation and traditional Hard Computing, but on neural networks, fuzzy computing, evolutionary programming, and chaotic computing. There is also an increase in the number of publications in proceedings of scientific conferences which are devoted to fuzzy logic, genetic algorithms, artificial life, biological computing, neural computing etc. This increase provides the evidence that the focus of the investigations and implementations of real artificial intelligence systems makes a shift towards Soft Computing. Figure 1.1 shows the structure of Soft Computing technology forming the basis for computational intelligence. The following main components of Soft Computing are known by now: fuzzy logic (FL), neural networks (NN), probabilistic reasoning (PR), genetic algorithms (GA), and chaos theory (ChT) (Figure 1.1). In our (and not only our) framework of SC, FL is the kernel of SC. FL's main characteristic is the robustness of its interpolative reasoning mechanism. Within Soft Computing, Fuzzy Logic occupies a special place bacause it can be used as a springboard for generalization of any theory, including its partners in SC consortium. In SC FL is mainly concerned with imprecision and approximate reasoning, NN with learning, PR with uncertainty and propagation of belief, GA with global optimization and search and ChT with nonlinear dynamics. Each of these computational paradigms (emerging reasoning technologies) provide us with complementary reasoning and searching methods to solve complex, real-world problems. In large scope, FL, NN, PR, and GA are complementary rather that competitive [5,12,15]. The interrelations between the components of SC, shown in Figure 1.1, make the theoretical foundation of Hybrid Intelligent Systems. As noted by L. Zadeh "currently the term hybrid intelligent systems is gaining currency as a descriptor of systems in which FL, NC, and PR are used in combination. In my view, hybrid intelligent systems are the wave of the future" [14]. The use of Hybrid Intelligent Systems are leading to the development of numerous

4 Soft Computing and Its Applications manufacturing system, multimedia system, intelligent robots, tradinig systems, which exhibits a high level of MIQ. Computing technologies Hard Computing - base of classical Artificial intelligence Soft Computing - base of Computational intelligence with high MIQ -^» Hybrid Systems <r Figure 1.1. The main components of Soft Computing 1.3. Comparative Characteristics of the Constituents of SC The constituents of SC can be used independently (fuzzy computing, neural computing, evolutionary computing etc.), and more often in combination [1,2,6-8,13]. Based on independent use of the constituents of Soft Computing, fuzzy technology, neural technology, chaos technology and others have been recently applied as emerging technologies to both industrial and non-industrial areas. Fuzzy logic is the leading constituent of Soft Computing. In Soft Computing, fuzzy logic plays a unique role. FL serves to provide a methodology for computing with words [1]. It has been successfully applied to many industrial spheres, robotics, complex decision making and diagnosis, data compression, and many other areas. To design a system processor for handling knowledge

5 Aims of Soft Computing 5 represented in a linguistic or uncertain numerical form we need a fuzzy model of the system. Fuzzy sets can be used as a universal approximator, which is very important for modeling unknown objects. If an operator cannot tell linguistically what kind of action he or she takes in a specific situation, then it is quite useful to model his/her control actions using numerical data. However, fuzzy logic in its socalled pure form is not always useful for easily constructing intelligent systems. For example, when a designer does not have sufficient prior information (knowledge) about the system, development of acceptable fuzzy rule base becomes impossible. As the complexity of the system increases, it becomes difficult to specify a correct set of rules and membership functions for describing adequately the behavior of the system. Fuzzy systems also have the disadvantage of not being able to extract additional knowledge from the experience and correcting the fuzzy rules for improving the performance of the system. Another important component of Soft Computing is neural networks. Artificial neural networks viewed as parallel computational models, are parallel fine-grained implementation of non-linear static or dynamic systems. A very important feature of these networks is their adaptive nature, where "learning by example" replaces traditional "programming" in problems solving. Another key feature is the intrinsic parallelism that allows fast computations. Artificial neural networks are viable computational models for a wide variety of problems including pattern classification, speech synthesis and recognition, curve fitting, approximation capability, image data compression, associative memory, and modeling and control of non-linear unknown systems [9,10]. Neural networks are favorably distinguished for efficiency of their computations and hardware implementations. Another advantage of neural networks is generalization ability, which is the ability to classify correctly new patterns. A significant disadvantage of neural networks is their poor interpretability. One of the main criticisms addressed to neural networks concerns their black box nature [15]. Evolutionary Computing (EC) is a revolutionary approach to optimization. One part of EC genetic algorithms are algorithms for global optimization. Genetic algorithms are based on the mechanisms of natural selection and genetics [11]. One advantage of genetic algorithms is that they effectively implement parallel multi-criteria search. The mechanism of genetic algorithms is simple. Simplicity of operations and powerful computational effect are the two main advantages of genetic algorithms. The disadvantages are the problem of convergence and absence of strong theoretical foundation. The requirement of coding the domain of the real variables' into bit strings also seems to be a drawback of genetic algorithms. It should be also noted that the computational speed of genetic algorithms is low. PR offers the mechanism to evaluate the outcome of systems affected by probabilistic uncertainty. PR uses the operation of conditioning to update the probability values and perform a probabilistic inference. The probabilistic

6 6 Soft Computing and Its Applications approach (objective or subjective) provides a rigorous framework for representation of a probabilistic knowledge, modeling random phenomena and for analyzing them. Moreover, PR approach does not distinguish between ambiguity of the knowledge and uncertainty generating errors and lack of complete knowledge. Fuzzy Logic is able to make this distinction. Here we should also note that human reasoning does not follow the axioms of probability theory and in case when the evidence is uncertain, the computational complexity significantly increases. At present Chaotic computing is a very important field of scientific research and treated as a basis for a new technology. A chaotic system is a deterministic system that exhibits random behavior. Chaos Theory deals with the non-linear dynamical systems that exhibit extreme sensitivity to initial conditions. Behavior of chaotic systems is characterized by a strange attractor, which has a fractal dimension bounded by the topological and the Euclidean dimensions. Table 1.1 presents the comparative characteristics of the components of Soft Computing. Fuzzy Sets Artificial Neural Networks Evolutionary Computing, GA Probabilistic Reasoning Chaotic computing Weaknp^cpc iivjacis Knowledge acquisition Learning Black Box interpretability Coding Computational speed Limitation of the axioms of Probability Theory Lack of complete knowledge Computational complexity Computational complexity Chaos identification complexity Strengths Interpretability Transparency Plausibility Graduality Modeling Reasoning Tolerance to imprecision Learning Adaptation Fault tolerance Curve fitting Generalization ability Approximation ability Computational efficiency Global optimization Rigorous framework Well understanding Nonlinear dynamics simulation Discovering chaos in observed data (with noise) Determining the predictability Prediction strategies formulation Table 1.1. Comparative characteristics of the components of Soft Computing Using chaotic analysis we determine the predictability and formulate prediction strategies of system's behavior. Chaotic computing deals also with nonlinear systems with unknown functional form and, possibly, noise. Chaotic

7 Aims of Soft Computing 7 computing gives a tool to determine a new perspective of nonlinear data analysis. No assumption is made about the behavior of the data. In addition to the aforesaid it should be noted that identifying chaos in realworld problems is a complex task. Another weakness of chaotic computing is that practical numerical analysis of chaotic systems in most cases is connected with computational difficulty. For each component of Soft Computing there is a specific class of problems, where the use of other components is inadequate. For example, the well-known problem of parking a car can be solved successfully by using only Fuzzy Logic, and not by using Neural Networks, GA etc Intelligent Combinations of the Components of SC As it was shown above, the components of Soft Computing Fuzzy Logic, Neuro Computing, Probabilistic Reasoning etc. complement each other, rather than compete. It becomes clear that FL, NC, PR, and GA are more effective when used in combinations. The following are known principal combinations of the components of Soft Computing: neuro computing + fuzzy logic (Neuro-Fuzzy: NF) fuzzy logic + genetic algorithms (FG), fuzzy logic + chaos theory (FCh), neural networks + genetic algorithms (NG); neural networks + chaos theory (NCh); fuzzy logic + neural networks + genetic algorithms (FNG), neural networks + fuzzy logic + genetic algorithms (NFG). fuzzy logic+probabilistic reasoning (FP) Other combinations of constituents of SC are possible as well. Lack of interpretability of neural networks on one hand and poor learning capability of fuzzy systems on the other hand are similar problems that limit the application of these tools. Neural Fuzzy systems are hybrid systems which try to solve this problem by combining the learning capability of connectionist models with the interpretability property of fuzzy systems. As it was noted above, in case of dynamic work environment, the automatic knowledge base correction in fuzzy systems becomes necessary. On the other hand, artificial neural networks are successfully used in problems connected to knowledge acquisition using learning by examples with the required degree of precision. Incorporating neural networks in fuzzy systems for fuzzification, construction of fuzzy rules, optimization and adaptation of fuzzy knowledge base, implementation of fuzzy reasoning, and denazification is the essence of the Neuro-Fuzzy approach. Chapter 9 of the book is devoted to Neuro-Fuzzy systems. The combination of rule-based fuzzy systems employing "rule of thumb" strategy used by humans in decision making, with genetic algorithms, which

8 8 Soft Computing and Its Applications allow to perform global search, enables creation of effective, robust, and adaptive systems. Often, the membership functions used in fuzzy rules in knowledge bases of fuzzy systems and fuzzy performance indices are represented as non-differentiable fuzzy numbers, e.g. trapezoids, triangles etc. Use of gradient-based methods for development of such systems becomes infeasible. An effective technique in this case is GA. The combination of FL and GA allows optimization of fuzzy knowledge base of fuzzy logic control system by defining optimal number of rules in knowledge base and optimal values for centers and shapes of membership functions. Here GA is used for constructing relational matrix and membership functions of the designed fuzzy systems. In turn, in the combination of FL with GA, theory of fuzzy systems can be used for improving the behavior of genetic operators or genetic algorithms on whole, i.e. it is possible to create fuzzy tools for improving effectiveness of GA via developing fuzzy genetic algorithms. The combination of genetic algorithms with neural networks yields promising results as well. It is known that one of main problems in development of artificial neural systems is selection of a suitable learning method for tuning the parameters of a neural network (weights, thresholds, and structure). The most known algorithm is the "error back propagation" algorithm. Unfortunately, there are some difficulties with "back propagation". First, the effectiveness of the learning considerably depends on initial set of weights, which are generated randomly. Second, the "back propagation", like any other gradient-based method, does not avoid local minima. Third, if the learning rate is too slow, it requires too much time to find the solution. If, on the other hand, the learning rate is too high it can generate oscillations around the desired point in the weight space. Fourth, "back propagation" requires the activation functions to be differentiable. This condition does not hold for many types of neural networks. Genetic algorithms used for solving many optimization problems when the "strong" methods fail to find appropriate solution, can be successfully applied for learning neural networks, because they are free of the above drawbacks. The models of artificial neurons, which use linear, threshold, sigmoidal and other transfer functions, are effective for neural computing. However, it should be noted that such models are very simplified. For example, reaction of a biological axon is chaotic even if the input is periodical. In this aspect the more adequate model of neurons seems to be chaotic. Model of a chaotic neuron can be used as an element of chaotic neural networks. The more adequate results can be obtained if using fuzzy chaotic neural networks, which are closer to biological computation. Fuzzy systems with If-Then rules can model non-linear dynamic systems and capture chaotic attractors easily and accurately. Combination of Fuzzy Logic and Chaos Theory gives us useful tool for building system's chaotic behavior into rule structure. Identification of chaos allows us to determine predicting strategies. If we use a Neural Network Predictor for predicting the system's behavior, the parameters

9 Aims of Soft Computing 9 of the strange attractor (in particular fractal dimension) tell us how much data are necessary to train the neural network. The combination of Neurocomputing and Chaotic computing technologies can be very helpful for prediction and control. Different methods exist for learning fuzzy neural networks, i.e. neural networks with fuzzy signals and/or fuzzy weights. In particular, direct fuzzification of the ordinary delta-rule is used. Another method is based on alpha-cuts of fuzzy sets, interval arithmetic, and consequent application of back propagation. In the latter case, the algorithm can fail to converge to correct values of weights. In either case it is necessary to calculate the gradient of the fuzzy error measure. The corresponding derivatives are too complex, especially in case of more general fuzzy sets for input and output signals and weights. Neuro-genetic algorithms for learning fuzzy neural networks based on a combination of FL, NC, and GA are free of the above drawbacks and produce effective results. It is necessary to note that Probabilistic Reasoning and Fuzzy Reasoning are complementary rather than competitive. The cooperation between these formalisms gives a useful tool for modeling and reasoning under uncertainty in complicated real-world problems. Such cooperation is of particular importance for constructing perception-based intelligent information systems. We hope that the mentioned intelligent combinations will develop further, and the new ones will be proposed. These SC paradigms will form the basis for creation and development of Computational Intelligence References 1. Zadeh L. A. Soft Computing and Fuzzy Logic. IEEE Software 11 (6): 48-58, Aliev R.A. Fuzzy Expert Systems. In Aminzadeh F. and Jamshidi M.(eds) SOFT COMPUTING: Fuzzy Logic, Neural Networks and Distributed Artificial Intelligence.pages NJ: PTR Prentice Hall, Zurada Y.M., Marks R.J., and Robinson C.Y.(eds) Computational Imitating life. NJ: Piscataway, IEEE Press, Pearson D.W., Steele N.C., and Albrecht R.F. Artificial Neural Nets and Genetic Algorithms. In Proc. of the Inter. Conf. in Ales, France, Zadeh L.A. The roles of fuzzy logic and soft computing in the conception, design and deployment of intelligent systems. BT Technol J. 14(4): 32-36, Zadeh L.A. Fuzzy logic, Neural Networks and Soft Computing. Comm of ACM 37(3): 77-84, Welstead S.T.(ed) Neural Networks and Fuzzy Logic Applications in C/C++, Professional Computing. NY: John Wiley, Yager R.R. and Zadeh L.A.(eds) Fuzzy sets, neural networks and Soft Computing. NY: VAN Nostrand Reinhold, Mohamad H.Hassoun, Fundamentals of artificial neural networks. Cambridge: MIT Press, Haykin S., Neural Networks: A Comprehensive Foundation. Marmillau and IEEE Computer Society, 1994.

10 10 Soft Computing and Its Applications 11. Goldberg D.E., Genetic algorithms in search, optimization and machine learning. Reading, MA: Addison-Wesley, Aliev R.A. and Aliev R.R., Soft Computing, volumes I, II, III. Baku: ASOA Press, (in Russian). 13. Nauck D., Klawonn F., and Kruse R., Foundations of Neuro-Fuzzy Systems.NY: John Wiley and Sons, Zadeh L.A. Foreword. In Proc. First European Congress on Intelligent Techniques and Soft Computing - EUFIT'95, page VII, Aliev R., Bonfig K., and Aliew F., Soft Computing. Berlin: Verlag Technic, 2000.