J. Casillas, O. Cordon, F. Herrera, 1. Magdalena (Eds.)

J. Casillas, O. Cordon, F. Herrera, 1. Magdalena (Eds.) Accuracy Improvements in Linguistic Fuzzy Modeling Springer-Verlag Berlin Heidelberg GmbH

Studies in Fuzziness and Soft Computing, Volume 129 http://www.springer.de/cgi-bin/search_book.pl?series=2941 Editor-in-chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland E-mail: kacprzyk@ibspan.waw.pl Further volumes of this series can be found on our homepage VoI. 110. E. Fink 1. 112. Y. Jin Advanced Fuzzy Systems Design and Applications, 2003 ISBN 3-7908-1523-3 VoI. 111. P.S. Szcepaniak, J. Segovia, J. Kacprzyk and L.A. Zadeh (Eds.) Intelligent Exploration of the Web, 2003 ISBN 3-7908-1529-2 VoI. 112. Y. Jin Advanced Fuzzy Systems Design and Applications, 2003 ISBN 3-7908-1537-3 VoI. 113. A. Abraham, L.C. Jain and). Kacprzyk (Eds.) Recent Advances in Intelligent Paradigms and Applications", 2003 ISBN 3-7908-1538-1 VoI. 114. M. Fitting and E. Orowska (Eds.) Beyond Two: Theory and Applications of Multiple Valued Logic, 2003 ISBN 3-7908-1541-1 VoI. 115. J.J. Buckley Fuzzy Probabilities, 2003 ISBN 3-7908-1542-X VoI. 116. C. Zhou, D. Maravall and D. Ruan (Eds.) Autonomous Robotic Systems, 2003 ISBN 3-7908-1546-2 Voi 117. O. Castillo, P. Melin Soft Computing and Fractal Theory for Intelligent Manufacturing, 2003 ISBN 3-7908-1547-0 VoI. 118. M. Wygralak Cardinalities of Fuzzy Sets, 2003 ISBN 3-540-00337-1 VoI. 119. Karmeshu (Ed.) Entropy Measures, Maximum Entropy Prin cip le and Emerging Applications, 2003 ISBN 3-540-00242-1 VoI. 120. H.M. Cartwright, L.M. Sztandera (Eds.) Soft Computing Approaches in Chemistry, 2003 ISBN 3-540-00245-6 VoI. 121.). Lee (Ed.) Software Engineering with Computational Intelligence, 2003 ISBN 3-540-00472-6 VoI. 122. M. Nachtegael, D. Van der Weken, D. Van de Viile and E.E. Kerre (Eds.) Fuzzy Filters for Image Processing, 2003 ISBN 3-540-00465-3 VoI. 123. V. Torra (Ed.) Information Fusion in Data Mining, 2003 ISBN 3-540-00676-1 VoI. 124. X. Yu, J. Kacprzyk (Eds.) Applied Decision Support with Soft Computing, 2003 ISBN 3-540-02491-3 VoI. 125. M. Inuiguchi, S. Hirano and S. Tsumoto (Eds.) Rough Set Theory and Granular Computing, 2003 ISBN 3-540-00574-9 VoI. 126. J.-L. Verdegay (Ed.) Fuzzy Sets Based Heuristics for Optimization, 2003 ISBN 3-540-00551-X Voi 127. L. Reznik, V. Kreinovich (Eds.) Soft Computing in Measurement and Information Acquisition, 2003 ISBN 3-540-00246-4 Voi 128. ). Casillas, O. Cordon, F. Herrera, L. Magdalena (Eds.) Interpretability Issues in Fuzzy Modeling, 2003 ISBN 3-540-02932-X

J. Casillas O. Cordon F. Herrera L. Magdalena (Eds.) Accuracy Improvements in Linguistic Fuzzy Modeling Springer

Dr. Jorge Casillas casillas@decsai.ugr.es Dr. Oscar Cordon E-mail: ocordon@decsai.ugr.es Dr. Francisco Herrera E-mail: herrera@decsai.ugr.es Dpto. Ciencias de la Computaci6n e Inteligencia Artificial Escuela Tecnica Superior de Ingenieria Informatica Universidad de Granada E - 18071 Granada Spain Dr. Luis Magdalena E-mail: llayos@mat.upm.es Dpto. Matematicas Aplicadas a las Tecnologfas de la Informaci6n Escuela Tecnica Superior de Ingenieros de Telecomunicaci6n Universidad Politecnica de Madrid E - 28040 Madrid Spain ISBN 978-3-642-05703-8 ISBN 978-3-540-37058-1 (ebook) DOI 10.1007/978-3-540-37058-1 Library of Congress Cataloging-in-Publication-Data applied for A catalog record for this book is available from the Library of Congress. Bibliographic information published by Die Deutsche Bibliothek Die Deutsche Bibliothek Iists this publication in the Deutsche Nationalbibliographie; detailed bibliographic data is available in the internet at <http://dnb.ddb.de>. This work is subject to copyright. AII rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitations, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. http://www.springer.de Springer-Verlag Berlin Heidelberg 2003 Origina1ly published by Springer-Verlag Berlin Heidelberg New York in 2003. Softcover reprint ofthe hardcover 1 st edition 2003 The use of general descriptive names, registered names trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: camera-ready by editors Cover design: E. Kirchner, Springer-Verlag, Heidelberg Printed on acid free paper 62/3020/M - 5 4 3 2 1 O

Foreword When I accepted the editors' invitation to write this foreword, I assumed that it would have been an easy task. At that time I did not realize the monumental effort that went into the organizat ion and compilation of these chapters, the depth of each contribution, and the thoroughness with which the book's theme had been covered. A foreword usually tries to impress upon the reader the importance of the book's main topic, placing the work within a comparative framework, and identifying the new trends or ideas that are pushing the state-of-the-art. While doing this, one also tries to relate the book's main theme to some personal experience that will help the reader understand the usefulness and applicability of the various contributions. I will do my best to achieve at least some of these lofty goals. The need for trading off interpretability and accuracy is intrinsic to the use of fuzzy systems. Before the advent of soft computing, and in particular of fuzzy logic, accuracy was the main concern of model builders, since interpretability was practically a lost cause. In a recent article in which I reviewed hybrid Soft Computing (SC) systems and compared them with more traditional approaches [1], I remarked that the main reason for the popularity of soft computing was the synergy derived from its components. In fact, SC's main characteristic is its intrinsic capability to create hybrid systems that are based on the integrat ion of constituent technologies. This integration provides complementary reasoning and searching methods that allow us to combine dom ain knowledge and empirical data to develop fiexible computing tools and solve complex problems. Soft Computing provides a different paradigm in terms of representation and methodologies, which facilitates these integration attempts. For instance, in classical control theory the problem of developing models is usually decomposed into system identificat ion (or system structure) and parameter estimation. The former determines the order of the differential equations, while the latter determines its coefficients. In these traditional approaches, the main goal is the construction of accurate models, within the assumptions used for the model construction. However, the models' interpretability is very limited, given the rigidity of the underlying representation language. The equation "model = structure + parruneters" 1, followed by the traditional approaches to model building, does not change with the advent of soft computing. However, with soft computing we have a much richer repertoire to represent the structure, to tune the parameters, and to iterate this process. This repertoire enables us to choose among different trade- 1 It is understood that the search method used to postulate the structures and find the parameter values is an important and implicit part of the above equation, and needs to be chosen carefully for efficient model construction. v

offs between the model's interpretability and accuracy. For instance, one approach aimed at maintaining the model's transparency might start with knowledge-derived linguistic models, where the dom ain knowledge is translated into an initial structure and parameters. Then the model's accuracy could be improved by using global or local data-driven search methods to tune the structure and/or the parameters. An alternative approach aimed at building more accurate models might start with data-driven search methods. Then, we could embed domain knowledge into the search operators to control or Iim it the search space, or to maintain the model's interpretability. Postprocessing approaches could also be used to extract more explicit structural information from the models. This book provides a comprehensive yet detailed review of ali these approaches. In the introduction the reader will find a general framework, within which these approaches can be compared, and a description of alternative methods for achieving different balances between models' interpretability and accuracy. The book is mainly focused on the achievement of the mentioned tradeoff by improving the accuracy while preserving interpretability in linguistic fuzzy modeling. Thus, it presents constrained optimization methods, as well as extensions to the modeling process and model structures to do so. These topics are germane to many applications and resonate with recent issues that I have addressed. Therefore, I would like to illustrate the pervasiveness of this book's main theme by relating it to a personal experience. By virtue of working in an industrial research center, I am constantly faced with the constraints derived from real-world problems. There are situations in which the use of black-box models is not acceptable, due to legal or compliance reasons. On the other hand, the same situations require a degree of accuracy that is usually prohibitive for purely transparent models. An example of such a situation is the automation of the insurance underwriting process, which consists in evaluat ing an applicant's medical and personal information to assess his/her potential risk and determine the appropriate rate class corresponding to such risk. To address this problem, we need to maintain full accountability of the model decisions, i.e. full transparency. This legal requirement, imposed by the states insurance commissioners, is necessary since the insurance companies need to notify their customers and explain to them the reasons for issuing policies that are not at the most competitive rates. Yet, the model must also be extremely accurate to avoid underestimating the applicants' risk, which would decrease the company's profitability, or overestimating it, which would reduce the company's competitive position in the market. We solved this problem by creating several hybrid se models, some of them transparent, for use in production, and some of them opaque, for use in quality assurance. The commonalities among these models are the tight integration of knowledge and data, leveraged in their construction, and the loose integrat ion of their outputs, exploited in their off-line use. In different parts VI

of this project we strived to achieve different balances between interpretability and accuracy. This project exemplifies the pervasiveness of the theme and highlights the timeliness of this book, which fills a void in the technical literature and describes a topic of extreme relevance and applicability. Piero P. Bonissone General Electric Global Research Center Schenectady, New York, 12308, USA [1] "Hybrid Soft Computing Systems: Industrial and Commercial Applications", P. P. Bonissone, Y-T Chen, K. Goebel and P. S. Khedkar, Proceedings of the IEEE, pp 1641-1667, voi. 87, no. 9, September 1999. VII

Preface System modeling with fuzzy rule-based systems, i.e. fuzzy modeling, usually comes with two contradictory requirements in the obtained model: the interpretability, capability to express the behavior of the real system in an understandable way, and the accuracy, capability to faithfully represent the real system. To obtain high degrees of interpretability and accuracy is a contradictory purpose and, in practice, one of the two properties prevails over the other. While linguistic fuzzy modeling (mainly developed by linguistic fuzzy systems) is focused on the interpretability, precise fuzzy modeling (mainly developed by Takagi-Sugeno-Kang fuzzy systems) is focused on the accuracy. Analyzing the research made from the former approach (linguistic fuzzy modeling), a large number of publications are found being oriented towards the use of new techniques and structures to extend the classical, rigid linguistic fuzzy modeling, whith the main aim of improving its accuracy. Thus, more flexible model structures with a larger number of freedom degrees (based on tools such as weights, hierarchical knowledge, or linguistic hedges) and advanced modeling processes (such as multicriteria optimizations or membership function learning) are performed. Of course, the flexibilizations made to enhance the precision should be performed under the assumption of preserving a good interpretability, otherwise, one of the most interesting features of linguistic fuzzy models would be ignored: its good capability to describe its intrinsic knowledge. From this perspective, this book focuses on showing a state-of-the-art on the recent proposals that attempt to obtain linguistic fuzzy models with a good interpretability-accuracy trade-off by improving their accuracy. The book is organized as follows. Section 1 introduces an overview of the different accuracy improvement mechanisms existing in the recent literature. Section 2 collects a set of contributions focused on using different accuracy improvements performed under some constrains that avoid an excessive interpretability loss; restrictions such as rigid structures, comprehensibility criteria of the membership functions, or compactness of the rule set are considered. Section 3 contains a set of contributions that propose more sophisticated modeling processes to attain a good accuracy while preserving interpretability. Finally, Section 4 introduces a different approach that performs the accuracy improvement extending the traditional model structure by using different methodologies such as importance factors for each rule, knowledge bases with different granularities, etc. We believe that this volume presents an up-to-date state of the current research that will be useful for non expert readers, whatever their background, to easily get some knowledge about this area of research. Besides, it will also support those specialists who wish to discover the latest results as well as the latest trends in research work in fuzzy modeling. IX

Finally, we would like to express our most sincere gratitude to Springer Verlag (Heidelberg, Germany) and in particular to Prof. J. Kacprzyk, for having given us the opportunity to prepare the text and for having supported and encouraged us throughout its preparation. We would also like to acknowledge our gratitude to ali those who have contributed to the books by producing the papers that we consider to be of the highest quality. We also like to mention the somehow obscure and altruistic, though absolutely essential, task carried out by a group of referees (ali the contributions have been reviewed by two of them), who, through their comments, suggestions, and criticisms, have contributed to raising the quality of this edited book. Granada and Madrid (Spain) January 2003 Jorge Casillas, Oscar Cordon, Francisco Herrera, and Luis Magdalena x

Table of Contents 1. OVERVIEW Accuracy improvements to find the balance interpretability-accuracy 3 in linguistic fuzzy modeling: an overview J. Casillas, O. Cord6n, F. Herrera, L. Magdalena 2. ACCURACY IMPROVEMENTS CONSTRAINED BY INTER PRETABILITY CRITERIA COR methodology: a simple way to obtain linguistic fuzzy models 27 with good interpretability and accuracy J. Casillas, O. Cord6n, F. Herrera Constrained optimizat ion of genetic fuzzy systems 46 F. Cheong, R. Lai Trade-off between the number of fuzzy rules and their classification 72 performance H. Ishibuchi, T. Yamamoto Generating distinguishable, complete, consistent and compact fuzzy 100 systems using evolutionary algorithms Y. Jin Fuzzy CoCo: balancing accuracy and interpretability offuzzy models 119 by means of coevolution C.A. Peiia-Reyes, M. Sipper On the achievement of both accurate and interpretable fuzzy systems 147 using data-driven design processes J. Valente de Oliveira, P. Fazendeiro XI

3. EXTENDING THE MODELING PROCESS TO IMPROVE THE ACCURACY Linguistic hedges and fuzzy rule based systems 165 C.- Y. Chen, B.-D. Liu Automatic construction of fuzzy rule-based fuzzy systems: A trade- 193 off between complexity and accuracy maintaining interpretability H. Pomares, I. Rojas, J. Gonzalez Using individually tested rules for the databased generat ion of inter- 220 pretable rule bases with high accuracy T. 8lawinski, P. K rause, H. Kiendl 4. EXTENDING THE MODEL STRUCTURE TO IMPROVE THE ACCURACY A description of several characteristics for improving the accuracy 249 and interpretability of the fuzzy rule learning algorithms E. Aguirre, A. Gonzalez, R. Perez An iterative learning methodology to design hierarchical systems of 277 linguistic rules for linguistic modeling R. Alcala, O. Cordon, F. Herrera, 1. Zwir Learning default fuzzy rules with general and punctual exceptions 302 P. Carmona, J.L. Castro, J.J. Castro-8chez, M. Laguia Integration of fuzzy knowledge 338 T.-P. Hong, C.-H. Wang, 8.-8. Tseng Thning fuzzy partitions or assigning weights to fuzzy rules: which is 366 better? L. 8anchez, J. Otero XII