Causality, Correlation and Artificial Intelligence for Rational Decision Making
This page intentionally left blank
Causality, Correlation and Artificial Intelligence for Rational Decision Making Tshilidzi Marwala University of Johannesburg, South Africa World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI
Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE Causality, Correlation and Artificial Intelligence for Rational Decision Making Downloaded from www.worldscientific.com Library of Congress Cataloging-in-Publication Data Marwala, Tshilidzi, 1971 Causality, correlation, and artificial intelligence for rational decision making / by Tshilidzi Marwala (University of Johannesburg, South Africa). pages cm Includes bibliographical references and index. ISBN 978-9814630863 (alk. paper) 1. Decision making. 2. Artificial intelligence. I. Title. T57.95.M375 2015 006.301'51--dc23 2014027464 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. Copyright 2015 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. In-house Editors: Sutha Surenddar/Chelsea Chin Typeset by Stallion Press Email: enquiries@stallionpress.com Printed in Singapore
Preface Causality, correlation and artificial intelligence for rational decision making covers five significant areas of intellectual inquiry which are causality, correlation, artificial intelligence, rationality, and decision making. It gives new insights on causality and correlations machines and proposes that these form as a fundamental unit for rational decision making. Causality has been a subject of study for many years and has often been confused with correlation. Human intuition has evolved such that it has learned to identify causality through correlation. This book proposes a model that claims that within any causality model is buried a correlation machine. This study uses artificial intelligence to build correlation and causality machines. Artificial intelligence methods that are used are the multi-layer perceptron (MLP), radial basis function (RBF), fuzzy inference systems, support vector machines, genetic algorithms (GAs), and simulated annealing (SA). The concepts of causality, correlation and artificial intelligence are then applied in interstate conflict, condition monitoring and several biomedical applications. This book introduces the concept of flexibly-bounded rationality and uses the correlation and causal machines to implement this. Flexibly-bounded rationality is a special case of bounded rationality. Furthermore, this book introduces the theory of the marginalization of irrationality for decision making and applies this to the problem of breast cancer diagnosis. This book is intended for philosophers, computer scientists, engineers, and other interested parties at both undergraduate and graduate levels as well as practitioners. v
This page intentionally left blank
Acknowledgments I would like to thank the following institutions for contributing toward the writing of this book: University of Cambridge, University of Pretoria, and University of Johannesburg. I also would like to thank my students for their assistance in developing this manuscript more particularly Fholisani Mashegana. I dedicate this book to the schools that gave me the foundation to always seek excellence in everything I do and these are: Mbilwi Secondary School, Case Western Reserve University, University of Pretoria, University of Cambridge (St. John s College), and Imperial College (London). I thank the Stellenbosch Institute of Advanced Study (STIAS) for the financial support as well as accommodation during the writing of this book. This book is dedicated to the following people: Nhlonipho Khathutshelo, Lwazi Thendo, Mbali Denga Marwala as well as Dr. Jabulile Vuyiswa Manana. Professor Tshilidzi Marwala, Ph.D. Johannesburg 1 February 2014 vii
This page intentionally left blank
Preface Acknowledgments Contents 1. Introduction to Artificial Intelligence based Decision Making 1 1.1 Introduction... 1 1.2 Correlation... 2 1.2.1 What is correlation?... 2 1.2.2 Correlation function... 2 1.3 Causality... 4 1.3.1 What is causality?... 4 1.3.2 Theories of causality... 4 1.3.3 What is a causal function?... 10 1.3.4 How to detect causation?... 11 1.4 Introduction to Artificial Intelligence... 11 1.4.1 Neural networks... 11 1.4.2 Hopfield networks... 12 1.4.3 Genetic algorithm... 13 1.4.4 Particle swarm optimization... 13 1.4.5 Simulated annealing... 14 1.5 Rational Decision Making... 14 1.6 Summary and Outline of the Book... 15 1.7 Conclusions... 16 References... 16 v vii 2. What is a Correlation Machine? 23 2.1 Introduction... 23 2.2 Correlation Machines... 24 2.2.1 Auto-associative memory network... 24 ix
x Contents Causality, Correlation and Artificial Intelligence for Rational Decision Making Downloaded from www.worldscientific.com 2.2.2 Principal component analysis... 26 2.2.3 Expectation maximization algorithm... 28 2.3 Genetic Algorithm... 32 2.3.1 Initialization... 33 2.3.2 Crossover... 33 2.3.3 Mutation... 34 2.3.4 Reproduction... 34 2.3.5 Termination... 35 2.4 Multi-layer Perceptron... 35 2.5 Experimental Comparison... 36 2.6 Conclusions... 38 References... 38 3. What is a Causal Machine? 43 3.1 Introduction... 43 3.2 Induction, Deduction, and Abduction... 44 3.3 What is Causality?... 45 3.4 Multi-layer Perceptron Causal Machine... 47 3.4.1 The architecture of the MLP causal machine... 48 3.4.2 Interstate conflict... 50 3.5 Radial Basis Function Causal Machine... 51 3.5.1 Theoretical foundation... 51 3.5.2 Applications to condition monitoring... 54 3.6 Fuzzy Inference System Causal Machine... 55 3.6.1 Theoretical foundation... 55 3.6.2 Application to a steam generator... 58 3.7 Conclusions... 59 References... 59 4. Correlation Machines Using Optimization Methods 65 4.1 Introduction... 65 4.2 Multi-layer Perceptron Neural Network... 66 4.3 Missing Data Estimation Technique... 67 4.4 Genetic Algorithms... 67 4.4.1 Initialization... 68 4.4.2 Crossover... 68 4.4.3 Mutation... 69 4.4.4 Reproduction... 69 4.4.5 Termination... 70
Contents xi Causality, Correlation and Artificial Intelligence for Rational Decision Making Downloaded from www.worldscientific.com 4.5 Particle Swarm Optimization... 70 4.6 Simulated Annealing... 73 4.6.1 SA parameters... 75 4.6.2 Transition probabilities... 75 4.6.3 Monte Carlo method... 75 4.6.4 Markov Chain Monte Carlo... 76 4.6.5 Acceptance probability function: Metropolis algorithm... 76 4.6.6 Cooling schedule... 77 4.7 Missing Data Estimation: Case Studies... 77 4.7.1 Mechanical system... 78 4.7.2 Modeling of beer tasting... 79 4.8 Conclusions... 79 References... 80 5. Neural Networks for Modeling Granger Causality 87 5.1 Introduction... 87 5.2 Granger Causality... 88 5.3 Multi-layer Perceptron for Granger Causality... 89 5.3.1 Bayesian statistics... 90 5.3.2 Hybrid Monte Carlo (HMC)... 91 5.4 RBF for Granger Causality... 94 5.4.1 The k-means... 96 5.4.2 Pseudo-inverse methods... 96 5.5 Example: Mackey Glass System... 97 5.6 Conclusions... 99 References... 99 6. Rubin, Pearl and Granger Causality Models: A Unified View 105 6.1 Introduction... 105 6.2 Neyman-Rubin Causal Model... 106 6.2.1 Missing data mechanism... 107 6.2.2 Missing data imputation methods... 108 6.3 Pearl Causality... 109 6.3.1 Directed acyclic graph... 109 6.3.2 Associations between variables... 110 6.3.3 d-separation... 112 6.3.4 Back-door adjustment... 113 6.3.5 Front-door adjustment... 114
xii Contents Causality, Correlation and Artificial Intelligence for Rational Decision Making Downloaded from www.worldscientific.com 6.3.6 Rules for do-calculus... 115 6.3.7 Pearl inferred causation algorithm... 116 6.3.8 Examples of using do-calculus... 117 6.4 Granger Causality... 118 6.5 Comparison: Neyman-Rubin, Pearl and Granger Causality... 119 6.6 Conclusions... 120 References... 120 7. Causal, Correlation and Automatic Relevance Determination Machines for Granger Causality 125 7.1 Introduction... 125 7.2 Causal Machine to Granger Causality... 126 7.2.1 Multi-layer perceptron... 128 7.2.2 Scaled conjugate gradient method... 129 7.3 Correlation Machine to Granger Causality... 131 7.3.1 Auto-associative network for missing data estimation.. 131 7.3.2 Nelder-Mead simplex optimization method... 132 7.3.3 Granger causality... 135 7.4 Automatic Relevance Determination for Granger Causality... 136 7.5 Experimental Investigation: Mackey Glass Time-Delay Differential Equation... 139 7.6 Conclusions... 141 References... 141 8. Flexibly-bounded Rationality 147 8.1 Introduction... 147 8.2 Rational Decision Making: A Causal Approach... 149 8.3 Rational Decision Making Process... 149 8.4 Bounded-Rational Decision Making... 150 8.5 Flexibly-bounded Rational Decision Making... 152 8.5.1 Advanced information processing... 154 8.5.2 Missing data estimation... 156 8.5.3 Intelligent machines... 156 8.6 Experimental Investigations... 157 8.6.1 Condition monitoring... 157 8.6.2 HIV modeling... 159 8.7 Conclusions... 160 References... 161
Contents xiii Causality, Correlation and Artificial Intelligence for Rational Decision Making Downloaded from www.worldscientific.com 9. Marginalization of Irrationality in Decision Making 167 9.1 Introduction... 167 9.2 Rational Decision Making... 168 9.3 What is Irrationality?... 170 9.4 Marginalization of Irrationality Theory... 172 9.5 Irrational Decision Making and the Theory of Marginalization of Irrationality in Decision Making... 173 9.6 Application of the Marginalization of Irrationality Theory for Breast Cancer Diagnosis... 175 9.6.1 MLP... 176 9.6.2 RBF... 178 9.6.3 Auto-associative neural network based on the MLP... 179 9.6.4 Auto-associative network based on the RBF... 180 9.7 Conclusions... 182 References... 182 10. Conclusions and Further Work 187 10.1 Introduction... 187 10.2 Way Forward... 188 References... 189 Index 191