Geometric Neurodynamical Classifiers Applied to Breast Cancer Detection Tijana T. Ivancevic Thesis submitted for the Degree of Doctor of Philosophy in Applied Mathematics at The University of Adelaide Discipline of Applied Mathematics School of Mathematical Sciences February 2008 Amended May 2008
0.0.1 Abstract................................. VI 0.1 Statement of Originality........................ VIII 0.2 List of Publications of Tijana T. Ivancevic........ IX 0.2.1 Co Authored Books....................... IX 0.2.2 Other Publications........................ X 0.3 Acknowledgments.............................. XI 1 Introduction and Overview...................... 1 1.1 Rationale of This Thesis........................ 1 1.2 Artificial Neural Networks....................... 4 1.2.1 The von Neumann Machine and the Symbolic Paradigm.................... 4 1.2.2 Analogy with the Brain.................... 6 1.2.3 Design.................................. 9 1.2.4 Feedforward Neural Networks.............. 16 1.2.5 Hopfield Networks........................ 18 1.2.6 Main Applications of ANNs................ 24 1.3 Introduction to Breast Cancer................... 26 1.3.1 Structure and Function of the Breast........ 26 1.3.2 Breast Cancer............................ 26 1.3.3 Causes and Risk of Breast Cancer........... 27 1.3.4 Early Detection of Breast Cancer........... 28 1.3.5 Diagnosis of Breast Change................ 28 1.4 Review of ANNs Application to Detection of Breast Cancer....................................... 29
IV 1.4.1 Studies in Improving the Performance of Screening Programs and Cost Reduction with Computer Aided Diagnosis................. 29 1.4.2 Classification Studies...................... 56 1.4.3 Prognosis Studies: Outcome and/or Prediction of Breast Cancer................ 64 2 Novel Geometric Dynamical Classifiers.......... 73 2.1 Tensorial Neurodynamics....................... 73 2.1.1 Mathematical Basis of Neurodynamics....... 73 2.1.2 Classical Neurodynamical Systems.......... 84 2.1.3 On Neurodynamical Stability............... 89 2.2 GBAM Neurodynamical Classifier................ 93 2.3 Lie-Derivative Neurodynamical Classifier.......... 100 2.3.1 Lie Derivative Operator in Differential Geometry................................ 100 2.3.2 Lie Derivatives of Various Tensor Fields...... 101 2.3.3 Self-Organizing Lie-Derivative Neuro-Classifier 103 2.4 Lie-Poisson Neurodynamical Classifier............ 106 2.4.1 Mathematical Basis of Lie-Poisson Dynamics. 106 2.4.2 Self-Organizing Lie Poisson Neuro-Classifier.. 113 2.5 Fuzzy Associative Dynamical Classifier........... 115 2.5.1 The Concept of Fuzziness.................. 115 2.5.2 Fuzzy Logic.............................. 125 2.5.3 Fuzzy Matrix Classifier-System Architecture.. 132 3 Simulation Results Applied to Breast Cancer Detection....................................... 141 3.1 Experimental Results Using the GBAM Classifier.. 141 3.2 Experimental Results Using the Lie-Derivative Classifier..................................... 144 3.3 Experimental Results Using the Lie-Poisson Classifier145 3.4 Experimental Results Using the FAM Classifier............................. 147 3.4.1 FAM Simulation Results Using the Database from the University of Wisconsin............ 147
V 3.4.2 FAM Simulation Results Using the Mammography Database.................. 148 3.5 Experimental results Using MLP Trained with Backpropagation.......................... 148 3.5.1 Data Test on MLP and Comparison with Geometric Dynamical Classifiers............ 148 4 Appendix: Mathematica Simulation Notebooks.. 151 4.1 Introductory Example: 3D Associative Memory Simulation............... 151 4.2 The GBAM Classifier Simulation................. 153 4.3 The Lie-Derivative Classifier Simulation........... 155 4.3.1 Lie-Linear Neuro-Classifier................. 156 4.3.2 Lie-Quadratic Neuro-Classifier.............. 159 4.4 The Lie-Poisson Classifier Simulation............. 162 4.4.1 The Lie-Poisson Neuro-Classifier with Lie-Poisson Learning...................... 162 4.4.2 The Lie-Poisson Neuro-Classifier with Differential Hebbian Learning.............. 166 4.5 General Fuzzy Logic Simulation.................. 169 4.5.1 Constructing Fuzzy Membership Functions... 169 4.5.2 The FAM Matrix Inference Simulation....... 172 4.5.3 The FAM Classifier Simulation............. 178 5 References...................................... 189
VI 0.0.1 Abstract This thesis proposes four novel geometric neurodynamical classifier models, namely GBAM, Lie-derivative, Lie-Poisson, and FAM, applied to breast cancer detection. All these models have been published in a paper and/or in a book form. All theoretical material of this thesis (Chapter 2) has been published in my monographs (see my publication list), as follows: 2.1 Tensorial Neurodynamics has been published in Natural Biodynamics (Chapters 3, 5 and 7), Geometrical Dynamics of Complex Systems; (Chapter 1 and Appendix), 2006) as well as Applied Differential Geometry: A Modern Introduction (Chapter 3) 2.2 GBAM Neurodynamical Classifier has been published in Natural Biodynamics (Chapter 7) and Neuro-Fuzzy Associative Machinery for Comprehensive Brain and Cognition Modelling (Chapter 3), as well as in the KES Conference paper with the same title; 2.3 Lie-Derivative Neurodynamical Classifier has been published in Geometrical Dynamics of Complex Systems; (Chapter 1) and Applied Differential Geometry: A Modern Introduction (Chapter 3); 2.4 Lie-Poisson Neurodynamical Classifier has been published in Geometrical Dynamics of Complex Systems; (Chapter 1) and Applied Differential Geometry: A Modern Introduction (Chapter 3); 2.5 Fuzzy Associative Dynamical Classifier has been published in Neuro-Fuzzy Associative Machinery for Comprehensive Brain and Cognition Modelling (Chapter 4), as well as in the KES Conference paper with the same title. Besides, Section 1.2 Artificial Neural Networks has been published in Natural Biodynamics (Chapter 7) and Neuro-Fuzzy Associative Machinery for Comprehensive Brain and Cognition Modelling (Chapter 3). Also, Sections 4.1. and 4.5. have partially been published in Neuro-Fuzzy Associative Machinery for Comprehensive Brain and Cognition Modelling (Chapters 3 and 4, respectively) and in the corresponding KES Conference papers.
VII A. The GBAM (generalized bidirectional associative memory) classifier is a neurodynamical, tensor-invariant classifier based on Riemannian geometry. The GBAM is a tensor-field system resembling a two-phase biological neural oscillator in which an excitatory neural field excites an inhibitory neural field, which reciprocally inhibits the excitatory one. This is a new generalization of Kosko s BAM neural network, with a new biological (oscillatory, i.e., excitatory/inhibitory) interpretation. The model includes two nonlinearly-coupled (yet non-chaotic and Lyapunov stable) subsystems, activation dynamics and self-organized learning dynamics, including a symmetric synaptic 2-dimensional tensorfield, updated by differential Hebbian associative learning innovations. Biologically, the GBAM describes interacting excitatory and inhibitory populations of neurons found in the cerebellum, olfactory cortex, and neocortex, all representing the basic mechanisms for the generation of oscillating (EEG-monitored) activity in the brain. B. Lie-derivative neurodynamical classifier is an associativememory, tensor-invariant neuro-classifier, based on the Lie-derivative operator from geometry of smooth manifolds. C. Lie-Poisson neurodynamical classifier is an associative-memory, tensor-invariant neuro-classifier based on the Lie-Poisson bracket from the generalized symplectic geometry. D. The FAM-matrix (fuzzy associative memory) dynamical classifier is a fuzzy-logic classifier based on a FAM-matrix (fuzzy phase-plane). All models are formulated and simulated in Mathematica computer algebra system. All models are applied to breast cancer detection, using the database from the University of Wisconsin and Mammography database. Classification results outperformed those obtained with standard MLP trained with backpropagation algorithm.
VIII 0.1 Statement of Originality This work contains no material which has been accepted for the award of any other degree or diploma in any university or other tertiary institution and, to the best of my knowledge and belief, contains no material previously published or written by another person, except where due reference has been made in the text. I give consent to this copy of my thesis being made available in the University Library. The author acknowledges that copyright of published works contained within this thesis (as listed below) resides with the copyright holders of those works. Tijana T. Ivancevic Date:
0.2 List of Publications of Tijana T. Ivancevic IX 0.2 List of Publications of Tijana T. Ivancevic 0.2.1 Co Authored Books 1. V. Ivancevic & T. Ivancevic: Human-Like Biomechanics: A Unified Mathematical Approach to Human Biomechanics and Humanoid Robotics (Springer, Dec. 2005) 2. V. Ivancevic & T. Ivancevic: Natural Biodynamics (World Scientific, 2006) 3. V. Ivancevic & T. Ivancevic: Geometrical Dynamics of Complex Systems: A Unified Modelling Approach to Physics, Control, Biomechanics, Neurodynamics and Psycho-Socio-Economical Dynamics (Springer, 2006) 4. V. Ivancevic & T. Ivancevic: High-Dimensional Chaotic and Attractor Systems (Springer, 2006) 5. V. Ivancevic & T. Ivancevic: Neuro-Fuzzy Associative Machinery for Comprehensive Brain and Cognition Modelling (Springer, 2007) 6. V. Ivancevic & T. Ivancevic: Computational Mind: A Complex Dynamics Perspective (Springer, 2007) 7. V. Ivancevic & T. Ivancevic: Applied Differential Geometry: A Modern Introduction (World Scientific, 2007) 8. V. Ivancevic & T. Ivancevic: Complex Dynamics: Advanced System Dynamics in Complex Variables (Springer, 2007) 9. V. Ivancevic & T. Ivancevic: Complex Nonlinearity: Chaos, Phase Transitions, Topology Change and Path Integrals (Springer, in press) 10. V. Ivancevic & T. Ivancevic: Quantum Leap: from Dirac and Feynman, across the Universe, to Human Body and Mind (World Scientific, in press)
X 0.2.2 Other Publications V. Ivancevic and T. Ivancevic: Human vs. Humanoid Biodynamics, Int. J. Humanoid Robotics (to appear) T. Ivancevic, C. Pearce, M. Bottema, L. Jain, A Differential Geometry Based Neurodynamical Classifier, J. Facta Universitatis, Series: Mechanics, Automatic Control and Robotics (in press) T. Ivancevic, B. Jovanovic, S. Djukic, M. Djukic, S. Markovic, N. Ivancevic: Tennis Champion of the Future: Revealing the Secrets of Future Sport Science. Union of Handball Camps, Novi Sad, Serbia, (2008) T. Ivancevic, L.C. Jain, M. Bottema: A New Two feature GBAM Neuro-dynamical Classifier for Breast Cancer Diagnosis. Proc. KES 99, IEEE Press, (1999) T. Ivancevic, L.C. Jain, M. Bottema: A New Two Feature FAM Matrix Classifier for Breast Cancer Diagnosis. Proc. KES 99, IEEE Press, (1999) T. Ivancevic: Some Possibilities of Multilayered Neural Networks Application in Biomechanics of Muscular Contractions, Human Motion and Sports Training. Master Thesis (in Serbian), Univ. Novi Sad, YU, (1995) V. Ivancevic, L. Lukman and T. Ivancevic: Selected Chapters in Human Biomechanics, University of Novi Sad, 1995 (in Serbian, awarded textbook, undergraduate level) V. Ivancevic & T. Ivancevic: Biomechanical Basis of Sports - Review with Software Implementation, Faculty of Physical Education, University of Novi Sad, 1994 (in Serbian) Tijana Jovanovic: Contribution to Formulation of a General Model of Control, Adaptation and Learning in Biomechanical Systems. Graduate Thesis, Univ. Novi Sad, YU, 1991 (in Serbian)
0.3 Acknowledgments XI 0.3 Acknowledgments This thesis would never have been completed without the assistance of people and institutions I would like to thank. I have particular gratitude to the University of Adelaide and the School of Mathematical Sciences for the Adelaide University Scholarship and other technical support. I wish to express my gratitude and appreciation to my Supervisor, Professor Charles E.M. Pearce, School of Mathematical Sciences, the University of Adelaide, as well as my Co-supervisor, Dr. Murk Bottema, School of Informatics & Engineering, Flinders University of South Australia. Their advice and support is highly appreciated.