MINE 432 Industrial Automation and Robotics

Size: px

Start display at page:

Download "MINE 432 Industrial Automation and Robotics"

Hugo Cummings
6 years ago
Views:

1 MINE 432 Industrial Automation and Robotics Part 3, Lecture 5 Overview of Artificial Neural Networks A. Farzanegan (Visiting Associate Professor) Fall 2014 Norman B. Keevil Institute of Mining Engineering

2 Today s Topics Introduction Biological neurons in humans Artificial Neuron Artificial Neural Networks Applications of ANNs MINE Industrial Automation and Robotics 2

3 Introduction

4 Introduction Artificial neural network (ANN) is a research subject in many scientific and engineering fields including machine learning, paralleled processing, modeling and simulation, image processing, etc. The idea of developing artificial neural networks was inspired by findings in neurosciences and biology about human brain and how it works MINE Industrial Automation and Robotics 4

5 Milestones in ANN Development 1943 McCulloch-Pitts neurons 1949 Hebb s law 1958 Perceptron (Rosenblatt) 1960 Adaline, better learning rule (Widrow, Huff) 1969 Limitations (Minsky, Papert) 1972 Kohonen nets, associative memory 1977 Brain State in a Box (Anderson) 1982 Hopfield net, constraint satisfaction 1985 ART (Carpenter, Grossfield) 1986 Backpropagation (Rumelhart, Hinton, McClelland) 1988 Neocognitron, character recognition (Fukushima) MINE Industrial Automation and Robotics 5

6 ANN as part of MATLAB Math, Statistics and Optimization Apps ANN Toolbox was added to MATLAB in 1992 MINE Industrial Automation and Robotics 6

7 Biological Neurons in Humans

Neurons can also be found in human spinal cord and peripheral nervous system.

8 Biological Neurons in Humans Most of neurons in human s body are located in brain. It is estimated that human s brain contains around 100 billions of interconnected neurons. Neurons can also be found in human spinal cord and peripheral nervous system. There are different types of neurons in humans including sensory neurons, brain and spinal cord neurons and motor neurons. MINE Industrial Automation and Robotics 8

9 Neurons Neurons gather and transmit electrochemical signals. They have the same characteristics and parts as other cells, but the electrochemical aspect lets them transmit signals over long distances (up to several feet or a few meters) and pass messages to each other. MINE Industrial Automation and Robotics 9

10 Neurons Network Neurons in brain are connected to each other at synapses. Signals are passed from one neuron to others through axons and terminal point synapse. MINE Industrial Automation and Robotics 10

11 Artificial Neurons

12 Artificial Neurons (Perceptron) An artificial neuron as the name suggests is artificial and process its input data much similar to a biological neuron such as human brain neuron cells. It is also known as perceptron. In brief, an artificial neuron is a piece of computer code which mathematically process numerical data and produces one or more output results. A biological neuron An artificial neuron MINE Industrial Automation and Robotics 12

13 Input Input Data Processing by An Artificial Neuron The weights represent how information being used by the network to solve a problem. x 1 w 1 x 2 x 3 w 2 w 3 z n wi x i 1 i ; y H ( z) Output y x n-1 x n w n-1 w n Activation function Weighted average MINE Industrial Automation and Robotics 13

14 Activation Functions Identity f(x) = x Binary step f(x) = 1 if x >= q f(x) = 0 otherwise Sigmoid f(x) = 1 / (1 + e -sx ) MINE Industrial Automation and Robotics 14

15 Artificial Neural Network (ANN)

16 Why Networks? A single disconnected artificial neuron cannot do much to solve real life problems Similar to human brain neurons, artificial neurons must be connected to each other in order to make a powerful computational network. A human brain can learn from new experiences and after learning can apply its knowledge to solve new problems. Similarly, an artificial neural network can be trained first to learn new rules or models from pre-defined examples, then it can be employed to solve new problems. ANN can learn supervised or unsupervised. MINE Industrial Automation and Robotics 16

17 Artificial Neural Network An artificial neural network is built by connecting several single artificial neurons in a specific structure to process input data. There are various types of ANN. The most classic one is a Multi-Layer Perceptron (MLP) in which a large number of artificial neurons or perceptron are highly interconnected in input, hidden and output layers. MINE Industrial Automation and Robotics 17

18 Types of ANNs Architecture Single layer feedforward Multilayer feedforward Recurrent Learning algorithm to determine connection weights Supervised Unsupervised Reinforcement Activation Function Function to compute output signal from input signal MINE Industrial Automation and Robotics 18

19 Training ANNs Given a set of input data, the goal of an ANN is to compute a correct output ( outputs). The output mainly depends on the weight of connections between artificial neurons and other internal mathematical parameters. x 1 x 2 x i w 1 w 2 y 1 w i If connection weights are not chosen accurately, then the ANN output will be incorrect or inaccurate. Therefore, an ANN needs to be trained and correct weights before it can be used for reliable predictions. x n w n y 2 MINE Industrial Automation and Robotics 19

20 Supervised Learning Learning is performed by presenting patterns with targets During learning, produced output is compared with the desired output The difference between both output is used to modify learning weights according to the learning algorithm Recognizing hand-written digits, pattern recognition, etc. Neural network models: perceptron, feed-forward, radial basis function, support vector machine. MINE Industrial Automation and Robotics 20

21 Unsupervised Learning Targets are not provided Appropriate for clustering task Find similar groups of documents in the web, content addressable memory, clustering. Neural network models: Kohonen, self organizing maps, Hopfield networks. MINE Industrial Automation and Robotics 21

22 Training ANN using Backpropagation The backpropagation algorithm learns (computes) the weights for a multilayer network. It employs a gradient descent to attempt to minimize the squared error between the network output values and the target values for these outputs. The backpropagation tries to minimize sum of the errors over all of the network output units E(w) = ½ (t kd o kd ) 2 d D and k outputs where outputs is the set of output units in the network, and t kd and o kd are the target and output values associated with the kth output unit and training example d. MINE Industrial Automation and Robotics 22

23 Applications of ANNs

24 Problem Types Solved by ANNs when we cannot formulate an algorithmic solution. when we can get lots of examples of the behavior we require. when we need to pick out the structure from existing data. Problem types suitable for ANN application: Storing and recalling patterns Classifying patterns Mapping inputs onto outputs Grouping similar patterns Finding solutions to constrained optimization problems MINE Industrial Automation and Robotics 24

25 Application of ANN in Mining As a new approach to model nonlinear relationships, artificial neural networks have found many applications in mining engineering such as blasting, drilling, rock mechanics and mineral processing, etc. In mining engineering, mostly, a MLP feedforward artificial neural network trained based on error back-propagation algorithm has been used to build a black-box model of relationship between one or several output variables and several inputs. MINE Industrial Automation and Robotics 25

26 ANN to Predict Flotation Efficiency Labidi et al., 2007 MINE Industrial Automation and Robotics 26

27 Defining Inputs and Outputs The first step in applying ANN approach is problem definition in terms of input and output variables and their ranges. Then, example data sets must be prepared for ANN training. The number of example data sets depends on input vector dimension. MINE Industrial Automation and Robotics 27

28 Optimizing ANN Architecture The optimal topologies of the network used to model the flotation stage was determined by a number of trial by varying the number of hidden layers, the number of neurons in each hidden layer, the type of function of neuron activation, the learning rate and momentum. The best neural network formed by two hidden layers each of which is constituted by 100 neurons. The activation function for both layers that gives the best result is the sigmoidal function MINE Industrial Automation and Robotics 28

29 ANN-Based Simulation of Flotation Process MINE Industrial Automation and Robotics 29

30 Simulated vs. Experimental Results MINE Industrial Automation and Robotics 30

31 Applying ANN in Blasting Optimization (Open Pit Mines) Monjezi et al., 2010 MINE Industrial Automation and Robotics 31

32 Optimized ANN Architecture MINE Industrial Automation and Robotics 32

33 ANN Predictions MINE Industrial Automation and Robotics 33

34 Blasting Optimization Based on ANN Simulation Fragmentation improvement: D 80 from 63 cm to 37 cm Flyrock reduction: from 110 m to 30 m MINE Industrial Automation and Robotics 34

35 Questions?

Course Objectives. This course gives a basic neural network architectures and learning rules.

Course Objectives. This course gives a basic neural network architectures and learning rules. Introduction Course Objectives This course gives a basic neural network architectures and learning rules. Emphasis is placed on the mathematical analysis of these networks, on methods of training them