Neural Turing Machines

Transcription

1 Neural Turing Machines Can neural nets learn programs? Alex Graves Greg Wayne Ivo Danihelka

2

3

4 Contents 1. IntroducBon 2. FoundaBonal Research 3. Neural Turing Machines 4. Experiments 5. Conclusions

5 IntroducBon First applicabon of Machine Learning to logical flow and external memory

6 IntroducBon First applicabon of Machine Learning to logical flow and external memory Extend the capabilibes of neural networks by coupling them to external memory

7 IntroducBon First applicabon of Machine Learning to logical flow and external memory Extend the capabilibes of neural networks by coupling them to external memory Analogous to TM coupling a finite state machine to infinite tape

8 IntroducBon First applicabon of Machine Learning to logical flow and external memory Extend the capabilibes of neural networks by coupling them to external memory Analogous to TM coupling a finite state machine to infinite tape RNN s have been shown to be Turing- Complete, Siegelmann et al 95

9 IntroducBon First applicabon of Machine Learning to logical flow and external memory Extend the capabilibes of neural networks by coupling them to external memory Analogous to TM coupling a finite state machine to infinite tape RNN s have been shown to be Turing- Complete, Siegelmann et al 95 Unlike TM, NTM is completely differenbable

10 FoundaBonal Research Neuroscience and Psychology Concept of working memory : short- term memory storage and rule based manipulabon Also known as rapidly created variables

11 FoundaBonal Research Neuroscience and Psychology Concept of working memory : short- term memory storage and rule based manipulabon Also known as rapidly created variables ObservaBonal neuroscience results in the pre- frontal cortex and basal ganglia of monkeys

12 FoundaBonal Research Neuroscience and Psychology CogniBve Science and LinguisBcs AI and CogniBve Science were contemporaneous in 1950 s s

13 FoundaBonal Research Neuroscience and Psychology CogniBve Science and LinguisBcs AI and CogniBve Science were contemporaneous in 1950 s s Two fields parted ways when neural nets received cribcism, Fodor et al. 88 Incapable of variable- binding eg Mary spoke to John Incapable of handling variable sized input

14 FoundaBonal Research Neuroscience and Psychology CogniBve Science and LinguisBcs AI and CogniBve Science were contemporaneous in 1950 s s Two fields parted ways when neural nets received cribcism, Fodor et al. 88 MoBvated Recurrent Networks research to handle variable binding and variable length input

15 FoundaBonal Research Neuroscience and Psychology CogniBve Science and LinguisBcs AI and CogniBve Science were contemporaneous in 1950 s s Two fields parted ways when neural nets received cribcism, Fodor et al. 88 MoBvated Recurrent Networks research to handle variable binding and variable length input Recursive processing hot debate topic in role inhuman evolubon (Pinker vs Chomsky)

16 FoundaBonal Research Neuroscience and Psychology CogniBve Science ad LinguisBcs Recurrent Neural networks Broad class of machines with distributed and dynamic state

17 FoundaBonal Research Neuroscience and Psychology CogniBve Science ad LinguisBcs Recurrent Neural networks Broad class of machines with distributed and dynamic state Long Short Term Memory RNN s designed to handle vanishing and exploding gradient

18 FoundaBonal Research Neuroscience and Psychology CogniBve Science ad LinguisBcs Recurrent Neural networks Broad class of machines with distributed and dynamic state Long Short Term Memory RNN s designed to handle vanishing and exploding gradient NaBvely handle variable length structures

19 Neural Turing Machines

20 Neural Turing Machines

21 Neural Turing Machines 1. Reading M t is NxM matrix of memory at Bme t

22 Neural Turing Machines 1. Reading M t is NxM matrix of memory at Bme t w t

23 Neural Turing Machines 1. Reading 2. WriBng involves both erasing and adding

24 Neural Turing Machines 1. Reading 2. WriBng involves both erasing and adding

25 Neural Turing Machines 1. Reading 2. WriBng involves both erasing and adding 3. Addressing

26 Neural Turing Machines 3. Addressing 1. Focusing by Content Each head produces key vector k t of length M Generated a content based weight w t c based on similarity measure, using key strength β t

27 Neural Turing Machines 3. Addressing 2. InterpolaBon Each head emits a scalar interpolabon gate g t

28 Neural Turing Machines 3. Addressing 3. ConvoluBonal shif Each head emits a distribubon over allowable integer shifs s t

29 Neural Turing Machines 3. Addressing 4. Sharpening Each head emits a scalar sharpening parameter γ t

30 Neural Turing Machines 3. Addressing (puhng it all together)

31 Neural Turing Machines 3. Addressing (puhng it all together) This can operate in three complementary modes A weighbng can be chosen by the content system without any modificabon by the locabon system

32 Neural Turing Machines 3. Addressing (puhng it all together) This can operate in three complementary modes A weighbng can be chosen by the content system without any modificabon by the locabon system A weighbng produced by the content addressing system can be chosen and then shifed

33 Neural Turing Machines 3. Addressing (puhng it all together) This can operate in three complementary modes A weighbng can be chosen by the content system without any modificabon by the locabon system A weighbng produced by the content addressing system can be chosen and then shifed A weighbng from the previous Bme step can be rotated without any input from the content- based addressing system

34 Neural Turing Machines Controller Network Architecture Feed Forward vs Recurrent

35 Neural Turing Machines Controller Network Architecture Feed Forward vs Recurrent The LSTM version of RNN has own internal memory complementary to M

36 Neural Turing Machines Controller Network Architecture Feed Forward vs Recurrent The LSTM version of RNN has own internal memory complementary to M Hidden LSTM layers are like registers in processor

37 Neural Turing Machines Controller Network Architecture Feed Forward vs Recurrent The LSTM version of RNN has own internal memory complementary to M Hidden LSTM layers are like registers in processor Allows for mix of informabon across mulbple Bme- steps

38 Neural Turing Machines Controller Network Architecture Feed Forward vs Recurrent The LSTM version of RNN has own internal memory complementary to M Hidden LSTM layers are like registers in processor Allows for mix of informabon across mulbple Bme- steps Feed Forward has bejer transparency

39 Experiments Test NTM s ability to learn simple algorithms like copying and sorbng

40 Experiments Test NTM s ability to learn simple algorithms like copying and sorbng Demonstrate that solubons generalize well beyond the range of training

41 Experiments Test NTM s ability to learn simple algorithms like copying and sorbng Demonstrate that solubons generalize well beyond the range of training Tests three architectures NTM with feed forward controller NTM with LSTM controller Standard LSTM network

42 Experiments 1. Copy Tests whether NTM can store and retrieve data Trained to copy sequences of 8 bit vectors Sequences vary between 1-20 vectors

43 Experiments 1. Copy

44 Experiments 1. Copy NTM

45 Experiments 1. Copy LSTM

46 Experiments 1. Copy

47 Experiments 2. Repeat Copy Tests whether NTM can learn simple nested funcbon Extend copy by repeatedly copying input specified number of Bmes Training is a random- length sequence of 8 bit binary inputs plus a scalar value for # of copies Scalar value is random between 1-10

48 Experiments 2. Repeat Copy

49 Experiments 2. Repeat Copy

50 Experiments 2. Repeat Copy

51 Experiments 3. AssociaBve Recall Tests NTM s ability to associate data references Training input is list of items, followed by a query item Output is subsequent item in list Each item is a three sequence 6- bit binary vector Each episode has between two and six items

52 Experiments 3. AssociaBve Recall

53 Experiments 3. AssociaBve Recall

54 Experiments 3. AssociaBve Recall

55 Experiments 4. Dynamic N- Grams Test whether NTM could rapidly adapt to new predicbve distribubons Trained on 6- gram binary pajern on 200 bit sequences Can NTM learn opbmal esbmator

56 Experiments 4. Dynamic N- Grams

57 Experiments 4. Dynamic N- Grams

58 Experiments 4. Dynamic N- Grams

59 Experiments 5. Priority Sort Tests whether NTM can sort data Input is sequence of 20 random binary vectors, each with a scalar rabng drawn from [- 1, 1] Target sequence is 16- highest priority vectors

60 Experiments 5. Priority Sort

61 Experiments 5. Priority Sort

62 Experiments 5. Priority Sort

63 Experiments 6. Details RMSProp algorithm Momentum 0.9 All LSTM s had three stacked hidden layers

64 Experiments 6. Details

65 Experiments 6. Details

66 Experiments 6. Details

67 Conclusion Introduced an neural net architecture with external memory that is differenbable end- to- end Experiments demonstrate that NTM are capable of leaning simple algorithms and are capable of generalizing beyond training regime

68