Marvin Minsky - the father of AI

Transcription

1 Marvin Minsky - the father of AI Johannes Verwijnen Seminar report UNIVERSITY OF HELSINKI Department of Computer Science Helsinki, April 27, 2016

2 Contents 1 Early life 1 2 Undergraduate studies at Harvard 2 3 Graduate studies at Princeton 4 4 Back to Harvard 6 5 At MIT 8 References 9 i

3 1 Early life Minsky was born in New York City on August 9, 1927 to Henry, an eye surgeon and future head of the ophthalmology department at Mount Sinai Hospital, and Fannie, of whom we only know that she was active in Zionist affairs. They lived in Manhattan for Minsky s first years in a home filled with lenses, prisms and diaphragms that Minsky took apart [8]. He showed exceptional intelligence already at the age of five as he tried to explain the nonoptimality of the standard answer to an intelligence test to the examiner. Based on this test he was sent to an experimental school for gifted children. The school had obligatory lessons in tap dancing, which Minsky demurred. Minsky s family moved to Riverdale, Bronx, and he was sent to a local public school. He disliked this school even more, as he was bullied and his teacher wanted him to repeat the third grade due to bad handwriting. Fortunately his parents intervened Figure 1: Marvin Minsky, courtesy MIT Museum and sent him to the local progressive private Ethical Culture Fieldston School, previously also attended by J. Robert Oppenheimer. While attending fifth grade at Fieldston, Minsky was already interested in electronics and organic chemistry thanks to his science teacher, Herbert Zim. Zim s teaching methods might have had a great influence on Minsky s choices of interest later on: I had been reading chemistry books, and I thought it would be nice to make some chemicals. In particular, I had read about ethyl mercaptan, which interested me because it was said to be the worst-smelling thing around. I went to Zim and told him that I wanted to make some. He said, Sure. How do you plan to do it? We talked about it for a while, and he convinced me that it 1

4 we were going to be thorough we should first make ethanol, from which we were to make ethyl chloride. I did make the ethanol and then the ethyl chloride, which instantly disappeared. It s about the most volatile thing thing there is. I think Zim had fooled me into doing this synthesis knowing that the product would evaporate before I actually got to make that awful mercaptan. I remember being sort of mad, and deciding that chemistry was harder than it looked on paper, because when you synthesize something it can just disappear. (Minsky in [1]) In 1941 after successfully finishing eighth grade at Fieldston Minsky continued his education at the Bronx High School of Science, now well-known for its academics, which was just founded in Minsky made many important connections during his studies, his schoolmates included Russell Kirsch, Anthony Oettinger and Frank Rosenblatt. His parents sent him to Phillips Academy in Androver, Massachusetts, for his final year which was a disappointment for Minsky, as he wasn t able to concentrate exclusively on science there. Upon graduation in 1945, the Second World War was still ongoing and Minsky enlisted in the Navy. He was sent to the Great Lakes Naval Training Center and after basic training found himself in radar school with a few other highly intelligent peers. Our little group was a strange kind of mini-harvard in the middle of the Navy., said Minsky [1]. As the war ended after a few months in the training center, the group didn t have much to do and Minsky was discharged in Undergraduate studies at Harvard Minsky went on to Harvard as a physics major and took freshman physics and advanced calculus. He was astonished at the ease of the coursework compared to what it was like at Bronx Science and also took courses in sociology and psychology and got interested in neurology. Zoology professor John Welsh let him do some laboratory work in solving how nerves in a crayfish claw worked. At one point, I had a crayfish claw mounted on an 2

5 apparatus in such a way that I could operate the individual nerves. I could get the several-jointed claw to reach down and pick up a pencil and wave it around. This got Minsky interested in instrumentation [1]. During his last year in high school Minsky had started to think about thinking. His motivation lied in his experience of learning mathematics. You take an hour a page to read this thing, and it still doesn t make any sense. Then, suddenly, it becomes so easy it seems trivial. [1] He invented some reinforcement theories and read B. F. Skinner s theories, which he disliked because of their lack of internal ideas. While Minsky was at Harvard the psychology laboratory was in the basement of Memorial Hall and he started hanging out there. During that time the basement was inhabited by B. F. Skinner and his group, George Miller, J. C. R. Licklider, and Georg von Békésy. Frustrated with the lack of imagination by the graduate students Minsky came up with an idea that the brain was composed of little relays, each having a probability to conduct an electric pulse. He tried to explain Skinner s results by a reward scheme that would change these probabilities to favor learning. This view is nowadays known as Hebbian learning, named after Donald Hebb, who came up with the idea earlier and published it in 1949 as The Organization of Behavior. In the late 1940s, still as an undergraduate, Minsky had laboratories in the psychology and biology department and was doing experimental work in optics in the physics department. His coursework included musical composition courses, taught by Irving Fine. Minsky s father was a painter and musician, but Minsky saw himself as more of an improviser than formal musician. Because of his low grades he wanted to make up for them by doing an undergraduate thesis. At Harvard this wasn t possible in physics, but the mathematics department had that option, so he switched majors. At this point he had taken enough mathematics courses for this not to be a problem. Minsky got acquainted with Andrew M. Gleason during his first years at Harvard. Gleason had broken German and Japanese codes for the Navy during the war and was a junior fellow at Harvard when Minsky met him. He was mesmerized by Gleason s nine-year plan to solve Hilbert s fifth problem (previously worked on by another acquaintance of this seminar, John von Neumann, for compact groups), in which he succeeded with Deane 3

6 Montgomery and Leo Zippin in During his senior year Minsky decided to tackle Brouwer s fixed-point theorem after reading Kakutani s generalization thereof. He was convinced that Kakutani had not gotten the most general result out of this logic and successfully wrote proofs in higher dimensions using the topology of knots. After reading the proof, Gleason called Minsky a mathematician, but Minsky never published his proof. At the time, I was influenced by the example of my father. When he made a surgical discovery, he would take six or seven years to write it up - correcting it and doing many more operations to make sure he was right. I felt that a successful scientist might publish three of four real discoveries in his lifetime, and should not load up the airwaves with partial results. I still feel that way. I don t like to take some little discovery and make a whole paper our of it. When I make a little discovery, either I forget about it or I wait until I have several things that fit together before I write them up. (Minsky in [1]) 3 Graduate studies at Princeton After graduation Minsky went to Princeton s mathematics department for his graduate studies. The department was run by Solomon Lefschetz and only admitted a handful of students each year, mostly by invitation. During his studies Minsky once got a peek at his transcript and found all A s in his coursework, many of which he had never taken. In Lefschetz s opinion one was either a mathematician or one wasn t and thus there were no exams at the department. While at Harvard, Minsky started planning an electronic machine that could learn. He read a paper by McCulloch and Pitts [5] that presented the idea of artificial neurons connected to carry out mental processes. While at Princeton he met Dean Edmonds, who was an electronics whizz, and talked him into trying to build this machine. They figured out a way to build the machine, consisting of interconnected circuits with loops and cycles, with 4

7 just six vacuum tubes and a motor per neuron and Minsky thought that the machine might learn to run mazes, like rats in Skinner s lab. George Miller, from the Harvard psychology department, managed to get funding for the idea from the Office of Naval Research and in the summer of 1951 the two graduate students went to Harvard and built the machine, the Stochastic Neural Analog Reinforcement Calculator (SNARC) - one of the first electronic learning machines [15]. Figure 2: One of SNARC s neurons. Image courtesy Gregory Loan. Minsky wanted to add a secondary memory to the network, making it able to predict the outcome of a choice. I had the naïve idea that if one could build a big enough network, with enough memory loops, it might get lucky and acquire the ability to envision things in its head [1]. Howard Aiken had taught a course in computers at Harvard that Minsky had taken so the though of simulating neurons using computers was not alien to him. However, Aiken s and later von Neumann s computers were too small to do anything interesting learning-wise. In 1953 a new systems analysis department was to be started at Tufts University and Minsky welcomed the idea to get back to Boston, so he joined them and finished his thesis there. For his dissertation Minsky researched on how a brain model based on neural networks might learn [11]. He had help from fellow graduate students Lloyd Shapley and John Nash as well as John von Neumann, who sat on his PhD committee. Once presented with the 5

8 dissertation in 1954 A. W. Tucker, the chair of the mathematics department at the time, went to von Neumann and said, This seems like very interesting work, but I can t evaluate it. I don t know whether it should really be called mathematics. Von Neumann replied, Well, if it isn t now, it will be someday - let s encourage it. [1] 4 Back to Harvard Andrew Gleason nominated Minsky as a junior fellow at Harvard and was supported by Claude Shannon, von Neumann and Norbert Wiener. The fellowship had very few responsibilities [8] which permitted him to make general theories about intelligence that didn t fit into any department. Minsky was disappointed in the microscopic technology of the times, since he couldn t make out systems of nerve cells. So he invented the confocal scanning microscope by shooting a zirconium arc light source through a pinhole, reading the reflected amount of light by a low-noise photomultiplier, displaying it on a military surplus long-persistence radar screen while moving the sample [8]. Minsky knew John McCarthy, now professor of mathematics at Dartmouth, from graduate school in Princeton and had worked for Claude Shannon at Bell Telephone Laboratories in the summer of In the spring of 1956 he helped them and Nathaniel Rochester from IBM to organize the Dartmouth Summer Research Project on Artificial Intelligence. Attendees included Trenchard More, Arthur Samuel, Ray Solomonoff and Oliver Selfridge, among others.[15] At this point Rochester had already implemented a neural network model with several hundred neurons based on Hebbian learning that ran on an IBM 701 and tried to find evidence of learning from a cubic foot of printouts of the network state. [...]If one didn t know what to look for [in the printouts] one might miss any evidence of self-organization of these nets, even if it did take place. I think that that is what I had been worried about when I decided not to use computers to study some of the ideas connected with my thesis, Minsky recalls [1]. Minsky spent most of the seminar trying to figure out a method for a 6

9 Figure 3: Trenchard More, John McCarthy, Marvin Minsky, Oliver Selfridge, and Ray Solomonoff at the 50th anniversary of the Dartmouth Summer Research Project in 2006 machine to create proofs for theorems. Simulating a machine on pen and paper he found a new proof for the base angle equality in an isosceles triangle by equality of two different triangles. Rochester was impressed by this proof and later hired Herbert Gelernter to further work on the problem. Once McCarthy had the idea to combine some of the ideas of Rand Corporation s programming language IPL with IBM s FORTRAN, Gelernter created FLPL (FORTRAN List-Processing Language), which then evolved into LISP. By 1959 Gelernter finished his Geometric Themorem Prover [4], which when given the job to find a proof for the base angle equality of an isosceles triangle, found Pappus proof. 7

10 5 At MIT After his fellowship, in 1957, Minsky went to work with Oliver Selfridge for MIT s Lincoln Laboratory. The year after that he was hired as an assistant professor by MIT s mathematics department and, together with John McCarthy who had also joined MIT s faculty, started the A.I. Group. Minsky s Bronx Science classmate Rosenblatt designed the first system to do sophisticated pattern matching, the Perceptron[14], which ran on an IBM 704 at Cornell Aeronautical Laboratory. Minsky, however, was not impressed: It remind[ed] me, in some ways, of a wonderful machine that J. C. R. Licklider made at Harvard early in the 1950s. It could recognize the word watermelon no matter who said it in no matter what sentence. With a simple enough recognition problem, almost anything will work with some reliability. [1] Minsky published his thoughts on heuristic methods in Steps Toward Artificial Intelligence [6]. McCulloch brought Seymour Papert, a former associate of Jean Piaget, together with Minsky in 1963 and within a few months the had started new research projects in human perception, child psychology, experimental robots and theory of computation. Their best known work is the book Perceptrons that was published in 1969 [9] and consists of a collections of proofs on the limits of the technology. There had been several thousand papers published on Perceptrons up to 1969, but our book put a stop to those. [1] The A.I. Group got about a million dollars of funding from ARPA in 1963 mostly thanks to work done on time-sharing by McCarthy and Corbató. Many of the students hired by the lab evolved from dropouts from other fields, especially mathematics and physics, to distinguished scientists. Minsky s idea was to use these talents to learn what computers could do in solving non-arithmetic problems. One of Minsky s students, James Slagle, worked on a symbolic automatic integrator (SAINT) as his thesis work [16] and two other students, William Martin and Joel Moses later developed that into MACSYMA - a groundbreaking computer algebra system[12]. Tom Evans s thesis was about making a computer learn to answer geometric analogies, like ones often used in I.Q. tests [3]. Two other themes Minsky s group was interested in were computer lin- 8

11 guistics and robotics. Bertram Raphael created a Semantic Information Retrieval -program [13] that could learn hierarchies of knowledge, whereas Daniel Bobrow worked on a system that would combine language and mathematics and thus solve word problems [2]. Henry Ernst created the A. I. Group s first computer-controlled robot arm. Most of these PhD theses were republished in Minsky s Semantic Information Processing [10]. Minsky also published a textbook in theory of computation in addition to several papers in the field. In the early 70s Minsky and Papert started working on the Society of Mind theory, that tries to explain how intelligence emerges from simpler actions. Minsky s theories are published in the book The Society of Mind [7], which actually is a collection of a few hundred one-page essays on different parts of the theory. Unfortunately Minsky died January 24, 2016 References [1] Bernstein, Jeremy: A.I. New Yorker, December 14th [2] Bobrow, Daniel G.: A heuristic program that solves symbolic integration problems in freshman calculus : symbolic automatic integrator (SAINT). PhD thesis, Massachusetts Institute of Technology, [3] Evans, Thomas G.: A Heuristic Program to Solve Geometric-Analogy Problems. PhD thesis, Massachusetts Institute of Technology, [4] Gelernter, Herbert: Realization of a geometry theorem proving machine. In Oldenbourg, R. (editor): Proceedings of the International Conference on Information Processing, pages , Unesco, Paris, [5] McCulloch, Warren S. and Walter Pitts: A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4): , ISSN BF [6] Minsky, Marvin: Steps toward artificial intelligence. Proceedings of the IRE, 49(1):8 30,

12 [7] Minsky, Marvin: Society of mind. Simon and Schuster, [8] Minsky, Marvin: Memoir on inventing the confocal scanning microscope. Scanning, 10(4): , [9] Minsky, Marvin and Seymour Papert: Perceptrons [10] Minsky, Marvin L.: Semantic Information Processing. The MIT Press, 1969, ISBN [11] Minsky, Marvin Lee: Theory of Neural-Analog Reinforcement Systems and Its Application to the Brain Model Problem. PhD thesis, Princeton University, [12] Moses, Joel: Macsyma: A personal history. Journal of Symbolic Computation, 47(2): , 2012, ISSN sciencedirect.com/science/article/pii/s [13] Raphael, Bertram: SIR : A Computer Program for Semantic Information Retrieval. PhD thesis, Massachusetts Institute of Technology, [14] Rosenblatt, Frank: The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory, [15] Russell, Stuart and Peter Norvig: Artificial Intelligence - A Modern Approach. Prentice Hall, 2nd edition, [16] Slagle, James R.: Natural Language Input for a Computer Problem Solving System. PhD thesis, Massachusetts Institute of Technology,