CSE5001(CS417)/ 高级人工智能 Advanced Artificial Intelligence

Transcription

1 CSE5001(CS417)/ 高级人工智能 Advanced Artificial Intelligence Xin Yao Fall 2017 Fall 2017 Artificial Intelligence: Introduction Xin Yao

2 Basic information Instructor: Xin Yao Credits: 3, Hours: 64 Language: Chinese, English Prerequisites: CS303/ 人工智能 Artificial Intelligence Fall 2017 Artificial Intelligence: Introduction Xin Yao

3 Teaching assistants Shaolong Shi ( 师少龙 ) @mail.sustc.edu.cn Room 911, Building A7, Nanshan i-park Yu Zhang ( 张宇 ) @mail.sustc.edu.cn Room 911, Building A7, Nanshan i-park Fall 2017 Artificial Intelligence: Introduction Xin Yao

4 Course description Outline: This course introduces recent advances in artificial intelligence. Topics covered include intelligent optimization and learning, as well as case studies in machine learning and pattern recognition. The assessment in the course will consist of homework assignments, a mid-term test and a final exam. Learning Outcomes: Upon finishing this course, students are expected to have a good understanding of challenging optimization and learning problems in AI, and different models and algorithms for tackling these problems. Teaching Methods: Two-hour lectures every week. Fall 2017 Artificial Intelligence: Introduction Xin Yao

5 Course assessment Homework (30%) Every week a homework will be assigned. Every homework will contribute 2% to the final grade. Late submission (15% for each day late). Mid-term test (20%) Final exam (50%) Fall 2017 Artificial Intelligence: Introduction Xin Yao

6 Course roadmap Week 1: Introduction Week 2: Basic search Week 3: Heuristic search Week 4: General search methods for optimization I Week 5: General search methods for optimization II Week 6: Supervised learning I Week 7: Supervised learning II Week 8: Theories of optimization and learning Fall 2017 Artificial Intelligence: Introduction Xin Yao

7 Course roadmap Week 9: Mid-term test Week 10: Feature engineering Week 11: Unsupervised learning Week 12: Markov decision process Week 13: Reinforcement learning Week 14: Fuzzy logic and fuzzy systems Week 15: Natural language processing Week 16: Pattern recognition and computer vision Fall 2017 Artificial Intelligence: Introduction Xin Yao

8 Recommended literature Books Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (Third edition), Cambridge University Press, The book resources: Reading materials Relevant papers as handed out at each lecture. Fall 2017 Artificial Intelligence: Introduction Xin Yao

9 AI Resources Major journals: Artificial Intelligence, IEEE Trans on Pattern Analysis and Machine Intelligence, International Journal of Computer Vision, Journal of Machine Learning Research, etc. Major conferences: AAAI Conference on Artificial Intelligence, IEEE Conference on Computer Vision and Pattern Recognition, International Conference on Computer Vision, International Conference on Machine Learning, International Joint Conference on Artificial Intelligence, Annual Conference on Neural Information Processing Systems, Annual Meeting of the Association for Computational Linguistics, etc. Fall 2017 Artificial Intelligence: Introduction Xin Yao

10 Webpage Fall 2017 Artificial Intelligence: Introduction Xin Yao

11 Introduction Xin Yao Fall 2017 Fall 2017 Artificial Intelligence: Introduction Xin Yao

12 Outline What is AI? The Foundation of AI The History of AI The Application of AI Summary Fall 2017 Artificial Intelligence: Introduction 1

13 What is AI? Fall 2017 Artificial Intelligence: Introduction 2

14 Definition of AI Intelligence: The ability to learn and solve problems Webster s Dictionary. Artificial intelligence (AI) is the intelligence exhibited by machines or software Wikipedia. The science and engineering of making intelligent machines McCarthy. The study and design of intelligent agents, where an intelligent agent is a system that perceives its environment and takes actions that maximize its chances of success. Russell and Norvig AI book. Fall 2017 Artificial Intelligence: Introduction 3

15 Why AI? Just as the Industrial Revolution freed up a lot of humanity from physical drudgery, I think AI has the potential to free up humanity from a lot of the mental drudgery. Andrew Ng. Fall 2017 Artificial Intelligence: Introduction 4

16 What is AI? Fall 2017 Artificial Intelligence: Introduction 5

17 Thinking humanly: cognitive approach Requires to determine how humans think! 1960 s cognitive revolution. Requires scientific theories of internal activities of the brain What level of abstraction? Knowledge or circuits? How to validate? Today, Cognitive Science and Artificial Intelligence are distinct disciplines. Fall 2017 Artificial Intelligence: Introduction 6

18 Acting humanly: Turing test Turing test (Alan Turing 1950): A computer passes the test of intelligence, if it can fool a human interrogator. Major components of AI: knowledge, reasoning, language, understanding, learning. Fall 2017 Artificial Intelligence: Introduction 7

19 AI passes Turing test in 'world first In 2014, a computer program called Eugene Goostman, which simulates a 13-year-old Ukrainian boy, is said to have passed the Turing test at an event organized by the University of Reading. BBC News ( Fall 2017 Artificial Intelligence: Introduction 8

20 Thinking rationally: laws of thought Codify right thinking with logic. Several Greek schools developed various forms of logic: notation and rules of derivation for thoughts. Problems: 1. Not all knowledge can be expressed with logical notations. 2. Computational blow up. Fall 2017 Artificial Intelligence: Introduction 9

21 Acting rationally: rational agent The right thing: that which is expected to maximize goal achievement, given the available information. A rational agent is one that acts so as to achieve the best outcome, or when there is uncertainty, the best expected outcome. Aristotle (Nicomachean Ethics): Every art and every inquiry, and similarly every action and pursuit, is thought to aim at some good. Fall 2017 Artificial Intelligence: Introduction 10

22 What is AI? Our Approach Acting Rationally: Our approach Fall 2017 Artificial Intelligence: Introduction 11

23 The Foundation of AI Fall 2017 Artificial Intelligence: Introduction 12

24 The Foundation of AI Philosophy Philosophy Mathematics Linguistics Mathematics Economics Neuroscience Psychology Control theory and cybernetics AI Economics Computer engineering Control theory and cybernetics Computer engineering Neuroscience Linguistics Psychology Fall 2017 Artificial Intelligence: Introduction 13

25 Philosophy Can formal rules be used to draw valid conclusions? How does the mind arise from a physical brain? Where does knowledge come from? How does knowledge lead to action? Logic, methods of reasoning. Mind as physical system that operates as a set of rules. Foundations of learning, language, rationality. Aristotle ( B.C.) Rene Descartes ( ) Fall 2017 Artificial Intelligence: Introduction 14

26 Mathematics What are the formal rules to draw valid conclusions? What can be computed? How do we reason with uncertain information? Logic: Formal representation and proof. Computation, algorithms. Probability. George Boole ( ) Thomas Bayes ( ) Fall 2017 Artificial Intelligence: Introduction 15

27 Economics How should we make decisions so as to maximize payoff? How should we do this when others may not go along? How should we do this when the payoff may be far in the future? Formal theory of rational decisions. Combined decision theory and probability theory for decision making under uncertainty. Game theory. Markov decision processes. Adam Smith ( ) Herbert Simon ( ) Fall 2017 Artificial Intelligence: Introduction 16

28 Neuroscience How do brains process information? How brains and computers are (dis)similar. Fall 2017 Artificial Intelligence: Introduction 17

29 Psychology How do humans and animals think and act? Cognitive psychology perceives the brain as an information processing machine. Led to the development of the field cognitive science: how could computer models be used to study language, memory, and thinking from a psychological perspective. Fall 2017 Artificial Intelligence: Introduction 18

30 Computer engineering How can we build an efficient computer? E.g., Self-driving cars are possible today thanks to advances in computer engineering. Fall 2017 Artificial Intelligence: Introduction 19

31 Control theory and cybernetics How can artifacts operate under their own control? Design simple optimal agents receiving feedback from the environment. Modern control theory design systems that maximize an objective function over time. Fall 2017 Artificial Intelligence: Introduction 20

32 Linguistics How does language relate to thought? Modern linguistics + AI = Computational linguistics (Natural language processing). Fall 2017 Artificial Intelligence: Introduction 21

33 The History of AI Fall 2017 Artificial Intelligence: Introduction 22

34 The History of AI The gestation of artificial intelligence ( ) The birth of artificial intelligence (1956) Early enthusiasm, great expectations ( ) A dose of reality ( ) Knowledge-based systems: The key to power? ( ) AI becomes an industry (1980 present) The return of neural networks (1986 present) AI adopts the scientific method (1987 present) The emergence of intelligent agents (1995 present) The availability of very large data sets (2001 present) Fall 2017 Artificial Intelligence: Introduction 23

35 The gestation of AI ( ) McCulloch and Walter Pitts, model of artificial neurons, Donald Hebb, Hebbian learning, Marvin Minsky and Dean Edmonds, neural network computer, Alan Turing, Computing Machinery and Intelligence, Alan Turing ( ) (Turing test, machine learning, genetic algorithms, and reinforcement learning ) Fall 2017 Artificial Intelligence: Introduction 24

36 The birth of AI (1956) Dartmouth Workshop John McCarthy (Lisp language) Marvin Minsky (SNARC) Claude Shannon (Information theory) Ray Solomonoff (Algorithmic probability) Allen Newell (General Problem Solver) Herbert Simon (Satisficing) Arthur Samuel (Computer checkers) And three others Oliver Selfridge (Pandemonium theory) Nathaniel Rochester (Designed IBM 701) Trenchard More (Natural deduction) Fall 2017 Artificial Intelligence: Introduction 25

37 Early enthusiasm ( ) Newell and Simon, General Problem Solver, 1961, physical symbol system, Herbert Gelernter, Geometry Theorem Prover, Arthur Samuel, checker programs, John McCarthy (MIT AI Lab), Lisp language, time sharing, Advice Taker, Marvin Minsky (Stanford AI Lab), Microworlds, Neural networks. Fall 2017 Artificial Intelligence: Introduction 26

38 A dose of reality ( ) Most early programs knew nothing of their subject matter; they succeeded by means of simple syntactic manipulations. The intractability of many of the problems that AI was attempting to solve. Some fundamental limitations on the basic structures being used to generate intelligent behavior. Fall 2017 Artificial Intelligence: Introduction 27

39 Knowledge-based systems ( ) General vs. Domain-specific knowledge. Buchanan et al., infer molecular structure, Feigenbaum, Buchanan, and Dr. Edward Shortliffe, diagnose blood infections, 1970s. Understanding natural language. Representation and reasoning languages. Fall 2017 Artificial Intelligence: Introduction 28

40 AI becomes an industry (1980 present) McDermott, R1, Fifth Generation project, Japan, Microelectronics and Computer Technology Corporation, United States, Overall, the AI industry boomed from a few million dollars in 1980 to billions of dollars in 1988, including hundreds of companies. AI Winter. Fall 2017 Artificial Intelligence: Introduction 29

41 Neural networks: le retour (1986 present) Reinvention of the back-propagation (Bryson & Ho, 1969) learning algorithm, mid-1980s. Modern neural network creating effective network architectures and algorithms and understanding their mathematical properties, or modeling of the empirical properties of actual neurons and ensembles of neurons. Fall 2017 Artificial Intelligence: Introduction 30

42 AI becomes scientific (1987 present) Speech recognition, hidden Markov models. Machine translation, sequences of words. Neural networks, data mining. Uncertain reasoning and expert systems, Bayesian network. robotics computer vision knowledge representation Fall 2017 Artificial Intelligence: Introduction 31

43 Intelligent agents (1995 present) Building complete agents (Newell, 1990; Laird et al., 1987) the isolated subfields of AI might need to be reorganized. AI has been drawn into much closer contact with other fields. Human-level AI (Minsky et al., 2004) machines that think, that learn and that create. Artificial General Intelligence (Goertzel and Pennachin, 2007) a universal algorithm for learning and acting in any environment Fall 2017 Artificial Intelligence: Introduction 32

44 Large datasets (2001 present) Banko and Brill, word-sense disambiguation, (100 million words of unlabeled training data). Hays and Efros, filling in holes in a photograph, (two million photos). Data will drive future discoveries and alleviate the complexity in AI. Fall 2017 Artificial Intelligence: Introduction 33

45 The Application of AI Fall 2017 Artificial Intelligence: Introduction 34

46 State-of-the-art applications Speech recognition Autonomous planning and scheduling Financial forecasting Game playing, video games Spam fighting Logistics planning Robotics (household, surgery, navigation) Machine translation Information extraction VLSI layout Automatic assembly Sentiment analysis Fraud detection Recommendation systems Web search engines Autonomous car Energy optimization Question answering systems Social network analysis Medical diagnosis, imaging Route finding Traveling salesperson Protein design Document summarization Transportation/scheduling Computer animation Many more! Fall 2017 Artificial Intelligence: Introduction 35

47 Speech recognition Virtual assistants: Siri (Apple), Echo (Amazon), Google Now, Cortana (Microsoft). Leverage deep neural networks to handle speech recognition and natural language understanding. Other technologies: Hidden Markov models Dynamic time warping Fall 2017 Artificial Intelligence: Introduction 36

48 Handwriting recognition State-of-the-art key technologies : recurrent neural networks and deep feedforward neural networks bi-directional and multi-dimensional long short-term memory Fall 2017 Artificial Intelligence: Introduction 37

49 Machine translation Historical motivation: translate Russian to English. First systems using mechanical translation (one-to-one correspondence) failed! Out of sight, out of mind -> Invisible, imbecile. Oops! Fall 2017 Artificial Intelligence: Introduction 38

50 Machine translation MT has gone through ups and downs. Today, Statistical Machine Translation leverages the vast amounts of available translated corpuses. While there is room for improvement, machine translation has made significant progress. Google Translate: 100+ languages Fall 2017 Artificial Intelligence: Introduction 39

51 Robotics Awesome robots today! NAO, ASIMO, and more! Robotics is an interdisciplinary branch of engineering and science Power source Actuation Sensing Manipulation Locomotion Fall 2017 Artificial Intelligence: Introduction 40

52 Recommendation systems Key technology: collaborative filtering Fall 2017 Artificial Intelligence: Introduction 41

53 Search engines Key technologies (near real time): 1. Web crawling 2. Indexing 3. Searching Fall 2017 Artificial Intelligence: Introduction 42

54 Spam filtering The baseline technology: Naive Bayes classifiers Fall 2017 Artificial Intelligence: Introduction 43

55 Face detection Viola-Jones algorithm Fall 2017 Artificial Intelligence: Introduction 44

56 Face recognition Popular technologies: principal component analysis using eigenfaces linear discriminant analysis elastic bunch graph matching using the Fisherface algorithm the hidden Markov model the multilinear subspace learning using tensorrepresentation and the neuronal motivated dynamic link matching. Fall 2017 Artificial Intelligence: Introduction 45

57 Cancer detection Skin Cancer Detection & Tracking using Deep Learning Fall 2017 Artificial Intelligence: Introduction 46

58 Logistics planning Key technologies heuristic search meta-heuristic search Fall 2017 Artificial Intelligence: Introduction 47

59 Autonomous driving DARPA Grand Challenge 2005: 132 miles 2007: Urban challenge 2009: Google self-driving car Fall 2017 Artificial Intelligence: Introduction 48

60 Autonomous planning and scheduling NASA s Remote Agent program (Jonsson et al., 2000). NASA s Mars Exploration Rovers (Al-Chang et al., 2004) European Space Agency s Mars Express (Cesta et al., 2007) Key technologies dynamic programming reinforcement learning combinatorial optimization Fall 2017 Artificial Intelligence: Introduction 49

61 Chess (1997) Garry Kasparov vs. IBM Deep Blue Powerful search algorithms! Fall 2017 Artificial Intelligence: Introduction 50

62 Jeopardy! (2011) Ken Jennings vs. IBM Watson Natural Language Understanding and information extraction! Fall 2017 Artificial Intelligence: Introduction 51

63 Go (2016) Lee Sedol versus Google AlphaGo Deep Learning, reinforcement learning, and search algorithms! Fall 2017 Artificial Intelligence: Introduction 52

64 Summary Fall 2017 Artificial Intelligence: Introduction 53

65 Summary AI is a hard (computational complexity, language, vision, etc), and a broad field with high impact on humanity and society. What can AI do for us is already amazing! AI systems do not have to model human/nature but can act like or be inspired by human/nature. How human think is beyond the scope of this course. Rational (do the right thing) agents are central to our approach of AI. Note that rationality is not always possible in complicated environment but we will still aim to build rational agents. Fall 2017 Artificial Intelligence: Introduction 54

66 Summary AI may be perceived as a scary area! Is AI a threat to our humankind? Professor Stephen Hawking, eminent scientist told BBC: The development of full artificial intelligence could spell the end of the human race. AI is a flourishing and exciting field: everyone can contribute. Looking forward for an exciting journey together! Fall 2017 Artificial Intelligence: Introduction 55

67 Reading materials Alan Turing, Computing Machinery and Intelligence, Artificial intelligence (Wikipedia) Online course (Prof. Ansaf Salleb-Aouissi, Columbia University) Fall 2017 Artificial Intelligence: Introduction 56

68 Homework Why would evolution tend to result in systems that act rationally? What goals are such systems designed to achieve? Is AI a science, or is it engineering? Or neither or both? Explain. Fall 2017 Artificial Intelligence: Introduction 57