Artificial Intelligence Academic year 2016/2017 Giorgio Fumera http://pralab.diee.unica.it fumera@diee.unica.it Pattern Recognition and Applications Lab Department of Electrical and Electronic Engineering University of Cagliari
Outline Part I: Historical notes Part II: Solving Problems by Searching Uninformed Search Informed search Part III: Knowledge-based Systems Logical Languages Expert systems Part IV: The Lisp Language Part V: Machine Learning Decision Trees Neural Networks
Part I Historical Notes
Artificial Intelligence A discipline born in the 1950s. Goal: building intelligent machines. A goal envisaged since a long time before, e.g.: I Leonardo da Vinci s robot knight (about 1495) I Automaton chess player ( Mechanical Turk, late 18th century) I science fiction
What is intelligence? Broad definition: a set of capabilities that allow humans to I learn I think I understand I communicate I be self-conscious I build abstract models of the world I plan I adapt to novel external conditions I... Some of these capabilities are exhibited also by animals (e.g., associative memory, reacting to stimuli, communicating).
Early investigations: clues from humans Goal: understanding human intelligence. I High-level manifestation: rationality Logic: Aristotle (4th cent. bc), G.W. Leibniz (17th-18th cent.), G. Boole (19th cent.), etc. I High-level manifestation: behavior and mind Psychology and cognitive science (since 19th cent.) I Low-level biological support: the brain Neuroanatomy and Neurophysiology (since 19th cent.): McCulloch and Pitt s model of neuron (1943), D.O. Hebb s theory on neurons as basic units of thought, etc.
Early investigations: clues from technology Goal: building machines capable of autonomously performing some task. I Automata (e.g., jacquard loom, 1804) Cybernetics (feedback and control): N. Wiener, W.R. Ashby (1940s) I Statistics and probability as tools to deal with uncertainty in reasoning and decision-making: T. Bayes (18th cent.); K. Pearson, R.A. Fisher, A. Wald, J. Neyman (late 19th 20th cent.)
Early investigations: clues from technology The computer: I precursors: mechanical devices: B. Pascal and G.W. Leibniz (17th cent.), C. Babbage s analytical engine (19th cent.) I from engineering: electromechanical and electronic devices (1940s): K. Zuse, J.P. Eckert, J.W. Mauchly, J. von Neumann I from logic and mathematics: computational theory, the foundation of computer science: A.M. Turing, A. Church (1930s)
Early investigations: thinking computers Electronic computers: the right tool for building intelligent machines? Alan M. Turing s (1912-1954) main contributions: I first investigations into the nature of computing I the logical computing machine (Turing machine): a universal computer I envisioning intelligent computers: Computing Machinery and Intelligence, Mind, Vol. LIX, No. 263, 433 460, 1950; operational definition of intelligence: the Turing Test
The birth of AI 1950s: many researchers from di erent disciplines investigate intelligence under two main viewpoints: I understanding human intelligence I building intelligent machines Dartmouth College Workshop (USA, summer 1956): o cial birth of AI, as a discipline aimed at building intelligent machines. The founders: J. McCarthy, M. Minsky, A. Newell, H. Simon, C. Shannon, O. Selfridge, R. Solomono,...
Early explorations: 1950s and 1960s Goals: identifying specific tasks that require intelligence, and figuring out how to get machines to do them. Great interest in mimicking high-level human thought and mental abilities, e.g.: I reasoning I understanding natural language I understanding images and some related low-level abilities: I recognizing speech sounds I distinguishing objects in images I reading cursive script But: how do humans do that?
Early explorations: 1950s and 1960s Starting point: toy problems, and some real-world ones I game playing 15-puzzle, checkers, chess, etc. I theorem proving I natural language processing (NLP) I recognizing objects in images
Early explorations: 1950s and 1960s First approaches and tools: I symbolic (high-level) approach: based on the assumption that the essence of intelligence is symbol processing heuristic search syntax analysis/generation symbolic knowledge representation (symbols, lists) symbolic knowledge processing: new programming languages (LISP, etc.) I non-symbolic (low-level) approach pattern recognition artificial neural networks
Heuristic search methods Symbol processing approach, applied to: I game playing: 15-puzzle, checkers, chess (the Drosophila of AI ) I geometric analogy problems I theorem proving I mechanizing problem solving: A. Newell and H. Simon s General Problem Solver (1959)
Heuristic search methods Common approach: I knowledge representation: lists of symbols (main feature of the LISP language, 1958, J. McCarthy) I search methods: search tree, heuristics
Natural language processing Symbol processing approach. Goals: understanding, generating, translating natural language. Linguistic levels: I morphology: word parts (e.g.: walking = walk + -ing) I syntax (grammar): rules that define well-formed sentences (e.g.: John hit the ball: Yes;ball the hit John: No) I semantics: meaning of a sentence I pragmatics: context and background knowledge, e.g.: John went to the bank John threw the ball to the window and broke it John threw the glass to the wall and broke it
Natural language processing Main focus of early research: syntactic level. Seminal work: N. Chomsky, Syntactic Structures, 1957. Grammar definition: syntax rules for analyzing/generating sentences; main tool: parse tree. Applications: I question answering (original goal: computer interfaces) I machine translation: early optimism, but a very hard task
Pattern recognition Goal: classifying input signals (images, sounds, electronic signals, etc.) into one of several categories. First problems: image classification. Early e orts: optical character recognition (OCR). Main approaches: I template matching I learning 1. image pre-processing (noise filtering, line thickening, edge enhancement,...) 2. feature extraction 3. classification rules learnt from examples
Artificial neural networks Non-symbolic (low-level), connectionist approach. F. Rosenblatt (psychologist): perceptron (1957 ) as a potential model of human learning, cognition and memory: I network of McCulloch-Pitts neural elements I learning algorithm for adjusting connection weights from examples First applications: pattern recognition I OCR I aerial image recognition
Great expansion: mid 1960s to early 1980s From toy/lab problems to real-world and commercial applications: I computer vision I mobile robots I game playing I speech recognition, NLP I knowledge representation and reasoning Research funding: I DARPA s Strategic Computing Program (USA) I Fifth Generation Computer Systems (Japan) I ESPRIT (Europe)
Great expansion: mid 1960s to early 1980s Computer vision: I Summer Vision project (MIT, 1966) I low-level, hierarchical image processing (hints from biology); image filters; line, corner, surface detection; 3D reconstruction I early application: guiding a robot arm to manipulate blocks I high-level vision: finding objects in scenes (templates, parts) I two main approaches emerge: whole scene reconstruction: di cult perceiving to guide robot action (purposive vision): easier
Great expansion: mid 1960s to early 1980s Mobile robots: I sensors, actuators, computer vision, environment modeling, planning I route finding: heuristic search, A* algorithm I first autonomous vehicles
Great expansion: mid 1960s to early 1980s Game playing: progress in chess I programs that attain human-level (not master) capability I investigations into human ability: accumulated knowledge vs massive search
Great expansion: mid 1960s to early 1980s NLP: greater success on less ambitious goals I improvements in grammars I machine translation with humans in the loop I declarative (logical languages, inference algorithms) vs procedural ( hard-wired ) knowledge I dialog systems (1971, T. Winograd s SHRDLU: blocks world, procedural knowledge) I speech recognition (easy), and understanding (di cult)
Great expansion: mid 1960s to early 1980s Knowledge representation and reasoning: I consulting / decision support / expert systems main idea: solving domain-specific problems by embedding expert knowledge in the form of IF-THEN rules I applications: chemistry, medical diagnosis, geology, military; since 1990s: business
Great expansion: mid 1960s to early 1980s Summing up, until the 70s AI research is mainly based on the symbol processing conception of human intelligence I main approach to AI: mimicking high-level human abilities through heuristic search and symbolic processing ( good, old-fashioned AI, GOFAI) I many successful applications through a pragmatic approach in specific tasks... I...but very limited achievements with respect to early expectations for a general AI
Mid 1980s: the AI winter Real-world tasks require much more intelligence than that achievable by heuristic search and symbolic processing (GOFAI). Two main issues emerge: I computational complexity: combinatorial explosion I need of a large body of knowledge (including common sense) Also the non-symbolic, connectionist approach (artificial neural networks) exhibits limitations. Main consequences: I drop of interest in AI I scaling back AI s goals I cut of research funds
Mid 1980s to 1990s: technical and theoretical advances New results in several fields, solid theoretical foundations from: I mathematics I statistics and probability theory I control engineering Concrete progress in real-world tasks (albeit still far from initial expectations).
Mid 1980s to 1990s: technical and theoretical advances Knowledge representation and reasoning: I new paradigms, e.g.; fuzzy logic, soft computing (inspired by human mind) I semantic networks, ontologies; e.g.: WordNet, http://wordnet.princeton.edu BabelNet, http://babelnet.org I probabilistic reasoning to overcome the limits of logic (probabilistic graphical models, Bayesian networks, learning) Search algorithms: evolutionary approach, genetic algorithms (inspired by evolution).
Mid 1980s to 1990s: technical and theoretical advances The rise of machine learning: I huge amount of data in digital form become available I main idea: automatically inferring knowledge (patterns, rules, etc.) from data instead of eliciting it from domain experts I data analysis methods: data mining, etc. I theoretical foundations: statistics I novel techniques: inductive logic programming, decision trees, resurgence of ANNs (1986: back-propagation algorithm), support vector machines, ensemble methods, etc. I applications to many fields: computer vision, natural language processing, etc.
Mid 1980s to 1990s: technical and theoretical advances Computer vision: I two main approaches persist: scene analysis, purposive vision I main achievements: surface, depth; tracking, object recognition I fruitful exchanges with research on animal/human vision I novel techniques: hierarchical models, ANNs, deep neural networks I extensive application of machine learning techniques
Mid 1980s to 1990s: technical and theoretical advances Intelligent Agent architectures: I sensor networks I autonomous, cooperating robots; emergent behavior I the intelligent agent paradigm A toy (?) example: soccer-playing robots
Mid 1990s today: main achievements Original goal (building intelligent machines): still far-reaching. But: successes in real-world problems, many commercial applications, many startup companies. Examples: I games: master-level programs in checkers, chess and (very recently) Go I robots: driverless automobiles, space vehicles I pervasive applications: home, cars, route finding in maps (search algorithms), recommender systems (machine learning, social/collaborative filtering), characters in video games,... I automated (high-frequency) trading I medicine I business rule management systems I translating systems I computer vision (face recognition,...)
Summary of the main approaches to AI Four distinct approaches to build intelligent machines: Human performance Rationality Mind/ Systems that think Systems that think thinking like humans rationally (cognitive modeling approach) ( law of thought approach) Systems that act Systems that act Behavior like humans rationally (Turing test approach) (rational agent approach) Rational agents (acting rationally): the most general approach, amenable to scientific/technological development (though, maybe, not useful enough for understanding human intelligence)
A snapshot of current AI research Associations: I Association for the Advancement of Artificial Intelligence (AAAI) www.aaai.org I European Coordinating Committee for Artificial Intelligence (ECCAI), www.eccai.org I Italian Association for Artificial Intelligence (AI*IA) www.aixia.it Conferences: I Int. Joint Conf. on Artificial Intelligence, ijcai.org Scientific journals: I Artificial Intelligence www.journals.elsevier.com/artificial-intelligence I J. of Artificial Intelligence Research, www.jair.org
A snapshot of current AI research Research topics: I Machine learning I Knowledge representation and reasoning I Reasoning under uncertainty or imprecision I Natural language I Vision I Pattern recognition I Heuristic search I Intelligent robotics I Multi-agent systems I Planning I...
Philosophical issues A long-standing question: Can machines be intelligent? Two main hypotheses: I Weak AI: machines can emulate intelligence (act intelligently) I Strong AI: machines can be intelligent (if they act intelligently, they are intelligent, e.g.: Turing test) Another long-standing question: Is (human) mind a machine?
Philosophical issues Some arguments raised against Weak AI: I machines can never do... (make mistakes, learn from experience, have a sense of humor, enjoy ice creams,... ) I machines are formal systems, and formal systems cannot establish the truth of every mathematical sentence (Gödel incompleteness theorem), whereas humans (in principle) can I human behavior can not be captured by a set of rules A. Turing s viewpoint (Mind, 1950): Can a machine think? is an ill-posed question. Consider this one: Can a machine fly/swim? I airplanes fly, but not as birds fly I ships swim in Russian, not in English or in Italian...
Philosophical issues Some issues about Strong AI: I even if machines can emulate intelligence, they cannot be self-conscious I relationship between mental states and body (brain) states (free will, consciousness, intentions): dualism (R. Descartes, 17th cent.) vs materialism ( brains cause mind ) I a machine running the right program (e.g., for natural language understanding) does not necessarily have a mind (the Chinese room thought experiment, J. Searle, 1980) I intelligence is an emerging behavior that can be only supported by biological brains
Ethical issues I Even if we could build intelligent machines, should we? I Consequences on humans: loss of jobs, loss of the sense of being unique, end of human race, etc. I Accountability (e.g., driverless cars) I... Ethical concerns are now re-flourishing, as many believe that human-level AI is now in reach (e.g., current focus of the Future of Life Institute, http://futureoflife.org).
Some recent projects Human Brain Project (EU) https://www.humanbrainproject.eu Overall goal: understanding the human brain and its diseases, and emulating its computational capabilities RoboLaw (EU) http://www.robolaw.eu Regulating Emerging Robotic Technologies in Europe: Robotics facing Law and Ethics