ai

1. CES 510 Intelligent System Design
B. Ravikumar
Department of Engg Science
116 I Darwin Hall
664 3335
ravi93@gmail.com

2. Textbook
 Chris Manning and Hinrich Shutze, Foundations of
Statistical Natural Language Processing, MIT Press,
1999.
 Various supplementary readings.
Other Useful Books:
Jurafsky & Martin, SPEECH and LANGUAGE
PROCESSING: An Introduction to Natural Language
Processing, Computational Linguistics, and Speech
Recognition.

3. Overview of Artificial Intelligence
• major applications
• image processing and vision
• robotics
• game playing
• speech recognition
• natural language understanding
• etc.

4. What is Artificial Intelligence
(John McCarthy , Basic Questions)
 What is artificial intelligence?
 It is the science and engineering of making intelligent machines, especially
intelligent computer programs. It is related to the similar task of using
computers to understand human intelligence, but AI does not have to
confine itself to methods that are biologically observable.
 Yes, but what is intelligence?
 Intelligence is the computational part of the ability to achieve goals in the
world. Varying kinds and degrees of intelligence occur in people, many
animals and some machines.
 Isn't there a solid definition of intelligence that doesn't depend on relating it to
human intelligence?
 Not yet. The problem is that we cannot yet characterize in general what
kinds of computational procedures we want to call intelligent. We
understand some of the mechanisms of intelligence and not others.
 More in: http://www-formal.stanford.edu/jmc/whatisai/node1.html

5. What is Artificial Intelligence?
 Human-like (“How to simulate humans intellect and behavior on
by a machine.)
• Mathematical problems (puzzles, games, theorems)
• Common-sense reasoning (if there is parking-space, probably illegal to
park)
• Expert knowledge: lawyers, medicine, diagnosis
• Social behavior
 Rational-like:
• achieve goals, have performance measure

6. What is Artificial Intelligence
 Thought processes
• “The exciting new effort to make computers think .. Machines with
minds, in the full and literal sense” (Haugeland, 1985)
 Behavior
• “The study of how to make computers do things at which, at the
moment, people are better.” (Rich, and Knight, 1991)

7. The Turing Test
(Can Machine think? A. M. Turing, 1950)
 Requires
• Natural language
• Knowledge representation
• Automated reasoning
• Machine learning
• (vision, robotics) for full test

8. What is AI?
 Turing test (1950)
 Requires:
• Natural language
• Knowledge representation
• automated reasoning
• machine learning
• (vision, robotics.) for full test
 Thinking humanly:
• Introspection, the general problem solver (Newell and Simon 1961)
• Cognitive sciences
 Thinking rationally:
• Logic
• Problems: how to represent and reason in a domain
 Acting rationally:
• Agents: Perceive and act

9. History of AI
 McCulloch and Pitts (1943)
• Neural networks that learn
 Minsky (1951)
• Built a neural net computer
 Darmouth conference (1956):
• McCarthy, Minsky, Newell, Simon met,
• Logic theorist (LT)- proves a theorem in Principia Mathematica-Russel.
• The name “Artficial Intelligence” was coined.
 1952-1969
• GPS- Newell and Simon
• Geometry theorem prover - Gelernter (1959)
• Samuel Checkers that learns (1952)
• McCarthy - Lisp (1958), Advice Taker, Robinson’s resolution
• Microworlds: Integration, block-worlds.
• 1962- the perceptron convergence (Rosenblatt)

10. The Birthplace of
“Artificial Intelligence”, 1956
 Darmouth workshop, 1956: historical meeting of the perceived founders
of AI met: John McCarthy, Marvin Minsky, Alan Newell, and Herbert
Simon.
 A Proposal for the Dartmouth Summer Research Project on Artificial
Intelligence. J. McCarthy, M. L. Minsky, N. Rochester, and C.E. Shannon.
August 31, 1955. "We propose that a 2 month, 10 man study of artificial
intelligence be carried out during the summer of 1956 at Dartmouth
College in Hanover, New Hampshire. The study is to proceed on the basis
of the conjecture that every aspect of learning or any other feature of
intelligence can in principle be so precisely described that a machine can
be made to simulate it." And this marks the debut of the term "artificial
intelligence.“

11. History, continued
 1966-1974 a dose of reality
• Problems with computation
 1969-1979 Knowledge-based system
• Expert systems:
—Dendral:Inferring molecular structures
—Mycin: diagnosing blood infections
—Prospector: recomending exploratory drilling (Duda).
 1986-present: return to neural networks
 Machine learning theory
 Genetic algorithms, genetic programming
 Statistical approaches and data mining

12. State of the art
 Deep Blue defeated the reigning world chess champion Garry Kasparov
in 1997
 Proved a mathematical conjecture (Robbins conjecture) unsolved for
decades
 No hands across America (driving autonomously 98% of the time from
Pittsburgh to San Diego)
 During the 1991 Gulf War, US forces deployed an AI logistics planning
and scheduling program that involved up to 50,000 vehicles, cargo, and
people
 NASA's on-board autonomous planning program controlled the
scheduling of operations for a spacecraft
 Proverb solves crossword puzzles better than most humans
 DARPA grand challenge 2003-2005, Robocup

13. What’s involved in Intelligence?
Intelligent agents
 Ability to interact with the real world
• to perceive, understand, and act
• e.g., speech recognition and understanding and synthesis
• e.g., image understanding
• e.g., ability to take actions, have an effect
 Knowledge Representation, Reasoning and Planning
• modeling the external world, given input
• solving new problems, planning and making decisions
• ability to deal with unexpected problems, uncertainties
 Learning and Adaptation
• we are continuously learning and adapting
• our internal models are always being “updated”
— e.g. a baby learning to categorize and recognize animals

14. Course overview
 Intelligent systems are autonomous systems (hardware / software or a
combination) that behaves as if it exhibits some form of intelligence.
 Concept goes back to Alan Turing who thought about machine
intelligence and devised Turing test to distinguish a machine from a
human through interaction.
 Some major areas:
 Symbolic information processing – deductive systems
 Game playing – chess, backgammon etc.
 natural language understanding – answering queries, translation, text
classification etc.
 Machine learning - adaptive behavior through stimulus
 Neural networks
 Statistical modeling
 Fuzzy logic, genetic programming etc.

15. Course overview
 In this course we will introduce statistical
techniques for inferring structure from text. The
aim of the course is to introduce existing
techniques in statistical NLP and to stimulate
thought into bettering these.
 Applications of NLP
 Information Retrieval
 Information Extraction
 Natural language interface to database
 Statistical Machine Translation

16. Tools
 Probability Theory
 Information Theory
 Algorithms
 Data Structures
 Probabilistic AI
 Grammars and automata

17. The Steps in NLP
Pragmatics
Syntax
Semantics
Pragmatics
Syntax
Semantics
Discourse
Morphology

18. The steps in NLP (Cont.)
 Morphology: Concerns the way words are built up from
smaller meaning bearing units. (come(s),co(mes))
 Syntax: concerns how words are put together to form
correct sentences and what structural role each word
has.
 Semantics: concerns what words mean and how these
meanings combine in sentences to form sentence
meanings.
 Pragmatics: concerns how sentences are used in
different situations and how use affects the
interpretation of the sentence.
 Discourse: concerns how the immediately preceding
sentences affect the interpretation of the next
sentence.

19. Parsing (Syntactic Analysis)
 Assigning a syntactic and logical form to an input sentence
 uses knowledge about word and word meanings (lexicon)
 uses a set of rules defining legal structures (grammar)
(S (NP (NAME Sam))
(VP (V ate)
(NP (ART the)
(N apple))))
I made her duck.

20. Word Sense Resolution
 Many words have many meanings or senses.
 We need to resolve which of the senses of an ambiguous
word is invoked in a particular use of the word.
 I made her duck. (made her a bird for lunch or made her
move her head quickly downwards?)

21. Reference Resolution
 Domain Knowledge (banking transaction)
 Discourse Knowledge
 World Knowledge
U: I would like to open a fixed deposit account.
S: For what amount?
U: Make it for 800 dollars.
S: For what duration?
U: What is the interest rate for 3 months?
S: Six percent.
U: Oh good then make it for that duration.

22. Why NLP is difficult?
 Different ways of Parsing a sentence
 Word category ambiguity
 Word sense ambiguity
 Words can mean more than their sum of parts (The Times of India)
 Imparting world knowledge is difficult ("the blue pen ate the ice-
cream")
 Fictitious worlds ("people on mars can fly")
 Defining scope ("people like ice-cream," does this mean all people
like ice cream?)
 Language is changing and evolving
 Complex ways of interaction between the kinds of knowledge
 exponential complexity at each point in using the knowledge

23. Inferring Knowledge from text
 Words
 word frequencies
 collocations
 word sense
 n-grams (words appear in certain order)
 Grammar
 word categories
 syntactic structure
 Discourse
 Sentence meanings
 Applications
 Information Retrieval
 Information Extraction
 Natural language interface
 Statistical Machine Translation

24. Simple Applications
 Word counters (wc in UNIX)
 Spell Checkers, grammar checkers
 Predictive Text on mobile handsets

25. More significant Applications
 Intelligent computer systems
 NLU interfaces to databases
 Computer aided instruction, automatic graders
 Information retrieval
 Intelligent Web searching
 Data mining
 Machine translation
 Speech recognition
 Natural language generation
 Question answering

26. Spoken Dialogue System
Speech
Synthesis
Speech
Recognition
Semantic
Interpretation
Response
Generation
Dialogue
Management
Discourse
Interpretation
U
s
e
r

27. Parts of the Spoken Dialogue System
 Signal Processing:
 Convert the audio wave into a sequence of feature vectors.
 Speech Recognition:
 Decode the sequence of feature vectors into a sequence of words.
 Semantic Interpretation:
 Determine the meaning of the words.
 Discourse Interpretation:
 Understand what the user intends by interpreting utterances in
context.
 Dialogue Management:
 Determine system goals in response to user utterances based on
user intention.
 Speech Synthesis:
 Generate synthetic speech as a response.

28. Levels of Sophistication in a Dialogue
System
 Touch-tone replacement:
System Prompt: "For checking information, press or say one."
Caller Response: "One."
 Directed dialogue:
System Prompt: "Would you like checking account information
or rate information?"
Caller Response: "Checking", or "checking account," or "rates."
 Natural language:
System Prompt: "What transaction would you like to perform?"
Caller Response: "Transfer Rs. 500 from checking to savings."