A presentation I have delivered at University of Bahrain on intelligent systems and their current and future use in organisations and by consumers, iPhone’s Siri is used as an example of the mainstream adoption of such systems.
7. Why Beta? Apple Creates “Perfect” Products?
Because language is a complex tool and NLP
require more advancement
8. Natural Language Processing (NLP)
• A field of computer science and linguistics concerned with the interactions
between computers and human (natural) languages.
• Approaches
– Deep analysis (use linguistic resources such as grammar analysis, etc.)
– Shallow analysis (statistical, use large collections of text )
– Hybrid
• Major Tasks
– Automatic Summarisation (Microsoft Word?)
– Natural Language Understanding (Siri?)
– Speech Recognition (Siri?)
– Machine Translation (Google versus. Sakhr)
– Sentiment Analysis
– Information Retrieval
– Information Extraction
9. Steps in Natural Language Processing (NLP)
Deep Approach (Linguistic Analysis)
11. Natural Language Processing (NLP)
Shallow Approach (Statistical Analysis)
• Systems can use the large amount of textual
information readily available (e.g. on the web)
to produce statistical information and drive
generalisations that can be used as rules for
predictions/classifications (language
independent?)
14. Information Overload and Decision Making
“Intelligent (NLP) Systems to Assist”
Herbert Simon Information Overload and
(Nobel Prize in Economics, Awarded 1978) Bounded Rationality
“What information consumes is rather obvious: it consumes the attention of its
recipients. Hence a wealth of information creates a poverty of attention, and a
need to allocate that attention efficiently among the overabundance of
information sources that might consume it.”
[Herbert Simon]
15. News, Blogs, Chats, etc. and their Influence
“Intelligent (NLP) Systems to Assist”
Daniel Kahneman
(Nobel Prize in Economics, Awarded 2002) Human Psychology
- “Traces of what people feel and believe can be found in text” [Daniel Kahneman]
- “Frequency Correlates with acceptability” [Richard Kittredge]
20. Natural Language Processing - Language is Complex
falling or rising
ارتفاع أم انخفاض
sick or healthy
صحية أم مريضة
ascending or descending
- Positive for Stock Market or Commodities Market (Oil)? صعود أم ھبوط
- Positive for Buyer or seller?
Cartoons Source: http://www.aleqt.com/ 20
21. Systems Need to be Cross Language
• Arabic:
– ( سنستخدمهrich morphology, absence of vowels)
• English:
• We will use it (simple morphology).
• Chinese:
– 我们将使用它 (no spaces between words)
22. Globalisation and Language - Banking and Finance
Demand in Europe and USA for market researchers and analysts
fluent in Arabic, Chinese, Hindi, Russian or Turkish
22
23. In the Business World
“I have been dealing with these big mathematical models of forecasting the economy and
I am looking at what was going on in the last few weeks (credit crunch), if i can figure out
whether or not people are more fearful or changing to be euphoric and I have a third way
of figuring out which of the two things is working (stronger), i don't need any these of
those stuff (mathematical models), i can forecast the economy better than any way i
know, the trouble is that we can’t figure that out.“
Alan Greenspan on the Daily Show (2007)
Chairman of the Federal Reserve 1987-2006
Position: “Text Data Mining”
Job Description: The successful candidate will mine text data from numerous news
sources and incorporate the information into the proprietary trading systems.
Posted on www.efinancialcareers.com (2007)
“We are exploring how to incorporate techniques of parsing texts in our trading systems.”
Head of Trading Research Team (2007)
Major Global Investment Bank
(Personal Communication)
28/05/2012 09:09 23
25. Content Analytics – NLP in Action
• Content analytics applications process content to derive
answers to specific questions:
– Shall I sell my “Apple” stocks?
– Shall I buy a Mercedes or a BMW?
• Sources of Content include:
– blogs
– news portals
– databases (corporate)
– instant messaging
– social networks
– search engines
– images, audio and video (public and private)
28. Text Analytics Market
• Text analytics solutions market is still
immature (the vendors with the longest track
records do not yet have software-as-a-service
offerings), and some of the underlying
capabilities are not as robust as the attractive
user interface may suggest (dealing with non-
English-language documents and documents
with many domain-specific terms?).
[Gartner]
29. Wrap Up – How Could Intelligent Systems
Change the way we take decisions
• NLP Intelligent systems will be more:
- Accurate
- Cross Language
- Cross Domain (e.g. finance, sport, medicine)
As individuals, we will depend more on such systems
deployed on Phones, Cars, Tablets, PCs, TVs, Kiosks, etc. and
they will not be perceived as intelligent systems (as least not
as now) in many daily tasks including:
- Buy/Sell decisions.
- Advise.
- Entertainment.
- Education.