Text Mining in Business Intelligence โดย รศ.ดร.โอม ศรนิล ในงาน THE FIRST NIDA BUSINESS ANALYTICS AND DATA SCIENCES CONTEST/CONFERENCE จัดโดย คณะสถิติประยุกต์และ DATA SCIENCES THAILAND

1. The First NIDA Business Analytics and Data Sciences Contest/Conference
วันที่ 1-2 กันยายน 2559 ณ อาคารนวมินทราธิราช สถาบันบัณฑิตพัฒนบริหารศาสตร์
https://businessanalyticsnida.wordpress.com
https://www.facebook.com/BusinessAnalyticsNIDA/
โดย รศ. ดร. โอม ศรนิล
สาขาวิชาวิทยาการข้อมูล
คณะสถิติประยุกต์ สถาบันบัณฑิตพัฒนบริหารศาสตร์
Text Mining in Business Intelligence
การทาเหมืองข้อความทาได้อย่างไร มีหลักการอย่างไร
ทาเหมืองข้อความภาษาไทยได้หรือไม่
เราจะประยุกต์ใช้การทาเหมืองข้อความกับธุรกิจได้อย่างไร
ต้องเขียนโปรแกรมเป็นหรือไม่หากจะทาเหมืองข้อความ
ทาเหมืองข้อความแล้วจะได้ความรู้อะไรบ้าง
นวมินทราธิราช 3003 วันที่ 1 กันยายน 2559 9.30-10.00 น.

2. TEXT MINING
IN BUSINESS INTELLIGENCE
OHM SORNIL, Ph.D.
Department of Computer Science, NIDA

3. BUSINESS INTELLIGENCE
“the ability to apprehend the interrelationships of presented facts in
such a way as to guide action towards a desired goal.”
(H. P. Luhn, 1958)
“a set of techniques and tools for the acquisition and transformation of
raw data into meaningful and useful information for business analysis
purposes.”
(D. M. Turner, 2016)

4. UNSTRUCTURED DATA
◉ Unstructured data is like Text, video, a voice recording of a
customer service transaction
◉ Generally accepted maxim is that structured data represents
only 20%. The rest is unstructured.
◉ If it can be counted, it can be analyzed.
◉ If it can be analyzed, it can be interpreted.

5. Source: http://www.csc.com/insights/flxwd/78931-big_data_universe_beginning_to_explode

6. JUST MARKETING TERMS
◉ Text mining = Text analytics = Natural language processing (NLP)
◉ A move from university research to real-world business problems

7. Internal
◉ Company documents
◉ Emails
◉ Reports
◉ Media releases
◉ Customer records and communication
SOURCES OF TEXTUAL DATA
External
◉ News
◉ Websites
◉ Blogs
◉ Social media posts

8. CHALLENGES
◉ Text is generally unstructured
◉ Large quantities and increasing rapidly
◉ Noisy (e.g., typoerrors, slangs, informal words, etc.)
◉ Synonymy and polysemy

9. TEXT MINING
◉ Process of extracting interesting information or patterns from
unstructured text
◉ An interdisciplinary field: computational linguistics, statistics,
and machine learning
◉ Can lead to the development of new opportunities in business

10. Business Applications

11. CUSTOMER RELATIONSHIP MANAGEMENT (CRM)
Input
◉ Text documents produced from
a variety of sources in contact
centers
Output
◉ Contents of client’s messages
◉ Routing specific requests to the
appropriate service
◉ Supplying immediate answers to
the most frequently asked
questions

12. OPINION ANALYSIS
Output
◉ Frequency of words mentioned is an indicator for concept
salience, e.g., “unbreakable”, “fragile”
◉ Frequency of co-occurrence represents the strength of
connection in the customer‘s mind,
e.g., <“Samsung”, “camera”>, <“iPhone”, “expensive”>
Input
◉ Customers’ messages in websites, blogs, Tweeter,
Facebook, etc.

13. MEDICAL RECORD ANALYSIS
Input
◉ Doctors’ comments
Output
◉ An early warning regarding
specific diseases
If frequency of “lungs” or “breathing” appears more than
45 appearances in the last 30 days for a given ZIP code or
region, it can be a clue to excessive environmental
conditions which are resulting in respiratory problems. A
proactive intervention can be activated to remedy the
situation.

14. SENTIMENT ANALYSIS
Input
◉ Customers’ messages in
websites, blogs, Tweeter,
Facebook, etc.
Output
◉ Positive, negative or neutral
opinions/feelings (polarity)
expressed by a writer in a
document collection

15. SENTIMENT ANALYSIS (FEATURE-BASED)

16. EMOTIONAL STATE CLASSIFICATION
SOURCE: http://emotion-research.net/toolbox/toolboxlabellingtool.2006-09-26.9095478150
https://annaszymanska1324161.wordpress.com/2014/04/28/very-emotional-research/

17. HUMAN RESOURCE MANAGEMENT
Input
◉ Staff’s opinions
◉ CVs from applicants
Output
◉ Level of employee satisfaction
◉ Selection of new personnel

18. INSURANCE CLAIM DIAGNOSIS
Input
◉ Note of all the details related to
the claim/health issues in the
form of a brief description
Output
◉ Identified a common group of
problems

19. CORPORATE FINANCE
Input
◉ Publicly available descriptions of any startups' business
- products/services, investors and social links between
individuals in 2 firms
Output
◉ Targets for mergers and acquisitions
Source: http://phys.org/news/2016-07-text-mining-intelligence-startups.html#jCp

20. INVESTMENT
Input
◉ Security related newsfeed
Output
◉ A model to predict movements of markets for everything
from government bonds to commodities.

21. MEANINGThe key is to capture the meaning of text.

22. TEXT MINING PROCESS
Text Sources Preprocessing
Presentation
(Visualization/
Browsing)
Modeling

23. COMMON PREPROCESSING
◉ Extracting text
◉ Tokenization
◉ Stopword elimination: is, am, are, the, of, for, … (http://www.ranks.nl/stopwords/thai-stopwords)
◉ Stemming: run, runs, ran, running  run

24. TEXT REPRESENTATION FOR MINING

25. INVERSE DOCUMENT FREQUENCY
SOURCE: http://nlp.stanford.edu/IR-book/pdf/06vect.pdf

26. TF-IDF TERM WEIGHTING

27. REAL-VALUED VECTOR

28. COSINE SIMILARITY BETWEEN 2 VECTORS

29. WORD CO-OCCURRENCE STRENGTH
◉ Mutual Information (MI) between words x and y

30. ADD-ON COMPONENTS
◉ WordNet
◉ Feature selection/reduction

31. WordNet
◉ WordNet is essentially Dictionary + Thesaurus
Relations: hyponymy, meronymy, antonymy

32. TASK SPECIFIC COMPONENTS
◉ Part-of-Speech (POS) tagging
◉ SentiWordNet
- Results of automatic annotation of all synsets of WordNet
according to the notions of “positivity”, “negativity” and “neutral”
◉ Emoticons

33. MINING ALGORITHMS
◉ General machine learning algorithms are applicable
Classification
Naïve Bayes
Support Vector Machine
Bayesian Network
Neural Network
Logistic Regression
etc.
Clustering
K-means
Fuzzy C-means
Hierarhical Clustering
Self-Organizing Map
etc.
Association Analysis
and Sequence Analysis
Apriori
Generalized Rule Induction
Influential Apriori
FP-Growth
etc.

34. Analysis Tasks

35. GENERAL DATA MINING TASKS
◉ Classification
◉ Clustering
◉ Association Analysis
◉ Prediction
◉ Sequence Analysis

36. INFORMATION EXTRACTION

37. Analytics Tools with Text Mining
Capabilities

38. OPEN SOURCED SOFTWARE
SOURCE: http://www.predictiveanalyticstoday.com/top-free-software-for-text-analysis-
text-mining-text-analytics/
R package TM

39. COMMERCIAL SOFTWARE
SOURCE: http://www.predictiveanalyticstoday.com/top-free-software-for-text-analysis-
text-mining-text-analytics/

40. THANKS !http://as.nida.ac.th
Email: osornil@as.nida.ac.th
Phone: 081-731-7175