This document provides an introduction and instructions for using text mining and network analysis tools in Python. It discusses text mining concepts like document clustering and summarization. It also provides instructions for installing Python, IPython, and relevant libraries. Specific sections cover morpheme analysis of Korean text and exploring network analysis for future work.

1. Korean Word Network
v.0.1.0
August 7, 2015
Kyunghoon Kim
kyunghoon@unist.ac.kr
Department of Mathematical Sciences
Ulsan National Institute of Science and Technology
Republic of Korea

2. 1
Contents
What is a text mining?
Text is the most common vehicle
Features of Text Mining
Document clustering
Warming up
Installation1. Python
Installation2. pip upgrade and library install
Python
IPython
Python Basic Coding
Text Mining Example
Morpheme Analysis
Network Analysis

3. 2
Introduction
What is a text mining?
▶ Data mining is about looking for patterns in data.

4. 2
Introduction
What is a text mining?
▶ Data mining is about looking for patterns in data.
▶ Text mining is about looking for patterns in text.

5. 2
Introduction
What is a text mining?
▶ Data mining is about looking for patterns in data.
▶ Text mining is about looking for patterns in text.
▶ It is the process of analyzing text to extract information that is
useful for particular purposes.

6. 3
Introduction
▶ Text is the most common vehicle for the formal exchange of
information.

7. 4
Introduction
▶ Comprehensible

8. 4
Introduction
▶ Comprehensible
▶ → Text summarization

9. 4
Introduction
▶ Comprehensible
▶ → Text summarization
▶ Document classification [supervised learning]

10. 4
Introduction
▶ Comprehensible
▶ → Text summarization
▶ Document classification [supervised learning]
▶ Document clustering [unsupervised learning]

11. 4
Introduction
▶ Comprehensible
▶ → Text summarization
▶ Document classification [supervised learning]
▶ Document clustering [unsupervised learning]
▶ Metadata extraction
▶ etc.

12. 5
Introduction
Document clustering can assist information retrieval by creating links
between similar documents.
http://www.codeproject.com/Articles/439890/Text-Documents-Clustering-using-K-Means-Algorithm
http://www.nature.com/nmeth/journal/v8/n6/fig_tab/nmeth.1619_F1.html

13. 6
Warming up
Installation1. Python
▶ Download Anaconda at http://continuum.io/downloads
▶ Install it. (BECAREFUL! Install with All User Option)
Anaconda is a completely free Python distribution. It includes over 195 of the
most popular Python packages for science, math, engineering, data analysis.
e.g., IPython, Numpy, Scipy, Pandas, Scikit-learn, especially Networkx

14. 7
Warming up
Installation2. pip upgrade and library install
If you are windows user, edit your code
▶ Open a file ‘C:AnacondaLibsite.py’
▶ Replace
def setencoding():
encoding = “ascii”
▶ to
def setencoding():
encoding = “mbcs”

15. 8
Warming up
Installation2. pip upgrade and library install
Click windows key and type ‘cmd’, enter.
▶ python -m pip install -U pip
▶ pip install umorpheme

16. 9
IPython
Interactive Computing
IPython
▶ IPython is a command shell for interactive computing in multiple
programming languages, originally developed for the Python
programming language, that offers enhanced introspection, rich
media, additional shell syntax, tab completion, and rich history.
▶ IPython Notebook is a web-based interactive computational
environment for creating IPython notebooks. An IPython notebook
is a JSON document containing an ordered list of input/output cells
which can contain code, text, mathematics, plots and rich media.

17. 10
IPython
Interactive Computing
Execution
1. cmd
2. mkdir test
3. cd test
4. ipython notebook
If your logo is , command ‘pip install -U ipython’ for version 3(jupyter)

18. 11
IPython
Interactive Computing
New ipython notebook

19. 12
IPython
Interactive Computing

20. 13
IPython
Interactive Computing

21. 14
IPython
Interactive Computing

22. 15
IPython
Interactive Computing

23. 16
IPython
Interactive Computing

24. 17
IPython
Interactive Computing

25. 18
Python
Basic Coding
References
1. English :
https://wakari.io/nb/url///wakari.io/static/notebooks/
Lecture_1_Introduction_to_Python_Programming.ipynb
2. Korean :
https://wikidocs.net/book/1
3. Interactive Tutorial :
http://interactivepython.org/runestone/static/
pythonds/index.html
4. List of Python Learning :
https://www.codementor.io/learn-python-online

26. 19
Python
Basic Coding
References
5. Online Python :
https://www.pythonanywhere.com/try-ipython/
6. Python Visualization :
http://www.pythontutor.com/visualize.html

27. 20
Morpheme Analysis
In order to use a online morpheme analyzer, register your information at
http://information.center/korean,
After ‘Service Registration’, you can get a API Key(string with 14
length). Don’t share it with others.

28. 21
Morpheme Analysis
1 s = '여러분 반갑습니다, 유니스트에 오신 것을 환영합니다.'
2 api_url = 'http://information.center/api/korean'
3 api_key = '3WOK8DKWKS2I59'
4 data = um.analyzer(s, api_url, api_key, '유니스트', 1)

29. 22
Morpheme Analysis

30. 23
Morpheme Analysis

31. 24
Morpheme Analysis

32. 25
Morpheme Analysis
https://docs.google.com/spreadsheets/d/
1-9blXKjtjeKZqsf4NzHeYJCrr49-nXeRF6D80udfcwY/

33. 26
Morpheme Analysis

34. 27
Morpheme Analysis

35. 28
Morpheme Analysis

36. 29
Network Analysis
Next Time ...