Text Processing with KNIME

Copyright © 2014 KNIME.com AG
Boston KNIME Users
Text Processing Applications
Kilian Thiel
KNIME

Agenda
• KNIME Crash Course
• Text Mining with KNIME: Mining Tripadvisor Data
• Text Mining with KNIME: Mining Amazon Reviews
(Anil Tarachandani)
• Networking Apero
2

Text Mining with KNIME: Mining Tripadvisor Data
Agenda
• The KNIME Textprocessing Extension
– Preliminaries
– Philosophy & Usage
• Classification of Tripadvisor Reviews
– Tripadvisor data
– Classification of reviews
3

Resources
http://tech.knime.org/knime-text-processing
• Documentation
• Examples
• Forum
• White Papers
4

Installation
5
1.) 2.)

Requirements
Requirements to import and run demo workflows
• KNIME 2.10
• Textprocessing (labs)
• Distance Matrix (KNIME)
• Palladian (Community)
6

Tips
• Settings (knime.ini)
– Set maximum memory for KNIME
– -Xmx3G
7

Demo
Prepare KNIME
• Go to KNIME directory
• Change knime.ini file (optional)
– -Xmx3G
• Start KNIME
• Install Textprocessing Extension
– (or better have it already installed)
8

Philosophy
9
… perhaps your name
is
Rumpelstiltskin[Perso
n] ? …
… perhaps your name
is
Rumpelstiltskin[Perso
n] ? … Visualization
Cluster-
ing
Classifi-
cation
1 1 1 0 1 0 0 1 1
0 1 1 0 0 1 0 0 0
0 0 1 1 1 0 1 1 0

Additional Data Types
• Document Cell
– Encapsulates a document
• Title, sentences, terms, words
• Authors, category, source
• Generic meta data (key, value pairs)
• Term Cell
– Encapsulates a term
• Words, tags
10

Data Table Structures
• Document table
– List of documents
• Bag of words
– Tuples of documents
and terms
• Document vectors
– Numerical
representations of
documents
11

Philosophy and Data Table Structures
12
Enrichment Preprocessing
1 1 1 0
1 0 0 1
Documents Bow VectorsDocuments Documents

Tripadvisor Data
13
Title
Author
Rating
Fulltext

Tripadvisor Data
14
Reviews about italian and chinese restaurants in
Boston
• Chinese: 272
• Italian: 268

Tripadvisor Data
15
Goal:
• Build classifier to distinguish between chinese and
italian restaurants, based on their reviews.
Review about italian or
chinese restaurant?

Tripadvisor Data
16
Goal:

1.) Reading
Read/Parse textual data
17

Demo
Reading
• Read Tripadvisor data (.table file)
• Filter rows with missing restaurant value
• Convert strings to documents
• Filter all but the document column
18

2.) Enrichment
Enrich documents with semantic information
19

Demo
Enrichment / Tagging
• Apply POS Tagger node
• Use Bag of Words node to inspect tagging result
20

3.) Preprocessing
Preprocess documents and filter words
21

Demo
Preprocessing
• Filter
– Numbers
– Punctuation marks
– Stop Words
• Convert to lower case
• Stemming
• Keep only nouns, verbs, adjectives
22

4.) Transformation
Creation of numerical representation of documents
23

Demo
Transformation
• Transform to bag of word
• Compute TF value for terms
• Transform to document vectors
• Extract category (class) value
24

5.) Classification
Training of a model (decision tree) and scoring
25

Demo
Classification
• Append color based on class
• Partition data into training and test set
• Train decision tree model in training data
• Apply decision tree model on test data
• Score model, measure accuracy
26

Additional Workflows
• Multi Word Tagging
– Detection of frequent Ngrams
– Creation of dictionary from Ngrams
– Applying Dictionary Tagger
• Classification with Multi Words
• Clustering of documents
27

Thank You
40k
60k
20k
28
Questions
• http://tech.knime.org/forum
• Kilian.Thiel@knime.com
Follow us
• Twitter: @KNIME
• LinkedIn: https://www.linkedin.com/groups?gid=2212172
• KNIME Blog: http://www.knime.org/blog

Text Processing with KNIME

More Related Content

What's hot

Viewers also liked

Similar to Text Processing with KNIME

More from KNIMESlides

Recently uploaded

Text Processing with KNIME