Download to read offline
![11
Scenario
[ 0.243, 0.145, 0.600, 0.022]
corpus Prob. Topic Model
Topic 1
Topic 2
Topic 3
Dirichlet Distribution
• Exponential family distribution over the simplex,
i.e. positive vectors that sum to one](https://image.slidesharecdn.com/efficientclustering-171215125442/85/Efficient-Clustering-from-Distributions-over-Topics-11-320.jpg)
![13
Approach 2: RDC
Ranking on Dirichlet-based Clustering (RDC)
• Only considering the top n topics from the ranked list of probability distribu7ons [29]
• Based on the assump7on that the highest weighted topics have a high influence in the
rest of topics when calcula7ng distances
0.23 0.18 0.33 0.13 0.13 n=2, P =
2 0 R =
1 0 2 3 4](https://image.slidesharecdn.com/efficientclustering-171215125442/85/Efficient-Clustering-from-Distributions-over-Topics-13-320.jpg)
Uploaded byCarlos Badenes-Olmedo
Efficient Clustering from Distributions over Topics
This document presents an efficient approach for clustering large document collections based on topic distributions generated by probabilistic topic models like LDA. It proposes three approaches - TDC, RDC, and CRDC - that cluster documents based on variations or rankings of their topic distributions rather than the distributions directly. An evaluation on Apache Commons Math dataset shows CRDC improves over baselines in effectiveness, cost and efficiency metrics. Future work will explore hybrid methods combining these approaches with existing techniques.
More Related Content
Efficient Clustering from Distributions over Topics
- 1. Badenes-Olmedo, Carlos Redondo Garcia,Jose Luís Corcho, Oscar Ontology Engineering Group (OEG) Universidad Politécnica de Madrid (UPM) cbadenes@fi.upm.es @carbadol github.com/librairy oeg-upm.net Efficient Clustering from Distributions over Topics K-CAP 2017 Knowledge Capture December 4th-6th, 2017 Austin, Texas, United States
- 2.
- 3. 3 Connected Documents ina Collection
- 4. 4 From Sets ofTextual documents to Graphs
- 5.
- 6. 6 Solution: Partition ofthe Search Space
- 7. 7 Use-Case: Digital Publisher 7,648Books 68,653 Chapters 4x1.6 Ghz 16GB RAM 4x1.6 Ghz 16GB RAM 2,910,883,150 similarities aprox 8 hours 76,301 Documents All pairwise similarities approx 3.5 days
- 8. 8 Hypothesis Topic 1 Topic 2 Topic 3 Topic 4 LDA LDA for efficientspace partition Probabilis)c Topic Models (PTM) and in par)cular, on Latent Dirichlet Alloca-on (LDA) can efficiently divide the search space and speed up the process of finding rela)ons among documents inside big collec)ons.
- 9. 9 • Each topicis a distribution over words • Each document is a mixture of corpus-wide topics • Each word is drawn from one of those topics Probabilistic Topic Models source: David Blei, Probabilistic Topic Models
- 10. 10 Topic Model Similarity Valuetopic0:0.520, topic1: 0.327 topic2: 0.081 … topic122: 0.182 topic1: 0.573, topic2: 0.172 topic3: 0.136 … topic122: 0.099 0.595 .. Topic0 Topic1 Topic2 Topic122 DOCUMENT ‘A’ DOCUMENT ‘B’ Topic-based Similarity Jensen-Shannon Divergence
- 11. 11 Scenario [ 0.243, 0.145,0.600, 0.022] corpus Prob. Topic Model Topic 1 Topic 2 Topic 3 Dirichlet Distribution • Exponential family distribution over the simplex, i.e. positive vectors that sum to one
- 12. Approach 1: TDC 12 • Instead of directly relying on the topic distribu5on’s scores, it considers their varia5ons across consecu5ve topics inside a document’s topic distribu5on 0.23 0.18 0.33 0.13 0.13 P = > < = > 2 1 2 0 T = Trends on Dirichlet-based Clustering (TDC)
- 13. 13 Approach 2: RDC Rankingon Dirichlet-based Clustering (RDC) • Only considering the top n topics from the ranked list of probability distribu7ons [29] • Based on the assump7on that the highest weighted topics have a high influence in the rest of topics when calcula7ng distances 0.23 0.18 0.33 0.13 0.13 n=2, P = 2 0 R = 1 0 2 3 4
- 14. 14 Approach 3: CRDC CumulativeRanking on Dirichlet-based Clustering (CRDC)
- 15.
- 16. 16 Experiments: Baselines • 1: h%p://commons.apache.org/proper/commons-math/ SimilarityMetrics: •Jensen-Shannon Divergence •Hellinger Distance Clustering Algorithms:
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27. 27 Conclusions • Unsupervised clustering algorithms, TDC, RDC and CRDC. • CRDC is a promising approach, which improves the efficiency obtained by other centroid-based and density-based approaches such as K-Means • Hierarchical approach for RDC algorithm was also considered but it did not produce good results
- 28.
- 29. 29 Use-Case: Digital Publisher 7,648Books 68,653 Chapters 4x1.6 Ghz 16GB RAM 4x1.6 Ghz 16GB RAM 2,910,883,150 similarities aprox 8 hours 76,301 Documents by using CRDC aprox 2 hours 64,635,080 similarities threshold = 0.9 approx 3.5 days
- 30. K-CAP 2017 Knowledge Capture December4th-6th, 2017 Austin, Texas, United States Badenes-Olmedo, Carlos Redondo Garcia, Jose Luís Corcho, Oscar Ontology Engineering Group (OEG) Universidad Politécnica de Madrid (UPM) Efficient Clustering from Distributions over Topics cbadenes@fi.upm.es @carbadol github.com/librairy oeg-upm.net