SlideShare a Scribd company logo
Interactive Analysis of
Word Vector Embeddings
Florian Heimerl and Michael Gleicher
Department of Computer Sciences
University of Wisconsin - Madison
Florian Heimerl and Michael Gleicher
Department of Computer Sciences
University of Wisconsin - Madison
Interactive Analysis of
Word Vector Embeddings
Summary:
Word vector embeddings offer unique challenges
Task analysis of needs 3 Designs for unmet needs
Buddy Plots
Concept Axis Plots
Co-occurance Matrices
What is an embedding?
General mathematics:
Place a smaller structure into a
larger structure
Computer science:
Place a discrete set of objects
into a vector space
Encode relationships between
objects
Paris
4,8,1,3,…
Atlanta
5,2,1,7,…New York
4,8,1,3,…
London
5,2,1,7,…
Boston
3,2,5,1,…
Tokyo
9,2,6,4,…
Beijing
7,3,2,7,…
San Jose
5,2,1,7,…
Jakarta
3,2,5,1,…
Sydney
9,2,6,4,…
Munich
7,3,2,7,…
High Dimensional Data
Objects have associated Vectors
Kinds of relationships in embeddings
Distance
A is closer to B than to C
B
C
D
A
E
Linear Structure
A is to B as C is to D
Semantic Directions
A is more X than C
B
C
D
AF
F
E
B
A
C
D
Relationships are interesting even if global positions are not
Word vector embeddings
Place words in a high-dimensional vector space
Words similar in meaning should be close in space
Infer similarity by distributional semantics:
similar context implies similar meaning
Construct embeddings by processing a corpus of text
my pet cat is brown
my pet dog is brown
my big car is brown
Several ways to build embeddings
Word2Vec
Skip-gram model
Neural embedding
GLoVE
Co-occurrence model
Factor matrix by optimization
Why use Word Vector Embeddings?
Learn about
Language or Corpora (Texts)
Find similar words/synonyms
Track changes of word usage
Exploring polysemy
Creating lexical resources
Evidence of bias
…
Natural Language Applications
Pre-Processing
Translation
Sentiment Analysis
Interpretation
…
Challenges of Word Vector Embeddings
Large numbers of Words
High-dimensional spaces
Complex relationships – meaningless positions
Complex processes for building embeddings
Complex downstream applications
No ground truth - subjective aspects
Why visualization?
Tasks are inherently human-centric
Variety of tasks involving interpretation
Linguistic and domain knowledge for applications
But what are those tasks?
Task Analysis:
What do people do with Word Embeddings?
Literature survey
111 papers from diverse communities
Consider use cases in Linguistics, HCI, Digitial Humanities, etc.
Augment this list by extrapolation
what tasks would users want – but aren’t doing yet
Use cases suggest tasks
Learn about
Language or Corpora (Texts)
Find similar words/synonyms
Track changes of word usage
Exploring polysemy
Creating lexical resources
Natural Language Applications
Pre-Processing
Translation
Sentiment Analysis
Interpretation
Evaluation
Intrinsic (good embedding?)
Extrinsic (applications success?)
Interpretation
Identify items of interest
Probe values of interest
Linguistic Tasks and Characteristics
We identified 7 distinct linguistic tasks within the literature
Two ways those tasks are relevant
• Automatically test embeddings through human-curated resources
• Users probe embedding
4 characteristics pertinent to those tasks: similarity, arithmetic
structures, concept axis, and co-occurrences
13
Prior Work:
Word Vector Embedding Visualizations
Liu et al., InfoVis 2017
Analogy Relationships
Smilkov et al., 2016
Distance Relationships
Rong and Adar, 2016
Training Process
Tasks and Characteristics vs. Visualizations
Rank word pairs similarity
View neighbors similarity
Select synonyms similarity
Compare concepts average, similarity
Find analogies offset, similarity
Project on Concept concept axis
Predict Contexts co-oc. probability
Smilkov, et al 2016
Liu, et al 2017
Tasks and Characteristics vs. Visualizations
Rank word pairs similarity
View neighbors similarity
Select synonyms similarity
Compare concepts average, similarity
Find analogies offset, similarity
Project on Concept concept axis
Predict Contexts co-oc. probability
Smilkov, et al 2016
Liu, et al 2017
Tasks we seek designs for in this paper
Buddy Plots
Concept Axis Plots
Co-occurance Matrices
1. Similarities (local distances)
2. Co-Occurrances
3. Concept Axes
Buddy Plots
Concept Axis Plots
Co-occurance Matrices
1. Similarities (local distances)
2. Co-Occurrances
3. Concept Axes
Similarities:
Understanding local distances
Distances are meaningful
even if absolute values are not
What is close to a word?
Are there groups of words that are similar?
Ordered lists are useful
Density (how many can you show)
Sense of relative distances
Comparison between words
Embedding Projector
Smilkov, et al. 2016
Buddy Plots (1D lists)
Alexander and Gleicher, 2016 – for Topic Models
Map distance (to selected reference) to horizontal axis
Reference
object
Closest
to ref
Not dimensionality reduction?
Map distance to word to horizontal axis
Focus on a single point – other relations not preserved
?
Embedding Projector
Smilkov, et al. 2016
Stacked/Chained buddy plots
Use color to encode distance
Stacked/Chained buddy plots
Use color to encode distance in the reference row
A
AB
B
Stacked/Chained buddy plots
Use color to encode distance in the reference
Interactive Analysis of Word Vector Embeddings
Same word… different embedding
Broadcast
Buddy Plots
Concept Axis Plots
Co-occurance Matrices
1. Similarities (local distances)
2. Co-Occurrances
3. Concept Axes
Why are words similar?
Understandng word co-occurence
Similarity based on co-occurrence
count how often one word occurs near another
Co-occurrence matrix
main form of input data
many models approximate the matrix (reconstruct)
Useful for understanding and diagnosing models
Co-occurrence view
How do we view the massive matrix?
color encoding (heat map) – density, relative values
select rows (specify words of interest)
select columns (metrics of interestingness – given rows)
highest values
highest variance
Co-occurrence matrix view
Matrix comparison view
High variance words in one embedding may be low in the other
Buddy Plots
Concept Axis Plots
Co-occurance Matrices
1. Similarities (local distances)
2. Co-Occurrances
3. Concept Axes
Concept Axes:
Understanding Semantic Directions
Opposing concepts
make an axis
Food
Pet
Concept Axes:
Understanding Concept Axes
Define an axis from
one concept to
another
Food
Pet
Dog
Cat
Goldfish
Chicken
Cow
Trout
Ways to define axes
Vector between two concepts
Interaxis - Kim et al., 2015
Classifier between two groups
Explainers – Gleicher, 2013
Food
Pet
Dog
Cat
Goldfish
Chicken
Cow
Trout
Multiple Concept Axes
Use multi-variate plots
2D = Scatterplot
Pet Food
SmallBig
Scatterplot Interaction
man woman
academic
worker
Interactive Analysis of Word Vector Embeddings
man woman
academic
worker
Interactive Analysis of Word Vector Embeddings
Interactive Analysis of Word Vector Embeddings
Application: Word meaning change
Application: Stability Assessment
Implementation
http://graphics.cs.wisc.edu/Vis/EmbVis/
Everything runs on line (simplified interfaces)
cloud version uses small models
Python backend / D3 frontend
Limitations
Implementation
Effectiveness
Completeness
Usability and Scalability
Evaluation (of designs)
More Tasks
Identifying Probes
Explicit Comparison
Connection to (model) evaluation
Feedback to model building
Summary:
Word vector embeddings offer unique challenges
Task analysis of needs Designs for 3 unmet needs
Buddy Plots
Concept Axis Plots
Co-occurance Matrices
Acknowledgments
This work is funded in part by NSF 1162037
and DARPA FA8750-17-2-0107
http://graphics.cs.wisc.edu/Vis/EmbVis/

More Related Content

What's hot

A Comprehensive Review of Large Language Models for.pptx
A Comprehensive Review of Large Language Models for.pptxA Comprehensive Review of Large Language Models for.pptx
A Comprehensive Review of Large Language Models for.pptx
SaiPragnaKancheti
 
Probabilistic information retrieval models & systems
Probabilistic information retrieval models & systemsProbabilistic information retrieval models & systems
Probabilistic information retrieval models & systems
Selman Bozkır
 
Fraud Detection and Prevention on AWS using Machine Learning
Fraud Detection and Prevention on AWS using Machine LearningFraud Detection and Prevention on AWS using Machine Learning
Fraud Detection and Prevention on AWS using Machine Learning
Amazon Web Services
 
Tutorial on Question Answering Systems
Tutorial on Question Answering Systems Tutorial on Question Answering Systems
Tutorial on Question Answering Systems
Saeedeh Shekarpour
 
Recommendation Systems - Why How and Real Life Applications
Recommendation Systems - Why How and Real Life ApplicationsRecommendation Systems - Why How and Real Life Applications
Recommendation Systems - Why How and Real Life Applications
Liron Zighelnic
 
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Edureka!
 
Netflix Recommendations - Beyond the 5 Stars
Netflix Recommendations - Beyond the 5 StarsNetflix Recommendations - Beyond the 5 Stars
Netflix Recommendations - Beyond the 5 Stars
Xavier Amatriain
 
Knowledge graph use cases in natural language generation
Knowledge graph use cases in natural language generationKnowledge graph use cases in natural language generation
Knowledge graph use cases in natural language generation
Elena Simperl
 
Conditional Random Fields
Conditional Random FieldsConditional Random Fields
Conditional Random Fields
lswing
 
Object Detection with Transformers
Object Detection with TransformersObject Detection with Transformers
Object Detection with Transformers
Databricks
 
Knowledge graphs + Chatbots with Neo4j
Knowledge graphs + Chatbots with Neo4jKnowledge graphs + Chatbots with Neo4j
Knowledge graphs + Chatbots with Neo4j
Christophe Willemsen
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filtering
Viet-Trung TRAN
 
Understanding GenAI/LLM and What is Google Offering - Felix Goh
Understanding GenAI/LLM and What is Google Offering - Felix GohUnderstanding GenAI/LLM and What is Google Offering - Felix Goh
Understanding GenAI/LLM and What is Google Offering - Felix Goh
NUS-ISS
 
Applied Data Science Course Part 1: Concepts & your first ML model
Applied Data Science Course Part 1: Concepts & your first ML modelApplied Data Science Course Part 1: Concepts & your first ML model
Applied Data Science Course Part 1: Concepts & your first ML model
Dataiku
 
Approximate nearest neighbor methods and vector models – NYC ML meetup
Approximate nearest neighbor methods and vector models – NYC ML meetupApproximate nearest neighbor methods and vector models – NYC ML meetup
Approximate nearest neighbor methods and vector models – NYC ML meetup
Erik Bernhardsson
 
How to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on SparkHow to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on Spark
Caserta
 
Sentiment Analysis
Sentiment AnalysisSentiment Analysis
Sentiment Analysis
Data Science Society
 
Language models
Language modelsLanguage models
Language models
Maryam Khordad
 
Featurizing log data before XGBoost
Featurizing log data before XGBoostFeaturizing log data before XGBoost
Featurizing log data before XGBoost
DataRobot
 
Sentiment analysis
Sentiment analysisSentiment analysis
Sentiment analysis
Makrand Patil
 

What's hot (20)

A Comprehensive Review of Large Language Models for.pptx
A Comprehensive Review of Large Language Models for.pptxA Comprehensive Review of Large Language Models for.pptx
A Comprehensive Review of Large Language Models for.pptx
 
Probabilistic information retrieval models & systems
Probabilistic information retrieval models & systemsProbabilistic information retrieval models & systems
Probabilistic information retrieval models & systems
 
Fraud Detection and Prevention on AWS using Machine Learning
Fraud Detection and Prevention on AWS using Machine LearningFraud Detection and Prevention on AWS using Machine Learning
Fraud Detection and Prevention on AWS using Machine Learning
 
Tutorial on Question Answering Systems
Tutorial on Question Answering Systems Tutorial on Question Answering Systems
Tutorial on Question Answering Systems
 
Recommendation Systems - Why How and Real Life Applications
Recommendation Systems - Why How and Real Life ApplicationsRecommendation Systems - Why How and Real Life Applications
Recommendation Systems - Why How and Real Life Applications
 
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
Stemming And Lemmatization Tutorial | Natural Language Processing (NLP) With ...
 
Netflix Recommendations - Beyond the 5 Stars
Netflix Recommendations - Beyond the 5 StarsNetflix Recommendations - Beyond the 5 Stars
Netflix Recommendations - Beyond the 5 Stars
 
Knowledge graph use cases in natural language generation
Knowledge graph use cases in natural language generationKnowledge graph use cases in natural language generation
Knowledge graph use cases in natural language generation
 
Conditional Random Fields
Conditional Random FieldsConditional Random Fields
Conditional Random Fields
 
Object Detection with Transformers
Object Detection with TransformersObject Detection with Transformers
Object Detection with Transformers
 
Knowledge graphs + Chatbots with Neo4j
Knowledge graphs + Chatbots with Neo4jKnowledge graphs + Chatbots with Neo4j
Knowledge graphs + Chatbots with Neo4j
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filtering
 
Understanding GenAI/LLM and What is Google Offering - Felix Goh
Understanding GenAI/LLM and What is Google Offering - Felix GohUnderstanding GenAI/LLM and What is Google Offering - Felix Goh
Understanding GenAI/LLM and What is Google Offering - Felix Goh
 
Applied Data Science Course Part 1: Concepts & your first ML model
Applied Data Science Course Part 1: Concepts & your first ML modelApplied Data Science Course Part 1: Concepts & your first ML model
Applied Data Science Course Part 1: Concepts & your first ML model
 
Approximate nearest neighbor methods and vector models – NYC ML meetup
Approximate nearest neighbor methods and vector models – NYC ML meetupApproximate nearest neighbor methods and vector models – NYC ML meetup
Approximate nearest neighbor methods and vector models – NYC ML meetup
 
How to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on SparkHow to Build a Recommendation Engine on Spark
How to Build a Recommendation Engine on Spark
 
Sentiment Analysis
Sentiment AnalysisSentiment Analysis
Sentiment Analysis
 
Language models
Language modelsLanguage models
Language models
 
Featurizing log data before XGBoost
Featurizing log data before XGBoostFeaturizing log data before XGBoost
Featurizing log data before XGBoost
 
Sentiment analysis
Sentiment analysisSentiment analysis
Sentiment analysis
 

Similar to Interactive Analysis of Word Vector Embeddings

Interpreting Embeddings with Comparison
Interpreting Embeddings with ComparisonInterpreting Embeddings with Comparison
Interpreting Embeddings with Comparison
gleicher
 
Interpreting Embeddings with Comparison
Interpreting Embeddings with ComparisonInterpreting Embeddings with Comparison
Interpreting Embeddings with Comparison
gleicher
 
The Duet model
The Duet modelThe Duet model
The Duet model
Bhaskar Mitra
 
Data Integration Ontology Mapping
Data Integration Ontology MappingData Integration Ontology Mapping
Data Integration Ontology Mapping
Pradeep B Pillai
 
The Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for HindiThe Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for Hindi
singhg77
 
Chat bot using text similarity approach
Chat bot using text similarity approachChat bot using text similarity approach
Chat bot using text similarity approach
dinesh_joshy
 
A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...
A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...
A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...
acijjournal
 
Semantic Based Model for Text Document Clustering with Idioms
Semantic Based Model for Text Document Clustering with IdiomsSemantic Based Model for Text Document Clustering with Idioms
Semantic Based Model for Text Document Clustering with Idioms
Waqas Tariq
 
Dissertation defense slides on "Semantic Analysis for Improved Multi-document...
Dissertation defense slides on "Semantic Analysis for Improved Multi-document...Dissertation defense slides on "Semantic Analysis for Improved Multi-document...
Dissertation defense slides on "Semantic Analysis for Improved Multi-document...
Quinsulon Israel
 
Meaningful Interaction Analysis
Meaningful Interaction AnalysisMeaningful Interaction Analysis
Meaningful Interaction Analysis
fridolin.wild
 
The Geometry of Learning
The Geometry of LearningThe Geometry of Learning
The Geometry of Learning
fridolin.wild
 
Eurolan 2005 Pedersen
Eurolan 2005 PedersenEurolan 2005 Pedersen
Eurolan 2005 Pedersen
University of Minnesota, Duluth
 
{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...
{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...
{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...
Amit Sheth
 
IRJET- Short-Text Semantic Similarity using Glove Word Embedding
IRJET- Short-Text Semantic Similarity using Glove Word EmbeddingIRJET- Short-Text Semantic Similarity using Glove Word Embedding
IRJET- Short-Text Semantic Similarity using Glove Word Embedding
IRJET Journal
 
5 Lessons Learned from Designing Neural Models for Information Retrieval
5 Lessons Learned from Designing Neural Models for Information Retrieval5 Lessons Learned from Designing Neural Models for Information Retrieval
5 Lessons Learned from Designing Neural Models for Information Retrieval
Bhaskar Mitra
 
IDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATA
IDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATAIDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATA
IDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATA
ijistjournal
 
Identifying the semantic relations on
Identifying the semantic relations onIdentifying the semantic relations on
Identifying the semantic relations on
ijistjournal
 
Diagrammatic knowledge modeling for managers – ontology-based approach
Diagrammatic knowledge modeling for managers  – ontology-based approachDiagrammatic knowledge modeling for managers  – ontology-based approach
Diagrammatic knowledge modeling for managers – ontology-based approach
Dmitry Kudryavtsev
 
Neural Models for Information Retrieval
Neural Models for Information RetrievalNeural Models for Information Retrieval
Neural Models for Information Retrieval
Bhaskar Mitra
 
Ontology
OntologyOntology

Similar to Interactive Analysis of Word Vector Embeddings (20)

Interpreting Embeddings with Comparison
Interpreting Embeddings with ComparisonInterpreting Embeddings with Comparison
Interpreting Embeddings with Comparison
 
Interpreting Embeddings with Comparison
Interpreting Embeddings with ComparisonInterpreting Embeddings with Comparison
Interpreting Embeddings with Comparison
 
The Duet model
The Duet modelThe Duet model
The Duet model
 
Data Integration Ontology Mapping
Data Integration Ontology MappingData Integration Ontology Mapping
Data Integration Ontology Mapping
 
The Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for HindiThe Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for Hindi
 
Chat bot using text similarity approach
Chat bot using text similarity approachChat bot using text similarity approach
Chat bot using text similarity approach
 
A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...
A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...
A COMPARATIVE STUDY OF ROOT-BASED AND STEM-BASED APPROACHES FOR MEASURING THE...
 
Semantic Based Model for Text Document Clustering with Idioms
Semantic Based Model for Text Document Clustering with IdiomsSemantic Based Model for Text Document Clustering with Idioms
Semantic Based Model for Text Document Clustering with Idioms
 
Dissertation defense slides on "Semantic Analysis for Improved Multi-document...
Dissertation defense slides on "Semantic Analysis for Improved Multi-document...Dissertation defense slides on "Semantic Analysis for Improved Multi-document...
Dissertation defense slides on "Semantic Analysis for Improved Multi-document...
 
Meaningful Interaction Analysis
Meaningful Interaction AnalysisMeaningful Interaction Analysis
Meaningful Interaction Analysis
 
The Geometry of Learning
The Geometry of LearningThe Geometry of Learning
The Geometry of Learning
 
Eurolan 2005 Pedersen
Eurolan 2005 PedersenEurolan 2005 Pedersen
Eurolan 2005 Pedersen
 
{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...
{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...
{Ontology: Resource} x {Matching : Mapping} x {Schema : Instance} :: Compone...
 
IRJET- Short-Text Semantic Similarity using Glove Word Embedding
IRJET- Short-Text Semantic Similarity using Glove Word EmbeddingIRJET- Short-Text Semantic Similarity using Glove Word Embedding
IRJET- Short-Text Semantic Similarity using Glove Word Embedding
 
5 Lessons Learned from Designing Neural Models for Information Retrieval
5 Lessons Learned from Designing Neural Models for Information Retrieval5 Lessons Learned from Designing Neural Models for Information Retrieval
5 Lessons Learned from Designing Neural Models for Information Retrieval
 
IDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATA
IDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATAIDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATA
IDENTIFYING THE SEMANTIC RELATIONS ON UNSTRUCTURED DATA
 
Identifying the semantic relations on
Identifying the semantic relations onIdentifying the semantic relations on
Identifying the semantic relations on
 
Diagrammatic knowledge modeling for managers – ontology-based approach
Diagrammatic knowledge modeling for managers  – ontology-based approachDiagrammatic knowledge modeling for managers  – ontology-based approach
Diagrammatic knowledge modeling for managers – ontology-based approach
 
Neural Models for Information Retrieval
Neural Models for Information RetrievalNeural Models for Information Retrieval
Neural Models for Information Retrieval
 
Ontology
OntologyOntology
Ontology
 

Recently uploaded

New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
tanupasswan6
 
Celonis Busniess Analyst Virtual Internship.pptx
Celonis Busniess Analyst Virtual Internship.pptxCelonis Busniess Analyst Virtual Internship.pptx
Celonis Busniess Analyst Virtual Internship.pptx
AnujaGaikwad28
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
ginni singh$A17
 
Willis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdfWillis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdf
LINAT
 
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
6459astrid
 
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
NABLAS株式会社
 
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
kuldeepsharmaks8120
 
DataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptxDataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptx
Kanchana Weerasinghe
 
Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...
Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...
Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...
kinni singh$A17
 
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
saadkhan1485265
 
CHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptxCHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptx
girewiy968
 
Potential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriatePotential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriate
huseindihon
 
potential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in generalpotential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in general
huseindihon
 
Oracle Database Desupported Features on 23ai (Part A)
Oracle Database Desupported Features on 23ai (Part A)Oracle Database Desupported Features on 23ai (Part A)
Oracle Database Desupported Features on 23ai (Part A)
Alireza Kamrani
 
Harendra Singh, AI Strategy and Consulting Portfolio
Harendra Singh, AI Strategy and Consulting PortfolioHarendra Singh, AI Strategy and Consulting Portfolio
Harendra Singh, AI Strategy and Consulting Portfolio
harendmgr
 
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
sharonblush
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
ginni singh$A17
 
transgenders community data in india by govt
transgenders community data in india by govttransgenders community data in india by govt
transgenders community data in india by govt
palanisamyiiiier
 
Nipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma TranscriptNipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma Transcript
zyqedad
 
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeliveryBDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
erynsouthern
 

Recently uploaded (20)

New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
 
Celonis Busniess Analyst Virtual Internship.pptx
Celonis Busniess Analyst Virtual Internship.pptxCelonis Busniess Analyst Virtual Internship.pptx
Celonis Busniess Analyst Virtual Internship.pptx
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
 
Willis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdfWillis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdf
 
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
 
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Nashik  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Nashik 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
 
DataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptxDataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptx
 
Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...
Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...
Noida Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Ava...
 
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
 
CHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptxCHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptx
 
Potential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriatePotential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriate
 
potential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in generalpotential usefulness of multi-agent maze-solving in general
potential usefulness of multi-agent maze-solving in general
 
Oracle Database Desupported Features on 23ai (Part A)
Oracle Database Desupported Features on 23ai (Part A)Oracle Database Desupported Features on 23ai (Part A)
Oracle Database Desupported Features on 23ai (Part A)
 
Harendra Singh, AI Strategy and Consulting Portfolio
Harendra Singh, AI Strategy and Consulting PortfolioHarendra Singh, AI Strategy and Consulting Portfolio
Harendra Singh, AI Strategy and Consulting Portfolio
 
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
 
transgenders community data in india by govt
transgenders community data in india by govttransgenders community data in india by govt
transgenders community data in india by govt
 
Nipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma TranscriptNipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma Transcript
 
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeliveryBDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
 

Interactive Analysis of Word Vector Embeddings

  • 1. Interactive Analysis of Word Vector Embeddings Florian Heimerl and Michael Gleicher Department of Computer Sciences University of Wisconsin - Madison
  • 2. Florian Heimerl and Michael Gleicher Department of Computer Sciences University of Wisconsin - Madison Interactive Analysis of Word Vector Embeddings
  • 3. Summary: Word vector embeddings offer unique challenges Task analysis of needs 3 Designs for unmet needs Buddy Plots Concept Axis Plots Co-occurance Matrices
  • 4. What is an embedding? General mathematics: Place a smaller structure into a larger structure Computer science: Place a discrete set of objects into a vector space Encode relationships between objects Paris 4,8,1,3,… Atlanta 5,2,1,7,…New York 4,8,1,3,… London 5,2,1,7,… Boston 3,2,5,1,… Tokyo 9,2,6,4,… Beijing 7,3,2,7,… San Jose 5,2,1,7,… Jakarta 3,2,5,1,… Sydney 9,2,6,4,… Munich 7,3,2,7,… High Dimensional Data Objects have associated Vectors
  • 5. Kinds of relationships in embeddings Distance A is closer to B than to C B C D A E Linear Structure A is to B as C is to D Semantic Directions A is more X than C B C D AF F E B A C D Relationships are interesting even if global positions are not
  • 6. Word vector embeddings Place words in a high-dimensional vector space Words similar in meaning should be close in space Infer similarity by distributional semantics: similar context implies similar meaning Construct embeddings by processing a corpus of text my pet cat is brown my pet dog is brown my big car is brown
  • 7. Several ways to build embeddings Word2Vec Skip-gram model Neural embedding GLoVE Co-occurrence model Factor matrix by optimization
  • 8. Why use Word Vector Embeddings? Learn about Language or Corpora (Texts) Find similar words/synonyms Track changes of word usage Exploring polysemy Creating lexical resources Evidence of bias … Natural Language Applications Pre-Processing Translation Sentiment Analysis Interpretation …
  • 9. Challenges of Word Vector Embeddings Large numbers of Words High-dimensional spaces Complex relationships – meaningless positions Complex processes for building embeddings Complex downstream applications No ground truth - subjective aspects
  • 10. Why visualization? Tasks are inherently human-centric Variety of tasks involving interpretation Linguistic and domain knowledge for applications But what are those tasks?
  • 11. Task Analysis: What do people do with Word Embeddings? Literature survey 111 papers from diverse communities Consider use cases in Linguistics, HCI, Digitial Humanities, etc. Augment this list by extrapolation what tasks would users want – but aren’t doing yet
  • 12. Use cases suggest tasks Learn about Language or Corpora (Texts) Find similar words/synonyms Track changes of word usage Exploring polysemy Creating lexical resources Natural Language Applications Pre-Processing Translation Sentiment Analysis Interpretation Evaluation Intrinsic (good embedding?) Extrinsic (applications success?) Interpretation Identify items of interest Probe values of interest
  • 13. Linguistic Tasks and Characteristics We identified 7 distinct linguistic tasks within the literature Two ways those tasks are relevant • Automatically test embeddings through human-curated resources • Users probe embedding 4 characteristics pertinent to those tasks: similarity, arithmetic structures, concept axis, and co-occurrences 13
  • 14. Prior Work: Word Vector Embedding Visualizations Liu et al., InfoVis 2017 Analogy Relationships Smilkov et al., 2016 Distance Relationships Rong and Adar, 2016 Training Process
  • 15. Tasks and Characteristics vs. Visualizations Rank word pairs similarity View neighbors similarity Select synonyms similarity Compare concepts average, similarity Find analogies offset, similarity Project on Concept concept axis Predict Contexts co-oc. probability Smilkov, et al 2016 Liu, et al 2017
  • 16. Tasks and Characteristics vs. Visualizations Rank word pairs similarity View neighbors similarity Select synonyms similarity Compare concepts average, similarity Find analogies offset, similarity Project on Concept concept axis Predict Contexts co-oc. probability Smilkov, et al 2016 Liu, et al 2017 Tasks we seek designs for in this paper
  • 17. Buddy Plots Concept Axis Plots Co-occurance Matrices 1. Similarities (local distances) 2. Co-Occurrances 3. Concept Axes
  • 18. Buddy Plots Concept Axis Plots Co-occurance Matrices 1. Similarities (local distances) 2. Co-Occurrances 3. Concept Axes
  • 19. Similarities: Understanding local distances Distances are meaningful even if absolute values are not What is close to a word? Are there groups of words that are similar? Ordered lists are useful Density (how many can you show) Sense of relative distances Comparison between words Embedding Projector Smilkov, et al. 2016
  • 20. Buddy Plots (1D lists) Alexander and Gleicher, 2016 – for Topic Models Map distance (to selected reference) to horizontal axis Reference object Closest to ref
  • 21. Not dimensionality reduction? Map distance to word to horizontal axis Focus on a single point – other relations not preserved ?
  • 23. Stacked/Chained buddy plots Use color to encode distance
  • 24. Stacked/Chained buddy plots Use color to encode distance in the reference row A AB B
  • 25. Stacked/Chained buddy plots Use color to encode distance in the reference
  • 27. Same word… different embedding Broadcast
  • 28. Buddy Plots Concept Axis Plots Co-occurance Matrices 1. Similarities (local distances) 2. Co-Occurrances 3. Concept Axes
  • 29. Why are words similar? Understandng word co-occurence Similarity based on co-occurrence count how often one word occurs near another Co-occurrence matrix main form of input data many models approximate the matrix (reconstruct) Useful for understanding and diagnosing models
  • 30. Co-occurrence view How do we view the massive matrix? color encoding (heat map) – density, relative values select rows (specify words of interest) select columns (metrics of interestingness – given rows) highest values highest variance
  • 32. Matrix comparison view High variance words in one embedding may be low in the other
  • 33. Buddy Plots Concept Axis Plots Co-occurance Matrices 1. Similarities (local distances) 2. Co-Occurrances 3. Concept Axes
  • 34. Concept Axes: Understanding Semantic Directions Opposing concepts make an axis Food Pet
  • 35. Concept Axes: Understanding Concept Axes Define an axis from one concept to another Food Pet Dog Cat Goldfish Chicken Cow Trout
  • 36. Ways to define axes Vector between two concepts Interaxis - Kim et al., 2015 Classifier between two groups Explainers – Gleicher, 2013 Food Pet Dog Cat Goldfish Chicken Cow Trout
  • 37. Multiple Concept Axes Use multi-variate plots 2D = Scatterplot Pet Food SmallBig
  • 46. Implementation http://graphics.cs.wisc.edu/Vis/EmbVis/ Everything runs on line (simplified interfaces) cloud version uses small models Python backend / D3 frontend
  • 47. Limitations Implementation Effectiveness Completeness Usability and Scalability Evaluation (of designs) More Tasks Identifying Probes Explicit Comparison Connection to (model) evaluation Feedback to model building
  • 48. Summary: Word vector embeddings offer unique challenges Task analysis of needs Designs for 3 unmet needs Buddy Plots Concept Axis Plots Co-occurance Matrices Acknowledgments This work is funded in part by NSF 1162037 and DARPA FA8750-17-2-0107 http://graphics.cs.wisc.edu/Vis/EmbVis/