Towards Twitter hashtag recommendation using distributed word representations and a deep feed forward neural network

ELIS – Multimedia Lab
Towards Twitter Hashtag Recommendation Using
Distributed Word Representations and a Deep Feed
Forward Neural Network
CSSC-2014
New Delhi, 24 September 2014
Abhineshwar Tomar, Frederic Godin, Baptist Vandersmissen,
Wesley De Neve, Rik Van de Walle
Multimedia Lab, Ghent University – iMinds, Belgium
Image and Video Systems Lab, KAIST, South Korea

2
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

3
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

4
Twitter
• An online social network service that enables users to send and read
short 140-character text messages, called "tweets" or "microposts"
Hashtag
(starts with #)
Tweet or
Mention
(starts with @)
Favorite
(like or
bookmark)
Retweet micropost
(sharing)

5
Famous Tweets
Note the presence of both textual and (embedded) visual information!

6
• Usage in general
Twitter Statistics
- 271 million monthly active users
- 500 million Tweets are sent per day
• Hashtags
- Only 8% of the tweets contain hashtags
- 3% of the hashtags are used more than 5 times

7
Hashtags on Twitter
Hashtag usage:
- topic-based indexing & search
• #socialnetwork
• #Reddit
- conversational/event clustering
• #www2014
Observation: only 8% of tweets contain a hashtag

8
 Introduction
 Goal
 Why
 Methodology
 Results
 Conclusion
 Future work
Overview

9
Goal
Generate hashtags that adhere to the
semantic and linguistic regularity of a tweet

10
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

11
• Hashtags
Why
- Content categorization and discovery
- Effective search of tweets
• Our approach
- Connect similar hashtags (topics)
- Promote the use of hashtags
• By understanding the semantics of the tweet

12
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

13
• Preprocessing
- Remove non-English words
- Remove non-ASCII characters
- Remove mentions (@USER)
- Remove URLs
- Remove RT @from retweets
• Feature vector generation
• Training of a feed forward neural network
• Evaluation
Methodology (1/3)

14
Methodology (2/3)
• Training: learning the relation between tweets and hashtags
Tweet Hashtag
word2vec
300-D
tweet
vector
word2vec
300-D
hashtag
vector
Deep feed-forward
neural
network
300-D input layer
1000-D hidden layer
500-D hidden layer
400-D hidden layer
300-D output layer
Elizabeth Warren Taking on
Hillary as New Democratic
Powerhouse
#politics

15
Methodology (3/3)
• Testing: recommending hashtags to tweets
word2vec
300-D
tweet
vector
300-D
hashtag
vector
Deep feed-forward
neural
network
300-D input layer
1000-D hidden layer
500-D hidden layer
400-D hidden layer
300-D output layer
Tweet
House Democrats suggest
Obama impeachment is
imminent to raise cash
vec2word
Hashtag
Hashtag
Hashtag
Hashtags
#politics
#crisis

16
word2vec
• Developed by Google Research
• Computes vector representations for words
- Through the use of neural network technology
• Trained on part of the Google News dataset (+/- 100 billion words)
• The model contains vectors for 3 million words and phrases
- Capture the semantic meaning of a word
• Example word vector properties
- vector('Paris') - vector('France') + vector('Italy') ≈ vector('Rome')
- vector('king') - vector('man') + vector('woman') ≈ vector('queen')

17
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

18
Results (1/3)
Tweet Recommended hashtags
1 Someone dm/text me bc I’m so bored madd, Oh noes, rainnwilson,
sooooooo, fricken
2 The good life is one inspired by love and guided by
knowledge.
Ahh yes, FIVE THINGS About,
YANKEES TALK, Kinder gentler,
Ya gotta love
3 Method of Losing Weight http://t.co/rs64CEuo5W Shape Shifting, Treat Acne, Detect
Cancer, Warps, Calorie Burn
4 I hate today cause its room cleaning day for me!!! FAN ’S ATTIC, Puh leez, Mopping
robot, % #F######## 3v.jsn, Interest
EURO JAP
5 SPELLS AND SPELL-CASTING:ENCYCLOPEDIA OF
5000 SPELLS ( JUDIKA ILLES ):BLACKSMITH’S
WATER HEALING SPELL: A...
http://t.co/k0TfrqJFQW
DEBUTS NEW, NOW AVAILABLE FOR,
TO PUBLISH, DESIGNED TO,
IS READY TO

19
Results (2/3)

20
Results (3/3)
Top-k recommendation Hit-rate
She et al. Our approach
1 Top-5 82% 83.33%
2 Top-10 89% 86.67%

21
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

22
Conclusion
• Introduced a novel approach for hashtag recommendation, using
distributed word representations and a feed forward neural network
• Learns semantic and linguistic regularities without requiring careful
feature engineering
• Can easily take advantage of temporal information
• Supports the automatic creation of new hashtags/trends

23
 Introduction
 Goal
 Motivation
 Methodology
 Results
 Conclusion
 Future work
Overview

24
Future Work
• Use of more than four days of data
• Use word representations from different data sources
• Investigate impact of the quality of the word representations created
• Investigate impact of the use of DBpedia and Freebase

Towards Twitter hashtag recommendation using distributed word representations and a deep feed forward neural network

Recommended

Recommended

More Related Content

Similar to Towards Twitter hashtag recommendation using distributed word representations and a deep feed forward neural network

Similar to Towards Twitter hashtag recommendation using distributed word representations and a deep feed forward neural network (20)

More from Wesley De Neve

More from Wesley De Neve (20)

Recently uploaded

Recently uploaded (20)

Towards Twitter hashtag recommendation using distributed word representations and a deep feed forward neural network