Slides from my lightning talk at satRDay Amsterdam, 1 sep 2018. Two hobby projects with R package text2vec. 1. Predicting house prices from house descriptions. 2. Word embeddings from the soap series The Bold and The Beautiful

1. TWO HOBBY PROJECTS WITH THE PACKAGE TEXT2VEC
https://www.linkedin.com/in/longhowlam
https://longhowlam.wordpress.com
@longhowlam
Longhow Lam -- Freelance Data Scientist

2. AGENDA
 TEXT2VEC INTRODUCTION
 JAAP.NL HOUSE PRICE PREDICTION
 SOAP ANALYTICS: THE BOLD

3. Just visit the website of this R package
www.text2vec.org

4. WWW.JAAP.NL HUIZEN ANALYTICS

5. https://github.com/longhowlam/jaap
https://www.linkedin.com/pulse/huis-te-koop-zet-beleggingsobject-je-huisomschrijving-longhow-lam/
HOUSE PRICES NOW CREATE MODEL ON HOUSE DESCRIPTIONS / TEXTS

6. PREDICT HOUSE PRICE WITH LASSO REGRESSION OR XGBOOST
TERM DOCUMENT MATRIX
Super sparse: 65.000 rows ~50.000 columns
house price kitchen big_garden garage ...(many more terms)... swimming_pool
house 1 235.000 1 0 1 ... 0
house 2 450.000 0 1 0 ... 0
house 3 376.000 1 0 0 ... 0
... ... ... ... ... ... ...
... ... ... ... ... ... ...
house 65.000 621.000 1 1 ... ... 1
Data.frame jaap with 65000 rows, column huisbeschrijvingen and column prijs

7. PREDICT HOUSE PRICE WITH LASSO REGRESSION OR XGBOOST
TERM DOCUMENT MATRIX
Too many columns for a normal linear regression, regularization is needed.
For example “lasso” regression

8. PREDICT HOUSE PRICE WITH LASSO REGRESSION OR XGBOOST

9. PREDICT HOUSE PRICE WITH LASSO REGRESSION OR XGBOOST

10. PREDICT HOUSE PRICE WITH LASSO REGRESSION OR XGBOOST

11. PREDICT HOUSE PRICE WITH LASSO REGRESSION OR XGBOOST

12. LASSO REGRESSION NEGATIVE AND POSITIVE COEFFICIENTS
R2
= 0.66
Intercept € 238.260
parkkosten € 39.644- familiehuis € 60.168
recreatiebungalow € 32.614- vrijstaande_villa € 48.180
bungalowpark € 31.801- belegging € 45.814
limburgse € 23.483- beleggingsobject € 42.543
2_kamer € 23.034- entree_vestibule € 41.674
plinten € 22.510- rijksmonument € 39.379
overdekt_zwembad € 21.971- recreatief € 39.142
2_kamerappartement € 20.625- verhuurd € 36.171
aannemer € 20.314- detaillering € 35.000
recreatiewoning € 19.748- visgraat € 33.589
proeven € 19.631- eigen_badkamer € 33.454
betaalbaar € 19.621- woningen_1 € 33.321
starterswoning € 19.502- toiletten € 32.836
volwassen € 19.476- rietgedekte € 32.096
kunststofkozijnen € 18.775- representatieve € 31.904
helder € 18.594- alarm € 31.841
verbeterd € 18.488- toplocatie € 31.821
eigen_gebruik € 18.430- gezinshuis € 31.297

13. ANALYTICS
SOAP

14. WORD EMBEDDINGS IN BOLD & BEAUTIFUL RECAPS
Term Document Matrix
Each document / recap is a vector of numbers
Word embedding
Each word is a vector of numbers
A word embedding has to be trained from a collection of documents / recaps
Amsterdam = (0.83, 0.89, 0.34, … , 0.63, 0.19)
Steffy = (0.33, 0.19, 0.79, … , 0.13, 0.01)
Germany = (0.72, 0.65, 0.43, … , 0.36, 0.57)
Laugh = (0.85, 0.77, 0.24, … , 0.88, 0.29)
…
…
https://github.com/longhowlam/TBATB

15. WORD EMBEDDINGS LINGUISTIC REGULARITIES
Closest words
Word relations
250 dimensional space
president
trump
car media
press
house
man
woman
king
queen
vector(“man") − vector(“woman")
is roughly
vector(“king”) − vector(“queen")
Trump speaks with the press
The president talks to the media

16. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS
➢
4000 daily recaps of TBTB over the last 15 years
➢
We have around 10.000 unique words in these recaps
➢
I am generating word vectors of dimension 250
First a simple word cloud to get a
general idea of term importance

17. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS

18. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS

19. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS

20. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS

21. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS

22. WORD EMBEDDINGS BOLD & BEAUTIFUL RECAPS
Stanford’s GloVe: Global Vectors for Word Representation

23. 1 steffy steffy 1.00
2 steffy liam 0.82
3 steffy hope 0.79
4 steffy said 0.78
5 steffy wyatt 0.76
6 steffy bill 0.69
7 steffy asked 0.68
8 steffy quinn 0.67
9 steffy agreed 0.65
10 steffy rick 0.65
WORD EMBEDDINGS LINGUISTIC REGULARITIES

24. WORD EMBEDDINGS BOLD & BEAUTIFUL EXAMPLE
death furious lastly excused frustration onset
0.223 0.2006 0.1963 0.1958 0.1950 0.1937
Word vectors voor:
Steffy − Liam

25. WORD EMBEDDINGS BOLD & BEAUTIFUL EXAMPLE
liam katie wyatt steffy quinn said
0.5550 0.4845 0.4829 0.4645 0.4491 0.4201
Word vectors voor:
Bill − anger

26. Thanks for your attention. QUESTIONS?
https://www.linkedin.com/in/longhowlam
https://longhowlam.wordpress.com/
@longhowlam