Esta charla pretende introducir a la audiencia al mundo del procesamiento del lenguaje natural, o NLP por sus siglas en inglés (Natural Language Processing). La charla en sí constará de 3 bloques.1. Estado del arte en NLP. ¿Qué se está usando hoy en día? ¿Qué problemas podemos solucionar y qué problemas no? Técnicas comúnmente usados en la industria2. Introducción a conceptos básicos a la hora de afrontar un proyecto ML con NLP: preprocesado, vectorización y embedding (word2vec, fastText, técnicas básicas como tf-idf, counting, etc). Clasificadores.3. Pequeño ejemplo práctico con despliegue usando Azure Machine Learning.

1. Introducción de NLP en
Azure

2. ORGANIZATION
Thank you!

3. @zNk
jasanchez@plainconcepts.com
• Interested in:
• Machine Learning (NLP, productionalize models)
• Highly scalable architectures (based on hadoop, spark, etc)
• Software Engineering (i really like C#)
• Gaming: Pubg, LoL
Jesús A. Sánchez Méndez
Data Engineer

4. @ematde
ematallanas@plainconcepts.com
• Knowmad, working with data long time ago
• Interested in:
• AI applied to industry
• Research in robotics, new models and general use of ML
• Love films and series
Eduardo Matallanas
Data Engineer

5. Things that you will learn today (hopefully)
1. What is NLP
2. How to face a NLP project
3. Different approaches to model NLP
4. Different techniques involved in the creation a NLP model
5. Libs and frameworks that we use and work/worked

6. What is NLP?

8. Language is a process of free creation; its laws and
principles are fixed; but the manner in which the
principles of generation are used is free and
infinitely varied. Even the interpretation and used
of words involves a process of free creation.
Noam Chomsky

9. Expectations vs Reality

10. Excitations Input channel
Processing
Output

11. Context
Recognition
Word
analysis
Sentiment
Analysis
Answer
Entity
recognition
Synonyms
Disambiguation
Lexical
semantics

12. How can a
virtual
assistant do it?

14. { }
Language
Undertanding (LUIS) Amazon Lex
Privada

15. 101: How to build
your own NLP

16. Facing a NLP project? Step to success
Data Collection
& Assembly
Data
Preprocessing
Data Exploration
& Visualization
Model Building
Model
Evaluation

17. The importance
of data
preprocessing

18. Step 1: Gather the
data!

19. Gather data
• Different data sources
• The nature of the data is important
• Which data is really important to build your
model?

20. Step 2:
Data preprocessing

21. Why preprocessing?

22. Tokenization
Easy, huh?
What about this? Jesus’ car wasn’t fined since he’s
resident that well-known neighbor
That’s not as easy as: Jesus’ car was not fined since
he is resident in that neighbor

23. Stemming
EATING -> EAT
RUNNING -> RUN
AMUSING -> AMUS
GRATEFULLY -> GRATE
STUDIES -> STUDI

24. LEMMATIZING
AM, ARE, IS -> BE
CARS, CAR, CAR’S -> CAR
DESTRUCTION -> DESTROY

25. Quite often used techniques
• Sentence to lower case
• Remove numbers (or even translate
them to text)
• Remove punctuation (this is usually
done in tokenization stage)
• Trimming
• Remove stopwords
• Depending on the domain, remove
sparse terms
• It was a pleasure David  it was a pleasure david
• My 15 birthday  My fifteenth birthday
• The house, in Madrid, was smaller.  The house in
Madrid was smaller
• the car broke in front of the building

26. Step 3:
Model building

28. Quick reminder: Machine Learning models?
𝑌 = 𝑓(𝑋)The output
we want to
predict
Our input
data (X)
The function
itself is the
model that we
are going to
use to predict
the output (Y)

29. Does this make sense?
𝑓 𝑋 = 𝑋2 𝑓 "ℎ𝑒𝑦 𝑤ℎ𝑎𝑡′𝑠 𝑢𝑝? " = ?
Of course it doesn’t…

30. From Words To vectors

31. How can we build the model?
1. What is needed?  Understand words (input)
2. What is expected?  labels, numbers, words, etc.
(output)
Need a vocabulary!!!
Model
Input Output

33. 𝑤𝑡ℎ𝑒,2 = 2 𝑥 log(
4
2
) = 0,6
𝑤𝑓𝑜𝑥,1 = 1 𝑥 log(
1
1
) = 0

34. Second approximation
Lots of words
High representation of general information
Encode the data based on the order of appearance  e.g. Hashing function
Too large dimensionality  PCA for reduction
Clustering the data  K-means
WIKIPEDIA1
2
3
4

37. Word
embedding
Words or phrases are mapped to vectors of real
numbers
Relationship of words
•CBOW  Predict a word given the context
•Skipgram  Predict the context
Dense and low-dimensional vectors
Their representation is learned by the usage of words

38. Sparse
Dense

39. Word2Vec
Two layer Neural Network NOT A DEEP NN
Applied to any written text
(words, sentence, paragraph, etc)

40. Word2Vec, CBOW? Skip-Gram?
Predicts a word Predicts context

41. Word2Vec, CBOW? Skip-Gram?
Predicts a word Predicts context

42. Word2Vec, CBOW? Skip-Gram?
Input  a corpus
Approach  Skip-gram
Outpout  a dictionary [word,vector]
Word Distance
Portugal 0,83447
France 0,72341
Moroco 0,69847
Andorra 0,653456
e.g. Spain

43. • Created by Facebook AI Research lab
• Pretrained models for 294 languages
• Extract a relationship even if the word is not in the
corpus
• Enrich an existing model by assigning more weight
to your own trained vectors
• Fastest lib
• Allows training with CBOW or skipgram
• Provides classification and clustering capabilities

44. Nice, this is a solution for
everything!
• Indeed, whatever floats your boat
• Quite often embedding is used to feed another model
• Classifier o regression model
• Deep neural network
Model
Input OutputWord
Embedding ?

45. Sounds good but it has to have a disadvantage
[0,0,0,0,0,0,0,0,0] (needs retrain)
What happens when you try get a vector of a word that does not
appear in the corpus?
Coupled to the language

46. Applications
• Multilanguage support • Relationship from language to language

47. Logic arithmetic (analogies)

48. king:queen [woman,
Attempted abduction,
teenager, girl]
man

49. China:Taiwan [Ukraine, Moscow,
Moldova, Armenia]
Rusia

50. Republican:Donald Trump
[Democratic, GOP,
Democrats, McCain]
Barack Obama

51. Building:Architech Software
[programmer,
SecurityCenter,
WinPcap]

52. Logic arithmetic (analogies)
Geopolitics: Iraq - Violence = Jordan
Distinction: Human - Animal = Ethics
President - Power = Prime Minister
Library - Books = Hall
Analogy: Stock Market ≈ Thermometer
Other examples
house:roof::castle:[dome, bell_tower, spire, crenellations,
turrets]
knee:leg::elbow:[forearm, arm, ulna_bone]
love:indifference::fear:[apathy, callousness, timidity,
helplessness, inaction]

53. Other
applications
• Information retrieval
• Text classification
• Sentiment analysis
• Entity recognition
• Extract keywords

54. Conclusion
Data and analysis are key
NLP models are complex structures
A general purpose model does not exist
Use framework and library to begin something small and
grow it
Hopefully you are now an expert in NLP algorithms

55. Now is your turn

56. Questions & Answers

57. Thanks and …
See you soon!
Thanks also to the organization
Without whom this would not have been posible.

58. Biblio
• Preprocessing
• http://www.nltk.org/
• https://pandas.pydata.org/
• Word Embedding
• https://skymind.ai/wiki/word2vec
• https://www.tensorflow.org/tutorials/representation/word2vec
• https://fasttext.cc/
• https://nlp.stanford.edu/projects/glove/
• LSTM
• http://colah.github.io/posts/2015-08-Understanding-LSTMs/
Pandas, numpyTrabajo en python
Fasttext