Ponencia sobre Escucha Inteligente en el Master Universitario en Ingeniería Informática (MUIinf). Como caso práctico se explica el geoposicionamiento basado en la identificación de variedad del lenguaje.

1. AUTORITAS
Classification of Spanish by Regions
Universitat Politècnica de València, Apr 24th

2. consulting, s.a.autoritas
2
“When you’re able to see
the subtle, it’s easy to win”
Sun Tzu

3. consulting, s.a.autoritas
3
Smart Listening
http://hdwall.co/facebook-wallpaper-digg/
DATA -> INFORMATION -> KNOWLEDGE -> INTELLIGENCE
Tools and
Methods
Human
Interpretation
Strategic
Application

4. consulting, s.a.autoritas
4
How does
Autoritas do it?
Technology
Method
Team

5. consulting, s.a.autoritas
5
DECISION MAKING
STRATEGY
INFORATION
RETRIEVAL
FILTERING, CLEANING
AND NOISE DELETION
KNOWLEDGE
EXTRACTION
Smart
Listening
The Smart Cycle

6. consulting, s.a.autoritas
6
THINK, THINK, THINK...
...since thought precedes
actionDECISION MAKING
STRATEGY
INFORATION
RETRIEVAL
FILTERING, CLEANING
AND NOISE DELETION
KNOWLEDGE
EXTRACTION
Smart
Listening

7. consulting, s.a.autoritas
7
Mi Entorno
What do you expect
from Internet / Social
Networks for you
Business
Objectives?

8. consulting, s.a.autoritas
8
Mi Entorno
Where is your
ROI?

9. consulting, s.a.autoritas
9
Catching the
bad
Police Investigation Bureau
¡They have it
very clear!

10. consulting, s.a.autoritas
10
Demand
prediction
Tourist Agency

11. consulting, s.a.autoritas
11
Measuring Reputation
in risk management
Insurance Companies

12. consulting, s.a.autoritas
12
Internet as
Information source
Marketing Agencies

13. consulting, s.a.autoritas
13
Governments
Open
Government
Participation
Collaboration Transparency
Sm
artListening

14. consulting, s.a.autoritas
14
To improve
customer
service
Telcos

15. consulting, s.a.autoritas
15
Social content generation
Audience meassurement
Mass media

16. consulting, s.a.autoritas
16
And yours?
Think, think, think...

17. consulting, s.a.autoritas
17
“Truth is out there”
X-Files
DECISION MAKING
STRATEGY
INFORATION
RETRIEVAL
FILTERING, CLEANING
AND NOISE DELETION
KNOWLEDGE
EXTRACTION
Smart
Listening

18. consulting, s.a.autoritas
18
OBJECTIVE:
To retrieve all that we
should retrieve and/but
without retrieving nothing
that we should not
retrieve

19. consulting, s.a.autoritas
19
Information Sources (channels)

20. consulting, s.a.autoritas
20
Concept-query mapping
Intelligence
Concepts
Queries
1 N
Brands
People
Products
Campaigns
Competitors
Referrals
Critics
Topics
Countries

21. consulting, s.a.autoritas
21
1 N
A real example: Spanish public television
5 brands
10 presenters
10 programas
10 campañas
6.856 queries8 channels
Concept-query mapping
Intelligence
Concepts
Queries

22. consulting, s.a.autoritas
22
API vs. Crawling

23. consulting, s.a.autoritas
23
information = data - noise
DECISION MAKING
STRATEGY
INFORATION
RETRIEVAL
FILTERING, CLEANING
AND NOISE DELETION
KNOWLEDGE
EXTRACTION
Smart
Listening

24. consulting, s.a.autoritas
24
Why is there noise?
Users lie, plagiarize or say foolishness...

25. consulting, s.a.autoritas
25
For example...
findability relevance

26. consulting, s.a.autoritas
26
Advertisements
without relevance
for the contents
Latest news
section that
distorts the
semantics of the
page
Usable contents
Usable content retrieval

27. consulting, s.a.autoritas
27
ORDEN
If the url includes the date,
it’s easy to know it
That’s relative. Is
this url from july or
january:
http://xxx/07/01/2010/
crawler-403-
forbidden.html
The importance (and difficulty) of
obtaining the right date

28. consulting, s.a.autoritas
28
English
estoy sin internet ¬¬ fuuuuck!!!
Finnish
... euskocaja, como euskolabel, euskotren,
euskomueble... XDDD
Portuguese
Flowah Powah!
German
Vierrrrrrrrrrrrnes, egunon!!
Language Models vs. n-Gramms vs. Machine Learning
Language filtering

29. consulting, s.a.autoritas
29
Source geography vs. contents geography vs. profile geography
Geography filtering

30. consulting, s.a.autoritas
30
resultados
elimina url
prescindibles
filtra palabras
marca url’s como
Spam
Elimina url’s
Quita de la ‘vista’ los
antitesauros
filtra #hastags
filtra influenciadores
filtra localizaciones
“Limpia, fija y da explendor”Lema de la RAE
80%
ofthe
w
orkload

31. consulting, s.a.autoritas
31
I believe I
solved the
equation:

32. consulting, s.a.autoritas
32
Information Retrieval Evaluation...
...in Science
- Do you want that all
retrieved content is
good? -> Precision
- Don’t you want to leave
anything without
retrieving? -> Recall

33. consulting, s.a.autoritas
33
7.000 retrieved
54 wrong
99.23% precision
3.000 retrieved
50 not retrieved
98.36% recall
Information Retrieval Evaluation...
...in the Industry
We have a lack of
credibility!!

34. consulting, s.a.autoritas
34
Example
QUERIES: Mallorca, Menorca, Ibiza, Formentera, Baleares, ...
METHOD: 1.200 random tweets / 3 annotators
30%
70%
Labeled
Not labeled
70
80
90
100
Profile Detector Combination
94%
82%
85%
79%
87%
85%
Precision
Recall
Touristic project: 4.575.096 tweets
1.372.528
3.202.567
672.539 wrongly retrieved
204.419 good but not retrieved
1.372.528 that we don’t know...

35. consulting, s.a.autoritas
354
Bring order to the
information...
...to analyse it and
get value
DECISION MAKING
STRATEGY
INFORATION
RETRIEVAL
FILTERING, CLEANING
AND NOISE DELETION
KNOWLEDGE
EXTRACTION
Smart
Listening

36. consulting, s.a.autoritas
36
...to ask questions...
...and to know which new questions to ask

37. consulting, s.a.autoritas
37
ORDEN
WHAT are people talking about,
the consonant, the programming
language or the galician
telecommunication company?
Semantic ambiguity in millions of documents!!

38. consulting, s.a.autoritas
38
WHEN? -> Crisis management
When are things happening?
In real time!!
And let remember the issue of dates identification...

39. consulting, s.a.autoritas
39
WHERE/FROM WHERE is people talking about?
Aprox. 2% of geotagged contents!!

40. consulting, s.a.autoritas
40
ORDEN
HOW? -> Not only sentiment analysis
Polarity is onlye one
dimension:
- Emotions
- Motivations
- Values
- SWOT
All of them answer
the question “how?”

41. consulting, s.a.autoritas
41
ORDEN
An example: “The risk premium in Spain is 235”
Positive, negative, neutral o none?

42. consulting, s.a.autoritas
An example: “The risk premium in Spain is 235”
Positive, negative, neutral o none?
42
ORDEN
My question: For whom?
- For the president of the country?
- For the leader of the opposition?
- For the director of the Spanish Bank?
- For the foreigner investor?
- For the national capitalist?
- For whom has a morgage?
Subjectivity in transmitter,
and in receiver!!

43. consulting, s.a.autoritas
43
ORDEN
WHO? -> Social Network Analysis
If I want to
transmit a
successful
message, who
can help me?
If there is a
crisis, who do I
have to keep an
eye on?

44. consulting, s.a.autoritas
44
ORDENWHY -> Author Profiling
EMOTIONAL PROFILE
GENDER
AGE GROUP
NATIVE LANGUAGE
... political ideology, religious beliefs, and much more!

45. consulting, s.a.autoritas
45
Big Data is both the solution and the problem...
...but mainly the opportunity
80~120
tw/sec
=4.800~7.200
tw/min
=288.000~432.000
tw/hour
=6.912.000~10.368.000
tw/day
‣Retrieval & Storing
‣Evolution
‣Words & Topics
‣Tagging
‣Hashtags
‣People
‣Locations
‣Brands
‣Sentiment & Emotion
‣Users & Relationships
‣Influence
‣Gender & Age
‣LanguageVariety
‣...

46. consulting, s.a.autoritas
46
Precision vs. computational cost
0
375000
750000
1125000
1500000
Approach 1 Approach 4 Approach 7 Approach 10 Approach 13 Approach 16 Approach 19
10.26minutes28.83minutes38.31minutes54.03minutes1.04hours1.09hours2.66hours3.25hours4.66hours4.86hours5.08hours5.13hours6.37hours6.56hours6.82hours17.88hours4.44days
5.19days
6.68days
9.90days11.78days
0
10
20
30
40
Approach 1 Approach 4 Approach 7 Approach 10 Approach 13 Approach 16 Approach 19
29.06
34.72
39.86
24.20
28.34 27.98
35.06
25.14
27.03
38.58
31.25 32.34
7.87
24.65
22.87
27.03
32.22
24.67
33.04
15.40
19.82
%
AGE AND GENDER IDENTIFICATION ~240K AUTHORS

47. consulting, s.a.autoritas
47
Classification of Spanish
by Regions
Autoritas Case Study

48. consulting, s.a.autoritas
48
On the Internet there are no boundaries...
...except the language

49. consulting, s.a.autoritas
49
Show me how you talk...
...and I tell you where you are
HispaBlogs
TRAINING TEST
450 200
x 5 varietiesx 5 varieties
https://github.com/autoritas/RD-Lab/
tree/master/data/HispaBlogs

50. consulting, s.a.autoritas
50
State-of-the-art
Automatic Identification of Language Varieties: The Case of Portuguese.
Zampieri, M., Gebrekidan-Gebre, B.
In Proceedings of the Conference on Natural Language Processing 2012
Corpus (1000 documents from newsletters): 2 regional variations
Features: word and character n-grams
ML Algorithm: Language probability distributions with log-likelihood function for probability estimation
Evaluation method: 50/50 split
Accuracy:
Word uni-grams: 99.6%
Word bi-grams: 91.2%
Character 4-grams: 99.8%
Automatic Identification of Arabic Language Varieties and Dialects in Social Media.
Sadat, F., Kazemi, F., Farzindar, A.
In Proceeding of the 1st. International Workshop on Social Media Retrieval and Analysis SoMeRa 2014
Corpus (blogs and forum documents): 6 regional variations
Features: character n-grams
ML Algorithm: Markov language model vs. Naïve Bayes
Evaluation method: 50/50 split
Accuracy: 98% (78% F-measure)

51. consulting, s.a.autoritas
51
Our approach: compared results
0
20
40
60
80
100
char n-grams
word n-grams
TF/IDF n-grams
Probabilistic FW
Skip-grams
72.20%
58.52%
39.32%
52.74%51.51%
77.70%
91.69%
79.26%
70.15%69.22%
10-fold cross-validation Supplied test set

52. consulting, s.a.autoritas
52
Your turn...
- Course work
- Master project
- Scholarship, job positions, collaborations
- ...

53. Argentina - Brasil - Chile - España - México - Panamá - UK
@kicorangel
francisco.rangel@autoritas.es