The Good, The bad and the Ugly of Big Data

João Vazão Vasques
Large Scale Distributed Data Systems
Programming
Product
Math
Data Engineering Manager @ Unbabel
Taekwondo
Yoga
Music
Magic
Books
Photography
J o a o Va z ã o Va s q u e s

80% 
English
The internet, 1997

5% 
Portuguese
8% 
Spanish
20% 
Chinese
4% 
German
6% 
Japanese
3% 
Arabic
30% 
English
The internet, 2017

Expert Human Translation$0.3
$0.03  10X REDUCTION
$0.007
$0.003
PRICE/WORD
Unbabel over time
100X REDUCTION
SEP 2017
AUG 2016
JUN 2018

Original customer
request
Translated
customer request
High Q.E.
Unbabel Pipeline
Quality
Estimation
Q.E.
Machine
Translation
Low Q.E.
Re-Eval
Community
Translators

Let’s talk about Data

W h a t d o t h e s e c o m p a n i e s h a v e i n c o m m o n ?

W h a t a r e t h e c o m p a n i e s o f t h e f u t u r e ?

N o t t h e o n e s w h o c a p t u r e h u g e v o l u m e s o f d a t a b u t . .

C o l l e c t d a t a i n t h e m o r n i n g a n d c h a n g e t h e i r p r o d u c t
a n d o p e r a t i o n s i n t h e a f t e r n o o n

Building Data Products
has never been easier

Why?

Why?
C l o u d

Why?
C l o u d D a t a To o l s

Why?
C l o u d D a t a To o l s C a p i t a l

Google is doing this..
let’s do it too

O v e r t h i n k i n g & e n g i n e e r i n g

O v e r t h i n k i n g & e n g i n e e r i n g
U r g e n t v s I m p o r t a n t

We have lots of Data

E v e n w o r s t …

We have BIG DATA

W h a t ’ s B i g D a t a ?

Y o u c a n ’ t d o n o t h i n g u s e f u l w i t h c r a p p y d a t a

S i z e w i l l n o t h e l p y o u …

F o c u s o n q u a l i t y

We have full visibility,
dashboards and
metrics

O h r e a l l y ?

KISS

KISS
A r e y o u
m e a s u r i n g
t h e r i g h t K P I s

KISS
A r e y o u
m e a s u r i n g
C a n y o u a c t
o n t h e m ?

KISS
A r e y o u
m e a s u r i n g
C a n y o u a c t
o n t h e m ?
H o w f a s t c a n y o u
i t e r a t e ?

#MoveTheNeedle

T h e d a t a . . .

Different storage and
query types

Meaning is not clear to
everyone

Data Modelling is
(most of the times)
wrong

The reality of the
physical word..
The CAP theorem

“Data Science” with a
product goal

The Distributed
Monolith
“Your Distributed Monoliths are secretly plotting against you”

S u m m a r y
Data helps companies crush their competitors

S u m m a r y
We have lots of tools to build data products

S u m m a r y
Big Data solves nothing by itself. Focus on quality

S u m m a r y
Collect data that you can use to improve your
product

S u m m a r y
product
Data Science must solve a business/product
need

S u m m a r y
product
need
#MoveTheNeedle

S u m m a r y
product
need
#MoveTheNeedle
Urgent vs Important

The Good, The bad and the Ugly of Big Data

Recommended

Recommended

More Related Content

Similar to The Good, The bad and the Ugly of Big Data

Similar to The Good, The bad and the Ugly of Big Data (20)

Recently uploaded

Recently uploaded (20)

The Good, The bad and the Ugly of Big Data