Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
MlL trends 2018-2020 : Data Driven Decision World Berlin
1. Looking back at 50 years of AI summers and
winters, current and upcoming ML trends,
and extrapolating to the future
Dr. Andreas Merentitis
OLX Markets Global Data Science Team
2. 2
Outline
Riding the 3rd AI wave
Update on the world around us in AI
Outline
Inspiring and impactful applications of AI
Looking forward
8. 8
Outline
Riding the 3rd AI wave
Update on the world around us in AI
Outline Inspiring and impactful applications of AI
Looking forward
9. 9
Selected relevant trends in ML - already here
2018, the year of Natural Language Processing
Implications
▪ Translation becomes easier across multiple languages
▪ Transfer learning improves language modeling to other
languages
Language modeling (predict next word from previous word)
can be used for pre-training NLP tasks
10. 10
Selected relevant trends in ML - already here
Voice transcription and recognition
Voice transcription and recognition will become commodity
and will be offered from the big cloud providers
Implications
▪ Changes in downstream algorithms
like search and recommendations will
be needed to take advantage of this
▪ UX will have to be updated to
accommodate for this service
11. 11
Selected relevant trends in ML - already here
Attention and Self-attention
Attention allows determining what parts
of the input are more important
Implications
▪ Improved performance in many
sequence to sequence tasks
▪ Self-attention made possible the
transformer network architecture
12. 12
Selected relevant trends in ML - already here
New optimizer approaches and combinations
New optimization methods that are
more efficient (Ranger)
Implications
▪ More stable learning
▪ Performance increase and
convergence in fewer epochs
▪ Models can handle larger datasets
13. 13
Selected relevant trends in ML - already here
ML Innovation map: more 1-to-N
Deep Learning made practical
applications that rely on images, text,
and sound
Implications
▪ More effort is placed on scaling and
incrementally improving it
▪ 0->1 innovation is reduced
▪ “Forget about training (from scratch)”
14. 14
Selected relevant trends in ML - already (partially) here
Machine learning at the edge
Real-Time AR
Self-Expression with
Machine Learning
All-Neural
On-Device Speech
Recognizer
(final model is 80MB)
Several applications can be done in
mobile phones directly
Implications
▪ Benefits for privacy and performance
▪ Constraint of model size
will disappear
▪ More functionality in the device
Source: Google AI
15. 15
Selected relevant trends in ML - already (partially) here
Standardized model artifact formats - ONNX
ONNX is an open format independent of
ML library
Implications
▪ Easier to take the best features of ML
libraries and deploy in standard way
▪ AI developers can more easily move
and combine models
16. 16
Selected relevant trends in ML - already (partially) here
Neural architecture search is increasingly commoditized
A variety of commercial and open
source libraries take on NAS
Implications
▪ Less effort and expertise is needed in
this part
▪ More focus on solving the business /
ML model fit part of the process
17. 17
Selected relevant trends in ML - already (partially) here
ML with less samples - soft labels and crowdsourcing
Frameworks such as snorkel and
products such as Sagemaker GT make
gathering data easier
Implications
▪ Easier to kickstart the deep learning
flywheel
▪ Scope of applications approachable
with deep learning increases
18. 18
Selected relevant trends in ML - already (partially) here
Interpretable ML
Frameworks provide access to powerful
methods for interpreting ML models
Implications
▪ Easier to debug ML models
▪ Easier to explain predictions at the
instance level
19. 19
Selected relevant trends in ML - emerging trends
Privacy-preserving machine learning
Frameworks for distributed learning are
gaining maturity
Implications
▪ Data can be kept on device of user
▪ Shared learning across countries /
users
Google AI: Federated Learning
Google AI: Federated Learning
20. 20
Selected relevant trends in ML - emerging trends
ML with less samples - synthetic data and ML+rules
Synthetic sample generation decreases
the amount of training labels and makes
models that are more robust
Implications
▪ Easier to kickstart the deep learning
flywheel
▪ Easier to integrate practical
knowledge in ML
21. 21
Selected relevant trends in ML - emerging trends
Machine comprehension
Machine Comprehension focuses on AI
models that can read a document and
answer questions against it
Implications
▪ A lot of new applications can be
automated
▪ Humans and computers can also
work together combining their
strengths
22. 22
Selected relevant trends in ML - emerging trends
Reinforcement Learning
RL is a paradigm in which agents take
actions in an environment so as to
maximize some notion of reward
Implications
▪ Can learn with sparse and delayed
rewards
▪ Improvements in robotics,
configurations problems,
personalization, types of optimization
23. 23
Selected relevant trends in ML - emerging trends
Self-supervised learning
Learning using (part of) the data as the
supervision signal coupled with a proxy
task and loss
Implications
▪ Can extract more signal from high
dimensional data like images, audio,
and video
▪ Less need for labels, increasing the
scope of possible applications
24. 24
Selected relevant trends in ML - emerging trends
Fairness in ML is in the spotlight
A ML model is typically as good as the
data it trained on
Implications
▪ Several examples it went bad
(Facebook, Tesla) gained attention
▪ More effort to address problems on
the regulation and technical side
25. 25
Outline
Riding the 3rd AI wave
Update on the world around us in AI
Outline Inspiring and impactful applications of AI
Looking forward
26. 26
Search2Vec for Synonyms
Capability Description
Semantic
synonyms
▪ Maps queries and items in a common space
▪ Enables returning the results of synonyms
Why is it cool?
▪ No need for finding synonyms with lexicons
▪ No manual work on adding e.g. new iphone model
▪ Largely language agnostic
Naspers
Innovation
award
Effect ▪ 15% increase in replies / DAU in OLXZA
Impact
28. 28
Example: BBC “Talking with the Machines” Initiative
Foice Interactive Stories
▪ Focus on spoken interfaces
▪ Original, interactive audio drama,
created especially for smart speakers
Amazon Echo and Google Home
▪ Listeners get to be part of the story by
answering questions and inserting
their own lines into the story
Smart speaker API
BBC Story engine
BBC audio drama: The
Inspection Chamber
29. 29
Example: No lines, no checkout, no problem!
No checkout retail
▪ Use cameras and ML to account for
what people are buying
▪ The items get charged to their
Amazon account automatically
30. 30
Outline
Riding the 3rd AI wave
Update on the world around us in AI
Outline Inspiring and impactful applications of AI
Looking forward
31. 31
Is it going to be different this time?
Positive (no winter)
▪ Several synergizing trends
▪ We have reached a tipping point of
“good enough” for many applications
▪ There is an “arms race” between the
biggest companies in the world
Negative (possible winter)
▪ Some of the “big bets” are risky
▪ There are a lot of expectations that
will be difficult to meet
▪ Ethical issues if not addressed can
cause a backlash
32. 32
Trend synergies
Several of the trends amplify each other, e.g.:
▪ New optimizers synergize with NAS
▪ Fairness benefits from ML interpretability
▪ ML at the edge helps with privacy preserving AI
33. 33
Take away message
☞ Dealing with the ethical issues of AI is one way we can at least reduce the
possibility of an upcoming “winter”
☞ We are currently riding the 3rd (some can argue 4th) AI “wave” or “summer”
☞ A lot of synergistic trends are currently empowering the ML flywheel
☞ There are many re-enforcing trends but also very high expectations and some
risky bets that can go wrong
