1.
Machine Learning on the web - moving from
Terminator to Star Trek
Chris Heilmann @codepo8, Future Decoded 2016 - 2nd November 2016

2.
CHRIS HEILMANN
@CODEPO8
CHRISTIAN.HEILMANN@MICROSOFT.COMWELCOME TO THE FUTURE (DECODED)

3.
CHRIS HEILMANN
@CODEPO8
CHRISTIAN.HEILMANN@MICROSOFT.COM

4.
Machine Learning, Artificial Intelligence, Connected
Devices, Security Breaches, Privacy and Surveillance…

5.
What does the future hold?
🔮🍵🍂

6.
😱 Constant machine
surveillance
😱 Replacement of humans
with cheaper, unfeeling
machines
😱 Robot takeover and end of
humankind unless we find a
saviour to kill them by
making them get into a
logic loop.

7.
😍 Ubiquitous computing as a resource
to call in on human decisions
😍 Machines taking on human tasks
that are dangerous or get us sick in
the long run
😍 Machines improving humans to
work around shortcomings
😍 A change of values: knowledge
instead of monetary goals

8.
Machine Learning and AI are incredibly
hot topics and people are worried…
🔥💵📈💯📢

9.
Expensive, creepy parrots…
https://www.youtube.com/watch?v=W0_DPi0PmF0

10.
Military research…

11.
It is time for us as developers to show
that it is not something in the future,
but something humans can benefit
from now.
🔓

12.
The future of computing needs first and
foremost people who are not afraid…

13.
My first computer wasn’t able to do much…

14.
This taught me to be not afraid of
machines.
😨

15.
Instead, it kept me curious and
interested about them. And it made
me always to look under the hood to
see what I can use them for.🤔

16.
Computers, programming
and algorithms to me are
tools to improve my work
and do things I have to do
in an easier fashion.
😨

17.
WELCOME TO THE FOURTH
INDUSTRIAL REVOLUTION!
https://en.wikipedia.org/wiki/July_Revolution

18.
Steam Engine
https://de.wikipedia.org/wiki/Hugo_Lentz

19.
Steam Engine > Electricity
https://de.wikipedia.org/wiki/Telefon

20.
Steam Engine > Electricity > Computer
https://de.wikipedia.org/wiki/Computer

21.
Each industrial revolution meant that
people lost their jobs…
😨

22.
At the same time the quality of life
improved…
😄

23.
Not for all, though, and this is where
the problems start…
😠

24.
What we call success these days turns
out to be more and more of a scam.
Making money by all means doesn’t
help humanity, the market or the
environment. And we only have one
planet to live on - for now.
📉

25.
And this is why this fourth industrial
revolution is ours to define. We’re in a
lucky position where our jobs are
secure and we control what’s
happening - for now.
📈

26.
http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

27.
http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

28.
http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

29.
http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
The robot takeover

30.
Robots and intelligent systems will
take jobs away from us…
It is not an if - it is a when.
And it is soon.
🤖

31.
http://www.bbc.com/news/technology-34066941
http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment.pdf
How safe is my job?
Abstract thinking
Working with humans
Images
Words
Numbers
Routine Variety
You’re out
Get nervous
You’re OK

32.
Steam Engine > Electricity > Computer > Digital, physical and biological symbiosis…
https://www.microsoft.com/microsoft-hololens/en-us

33.
This feels a lot of like science fiction…

34.
The Future,
1991
Star Trek TNG S04E19 - The nth degree - http://gb.imdb.com/title/tt0708813/

35.
Lifts…

36.
Tablet computers…

37.
Server synchronisation

38.
Language recognition

39.
Finding people

40.
Nowadays, we have supercomputers in
our pockets…

41.
http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/
And mobile computing
is constantly on the
rise…

42.
This means that the
interaction between
human and machine
changed drastically…

43.
Keyboard
and Mouse
Camera
Speaker
Microphone
Device Movement
Body functions

44.
Text, images,
sound and
video…
Speech recognition,
speech synthesis,
vibration,
notifications…

45.
Click and type
Touch, move, get
closer, asking
questions and
showing emotions…

46.
All this generates an amazing
amount of data that is not in a
simple to parse format and at
times very erratic…

47.
This is why we needed to find a way to get computers
to understand humans better…
Sound, images
and movement
00101011010010101…

48.
In the fourth industrial revolution, information is oil.

49.
Computers are great to
churn through large sets
of data and find patterns
without getting tired or
sloppy - very much what
humans are not.

50.
The more data we give the
machines, the better the results
will be…

51.
Our job right now is to
create mobile interfaces
that are simple, human
and fun to use…

52.
Let’s acknowledge the
elephant in the room first…
https://www.flickr.com/photos/135045262@N04/30459872622/

53.
Business and hype is all
about the chat bots and
personal assistants…
🗣🤖
Siri (Apple)
Google Now (Google)
Cortana (Microsoft)
Alexa (Amazon)
M (facebook)

54.
https://techcrunch.com/2015/09/29/forget-apps-now-the-bots-take-over/

55.
Explosive projections…

56.
If you want to play with bots, we got your back…
http://blog.decodedshow.com/2016/09/27/i-have-a-bot-for-that/
https://docs.botframework.com/en-us/node/builder/chat/UniversalBot/
@ritazh

57.
If you want to play with bots, we got your back…
http://blog.decodedshow.com/2016/09/27/i-have-a-bot-for-that/
https://docs.botframework.com/en-us/node/builder/chat/UniversalBot/
@ritazh

58.
However, more traditional interfaces like fat client and web
need more intelligent and simple interfaces and content,
too. They aren’t going away…

59.
And machine learning is
everywhere…

60.
SCRIPTING SOLUTIONS FOR
ALTERNATIVE CONTENTIntelligent image resizing

61.
https://github.com/jwagner/smartcrop.js/Smartcrop.js

62.
High contrast tricks
https://blog.imgix.com/2015/10/21/automatic-
point-of-interest-cropping-with-imgix.html

63.
Intelligent resizing http://cloudinary.com/blog/
automatically_art_directed_responsive_images

64.
Making photos findable… https://photos.google.com

65.
Making photos indexable and understandable…
Image may contain: dog,
outdoor and nature

66.
How does that work?
Image may contain: dog,
outdoor and nature
https://code.facebook.com/posts/561187904071636

67.
Learning from good data… http://image-net.org/

68.
Learning from *lots* of data… http://image-net.org/

69.
Learning from *lots* of data…
https://github.com/
openimages/dataset

70.
Refining the data with language
https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html

71.
Compare and enhance…

72.
Detect syntax, mix and match

73.
Add visual information

74.
http://captionbot.ai
APIs and demos to use yourself…

75.
Detection Result:
JSON:
[
{
"faceRectangle": {
"width": 109,
"height": 109,
"left": 62,
"top": 62
},
"attributes": {
"age": 31,
"gender": "male",
"headPose": {
"roll": "2.9",
"yaw": "-1.3",
"pitch": "0.0"
}
"faceLandmarks": {
"pupilLeft": {
"x": "93.6",
"y": "88.2"
},
"pupilRight": {
"x": "138.4",
"y": "91.7"
},
...
Detect Faces…

76.
Verification Result:
JSON:
[
{
"isIdentical":false,
"confidence":0.01
}
]
Verify Faces

77.
Cluster automatically…

78.
https://www.microsoft.com/cognitive-services/en-us/video-api
Understanding and fixing video automatically…
📹 Stabilise shaky videos
📹 Detect and track faces
📹 Detect motion
📹 Generate video thumbnails

79.
Machines and code are there to
help humans communicate better.
The more useful our “intelligent”
solutions are, the easier it will be for
people to stop fearing innovation…
🎙

80.
Fear of innovation in many cases
shows itself by making sure the first
use of an “intelligent” system leads
to a failure. We love to show our
supremacy over technology.
🍌

81.
In the worst case, it is flat our
trolling and sabotage - and the
press loves this.
😒

82.
🤔

83.
Quality is dependent on data and context…
https://www.microsoft.com/cognitive-services/en-us/custom-recognition-intelligent-service-cris

84.
Get inspired by the amazing stuff
that is already happening - and
create more human solutions. 🦄

85.
https://www.youtube.com/watch?v=R2mC-NUAmMk
Smart Glasses listening
to questions, using OCR
and image analysis…

86.
https://channel9.msdn.com/Events/Build/2016/KEY01
“Live” translation of calls.

87.
Writing and reading
tools that change
display of text catered
to the needs of people
with learning
disabilities…
https://blogs.office.com/2016/01/19/learning-
tools-for-onenote-improves-learning-for-all/
https://www.youtube.com/watch?v=3Ztr44aKmQ8

88.
Adding automated content in context…
https://research.googleblog.com/2016/05/aw-so-cute-allo-helps-you-respond-to.html

89.
It is up to us to make this revolution
one that improves all mankind, and
not only makes some quick money
and shows us more ads.
8

90.
Humans are great people :)
No need to make them do things
they are not meant to do.🤕

91.
Lift assistant…
https://www.flickr.com/photos/breynolds/353144443/

92.
CHRIS HEILMANN
@CODEPO8
CHRISTIANHEILMANN.COM
THANKS!