The document discusses how machines and software can help humans by doing tasks like preventing mistakes, performing repetitive tasks, filling information gaps, remembering and categorizing information, improving understanding, enabling new communication methods, and providing protection. It describes how advances in AI, APIs, cloud services, and data processing have made it possible to build useful and helpful interfaces. The conclusion encourages developers to use these capabilities to create simple, human-centric interfaces that benefit users.

1. The Soul in The Machine - Developing for Humans
Chris Heilmann @codepo8 - May 2017

2. PROLOGUE
“VISIONS OF THE FUTURE”

3. Arthur C
Clarke
Aldous
Huxley
George
Orwell
Alan
Moore

5. http://www.independent.co.uk/life-style/gadgets-and-tech/news/theresa-may-internet-conservatives-government-a7744176.html

7. https://www.newscientist.com/article/2113299-
googles-deepmind-ai-can-lip-read-tv-shows-
better-than-a-pro/?sf43896303=1
https://arxiv.org/pdf/1611.05358v1.pdf
46.8% vs 12.4% success rate

8. Technological progress
has merely provided
us with more efficient
means for going
backwards.
— Aldous Huxley

10. Duophonix @ Deviantart http://duophonix.deviantart.com/art/Ihal-9000-184254683

11. CHRIS HEILMANN
@CODEPO8
HTTPS://CHRISTIANHEILMANN.COM

12. ACT ONE
“CREATURE COMFORTS WE ALL ENJOY”

13. Things machines and software can do for humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

14. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

15. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://www.bloomberg.com/news/articles/
2017-02-28/jpmorgan-marshals-an-army-of-
developers-to-automate-high-finance

16. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

17. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://arxiv.org/abs/1702.00783

18. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://gwmt.nvidia.com/
http://www.trustedreviews.com/news/
nvidia-super-resolution-tool-launch

19. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

20. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

21. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

22. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

23. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

24. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
Image may contain: dog,
outdoor and nature

25. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://blogs.office.com/2016/12/02/accessibility-in-
office-365-enabling-greater-digital-inclusion/

26. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm http://captionbot.ai

27. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://www.youtube.com/watch?v=XMdct-5bERQ

28. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

29. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm
https://research.googleblog.com/2016/05/aw-so-cute-
allo-helps-you-respond-to.html

30. Things machines and
software can do for
humans:
• Prevent us from making mistakes
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise for us
• Make us understand better
• Allow us to communicate more
• Protect us from harm

31. ACT TWO
“ENTER THE ENABLERS”

32. A few years ago all this was pure science
fiction…

33. And even today it feels like it is too hard
to do, right?

34. Wrong. The trick is to understand that we
live in a world of services, clouds,
databases and products.

35. All players are doing a great job giving you access…
https://www.youtube.com/watch?v=ETeeSYMGZn0

36. Almost every company offers APIs for this.

37. REST endpoints with support for Node or
even client-side JavaScript.

38. Images
Audio
Video
(Foreign) Text
Gestures
Faces
Emotions
…
build-stuff.json

39. A lot of work went into these APIs, work
you might not want to do yourself…

40. Machines, corporations, services and
products have harvested tons and tons of
data over the last years.

41. The biggest problem was to understand
the data – as it was a mess.

42. So we trained machines in thousands of
iterations to get the best information out
of a small set of fata.

43. What’s in a sentence?

44. Word dependencies

45. Labels

46. Speech / Language Syntax

47. Language stems

48. Morphology / Grammar

49. Better translation through sentences vs. words

50. Better translation through sentences vs. words

51. Better translation through sentences vs. words

52. Bad results? Use the same platforms to
train your own data according to your
specifications.

53. Add quality by filtering with context…
https://www.microsoft.com/cognitive-services/en-us/custom-recognition-intelligent-service-cris

54. The good news for us is that we can now
use these findings to build better
interfaces.

55. ACT THREE
“GO FORTH AND DO SOME GOOD”

56. The interfaces we
build today condition
the users of now and
of tomorrow…

57. The interfaces we
build today condition
the users of now and
of tomorrow…

58. It is time for us to use and build better, more
human interfaces…

59. There is no such thing as a perfect user - think inclusive.
https://www.microsoft.com/en-us/Design/inclusive
It isn’t about allowing access but about avoiding barriers.

60. Our job right now is to
create interfaces that are
simple, human and fun
to use…

61. What we can do with technology for us and our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

62. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

63. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

64. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

65. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

66. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
http://code.visualstudio.com/

67. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
https://channel9.msdn.com/Events/Build/2017/T6078

68. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
JSON:
{
"language": "en",
"orientation": "Up",
"regions": [
{
"boundingBox": "41,77,918,440",
"lines": [
{
"boundingBox": "41,77,723,89",
"words": [
{
"boundingBox": "41,102,225,64",
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62",
"text": "IS"
},
{
"boundingBox": "539,77,225,64",
"text": "LIKE"
}
. . .
TEXT:
LIFE IS LIKE
RIDING A BICYCLE
TO KEEP YOUR BALANCE
YOU MUST KEEP MOVING
JSON:
{
"language": "en",
"orientation": "Up",
"regions": [
{
"boundingBox": "41,77,918,440",
"lines": [
{
"boundingBox": "41,77,723,89"
"words": [
{
"boundingBox": "41,102,225,6
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62
"text": "IS"
},
{
"boundingBox": "539,77,225,6
"text": "LIKE"
}
. . .
TO KEEP YOUR BALANCE
YOU MUST KEEP MOVING

69. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

70. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
http://cloudinary.com/blog/
automatically_art_directed_responsive_images

71. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

72. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

73. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
https://www.videobreakdown.com/

74. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

75. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

76. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

77. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

78. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

79. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

80. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

81. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

82. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

83. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users
🎙"
🔊$

84. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

85. What we can do with
technology for us and
our users…
• Prevent mistakes before they happen
• Do boring, repetitive tasks
• Fill gaps in information
• Remember and categorise
• Allow for better understanding
• Offer new ways to communicate
• Protect us and our end users

86. EPILOGUE
“OH THE PLACES YOU WILL GO…”

87. The saddest aspect of
life right now is that
science gathers
knowledge faster than
society gathers wisdom.
— Isaac Asimov

88. The future is already
here – it’s just not very
evenly distributed.
— William Gibson

89. Let’s fix that! Right now!

90. CHRIS HEILMANN
@CODEPO8
CHRISTIANHEILMANN.COM
THANKS!