Building Deep Learning applications

1. @rinofm

2. How we scaled our deep learning
infrastructure to process 1000
photos / second while cutting our
AWS bill by 10%
Rino Montiel, VP Engineering
@rinofm

4. EyeEm is a photography company that
builds the world’s leading computer
vision technology to connect its global
creative community with iconic brands.

5. What we’ve automated
Photo quality rating
Popular
Auto tagging
Model release detection
Keywording
Captioning
Search
Personalized aesthetics
Concept training platform

6. Understanding
photography

7. Man in Abandoned Building

10. Research, Production, Data
Three Parts

11. Where the models (and some trouble) come from
Beginning:
Research

12. Training and inference

13. “car”
“human”

14. “human”

15. GPUs run deep learning training
and inference very efficiently
Deep learning and GPUs

16. More data, more GPUs
Training with millions of
photos needs GPUs

18. Model trained and
production ready

19. Thumbnail Service
S3 - Original photo
Deep Learning
Service
Keywords
Aesthetics
Captions

20. Problem:
Input size changes

21. Problem:
Resizing algorithm

22. Scaling thumbnail generation for training
Problem:
Thumbnail Service

23. Scaling thumbnail generation for training
Problem:
Thumbnail Service
Thumbnail Service

24. Scaling thumbnail generation for training
Problem:
Thumbnail Service
Thumbnail ServiceThumbnail Internal
R&D

25. Training: Learning new functionality from past data
Inference: Applying functionality to new data
Input size can change
Resizing algorithm affects performance
Thumbnail generation is load intensive
Rely on a mix of on-demand and spot instances $

26. Middle:
Production
Dealing with scale

27. S3 - Original photos
Edge

28. S3 - Original photos
Edge
…
Deep Learning Service
Search

29. S3 - Original photos
Edge
…
Espresso
Search

30. Espresso
aesthetics concepts quality captions …
personalised
aesthetics
G2 machines

31. What we’ve automated Search

32. S3 - Original photos
Edge
Espresso
Search

33. S3 - Original photos
Edge
Espresso
Search
Keywords
Photo uploaded
Keywords ready

35. What we’ve automated Auto-tagging

37. Problem:
Latency is too high

38. Problem:
This is getting expensive…

39. "GPU-Based Deep Learning Inference: A Performance and Power Analysis”, NVIDIA
2015
Batch inference

40. Espresso
Batch of base64 encoded images
Array of concepts and aesthetic scores
Edge

41. Image reviewing
pipeline

42. The traditional stock
photography

44. What we’ve automated
Keywording
Captioning
Photo quality rating
Model release detection

45. Photo opted in
Espresso
Photo
reviewing
process
Model releases
Quality
Keywords and captions
Edge

47. Woman photographing through smartphone

48. Photo opted in
Photo reviewed
Espresso
Model releases
Quality
Keywords and captions
Confident?
Photo
reviewing
process
Edge

49. Asynchronous architectures are necessary to scale
Batch inference is essential for cost optimization and latency
90% accuracy can be misleading, but it can be optimized

50. Everything runs

51. Data pipeline for analytics and machine learning
End:
Data

52. Espresso
aesthetics concepts quality captions …
personalised
aesthetics

53. Problem:
Too much data!

54. One photo
Keywords
Captions
Confidence
Quality
Model releases
Aesthetics

55. {"photoId":"102394173","concepts":[["real people",0.99792659282684326],
["nature",0.97948437929153442],["sky",0.97900015115737915],["outdoors",
0.92300915718078613],["day",0.92132925987243652],["beach",
0.91038638353347778],["scenics",0.88209074735641479],["lifestyles",
0.84903037548065186],["two people",0.82873058319091797],["leisure
activity",0.80806028842926025],["sand",0.79079025983810425],
["tranquility",0.78834164142608643],["men",0.78776693344116211],["full
length",0.78717267513275146],["beauty in nature”,0.69489455223083496],
["landscape",0.64603018760681152],["tree",0.48800194263458252],["mammal",
0.31885015964508057]],"confidence":0.8376338593142145}
Concepts/Confidence

56. {"photoId":"93900842","captions":[{"caption":"STREAM FLOWING THROUGH
ROCKS","probability":0.656147889422701,"wordsProbability":
[["STREAM",-0.47248415835501062],["FLOWING",-0.29332975536168115],
["THROUGH",-0.59238967310011115],["ROCKS",-1.1257078392955155]]},
{"caption":"VIEW OF STREAM FLOWING THROUGH ROCKS","probability":
0.5243615522471853,"wordsProbability":[["VIEW",-1.8661552402619594],
["OF",-0.0041088119466486661],["STREAM",-0.087614953179970007],
["FLOWING",-1.3746558803077191],["THROUGH",-0.31274626783455156],
["ROCKS",-1.5081047112344315]]},{"caption":"VIEW OF STREAM IN
FOREST","probability":0.4874233842518491,"wordsProbability":
[["VIEW",-1.8661552402619594],["OF",-0.0041088119466486661],
["STREAM",-0.087614953179970007],["IN",-1.3630659191281458],
["FOREST",-1.690672657028897]]},{"caption":"VIEW OF STREAM IN
WATER","probability":0.43154049087804885,"wordsProbability":
Captions

57. Billions of events

58. = …
Redshift
HDFS/Spark
Apache Flume
Report

59. Our move to EMR

60. Analytics pipeline:
Spark/EMR/Redshift

61. = …
Redshift
HDFS/Spark
Apache Flume
Report

62. = …
Redshift
Apache Flume
Report
Events
EMR

63. ML pipeline:
Spark ML/EMR/ElastiCache

64. = …
Redshift
Apache Flume
Report
Events
EMR

65. = …
Apache Flume
Clients
Events
EMRElastiCacheAPI

66. Machines produce a ton of data
Hadoop management is time consuming, avoid it!
Event driven data architectures have many advantages

67. Thank you!
@rinofm