This document discusses Apache Zeppelin, an open-source notebook application that allows for data exploration and visualization. It can run Python, Scala/Spark, and bash code. The document then describes using Zeppelin to build a machine learning model for identifying household objects from images using MXNet and datasets stored on AWS S3. It discusses the infrastructure, ETL process, decisions around model type (classification vs object detection), and lessons learned around validation and explainability.

1. @garysieling
APACHE ZEPPELINAPACHE ZEPPELIN

3. @garysieling

4. @garysieling
Notebook Features
Notebook Uses
Machine Learning Demo

5. @garysieling
WHY ZEPPELIN?WHY ZEPPELIN?
Multi-language (Python, Scala/Spark, bash)
Runs on the JVM

6. @garysieling
LANGUAGE CONVERSIONLANGUAGE CONVERSION

7. @garysieling
DATA VISUALIZERDATA VISUALIZER

8. @garysieling
DATA VISUALIZERDATA VISUALIZER

9. @garysieling
CLOUD + SPARK INTEGRATIONCLOUD + SPARK INTEGRATION

10. @garysieling
JUPYTERJUPYTER
More Macros / languages
Older / larger community
Security / multi-user features

11. @garysieling
ZEPPELIN APIZEPPELIN API

12. @garysieling

13. @garysieling

14. @garysieling

15. @garysieling
ENVIRONMENT SETUPENVIRONMENT SETUP
Zeppelin Docker (Java / Jetty)
Pre-installed python libraries (opencv, mxnet)
Git Volume
Data Volume

16. @garysieling
IDENTIFYING HOUSEHOLD OBJECTSIDENTIFYING HOUSEHOLD OBJECTS
I.e. a home inspector
Reverse image search
No good datasets

17. @garysieling

18. @garysieling

19. @garysieling
INFRASTRUCTUREINFRASTRUCTURE
App to record videos
Stock Videos stored on S3
Pull videos to training machine
Extract frames with ffmpeg
Train with mxnet / gluon

20. @garysieling

21. @garysieling
ETL PROCESSETL PROCESS

22. @garysieling

23. @garysieling

24. @garysieling

25. @garysieling

26. @garysieling

27. @garysieling
DECISION POINTSDECISION POINTS

28. @garysieling
WHY MXNET?WHY MXNET?
AWS support
Multi-language
DeepLens

30. @garysieling
IMAGE SOURCESIMAGE SOURCES
Bing API
Stock videos
Idea: 3-D models?

31. @garysieling

32. @garysieling

33. @garysieling

34. @garysieling

36. @garysieling

37. @garysieling
DECISIONSDECISIONS
Classification
Object Detection
Segmentation

38. @garysieling

39. @garysieling
DEBUGGINGDEBUGGING
Run end-to-end with a small dataset
Debug every step
Ability to restart from a known point

40. SPOT CHECK TAGSSPOT CHECK TAGS

41. @garysieling

42. DATASET SIZEDATASET SIZE

43. @garysieling

44. PROCESSED IMAGE PREVIEWPROCESSED IMAGE PREVIEW

45. TAG AN IMAGETAG AN IMAGE

46. CONFUSION MATRIXCONFUSION MATRIX

47. @garysieling

48. @garysieling
READ A TSV FILEREAD A TSV FILE

49. @garysieling
TSV VISUALIZATIONTSV VISUALIZATION

50. @garysieling

51. MANUAL VALIDATION (863 ENTRIES)MANUAL VALIDATION (863 ENTRIES)

52. @garysieling

53. @garysieling
AUGMENTATION WITH MXNET FILTERSAUGMENTATION WITH MXNET FILTERS

54. @garysieling

55. @garysieling

56. @garysieling
EXPLAINABILITYEXPLAINABILITY

57. @garysieling

58. @garysieling

59. @garysieling

60. @garysieling
REPRODUCEABILITYREPRODUCEABILITY
Notebook lets you build a world around your
problem
Dockerized Scripts
Upload output to S3
Upload a testing notebook to S3

61. @garysieling
LESSONS LEARNEDLESSONS LEARNED
Do some manual tagging (challenge data vs. real
world problems)
Hold out a validation set that matches the
"business" problem

62. @garysieling
WHEN DOES IMAGE RECOGNITIONWHEN DOES IMAGE RECOGNITION
WORK?WORK?

63. @garysieling
NEXT STEPSNEXT STEPS
Cloud scaling
New Sensors
Video flicker

64. @garysieling
OTHER RESOURCESOTHER RESOURCES
mxboard (tensorboard integration)
Pyimagesearch
https://github.com/garysieling/apache-zeppelin-
talk-slides

65. @garysieling
MobileNet -
Resnet152 -
VGG16 -
https://www.youtube.com/watch?
v=IFRSQxhFoZk
https://www.youtube.com/watch?
v=aLPYIn_YnjM
https://youtu.be/z7oWboe5gHY

66. @garysieling
GARY SIELINGGARY SIELING
Element 84
@garysieling
gary.sieling@gmail.com

67. @garysieling