Using Kubernetes and TensorFlow to build the Fog Computing Platform that can dynamically deploy the deep learning applications on to the IoT devices (Raspberry PI).

1. Raspberry PI x Kubernetes x TensorFlow
蔡霈萱 Pei-Hsuan Tsai
!1

2. About Me
▸ Linker Networks Inc. , Software Engineer
▸ NTHU, Master program in Computer Science
▸ NCTU, Bachelor degree in Computer Science
▸ Github: https://github.com/WakeupTsai
▸ Email: phstsai@gmail.com
!2

3. !3
Independent Study Project（2015.03)

4. ⾞位在哪裡：⾞位尋找及預約系統
▸ OpenMTC
• A prototype implementation of an IoT/M2M middleware
aiming to provide a standard-compliant platform for IoT
services
▸ Devices
• Arduino UNO R3
• Distance Measuring Sensor
• Arduino Wifi Shield
• Wireless router
!4https://github.com/WakeupTsai/IOT_Projects

5. Architecture
!5

6. !6
Docker + Kubernetes (2016.02)

7. Motivation
▸ Internet of Things (IoT) grows rapidly
▸ Produce incredible amount of data
• Overload the data centers and congest the networks
seriously
!7

8. Limitations of Current Solution
!8
Analyze and Compute
Data
in Data center
Huge Amount of

9. Edge Analytics - Pre-processing
▸ Reduce latency
▸ Reduce network traffic
▸ Reduce the load of data centers
!9
REPORT

10. Shooter Tracking Usage Scenario
!10
• Face recognition
• Path estimation
• Sound recognition

11. ▸ Fog computing leverages devices in data centers, edge
networks, and end devices in simultaneously
End Devices
Edge Networks
Data Centers
Fog Computing
!11

12. Advantages: Fog >> Cloud
▸ Reduce network traffic
▸ Short response time
▸ Diverse kinds of resources
• Computations, communications, storage, and sensors
▸ Utilize wasted resources
▸ Low cost
▸ Low carbon foot print
▸ …
!12

13. Requirement of Dynamic Deployment
▸ Frequently updating or replacing the applications
• Application virtualization
▸ Managing lots of fog devices and applications
• Orchestration tool
▸ Triggering another application when something happened
• Event-driven mechanism
!13

14. Virtualization Technology
▸ Virtualized modules
• Dynamically placed on the fog devices
• Migrated among the fog devices
• Allocated the resource on-demand
• More private
!14

15. Traditional VM v.s Container
▸ Container
• Share the same OS kernel,
and use the namespaces
to distinguish one from
another
▸ Traditional Virtual Machine
• Need large storage space
and more computing
power
!15What is a Container [Digital image]. (n.d.). Retrieved from https://www.docker.com/what-container

16. Container-based Applications
▸ Lightweight
• Quick start
• Easy to replace the configuration of
the applications
!16

17. Docker Image
▸ A Docker image is built up from a series of layers. Each
layer represents an instruction in the image’s Dockerfile.
Each layer except the very last one is read-only
!17

18. Example
!18

19. Raspberry PI
▸ A small and affordable computer that you can use to learn
programming
!19

20. HypriotOS
▸ The fasted way to get Docker up and running on any
Raspberry Pi
!20
https://blog.hypriot.com

21. Kubernetes Architecture
▸ Master / Minion
▸ Each devices hosts several containers, can be assembled
into pod
!21
Minion 1 Minion 2

22. !22
Location A Location B
Resource Usage: 5%
Resource Usage: 29%
Resource Usage: 79%
Resource Usage: 11%
Resource Usage: 36%
Resource Usage: 44%
Deploy the tasks on suitable devices

23. !23
Example

24. Master
▸ i5 CPU NUC
▸ Kubernetes v1.5.0
!24
https://github.com/kubernetes/kubernetes

25. ▸ Raspberry PIs
▸ kubernetes-on-arm
Minion
!25
https://github.com/luxas/kubernetes-on-arm

26. !26
SCALE (2016.05)

27. ▸ Safe Community Alert Network
• Using the available, cheap sensors we have to build an
affordable, practical security system
SCALE
!27
https://github.com/WakeupTsai/SmartAmericaSensors

28. MQTT
▸ MQTT is a M2M/IoT connectivity protocol. It was
designed as an extremely lightweight publish/
subscribe messaging transport
!28

29. !29
Monitoring (2016.11)

30. ▸ Fog devices situation monitoring
• Resource usage (CPU, Memory)
• Containers status
Providing the important information for deployment
strategy.
Kubernetes Dashboard
!30https://github.com/WakeupTsai/FogComputingPlatform-dashboard
https://github.com/kubernetes/dashboard

31. Kubernetes Dashboard
!31https://github.com/WakeupTsai/FogComputingPlatform-dashboard
https://github.com/kubernetes/dashboard

32. MQTT Panel
!32https://github.com/WakeupTsai/FogComputingPlatform-MQTT-Panel
https://github.com/fabaff/mqtt-panel

33. !33
TensorFlow (2017.03)

34. Requirement of Edge IoT Analytics
▸ Raw sensor data are huge
• Deep learning
▸ Resource-constrained fog computing devices
• Distributed computing
!34

35. Pre-processing with Deep Learning
▸ TensorFlow
• An open-source software library for
Machine Intelligence
• Data flow graphs
❖ Nodes - mathematical operations
❖ Edges - multidimensional data arrays
(tensors)
!35
Cut

36. Example
!36

37. ▸ Air quality monitor
▸ Sound classification
▸ Object detection
3 Applications
!37
https://github.com/WakeupTsai/tensorflow-applications
https://github.com/nextco/audio-classification
https://github.com/nilboy/tensorflow-yolo

38. ▸ Collecting the resource from several heterogeneous fog
devices
Distributed Computing
!38

39. !39
Fog Computing Platform (2017.07)

40. System Overview
!40

41. ▸ User interface
▸ Operator deployment algorithm
• Decide deploying which
operators on which minions
▸ Device manager
• Collect crucial device status
▸ Deployment manager
• Launch specific Docker images on chosen minions
▸ Image pool
• Docker images are stored in the image pool at the server
Master
!41

42. Minions
▸ TensorFlow-enabled container
• Docker containers including
TensorFlow and its analytic
libraries
▸ Client agent
• Monitor and report the status of minions and pod to
device manager
▸ Local image pool
!42

43. Programming Model
!43

44. !44
3 Applications

45. How to Implement
▸ Kubernetes YAML
▸ Docker Images (TensorFlow base image)
▸ python code (Import TensorFlow library)
!45
https://hub.docker.com/r/wakeup706/rpi-tensorflow/
https://github.com/samjabrahams/tensorflow-on-raspberry-pi

46. ▸ Helm is a tool for managing Kubernetes charts. Charts are
packages of pre-configured Kubernetes resources.
Helm
!46
https://github.com/helm/helm
https://github.com/WakeupTsai/FogComputingPlatform-Auto-Deploy

47. !47
Master
Alerter
Kubernetes
MQTT Broker
Minion
Minion
Environment
Monitor
Environment
Monitor
Environment
Monitor Minion
Publish the motion detect event.
Subscribe the event from broker,
then determine it as an intrusion.
Ask Kubernetes to launch
the surveillance service.
Object
Detection
Image
Capture
Deploy an image
capture app.
Deploy an object
detection app.
Publish the result.
Deploy environment
monitor app on every
street light.
Capture an image, and publish it
Subscribe the image, and
then do the analyze.
Scenario

48. !48
Demo Video

49. Demo Video
!49
https://drive.google.com/file/d/1tRs355XXMSocAejSZ_E-4LkjiOQnu7-y/view?usp=sharing

50. Q&A