A scalable processing architecture for Internet of Things

1. chariot
A scalable processing architecture for Internet of Things
Abhilash Kumar and Saurav Kumar
April 19, 2016
Indian Institute of Technology Kanpur

2. introduction

3. introduction
Credits: bhartiyatech.org 2

4. introduction
Internet of Things
∙ Network of physical objects—devices, vehicles, buildings etc
Credits: Wikipedia
3

5. introduction
Internet of Things
∙ Network of physical objects—devices, vehicles, buildings etc
∙ Embedded with electronics, software, sensors, and network
connectivity
Credits: Wikipedia
3

6. introduction
Internet of Things
∙ Network of physical objects—devices, vehicles, buildings etc
∙ Embedded with electronics, software, sensors, and network
connectivity
∙ Objects can be sensed and controlled remotely across existing
network infrastructure
Credits: Wikipedia
3

7. introduction
Internet of Things
∙ Network of physical objects—devices, vehicles, buildings etc
∙ Embedded with electronics, software, sensors, and network
connectivity
∙ Objects can be sensed and controlled remotely across existing
network infrastructure
∙ IoT augmented with sensors and actuators: Smart grids, smart
homes, intelligent transportation and smart cities
Credits: Wikipedia
3

8. introduction
Credits: Clarice Technologies (2014)
4

9. chariot

10. chariot
The Idea
∙ Use the computational resources of powerful machines
6

11. chariot
The Idea
∙ Use the computational resources of powerful machines
∙ Small IoT devices need not have powerful processors or RAMs
6

12. chariot
The Idea
∙ Use the computational resources of powerful machines
∙ Small IoT devices need not have powerful processors or RAMs
∙ Saves resources, and thus money
6

13. chariot
The Idea
∙ Use the computational resources of powerful machines
∙ Small IoT devices need not have powerful processors or RAMs
∙ Saves resources, and thus money
∙ Saves battery power on devices
6

14. chariot
The Idea
∙ Use the computational resources of powerful machines
∙ Small IoT devices need not have powerful processors or RAMs
∙ Saves resources, and thus money
∙ Saves battery power on devices
∙ We present: A scalable architecture to solve this problem
6

15. chariot
The Idea
∙ Use the computational resources of powerful machines
∙ Small IoT devices need not have powerful processors or RAMs
∙ Saves resources, and thus money
∙ Saves battery power on devices
∙ We present: A scalable architecture to solve this problem
∙ Inspired from Pub-Sub architecture and RPC protocol
6

16. chariot
Nomenclature
Credits: sinetif.com
7

17. chariot
Nomenclature
Credits: sinetif.com
8

18. chariot
Nomenclature
∙ Also, CHAR is anagram for ARCH: Architecture for IOT
9

19. chariot
Nomenclature
∙ Also, CHAR is anagram for ARCH: Architecture for IOT
∙ Named our components based on horse-wagon analogy
9

20. chariot
Nomenclature
∙ Also, CHAR is anagram for ARCH: Architecture for IOT
∙ Named our components based on horse-wagon analogy
∙ IoT device - Yaan (Sanskrit for wagon)
9

21. chariot
Nomenclature
∙ Also, CHAR is anagram for ARCH: Architecture for IOT
∙ Named our components based on horse-wagon analogy
∙ IoT device - Yaan (Sanskrit for wagon)
∙ Processing Machine - Ashva (Sanskrit for horse)
9

22. chariot
Nomenclature
∙ Also, CHAR is anagram for ARCH: Architecture for IOT
∙ Named our components based on horse-wagon analogy
∙ IoT device - Yaan (Sanskrit for wagon)
∙ Processing Machine - Ashva (Sanskrit for horse)
∙ Supervising Machine - Prashti (Sanskrit for horse leader)
9

23. chariot
Nomenclature
∙ Also, CHAR is anagram for ARCH: Architecture for IOT
∙ Named our components based on horse-wagon analogy
∙ IoT device - Yaan (Sanskrit for wagon)
∙ Processing Machine - Ashva (Sanskrit for horse)
∙ Supervising Machine - Prashti (Sanskrit for horse leader)
∙ Zookeeper - Turagraksa (Sanskrit for patron of horse)
9

24. architecture diagrams

25. architecture diagrams
Logical View
11

26. architecture diagrams
Physical View
12

27. architecture diagrams
Process View
13

28. implementation

29. implementation
Technology Stack
∙ Source Language: Java 7
15

30. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
15

31. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
∙ RabbitMQ RPC Server
15

32. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
∙ RabbitMQ RPC Server
∙ Apache Avro Data Schema
15

33. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
∙ RabbitMQ RPC Server
∙ Apache Avro Data Schema
∙ Spark WebServer for D2
15

34. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
∙ RabbitMQ RPC Server
∙ Apache Avro Data Schema
∙ Spark WebServer for D2
∙ Docker
15

35. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
∙ RabbitMQ RPC Server
∙ Apache Avro Data Schema
∙ Spark WebServer for D2
∙ Docker
∙ MongoDB
15

36. implementation
Technology Stack
∙ Source Language: Java 7
∙ Gradle build system
∙ RabbitMQ RPC Server
∙ Apache Avro Data Schema
∙ Spark WebServer for D2
∙ Docker
∙ MongoDB
∙ JUnit Test Framework
15

37. implementation
Flow diagram
16

38. demonstration

39. demonstration
Demonstration
∙ Project page and Codebase
18

40. demonstration
Demonstration
∙ Project page and Codebase
∙ Screencast of deployment, running simple Java client, fault
tolerance testing
18

41. demonstration
Demonstration
∙ Project page and Codebase
∙ Screencast of deployment, running simple Java client, fault
tolerance testing
∙ Image Blur Android App demo
18

42. demonstration
Demonstration
∙ Project page and Codebase
∙ Screencast of deployment, running simple Java client, fault
tolerance testing
∙ Image Blur Android App demo
∙ Live demo of using chariot framework from Python shell as a
client
18

43. quality attributes

44. quality attributes
Quality Attributes
✓ Availability: Two Prashti (and Zookeeper) to make system
available
20

45. quality attributes
Quality Attributes
✓ Availability: Two Prashti (and Zookeeper) to make system
available
✓ Maintainability: Modular code with Java best practices.
20

46. quality attributes
Quality Attributes
✓ Availability: Two Prashti (and Zookeeper) to make system
available
✓ Maintainability: Modular code with Java best practices.
✓ Performance: Low latency by design. Load-balancing among
Ashvas
20

47. quality attributes
Quality Attributes
✓ Availability: Two Prashti (and Zookeeper) to make system
available
✓ Maintainability: Modular code with Java best practices.
✓ Performance: Low latency by design. Load-balancing among
Ashvas
× Security: Unencrypted data flow and common shared directory
for docker communication
20

48. quality attributes
Quality Attributes
✓ Availability: Two Prashti (and Zookeeper) to make system
available
✓ Maintainability: Modular code with Java best practices.
✓ Performance: Low latency by design. Load-balancing among
Ashvas
× Security: Unencrypted data flow and common shared directory
for docker communication
✓ Scalability: Devices and Ashvas can be added dynamically at
runtime
20

49. quality attributes
Quality Attributes
✓ Availability: Two Prashti (and Zookeeper) to make system
available
✓ Maintainability: Modular code with Java best practices.
✓ Performance: Low latency by design. Load-balancing among
Ashvas
× Security: Unencrypted data flow and common shared directory
for docker communication
✓ Scalability: Devices and Ashvas can be added dynamically at
runtime
✓ Usability: Simple interface for developers to integrate
applications and devices.
20

50. quality attributes
Quality Attributes
✓ Testability: JUnit Tests automatically runs on gradle build.
Integration Tests also provided with source code.
21

51. quality attributes
Quality Attributes
✓ Testability: JUnit Tests automatically runs on gradle build.
Integration Tests also provided with source code.
× Efficiency: Local procedure calls will outperform remote
procedure calls for small computations.
21

52. quality attributes
Quality Attributes
✓ Testability: JUnit Tests automatically runs on gradle build.
Integration Tests also provided with source code.
× Efficiency: Local procedure calls will outperform remote
procedure calls for small computations.
✓ Reliability: Automated recovery begins on faults. Uses active
redundancy to restore states.
21

53. quality attributes
Quality Attributes
✓ Testability: JUnit Tests automatically runs on gradle build.
Integration Tests also provided with source code.
× Efficiency: Local procedure calls will outperform remote
procedure calls for small computations.
✓ Reliability: Automated recovery begins on faults. Uses active
redundancy to restore states.
✓ Flexibility: By design, keeping in mind the use case.
21

54. quality attributes
Quality Attributes
✓ Testability: JUnit Tests automatically runs on gradle build.
Integration Tests also provided with source code.
× Efficiency: Local procedure calls will outperform remote
procedure calls for small computations.
✓ Reliability: Automated recovery begins on faults. Uses active
redundancy to restore states.
✓ Flexibility: By design, keeping in mind the use case.
✓ Reusability: Modular code with Java best practices. Avro schema
reusable for different languages
21

55. use cases

56. use cases
Possible use cases
∙ Internet of Things: By 2020, home networks will have many IoT
devices - sensors, smart TVs, washing machines, AI powered
mini-bots etc
23

57. use cases
Possible use cases
∙ Internet of Things: By 2020, home networks will have many IoT
devices - sensors, smart TVs, washing machines, AI powered
mini-bots etc
∙ As private cloud offering PaaS in institutions and organizations.
CSE students can put their docker images on Docker Hub and
simply run code from personal devices which will be executed in
CSE Lab Machines
23

58. use cases
Possible use cases
∙ Internet of Things: By 2020, home networks will have many IoT
devices - sensors, smart TVs, washing machines, AI powered
mini-bots etc
∙ As private cloud offering PaaS in institutions and organizations.
CSE students can put their docker images on Docker Hub and
simply run code from personal devices which will be executed in
CSE Lab Machines
23

59. Questions?
Slides created using Beamer(mtheme) on ShareLaTeX
24

60. Thank You
25