Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Open sourcing the IoT : EnMasse running on Kubernetes

4,056 views

Published on

What is messaging ? How messaging is strictly related to the IoT ? Why we should use an open source solution for the IoT ? During this session we'll see the EnMasse project (and Eclipse Hono), its architecture and how it can be deployed on Kubernetes or OpenShift providing a messaging infrastructure for the IoT.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Open sourcing the IoT : EnMasse running on Kubernetes

  1. 1. Open sourcing the IoT Running EnMasse on Kubernetes Paolo Patierno Senior Software Engineer @ Red Hat 05/06/2017
  2. 2. INSERT DESIGNATOR, IF NEEDED2 Who am I ? @ppatierno ● Senior Software Engineer @ Red Hat ○ Messaging & IoT team ● Lead/Committer @ Eclipse Foundation ○ Hono, Paho and Vert.x projects ● Microsoft MVP ● Technologies and protocols “globetrotter” ● Hacking low constrained devices in spare time ● Blogger and speaker about distributed systems, messaging, IoT and embedded “world”
  3. 3. INSERT DESIGNATOR, IF NEEDED3 Agenda ● Messaging … what ? ● Messaging … for IoT ● Messaging & IoT … in the cloud ● EnMasse : the open source MaaS ! ● Running EnMasse … ○ Kubernetes ○ Azure Container Service ○ OpenShift
  4. 4. INSERT DESIGNATOR, IF NEEDED4 ● It’s about messages exchange ○ Internally in distributed systems ○ Externally between systems ● Communication at the application level ● Messages go from sender/producer/publisher to receiver/consumer/subscriber ○ Asynchronously ○ Time decoupling ○ … or directly and synchronously What is messaging?
  5. 5. INSERT DESIGNATOR, IF NEEDED5 Messaging patterns C S P S S S msg X msg X msg X P C C C msg X msg Y msg Z Request/Response Publish/Subscribe Competing Consumers msg req msg resp
  6. 6. INSERT DESIGNATOR, IF NEEDED6 Quality of Service P C At Most Once At Least Once Exactly Once msg X P C msg Y P C msg Z P C msg X P C msg Y P C msg Y P C msg X P C msg Y P C msg Z
  7. 7. INSERT DESIGNATOR, IF NEEDED7 IoT : messaging vengeance ● … maybe in the past … ● … messaging was not so cool for developers ... ● … but today with IoT this is changed because … ● … IoT is all about messaging so … “Messaging vengeance” !
  8. 8. INSERT DESIGNATOR, IF NEEDED8 “give me a scalable messaging platform, and I shall move the Internet of Things world” (Archimedes) IoT : messaging as a “lever”
  9. 9. INSERT DESIGNATOR, IF NEEDED IoT Core Services 9 What makes an IoT platform ? Messaging infrastructure Device Provisioning Business Services Monitoring Real time streaming Machine Learning … Authentication & Authorization Device Registration
  10. 10. INSERT DESIGNATOR, IF NEEDED10 IoT : communication patterns Telemetry Inquiries Commands Notifications
  11. 11. INSERT DESIGNATOR, IF NEEDED11 IoT : communication patterns Messaging patterns & protocols ● Telemetry & Notifications are about … ○ …. messaging publish/subscribe ● Commands & Inquiries are about … ○ … messaging request/response ● Different protocols (AMQP, MQTT, HTTP, …) implement them in different way ○ As built-in support … ○ … or on top of it at application level ○ Read more on “Strengths And Weaknesses Of IoT Communication Patterns” * * DZone IoT Guide : https://dzone.com/guides/iot-applications-protocols-and-best-practices
  12. 12. INSERT DESIGNATOR, IF NEEDED12 IoT : interoperability Open standards AMQP 1.0 MQTT HTTP CoAP XMPP STOMP
  13. 13. INSERT DESIGNATOR, IF NEEDED13 ● Microsoft Azure ○ Service Bus + Event Hub ○ IoT Hub ● Amazon Web Services ○ Simple Queue Service (SQS) ○ AWS IoT ● Google ○ FireBase Cloud Messaging ○ IoT Core Messaging & IoT in the cloud
  14. 14. INSERT DESIGNATOR, IF NEEDED14 ● They are not open source ! ● Freedom of choice ○ On-premise or in the cloud ○ Ability to choose which cloud ○ Open Standards protocols allows users to choose client freely ● Migrating from one to the other can be complex Cloud provider limitations
  15. 15. INSERT DESIGNATOR, IF NEEDED15 ● Open source cloud messaging running on Kubernetes and OpenShift ● enmasse.io EnMasse Messaging-as-a-Service
  16. 16. INSERT DESIGNATOR, IF NEEDED16 ● Multiple communication patterns: request/response, publish/subscribe and competing consumers ● Support for “store and forward” and direct messaging mechanisms ● Scale and elasticity of message brokers ● AMQP 1.0 and MQTT support ● Simple setup, management and monitoring ● Multitenancy: manage multiple independent instances ● Deploy “on premise” or in the cloud EnMasse Features
  17. 17. INSERT DESIGNATOR, IF NEEDED17 ● Authentication and authorization ● Service broker API ● HTTP(S) ● Message grouping ● Distributed transactions ● Message ordering ● Multiple flavors ○ Apache Kafka ● ... EnMasse Coming features
  18. 18. INSERT DESIGNATOR, IF NEEDED18 Architecture
  19. 19. INSERT DESIGNATOR, IF NEEDED19 Routing vs “Broking” Broker Producer Broker Consumer Send message Accepted Send message Accepted
  20. 20. INSERT DESIGNATOR, IF NEEDED20 Routing vs “Broking” Router Producer Router Consumer Send message Accepted Send message Accepted
  21. 21. INSERT DESIGNATOR, IF NEEDED MQTT over AMQP 21 ● MQTT gateway ○ Handles connections with remote MQTT clients ○ Bridges MQTT - AMQP protocols ● MQTT lwt ○ Provides the “will testament” feature ○ In charge to recover & send the “will” if client dies ● It brings MQTT features over AMQP so … ○ … “will testament” works for AMQP clients as well
  22. 22. INSERT DESIGNATOR, IF NEEDED22 Configuration distribution
  23. 23. INSERT DESIGNATOR, IF NEEDED23 Configuration interface { "apiVersion": "v3", "kind": "Address", "metadata": { "name": "myqueue" }, "spec": { "store_and_forward": true, "multicast": false, "flavor": "vanilla-queue" } }
  24. 24. INSERT DESIGNATOR, IF NEEDED { "apiVersion": "v3", "kind": "Flavor", "metadata": { "name": "vanilla-queue" }, "spec": { "type": "queue", "Description": "Simple in-memory queue", "templateName": "queue-inmemory", "templateParameters": {} } } 24 Configuration interface
  25. 25. INSERT DESIGNATOR, IF NEEDED25 Continuous integration
  26. 26. INSERT DESIGNATOR, IF NEEDED Continuous integration 26 Component build pipeline build test Build docker image & push success failure trigger systemtests
  27. 27. INSERT DESIGNATOR, IF NEEDED AMQP 1.0 27 Eclipse Hono Connect. Command. Control API Endpoints Auth Service Device Registry Protocol Adapters Business Applications Devices Devices AMQP 1.0 AMQP 1.0 AMQP 1.0 HTTP, MQTT
  28. 28. INSERT DESIGNATOR, IF NEEDED28 Eclipse Hono IoT API ● Telemetry ○ used by devices to send data downstream ○ leverages on “direct messaging” ● Device Registration ○ used to make Hono aware of devices that will connect to the service ○ register, deregister, get information … ● Event ○ used by devices to send event downstream ○ differ from Telemetry on using “store and forward” (with TTL) ● Command & Control ○ used by applications to send commands to devices ○ command execution can be “just in time” or “deferred”
  29. 29. INSERT DESIGNATOR, IF NEEDED29 IoT : how to deploy ? ● “On premise” … ○ … maybe for a not so big solution ○ … ingesting few data and handling few devices ● “Cloud” … ○ … needs for more scalability ○ … don’t want to manage the infrastructure ● “Hybrid” … ○ … needs for processing at the edge ○ … needs for not making sensible data public
  30. 30. INSERT DESIGNATOR, IF NEEDED30 Azure Container Service ● A containers hosting solution ● Scale and orchestrate using … ○ Kubernetes ○ Docker Swarm ○ DC/OS ● Deploying a cluster using Azure CLI / portal ○ Resource group with VMs, load balancer, ... ● Managing directly your preferred “orchestrator” ○ ACS provides you “only” the infrastructure
  31. 31. INSERT DESIGNATOR, IF NEEDED31 Azure & OpenShift ● OpenShift Origin ○ the upstream open source project ● OpenShift Container Platform ○ the Red Hat productized version ○ enterprise grade container platform +
  32. 32. Demo : the deployment on Kubernetes Qpid Dispatch Router ActiveMQ Artemis Spark Streaming driver AMQP clients temperature max 24 → /temperature /temperature → … 23, 24, 23, … /max ← … 23, 25, … 25 ← /max
  33. 33. Demo : the deployment on OpenShift Qpid Dispatch Router ActiveMQ Artemis Spark Streaming driver AMQP clients temperature max 24 → /temperature /temperature → … 23, 24, 23, … /max ← … 23, 25, … 25 ← /max
  34. 34. INSERT DESIGNATOR, IF NEEDED34 DEMO
  35. 35. INSERT DESIGNATOR, IF NEEDED Resources 35 ● EnMasse : https://enmasseproject.github.io/ ● Qpid Dispatch Router : http://qpid.apache.org/components/dispatch-router/ ● ActiveMQ Artemis : https://activemq.apache.org/artemis/ ● Azure Container Service : https://azure.microsoft.com/en-us/services/container-service/ ● OpenShift on Azure : http://aka.ms/openshift ● Eclipse Hono : https://www.eclipse.org/hono/ ● Demo : https://github.com/ppatierno/enmasse-spark-demo ● My blog : https://paolopatierno.wordpress.com/
  36. 36. Thank you ! Questions ? @ppatierno

×