Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing and Implementing your IOT Solutions with Open Source

1,702 views

Published on

Designing and Implementing your IOT Solutions with Open Source

Published in: Technology
  • Be the first to comment

Designing and Implementing your IOT Solutions with Open Source

  1. 1. © 2016 MapR Technologies 1© 2016 MapR Technologies Designing & Implementing your IOT Solutions with Open Source
  2. 2. © 2016 MapR Technologies 2 Agenda • Use Case • Demo • Layers – Sensor – Controller – Protocol – Messaging – Computation – Persistence – Dashboard
  3. 3. © 2016 MapR Technologies 3© 2015 MapR Technologies Use Case – Temperature Monitoring
  4. 4. © 2016 MapR Technologies 4 Use Case – Temperature monitoring dashboard
  5. 5. © 2016 MapR Technologies 5 Do I need big data for IOT ? Sensor reads temperature once every second • Number of readings per year 3600*24*365 = 31,536,000 (31 million) • Every reading is 100 bytes 31,536,000 * 100 = 3,153,600,000 (3.1 GB) • Every store has 50 aisel(4 sensors) and 6000+ stores 200 * 6000 = 1,200,000 (1.2 million sensors) 3 * 200 * 6000 = 3,600,000 (3.6 PetaBytes)
  6. 6. © 2016 MapR Technologies 6© 2015 MapR Technologies Demo
  7. 7. © 2016 MapR Technologies 7 Demo – Data Flow
  8. 8. © 2016 MapR Technologies 8 Demo – Data Flow
  9. 9. © 2016 MapR Technologies 9 IOT Big Data Reference Architecture
  10. 10. © 2016 MapR Technologies 10© 2015 MapR Technologies Sensors/Actuators
  11. 11. © 2016 MapR Technologies 11 Why do we need Sensor/Actuator Layer ? Sensor/actuator will be responsible for 1. Interacting with physical world 1. Sensor – Measures or detects physical property 1. Thermistor 2. Temperature Oscillators 2. Actuator – Move or control mechanism or system 1. LED 2. Servo Motor
  12. 12. © 2016 MapR Technologies 12 Temperature Sensors DS18B20 DHT22
  13. 13. © 2016 MapR Technologies 13 Sensor Pin Layout DS18B20 DHT22
  14. 14. © 2016 MapR Technologies 14 Actuator Pin Layout
  15. 15. © 2016 MapR Technologies 15 Home Automation
  16. 16. © 2016 MapR Technologies 16 IOT Home Automation You can build home automation 1. Temp Monitoring – For AC 2. Auto turning of lights on and off 3. Sprinkler System 4. Primitive home security 5. Gas detection
  17. 17. © 2016 MapR Technologies 17© 2015 MapR Technologies Device
  18. 18. © 2016 MapR Technologies 18 Why do we need Device Layer ? The controller layer is responsible for 1. Reading data from sensor 2. Transform sensor data into message 3. Communicate with IOT Gateway using suitable protocol 4. Listening to message from IOT Gateway 5. Activating with actuators based on message from Gateway
  19. 19. © 2016 MapR Technologies 19 Arduino Arduino Uno NodeMCU V2 –ESP8266
  20. 20. © 2016 MapR Technologies 20 Arduino Application
  21. 21. © 2016 MapR Technologies 21 Install Sensor Library
  22. 22. © 2016 MapR Technologies 22 Temperature sensor – Sensor Library
  23. 23. © 2016 MapR Technologies 23 Temperature sensor – Start Sensor
  24. 24. © 2016 MapR Technologies 24 Arduino – Read temperature
  25. 25. © 2016 MapR Technologies 25 Upload code to Arduino
  26. 26. © 2016 MapR Technologies 26 Raspberry Pi
  27. 27. © 2016 MapR Technologies 27 Raspberry pi - Logic for reading temperature
  28. 28. © 2016 MapR Technologies 28 Raspberry Pi – Development Life Cycle
  29. 29. © 2016 MapR Technologies 29 IOT Device comparison Arduino Raspberry Pi Architecture Micro controller Micro Processor Operating System No Yes Power Can go to sleep Continuous 5V Network No Yes Sensor Analog + digital Digital Development Environment Arduino IDE – C code Java, Python,… etc
  30. 30. © 2016 MapR Technologies 30© 2015 MapR Technologies Protocols
  31. 31. © 2016 MapR Technologies 31 Why do we need Protocol Layer ? The protocol layer is responsible for 1. Communication between IOT Device and IT infrastructure 2. The IOT Devices operate in constrained environment 1. Limited processing power 2. Limited amount of memory (Less than 100K) 3. Network usage 1. Less Reliable network 2. Optimized usage of network 4. Operate on battery 1. Sleep mode
  32. 32. © 2016 MapR Technologies 32 MQTT – Message Queue Telemetry Transport MQTT is publish/subscribe messaging protocol designed for lightweight M2M communication. • Open: MQTT is OASIS standard created by IBM • Lightweight Client: Client libraries less than 30 KB • Optimized Network usage: Small packet size 2 bytes (header) – TCP Based. MQTT-SN is UDP based – Easy to parse packets • Reliable: Three QoS patterns to avoid packet loss on client disconnect – At-most-once – does not survive failures, but is never duplicated – At-least-once – survives connection loss and other failures, but can be duplicated – Exactly-once – survives connection loss and other failures, and is never duplicated
  33. 33. © 2016 MapR Technologies 33 MQTT vs. HTTPS 1. Performance is 80 times better 2. Battery usage 10 times less http://stephendnicholas.com/archives/1217
  34. 34. © 2016 MapR Technologies 34 MQTT Device 1 Device 2 Device 3 MQTT Broker Subscriber 1 Subscriber 2 /store/fremont/aisel1 /store/fremont/aisel2 /store/sanjose/aisel1 /store/fremont/# /store/#
  35. 35. © 2016 MapR Technologies 35 MQTT – Server & Broker Subscriber 1 Subscriber 2 /store/fremont/aisel1 /store/fremont/aisel2 /store/sanjose/aisel1 /store/fremont/# /store/#
  36. 36. © 2016 MapR Technologies 36 MQTT – Include MQTT client library
  37. 37. © 2016 MapR Technologies 37 MQTT – Setup MQTT Connection
  38. 38. © 2016 MapR Technologies 38 Arduino – Send Receive Message
  39. 39. © 2016 MapR Technologies 39 Arduino – Memory usage
  40. 40. © 2016 MapR Technologies 40 CoAP – Constrained Application Protocol The CoAP is request/response protocol similar to HTTP + REST • Open: CoAP is IETF standard - RFC 7252 IETF • Lightweight Client: Client libraries less than 30 KB • Optimized Network usage – Based on UDP (6LowPAN), SMS in addition to TCP – Compact 4 bytes header, compressed – CoAP uses familiar HTTP stuff like Method (Get, Put, Post, Put, Delete), status codes, URIs, content types/mapping – Can be used to carry different type of data. It uses content type to identify type of data being carried – HTTP Proxy can be used to map COAP to HTTP • Reliable: The HTTP proxy takes care of reliability
  41. 41. © 2016 MapR Technologies 41 Arduino – Read Temp. + Publish MQTT Message http://slideplayer.com/slide/9365664/
  42. 42. © 2016 MapR Technologies 42 COAP Device1 Device 2 Device 3 HTTP Proxy Subscriber 1 Subscriber 2 /store/fremont/aisel1 /store/fremont/aisel2 /store/sanjose/aisel1 coap://192.168.1.2:5683:/store/sanjose/aisel1 http://192.168.1.2:5683:/store/fremont/store1
  43. 43. © 2016 MapR Technologies 43 Arduino – Read Temp. + Publish COAP Message
  44. 44. © 2016 MapR Technologies 44© 2015 MapR Technologies Messaging
  45. 45. © 2016 MapR Technologies 45 Why do we need Messaging Layer ? The Messaging layer is responsible for 1. Highly available MQTT 1. Scale MQTT layer horizontally 2. Provide high availability 2. Decouple IOT layer from the computation layer 3. Allow multiple consumers to process the IOT data to meet different business requirements
  46. 46. © 2016 MapR Technologies 46 MQTT load balancing – No Kafka Device Device Device Device MQTT MQTT MQTT Spark Spark Spark
  47. 47. © 2016 MapR Technologies 47 MQTT load balancing - Kafka Device Device Device Device MQTT MQTT MQTT Spark Spark Spark Kafka
  48. 48. © 2016 MapR Technologies 48 MQTT Kafka Bridge MQTT Kafka Bridge 1 Bridge 2
  49. 49. © 2016 MapR Technologies 49© 2015 MapR Technologies Computation
  50. 50. © 2016 MapR Technologies 50 Why do we need computation layer ? The computation layer is responsible for 1. Process huge amount of data in linear scalable manner 1. Read streaming data 2. Transform/Enrich/Analyze data 3. Store the incoming data into persistence storage 2. Analyze IOT data 1. Business Rules 2. Predictive model
  51. 51. © 2016 MapR Technologies 51 Spark Streaming
  52. 52. © 2016 MapR Technologies 52 Business rules
  53. 53. © 2016 MapR Technologies 53© 2015 MapR Technologies Storage
  54. 54. © 2016 MapR Technologies 54 Why do we need Storage Layer ? The storage layer is responsible for 1. Providing persistent store for IOT data 2. Store data in time series format 1. Efficient storage of data 2. Ingest data at high speed 3. Retrieve/Query data
  55. 55. © 2016 MapR Technologies 55 Time Series Database • Data Point (time, temperature) • Metrics sensor.temp.1s • Tags sensorId=2, location=bed1 • Metrics + Tags = Time Series
  56. 56. © 2016 MapR Technologies 56 InfluxDB
  57. 57. © 2016 MapR Technologies 57© 2015 MapR Technologies Dashboard
  58. 58. © 2016 MapR Technologies 58 Why do we need Dashboard Layer ? The dashboard layer is responsible for 1. Interact with TSDB 2. Visualize TSDB data in easy to understand format
  59. 59. © 2016 MapR Technologies 59 Configure InfluxDB connection
  60. 60. © 2016 MapR Technologies 60 Temperature Variation Graph
  61. 61. © 2016 MapR Technologies 61 Temperature Variation Graph
  62. 62. © 2016 MapR Technologies 62 Resources • Streaming Processing for Real-Time Analytics and Dashboard • MapR Streams • MapR DB • Eclipse IOT • COAP Tutorial • GitHub For source code
  63. 63. © 2016 MapR Technologies 63© 2015 MapR Technologies Questions ? Sunil Patil, MapR Technologies sunilpatil@mapr.com
  64. 64. © 2016 MapR Technologies 64 Arduino - Logic for reading temperature

×