Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Internet of Things: Where Fast Data Meets Big Data: Couchbase Connect 2015

868 views

Published on

It’s one thing to collect a large amount user-generated data and process it. It’s another to ingest and store massive amounts of machine-generated data in real time. This session will focus on how machines and things are changing database requirements from the throughput and latency they require, to the sheer number of them, to their evolving data model, and the impact of location on network availability.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Internet of Things: Where Fast Data Meets Big Data: Couchbase Connect 2015

  1. 1. INTERNET OF THINGS: WHERE FAST DATA MEETS BIG DATA Shane Johnson, Couchbase
  2. 2. ©2015 Couchbase Inc. 2 Internet ofThings Agenda  The Introduction  The Numbers  The Challenge  More Challenges  The Convergence  The Solution
  3. 3. The Introduction
  4. 4. ©2015 Couchbase Inc. 4 Internet ofThings Landscape IoT Mobile M2M Wearable
  5. 5. ©2015 Couchbase Inc. 5 Internet ofThings Industry / Sector  Automotive  Consumer Electronics  Energy  Fitness  Fleet Management  Healthcare  HomeAutomation  Industrial  Manufacturing  Smart Appliances  Supply Chain Management  Transportation  Utilities
  6. 6. ©2015 Couchbase Inc. 6 Internet of Things The “Things”  Small or Big  Medical Implant  Railroad  Consumer or Industrial  Television  WindTurbine  Fixed or Mobile  Refrigerator  Car  Online or Offline  Thermostat  Offshore Oil Rig
  7. 7. The Numbers
  8. 8. ©2015 Couchbase Inc. 8 Internet ofThings Connected “Things” 2015 15 Billion 2020 50 Billion
  9. 9. ©2015 Couchbase Inc. 9 Internet ofThings Data Generated 2013 .35ZB (72 MillionTB) 2020 4.4ZB (320 MillionTB)
  10. 10. The Challenges
  11. 11. ©2015 Couchbase Inc. 11 Internet ofThings Internet ofThings == Big Data
  12. 12. ©2015 Couchbase Inc. 12 Internet ofThings Velocity – Faster Data  ConcurrentConnections  Hundreds toThousands  Thousands to Millions  Streaming Data  Hundreds per Second  Example: Sensor  Thousands per Second  Example:WindTurbine  Millions per Second  Example:Wind Farm
  13. 13. ©2015 Couchbase Inc. 13 Internet ofThings Volume – Bigger Data Gigabytes Terabytes Petabytes
  14. 14. ©2015 Couchbase Inc. 14 Internet ofThings Variety – Different Data  Heterogeneous  Refrigerator  Thermostat  Door Lock  Motion Sensor  Baby Monitor  Evolving  New Hardware  New Firmware  New Software
  15. 15. More Challenges
  16. 16. ©2015 Couchbase Inc. 16 Internet ofThings Global Economy = Global Data  Multiple Data Centers  Single Regions  Multiple Regions  Cloud Infrastructure  Multiple Regions  MultipleAvailability Zones  Hybrid Cloud  One (DC) to Many (Cloud)  ReplicateOut  Many (Cloud) to One (DC)  Replicate In
  17. 17. ©2015 Couchbase Inc. 17 Internet ofThings Never Ending Data  Storage  Not Infinite  Tiered  Current  Historical  Network Availability  Not Guaranteed  Unresponsive  Interrupted  Down
  18. 18. ©2015 Couchbase Inc. 18 Internet ofThings Bidirectional Data  Machines Generate  Actions  Readings  Events  Machines Consume  Commands  Configuration  Events
  19. 19. The Convergence
  20. 20. ©2015 Couchbase Inc. 20 Internet ofThings The Pinnacle of Convergence?
  21. 21. The Solution
  22. 22. ©2015 Couchbase Inc. 22 The Internet ofThings Architecture Kafka (Messaging) Couchbase Server Spark Core / SQL (Analysis)
  23. 23. ©2015 Couchbase Inc. 23 The Internet ofThings Architecture Couchbase Server Kakfa (Messaging) Spark Streaming (Analysis)
  24. 24. ©2015 Couchbase Inc. 24 The Internet ofThings Architecture Kafka (Messaging) Spark Streaming (Analysis) Couchbase Server Hadoop (Data Lake)
  25. 25. ©2015 Couchbase Inc. 25 The Internet ofThings Couchbase Mobile Couchbase Sync Gateway Couchbase Server Architecture Hadoop / Spark Connected Device
  26. 26. ©2015 Couchbase Inc. 26 The Internet ofThings MobileDevice MobileDevice Sync Gateway MobileDevice MobileDevice Sync Gateway Couchbase Server Couchbase Server Remote Site Remote Site Cloud Region Cloud Region Couchbase Server Local DC
  27. 27. ©2015 Couchbase Inc. 27 The Internet ofThings MobileDevice MobileDevice Sync Gateway MobileDevice MobileDevice Sync Gateway Couchbase Server Couchbase Server Cloud Region Cloud Region Couchbase Server Local DC
  28. 28. ©2015 Couchbase Inc. 28 The Internet ofThings Summary  Challenges  Concurrent Connections  Streaming Data  GBs toTBs to PBs of Data  Different Data (Evolving)  Global Data  Never Ending Data  Directional Data  Solutions  Couchbase Server and  Messaging (Kafka)  Stream Processing (Storm)  Hadoop  Couchbase Mobile  Embedded Database  Automatic Synchronization  Online / Offline
  29. 29. Thank you.

×