Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Accessing IoT Data with Couchbase Server, Couchbase Mobile and Kaa: Couchbase Connect 2015

3,856 views

Published on

We will discuss how IoT data can be easily accessed by user applications in real-time through the Couchbase Server and Mobile synchronization, while feeding sensory data through Kaa open source IoT platform.

Published in: Technology

Accessing IoT Data with Couchbase Server, Couchbase Mobile and Kaa: Couchbase Connect 2015

  1. 1. Accessing IoT data with Couchbase Server, Couchbase mobile, and Kaa Andrew Kokhanovskyi CTO, CyberVision, Inc. @akokhanovskyi kaaproject.org© 2015 CyberVision, Inc. All rights reserved.
  2. 2. kaaproject.org© 2015 CyberVision, Inc. All rights reserved.
  3. 3. kaaproject.org© 2015 CyberVision, Inc. All rights reserved. Endpoints Cloud Web dashboardsMobile applications Monitoring systems Data Aggregated Data Conceptual diagram
  4. 4. kaaproject.org© 2015 CyberVision, Inc. All rights reserved. Server / Lite
  5. 5. kaaproject.org© 2015 CyberVision, Inc. All rights reserved. ➢ Fully-featured IoT middleware platform ➢ 10 Kb RAM footprint (with C SDK) ➢ Guaranteed data delivery and reliable local storage ➢ Built-in transport security ➢ Efficient data serialization ➢ Horizontally scalable and fault tolerant ➢ 100% open-source (Apache license 2.0) ➢ Rapid application development using C / C++ / Java SDKs ➢ Integration with popular device platforms Why Kaa?
  6. 6. kaaproject.org© 2015 CyberVision, Inc. All rights reserved. ➢ Flexible document-oriented data model ➢ Elastic scalability ➢ Consistent high performance ➢ Real-time big data analytics ➢ Always-on availability ➢ Open-source (Apache license 2.0) ➢ Real-time Server synchronization with Couchbase Lite in user apps Why Couchbase?
  7. 7. Architecture kaaproject.org© 2015 CyberVision, Inc. All rights reserved. Endpoints Web dashboardsMobile applications Monitoring systems Telemetry data Lots of telemetry data Aggregated data Server Lite
  8. 8. Kaa cluster Data flow: Kaa perspective © 2015 CyberVision, Inc. All rights reserved. Your application Kaa SDK Sensors Raw Data Your device Operations server node Couchbase appender Kaa services Structured Data Structured Data kaaproject.org
  9. 9. Mobile application Couchbase Lite Application logic Data flow: Couchbase perspective © 2015 CyberVision, Inc. All rights reserved. Couchbase cluster Documents (huge volume) View Documents (summary) Structured Data Sync Gateway Couchbase Lite Application logic Mobile application kaaproject.org
  10. 10. The secret sauce: structured data e2e kaaproject.org© 2015 CyberVision, Inc. All rights reserved.
  11. 11. kaaproject.org© 2015 CyberVision, Inc. All rights reserved. Power plant
  12. 12. Power plant: telemetry data collection © 2015 CyberVision, Inc. All rights reserved. kaa::KaaUserLogRecord voltageReport; voltageReport.timestamp = getCurrentTime(); samples.reserve(configuration.panelCount); for (std::int32_t i = 0; i < configuration.panelCount; ++i) { kaa_log::VoltageSample sample; sample.zoneId = solarPanels_[i].getZoneId(); sample.panelId = solarPanels_[i].getPanelId(); // Analog-to-digital converter factor sample.voltage = solarPanels_[i].getVoltage(); samples.push_back(sample); } voltageReport.samples = std::move(samples); // upload log record kaaClient_.addLogRecord(report); Data record schema C++ client code snippet kaaproject.org { "type": "record", "name": "VoltageReport", "namespace": "org.kaaproject.kaa.examples.powerplant", "fields": [ {"name": "timestamp", "type": "long"}, {"name": "samples", "type": { "type": "array", "items": { "type": "record", "name": "VoltageSample", "fields": [ {"name": "zoneId", "type": "int"}, {"name": "panelId", "type": "int"}, {"name": "voltage", "type": "double"} ] }}}] }
  13. 13. © 2015 CyberVision, Inc. All rights reserved. kaaproject.org Power plant: same client code in Java and pure C Java snippet C snippet Long timestamp = System.currentTimeMillis(); List<VoltageSample> samples = new ArrayList<>(); for (int i = 0; i < KAA_DEMO_MAX_ZONE_ID; i++) { Integer zoneId = i; Integer panelId = getPanelId(i); Double voltage = getVoltage(panelId); VoltageSample sample = new VoltageSample(zoneId, panelId, voltage); samples.add(sample); } VoltageReport report = new VoltageReport(timestamp, samples); kaaClient.addLogRecord(report); kaa_user_log_record_t *log_record = kaa_logging_voltage_report_create(); log_record->timestamp = time(NULL) * 1000; log_record->samples = kaa_list_create(); for (size_t i = 0; i < KAA_DEMO_MAX_ZONE_ID; i++) { kaa_logging_voltage_sample_t *sample = kaa_logging_voltage_sample_create(); sample->zone_id = i; sample->panel_id = getPanelId(i); sample->voltage = getVoltage(sample->panel_id); kaa_list_push_back(log_record->samples, sample); } kaa_logging_add_record(log_collector, log_record); log_record->destroy(log_record);
  14. 14. © 2015 CyberVision, Inc. All rights reserved. kaaproject.org Power plant: sample log document
  15. 15. © 2015 CyberVision, Inc. All rights reserved. kaaproject.org Power plant: Couchbase view
  16. 16. © 2015 CyberVision, Inc. All rights reserved. kaaproject.org Power plant: Couchbase Java daemon
  17. 17. © 2015 CyberVision, Inc. All rights reserved. kaaproject.org database = manager.getDatabase(localDBName); URL dbURL = new URL(dbURLStr + gatewayName); pull = database.createPullReplication(dbURL); List<String> channels = Arrays.asList("totals"); Authenticator auth = new BasicAuthenticator(username, password); pull.setAuthenticator(auth); pull.setChannels(channels); pull.setContinuous(true); pull.start(); sync gateway pull replication View reportView = database.getView(VIEW_NAME); reportView.setMap(new Mapper() { @Override public void map(Map<String, Object> document, Emitter emitter) { Long timestamp = (Long) document.get (TIMESTAMP_FIELD); if (timestamp != null) { emitter.emit(timestamp, document); } } }, "1"); local view Power plant: Couchbase Lite Android
  18. 18. Video goes here kaaproject.org© 2015 CyberVision, Inc. All rights reserved.
  19. 19. kaaproject.org© 2015 CyberVision, Inc. All rights reserved. ➢ 2 days implementation ➢ Reliable telemetry data delivery ➢ Average e2e latency < 1-2 sec ➢ Horizontal scalability ➢ Clear and simple integration ➢ Collected endpoint data can be easily queried and analyzed using Couchbase views ➢ Couchbase Lite pull replication: synchronizing data to apps with ~ zero effort Results
  20. 20. Andrew Kokhanovskyi, CTO, CyberVision ak@cybervisiontech.com kaaproject.org cybervisiontech.com THANK YOU FOR YOUR ATTENTION
  21. 21. © 2015 CyberVision, Inc. All rights reserved. Memory usage Operations at 80% CPU utilization 1 node 2 nodes 4 nodes 16 nodes Notifications / sec 45 k 86 k 173k 710 k Configuration updates / sec 30 k 58 k 122 k 465 k Events / sec 120 k 150 k 410 k 1 500 k Endpoint count 100 k 200 k 400 k 800 k RAM usage 2Gb 3Gb 5Gb 9.5Gb Key performance indicators kaaproject.org
  22. 22. © 2015 CyberVision, Inc. All rights reserved. Zookeeper quorum Endpoints Control servers standby Bootstrap servers Operations servers Fault-tolerance and horizontal scalability kaaproject.org active
  23. 23. © 2015 CyberVision, Inc. All rights reserved. kaaproject.org Power plant: sync gateway configuration { "interface":":4984", "adminInterface":":4985", "log":["CRUD", "REST+", "Access"], "databases":{ "sync_gateway":{ "server":"http://localhost:8091", "bucket":"sync_gateway", "shadow": { "server": "http://localhost:8091", "bucket": "powerplant" }, "sync":`function(doc) { if (doc.zones != undefined) {channel ("totals"); } }` } } }

×