O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A
#DenodoDataFest
RAPID, AGILE DATA STRATEGIES
For Accelerating Analytics, Cloud, and Big Data Initiatives.
The Role of Data Virtualization in IoT
Integration
Lakshmi Randall
Head of Product Marketing, Denodo
Twitter: @LakshmiLJ
Agenda
1.What’s so important about IoT Integration?
2.How does Denodo support IoT Data Integration?
3.Customer Case Study
3
IoT is invading our kitchen!
4
What’s so important about IoT
Integration?
5
The Importance of IoT Integration
 Investment in IoT devices is soaring
 IoT is proliferating across all business and
consumer sectors
 Data generated in the IoT offers a Data
Monetization Model
6
IOT Investment
 2016: IoT hardware purchases surpass
$2.5 million per minute.
 2021: one million IoT devices procured
and installed per hour.
IoT Proliferation
 2018: 6 billion connected devices
require support.
 2020: >21 billion connected devices
in operation.
 2020: industry-specific devices number
2.9 billion (nearly 200% growth since
2015).
IoT Investment & Proliferation Milestones
7
Source: Gartner 2016
IoT Monetization
 Enhance traditional products with sensors
and connectivity
 Offer bundled services for connected things
(e.g., connected cars)
 Collect, Aggregate, Anonymize and
Monetize.
8
IoT Use Cases
9
Preventative &
Proactive
Maintenance
Data Monetization
(information-
oriented products
& services)
Customer
Satisfaction &
Retention
Operational
Efficiency (asset &
equipment
optimization)
Safety Security
Fraud Detection
Real-time
Analytics
Patient care
IoT Endpoints for 2020 by Sector
10
Source: Gartner, April 2016
How does Denodo support IoT
Data Integration?
11
Data-in-transit and Data-at-rest
Big Data Connectivity
BigData and Cloud Databases Connectivity
■ Hadoop Ecosystem:
■ SQL on Hadoop: Hive, Impala, Presto,…
■ HDFS, Parquet, Avro, CSV…
■ Execution of map/reduce Jobs
■ Certified with major Hadoop distributions
■ In-memory platforms: Apache Spark, Presto DB, HANA,…
■ Parallel DWs and Appliances: Vertica, Impala, Teradata, Greenplum,…
■ Cloud RDBMS: Redshift, Snowflake, DynamoDB,…
■ NoSQL (MongoDB, CouchDB, Neo4J, Redis, Oracle NoSQL, Cassandra, etc.)
■ Streaming data (Spark streams, Splunk, IBM Streams, Kafka,…)
12
Enhanced Adapters for Big Data ecosystem
13
Request-Response:
Named adapters for stream
services:
 Kafka
 IBM Streams
Streaming:
Extend current JMS support
with:
 Enhanced support for
temporary windows
 Support for MQTT
Enhanced Integration with IoT - Streaming
Enhanced Adapters for the Internet of Things Ecosystem
JMS
MQTT
JMS
MQTT
Data Ingestion
■ Batch, On-demand and Streaming Data
Ingestion
■ Simultaneously supports Batch and
Streaming data integration
■ Learns to extract structured data from
semi-structured content using Machine
Learning
■ Ingest the data in a schema-agnostic way
including schema-on-read and multiple
schemas
14
Batch, On-demand and Streaming Data Ingestions
Enrich Machine Data and Combine with Other Data
Ingest, Integrate & Deliver
Persisted
(In-memory, Hadoop)
Streams
(specific time window)
Message Queue
Machine-generated/Event data Alerts
Workflows
Operational
Processes
Analytical
Processes
Consumers
Visualization
Data Virtualization
Enrich and Combine IoT
Data with Other Data
Historians
Streams
ERP/SCM
DW
Analytical
DB
MDM
Apps
Data
Marts
Hadoop NoSQL
16
Security
Data in Motion – secure channels
• Using SSL/TLS
• Client-to-Denodo and Denodo-to-source
• Available for all protocols (JDBC, ODBC, ADO.NET and WS)
Data at Rest – secure storage
• Cache: third party database. Can leverage its own encryption mechanism
• Swapping to disk: serialized temporarily stored in a configurable folder that can be encrypted by the OS
Encryption/Decryption and Data Masking
• Support for custom decryption for files and web services
• Transparent integration with RDBMs encryption
Authentication and Authorization
• LDAP/AD, Kerberos support, Granular data security,
Securing data
Customer Case Study
17
Leading Construction Manufacturer - Telematics &
Predictive Maintenance
Dealer
Maintenance
Parts Inventory
OSI PI Hadoop Cluster
Tableau: Dealer / Customer Dashboard
Business Benefits
 Improved asset performance and proactive maintenance.
 Reduced warranty costs due to proactive maintenance of
parts preventing parts failure.
 Optimized pricing for services and parts among global service
providers.
 New Business Model opportunities based on real-time
analysis of detailed sensor data.
Data Virtualization Benefits
Implement a Single Logical Data Lake Using Data Virtualization
Improves the
enterprise func-
tionality of data
lakes by
combining one or
more physical
data lakes with
other enterprise
data
Provides a way to
access data from
separate systems
through an
abstraction layer
that makes it
appear as if the
data were in a
single data lake
Improves an
organization’s
ability to govern
and extract more
value from its
data lakes by
extending them
as logical data
lakes
20
Key Takeaways
 Identify if and how IoT data will benefit your organization
 Identify your potential IoT Data sources
 Employ Data Virtualization to combine IoT data with other data to
enhance the use and value of data assets
 Employ a Logical Data Lake/Logical Data Warehouse architecture to
eliminate the cost of storing information in multiple places, to
govern IoT data access, and to prevent IoT data from becoming
siloed.
21
Q&A
Thank you!
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and
microfilm, without prior the written authorization from Denodo Technologies.
O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A
#DenodoDataFest

Denodo DataFest 2016: The Role of Data Virtualization in IoT Integration

  • 1.
    O C TO B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A #DenodoDataFest RAPID, AGILE DATA STRATEGIES For Accelerating Analytics, Cloud, and Big Data Initiatives.
  • 2.
    The Role ofData Virtualization in IoT Integration Lakshmi Randall Head of Product Marketing, Denodo Twitter: @LakshmiLJ
  • 3.
    Agenda 1.What’s so importantabout IoT Integration? 2.How does Denodo support IoT Data Integration? 3.Customer Case Study 3
  • 4.
    IoT is invadingour kitchen! 4
  • 5.
    What’s so importantabout IoT Integration? 5
  • 6.
    The Importance ofIoT Integration  Investment in IoT devices is soaring  IoT is proliferating across all business and consumer sectors  Data generated in the IoT offers a Data Monetization Model 6
  • 7.
    IOT Investment  2016:IoT hardware purchases surpass $2.5 million per minute.  2021: one million IoT devices procured and installed per hour. IoT Proliferation  2018: 6 billion connected devices require support.  2020: >21 billion connected devices in operation.  2020: industry-specific devices number 2.9 billion (nearly 200% growth since 2015). IoT Investment & Proliferation Milestones 7 Source: Gartner 2016
  • 8.
    IoT Monetization  Enhancetraditional products with sensors and connectivity  Offer bundled services for connected things (e.g., connected cars)  Collect, Aggregate, Anonymize and Monetize. 8
  • 9.
    IoT Use Cases 9 Preventative& Proactive Maintenance Data Monetization (information- oriented products & services) Customer Satisfaction & Retention Operational Efficiency (asset & equipment optimization) Safety Security Fraud Detection Real-time Analytics Patient care
  • 10.
    IoT Endpoints for2020 by Sector 10 Source: Gartner, April 2016
  • 11.
    How does Denodosupport IoT Data Integration? 11 Data-in-transit and Data-at-rest
  • 12.
    Big Data Connectivity BigDataand Cloud Databases Connectivity ■ Hadoop Ecosystem: ■ SQL on Hadoop: Hive, Impala, Presto,… ■ HDFS, Parquet, Avro, CSV… ■ Execution of map/reduce Jobs ■ Certified with major Hadoop distributions ■ In-memory platforms: Apache Spark, Presto DB, HANA,… ■ Parallel DWs and Appliances: Vertica, Impala, Teradata, Greenplum,… ■ Cloud RDBMS: Redshift, Snowflake, DynamoDB,… ■ NoSQL (MongoDB, CouchDB, Neo4J, Redis, Oracle NoSQL, Cassandra, etc.) ■ Streaming data (Spark streams, Splunk, IBM Streams, Kafka,…) 12 Enhanced Adapters for Big Data ecosystem
  • 13.
    13 Request-Response: Named adapters forstream services:  Kafka  IBM Streams Streaming: Extend current JMS support with:  Enhanced support for temporary windows  Support for MQTT Enhanced Integration with IoT - Streaming Enhanced Adapters for the Internet of Things Ecosystem JMS MQTT JMS MQTT
  • 14.
    Data Ingestion ■ Batch,On-demand and Streaming Data Ingestion ■ Simultaneously supports Batch and Streaming data integration ■ Learns to extract structured data from semi-structured content using Machine Learning ■ Ingest the data in a schema-agnostic way including schema-on-read and multiple schemas 14 Batch, On-demand and Streaming Data Ingestions
  • 15.
    Enrich Machine Dataand Combine with Other Data Ingest, Integrate & Deliver Persisted (In-memory, Hadoop) Streams (specific time window) Message Queue Machine-generated/Event data Alerts Workflows Operational Processes Analytical Processes Consumers Visualization Data Virtualization Enrich and Combine IoT Data with Other Data Historians Streams ERP/SCM DW Analytical DB MDM Apps Data Marts Hadoop NoSQL
  • 16.
    16 Security Data in Motion– secure channels • Using SSL/TLS • Client-to-Denodo and Denodo-to-source • Available for all protocols (JDBC, ODBC, ADO.NET and WS) Data at Rest – secure storage • Cache: third party database. Can leverage its own encryption mechanism • Swapping to disk: serialized temporarily stored in a configurable folder that can be encrypted by the OS Encryption/Decryption and Data Masking • Support for custom decryption for files and web services • Transparent integration with RDBMs encryption Authentication and Authorization • LDAP/AD, Kerberos support, Granular data security, Securing data
  • 17.
  • 18.
    Leading Construction Manufacturer- Telematics & Predictive Maintenance Dealer Maintenance Parts Inventory OSI PI Hadoop Cluster Tableau: Dealer / Customer Dashboard
  • 19.
    Business Benefits  Improvedasset performance and proactive maintenance.  Reduced warranty costs due to proactive maintenance of parts preventing parts failure.  Optimized pricing for services and parts among global service providers.  New Business Model opportunities based on real-time analysis of detailed sensor data.
  • 20.
    Data Virtualization Benefits Implementa Single Logical Data Lake Using Data Virtualization Improves the enterprise func- tionality of data lakes by combining one or more physical data lakes with other enterprise data Provides a way to access data from separate systems through an abstraction layer that makes it appear as if the data were in a single data lake Improves an organization’s ability to govern and extract more value from its data lakes by extending them as logical data lakes 20
  • 21.
    Key Takeaways  Identifyif and how IoT data will benefit your organization  Identify your potential IoT Data sources  Employ Data Virtualization to combine IoT data with other data to enhance the use and value of data assets  Employ a Logical Data Lake/Logical Data Warehouse architecture to eliminate the cost of storing information in multiple places, to govern IoT data access, and to prevent IoT data from becoming siloed. 21
  • 22.
  • 23.
    Thank you! © CopyrightDenodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies. O C T O B E R 1 8 , 2 0 1 6 S A N F R A N C I S C O B A Y A R E A , C A #DenodoDataFest