Data is growing at an exponential pace. Yet our ability to derive any meaningful value increases only slightly. Simply put, it's not about the volume. This deck overviews the data opportunities and specific steps organizations can take to amplify data value to data consumer.

1. Machine Data Explosion
How to amplify the value of IoT data
Mark Chmarny
@mchmarny

2. DATA DYNAMICS
Data to grow 4.4ZB in 2015
from to 44ZB in 2020
10x growth, 90% of that
machine-generated, new device
…but, amount of data from which we can derive
value to increase only from 22% to 35% Sources: IDC

3. Sources: Gartner | Cisco | Intel | IDC
Gartner
Cisco
McKinsey
IDC
26B
50B
200B
212B
DEVICES
Manufacturing Health Care Retail Security Transportation
40% 30% 8% 7%
4%
VERTICALS Source: Intel
25+Million
New Apps
$4+Trillion
Business
Source: IDC
OPPORTUNITY
DATA GROWTH

4. • Value increases when data is related (more context, better)
• Single raw data source candrive many value chains (diff process, diff service)
• Some data has no value until integratedor even when delivered as a service (noise vs signal)
Raw
Data
Processed
Data
Integrated
Data
Data
Services
Symbol
What
Where
When
How
Why
Sources: SVDS
Lack of trust
Fear of sharing with partners, common
perception of incompetencyto protect
“their” data
Knee-jerk reaction
67% would rather lose the opportunityto
monetize than to risk losing control
Gray market
Yet, over 60% of service-delivery
companies already monetize collected
data without original providers concent
Source: Accenture
INHIBITORS VALUE CHAIN
Sources: Nate Silver’s book
DATA VALUE

5. Sales
Fact
Customer
Dimension
Supplier
Dimension
Store
Dimension
Geography
Dimension
Product
Dimension
TRADITIONAL
EDW DATA
NEW DEVICE DATA
EVENT FACT 14 53939807 2657 ABC 0.034 X: Y:Z…
When Where What Value
EVENT FACT
Dimension is the context so this is efficient:
get sales where product = ‘x’ and supplier = ‘y'
§ Data (most) born in an absence of context (narcissistic device?)
§ Observations, by default, are immutable (don’t change after reading)
§ Individual events insignificant, more interesting the longer observed (series)
Observation
Actuation
Persistence
Latency
Attributes
Ingestion bandwidth
important but “total latency”
most critical
NEW CONSUMPTION MODEL
VERTICALS
DATA TYPES

6. Raw
(as is)
Prepared
(cleansed, standardized)
Processed
(augmented, related)
Consumed
(acquired, served)
Health
(genomic)
Text-based files in columnar structure
Standardized formats (VCF, WDL, CDL…)
Small data variation across sources (deltas consumed)
Finance
(market)
• Primarily transactional
• RDBMS managed
• Diverse data structures
(schema, codes, relation)
• Requires transformation, standardization
• Comes with a lot of context (relationships)
• May benefit from out-of-domain links
• Batch (file) or service (API)
• Parameterized queries
(question/answer)
Industrial
(machine)
• Machine generated, minimal context
• Already highly standardized data per device type
• Immutable (doesn’t change after reading)
• Individual events
insignificant, long series
need management
• Need relation
• Derived value service
(trends, anomalies…)
• Best consumed as stream vs
batch
Data exchange format standardization opportunity
DATA METRIC (VERTICAL)

7. DISTRIBUTED
§ Federated queries return only summary/deltas
§ Best on common format data-sets
§ Deliver always latest data, no duplication
§ Demands support from individual partners
§ Better for async/batch requests due to latency
CENTRILIZED
§ Aggregates all data prior to query (duplication)
§ Queries over combined/indexed data
§ Perception of data out of provider’s control
§ Enables query by context not available at source
§ Supports real-time queries
Partner
Partners
MODEL CONSIDERATIONS
§ Data “schema” or format commonality (standard)
§ Consumer usage demands (async query)
§ Network bandwidth/latency, consistency tolerance
§ Context locality demand
§ Skillset, willingness to absorb opex (all providers)
§ Geofencing requirements (compliance)
NOT mutually exclusive -
ability to facilitate both is
an advantage.
store provider
store provider
store provider
exchange
consume
r
consume
r
consume
r
=
=
=
=
=
=
DATA ACCESS

8. Minimize data sharing OPEX through
automation. Make it convenient. No
data will be shared if the cost of its
exchange is higher thanmarket value
§ Reusable connectors (Drivers)
§ Gateway API for Scheduling,
Validation, Alerts, Audit
Create information abstraction layer
to deliver data in readily to consume
formats optimized for specific use-
case to assure maximumstickability
§ API management, bindings
§ Federated & granular ACLs
§ Deep metering & telemetry
Build new data views by connecting
related sets to expose otherwise not
obvious insights. Invest in becoming
birthplace of organic data
§ Mine for link & associations
§ Deliver data curation service
§ Augment on-read context
Create insightbazar, services beyond
data, enable bi-directional exchange,
enable sampling for value prior to
use or purchase
§ Model & service gamification
§ On-demand data scientists
§ Hackathon & competitions
LOWER
OPEX
ADD
CONTEXT
DIVERSE
APIS
CREATE
BAZAR
DATA EXCANGE

9. POSTPROCESSING
Data Provider Data Exchange Data Consumer
Batch
Stream
Data Set
1-1
Data Set Query
1-n sets
Answer Service
1-n sets
Data Events
1-n provider
n:n
1:n
n:n
n:n
USAGE PATTERNS
File
Query
Stream
$/File Download
$/Query or
$/Query Plan (Time)
$/Event or
$/Subscription (Time)
Job Exec
Distributed Exec
1-n provider
n:1
Job
$/Job (* Target) or
$/Job Exec Plan

10. Batch
Upload
Data
Stream
Federation
(Registration, ACL)
Gateway
(Management, Audit & Push/Pull)
Operations
(Management, Metering)
Manage
Monitor
Ingestion
(Push/Pull)
Processing
(Link Mining, Catalog & Service Creation)
Preparation
(Tokenization, Standardization, Validation)
Service
(Batch, Exec, Query, Management)
Data Provider
Service
(API)
Data Consumer
Batch
Download
Data
Stream
Data
Query
• Delivers data encryption at
rest and in motion
• Accommodate both
distributed and centralized
models
• Enable specialized access
& format requirements
• Lower provider OPEX with
turn-key gateway or SDK
• Deliver granular utilization
& access metrics to expand
monetization surface
Billing
(Invoice, Collect)
Partner Services
Persistent
(Distribution, Durability, Optimization)
DATA PLATFORM

11. Machine Data Explosion
How to amplify the value of IoT data
Mark Chmarny
@mchmarny