George Gilbert's Big Data Presentation at theCUBE Party at Big Data NYC 2015.

1. Systems of Intelligence:
The Biggest Change in Enterprise Applications in 50 Years
George Gilbert
@GGilbert41
Big Data Analyst

2. StoreE-Mail
Social
Media IM
Where Customers Should be Investing
Analogous to Pipelines in Systems of Record
Operational apps
Customer interactions
Agility: Speed of improving reports
weeks, months, quarters
Latency: Speed of reporting
Days, weeks, months
ETL Development
Operational
Data
Customer
“Breadcrumbs”
Production ETL

3. StoreE-Mail
Social
Media IM
Where Customers Should be Investing
Systems of Intelligence Pipelines are all About Speed and Agility
Retail
Consumer
Mobile
Call Center
eCommerce
Operational apps
Customer interactions
Agility: Improving predictions
seconds to days
Latency: Speed of predictions
ms to seconds
Machine learning pipeline
Operational
Data
Customer
“Breadcrumbs”
Predictions,
Recommendations
Improving
Predictions
(Machine
Learning)
Intelligence

4. Critical new data and analytic skills
are required
• Accuracy of predictions (Revenue)
Modernizing SoR can accelerate the
journey
• Speed of predictions (latency)
• Speed of improving predictions (agility)
Choice of new platform
• TCO/operational complexity
• Development complexity
• Existing infrastructure – technology and
skills
What Trade-Offs Should Customers Consider
When Deploying Systems of Intelligence
Incremental Revenue

5. Planning Your Customer Journey: Skills and Platform Progress
Smart Grid
Fraud prevention
Real-time loyalty
omni-channel
multi-touchpoint
Predictive model learns from and anticipates consumer
in near real-time
Continuously updated prediction
of energy supply, demand tunes
end-point consumptionIntelligent
systems management
System learns “normal” behavior of apps
and infrastructure and flags or fixes anomalies
Identify spending behavior out of the norm
ApplicationsApplications
TechnologyMaturity,EnterpriseCapabilitesTechnologyMaturity,EnterpriseCapabilites
Time

6. Big Data Platform Evolution – Simplifying Operations and Development:
Storage Consolidation, API’s > OSS Products, Spark Hollowing out Hadoop
Machine Learning
SQL: Join, filter, aggregate
Streaming
HDFS-Compatible File System
Resource + Workload Manager
HBase-Compatible DB
Polyglot Data API: SQL, KV, JSON
YARN Resource Manager
HDFS, HBase
Streaming
Spark, Flume,
Flink, Samza
Dataflow
Kafka-Like Messaging
SQL
Impala, Drill,
Hive, HAWQ…
Machine
Learning
Mahout,
Spark MLlib
Hadoop 2.0 Big Data 3.0
In-Memory Storage (DB or File System)
Dev Tools Dev Tools Dev Tools
Dev Tool
IntegratedDevops
Management,Governance

7. Operational Simplicity vs. Complexity:
Single Pane of Glass vs. Familiar Legacy Sprawl

8. Development Simplicity vs. Complexity:
Spreadsheet vs. Many Development Tools

9. Many data managers – optimal functionality
(Cassandra, Aerospike, MongoDB, Neo4j…)
Single vendor data platform
(Azure, AWS, Google Cloud Platform,
Bluemix, Pivotal)
Single multi-purpose engine
(Oracle, Spark)
Customers Must Balance Operational Simplicity and Development Simplicity
Relative to Existing Skills and InfrastructureOperationalSimplicity
Development Simplicity
Hadoop ecosystem
(Hortonworks, Cloudera, MapR)
SaaS: AirBnB,
Uber, Commerce
Fortune 500
Mainstream

10. Pro: Optimal functionality
Con: Complexity
Customer sweet spot
• Leading-edge Internet-
centric companies
• Netflix, Uber, ad-tech,
gaming, ecommerce
Many Data Managers – Optimal Functionality:
Highest Development and Operational Complexity

11. Many optimized data managers
(Cassandra, Aerospike, MongoDB, Neo4j…)
Single vendor data platform
(Azure, AWS, Google Cloud Platform,
Bluemix, Pivotal)
Single multi-purpose engine
(Oracle, Spark)
OperationalSimplicity
Development Simplicity
Hadoop ecosystem
(Hortonworks, Cloudera, MapR)
Hadoop 2.0
Big Data 3.0
Customers Must Balance Operational Simplicity and Development Simplicity
Relative to Existing Skills and Infrastructure

12. Critical new data and analytic skills
are required
• Accuracy of predictions
Modernizing SoR can accelerate the
journey
• Speed of predictions (latency)
• Speed of improving predictions (agility)
Choice of new platform
• TCO/operational complexity
• Development complexity
• Existing infrastructure – technology and
skills
Systems of Intelligence P&L Statement:
Designing with “Budgetary” Constraints
Technology
maturity:
• lowers cost
• opens new
application
possibilities
Incremental Revenue