This document discusses selecting the right big data platform for your needs. It covers key considerations like data processing concepts, analytic appliances, security, data movement and integration with existing systems. The document also provides an overview of the IBM BigInsights platform for Apache Hadoop, including its open source components, management capabilities and tools for data scientists and analysts. Integration with other systems and choosing the right vendor while avoiding oversizing are emphasized as important factors for a big data platform.

1. © 2015 IBM Corporation
Výběr vhodné Big Data
platformy pro vaše potřeby
Jan Sovka

2. Sizing
Integration
Purpose Vendor

3. Big Data Platform
Data processing concept
Analytic
Appliances
Security, Governance and Business Continuity
Information Movement, Matching & Transformation
Landing,
Exploration
& Archive Enterprise
Warehouse
Data Marts
Real-Time Analytics
Data Sources
Structured
Operational
Unstructured
External
Social
Sensor
Geospatial
Time Series
Streaming
Information & Insight
BI & Performance
Management
Predictive Analytics &
Modeling
Exploration &
Discovery

4. Few months ago…
Big Data platform
4

5. How the old approach looked like…
+
Enterprise
support

6. And now?
Big Data platform
6

7. Open Data Platform Initiative
All Standard Apache Open Source Components
HDFS
YARN
MapReduce
Ambari HBase
Spark
Flume
Hive Pig
Sqoop
HCatalog
Solr/Lucene
ODP
All Standard Apache Open Source Components
HDFS
YARN
MapReduce
Ambari HBase
Spark
Flume
Hive Pig
Sqoop
HCatalog
Solr/Lucene
ODP

8. Text Analytics
POSIX Distributed
Filesystem
Multi-workload, multi-tenant
scheduling
IBM BigInsights
Enterprise Management
Machine Learning on Big R
Big R (R support)
IBM Open Platform with Apache Hadoop
(HDFS, YARN, MapReduce, Ambari, Hbase, Hive, Oozie, Parquet, Parquet Format, Pig,
Snappy, Solr, Spark, Sqoop, Zookeeper, Open JDK, Knox, Slider)
IBM BigInsights
Data Scientist
IBM BigInsights
Analyst
Big SQL
BigSheets
Industry standard
SQL (Big SQL)
Spreadsheet-style
tool (BigSheets)
Overview of BigInsights
. . .
IBM BigInsights for Apache Hadoop

9. Integration
Big Data platform
9

10. Big Data Platform
Data processing concept
Analytic
Appliances
Security, Governance and Business Continuity
Information Movement, Matching & Transformation
Landing,
Exploration
& Archive Enterprise
Warehouse
Data Marts
Real-Time Analytics
Data Sources
Structured
Operational
Unstructured
External
Social
Sensor
Geospatial
Time Series
Streaming
Information & Insight
BI & Performance
Management
Predictive Analytics &
Modeling
Exploration &
Discovery

11. Connecting Big Data platform with the ecosystem
SQL layer
Oracle Teradata Big Data platform
Current applications

12. SQL on Big Data – does it run 100% of your queries?
Based on TPC-DS benchmark. Results for 10TB scale shown here

13. Clear purpose & heading
Big Data platform
13

14. Clear purpose & heading
 Select the proper tools
– Business & Spark, SQL, Pig
– It’s not just a storage
 Yes it’s flexible, but…
 Don’t be afraid to scale
You are not building Facebook
14

15. Conclusion
Big Data platform
15

16. Conclusion
 Integration: Do not build an island
 Vendor: Choose wisely, but don’t overthink
 Sizing: spend on people, not on machines
 Analysis: don’t expect a miracle
16

17. Jan Sovka
jan_sovka@cz.ibm.com