In-Place analytics with Unified Data Access

1© Copyright 2015 EMC Corporation. All rights reserved.
EMC Solutions are Powered by
Intel® Xeon® Processor Technology
ISILON ROADMAP 2015
STEFAN RADTKE
CTO, EMEA
EMERGING TECHNOLOGY DIVISION
IN-PLACE ANALYTICS WITH
UNIFIED DATA ACCESS
DR. STEFAN RADTKE

ANALYTICS TODAY IS ABOUT BIG DATA
Big Data needs Big storage
41%
38%
35%
32%
28%
0% 10% 20% 30% 40% 50%
Application…
Big Data/Business…
Security/Risk…
Business Process…
Cloud Computing

BIG DATA IS SPREAD OUT
Unstructured Data
Dark Data
Structured Data
Traditional
Social Networks, UGCPublic records
Location
Data
Internet Of Things
Emerging
Social Networks,
UGCPublic
records
Location
Data
Internet Of
Things
Emerging
Unstructured Data
Dark
Data
Structured Data
Traditional
Big Data is Growing FastBig Data is Siloed
Silos, Complexity, Data explosion, security, etc..
Hadoop Can add to the Data Problems
Store, manage, protect and analyzetraditional and emergingdata
Introducing the Data Lake Foundations
Data Lake
Infrastructurechoices: virtual,physical, converged
Data Lake Foundations
Virtual
Physical
Converged
Storage
Compute
APPLICATIONS

EMC DATA LAKE FOUNDATION
EMC Isilon
Type ECS-Appliance: ScaleOut Object Isilon: ScaleOut File + Object
Protocols Swift, S3, MS Azure, Block, HDFS Swift, NFS, SMB, HDFS
Use Case Scale out Block and Object access;
Data in place Analytics with Hadoop
Access data through NAS and Object;
Enterprise Storage Features; Data in place
Analytics with Hadoop
Capacity Multi-Datacenter, Exabyte Scale Single Instance, 50 PB
EMC Cloud Storage (ECS)

Complex Architecture
• Many special-purpose
systems
• Moving data around
• No complete views
Cost of Analytics
• Existing systems strained
• No agility
• BI backlog
Time to Data
• Up-front modeling
• Transforms slow
• Transforms lose data
Visibility
• Leaving data behind
• Risk and compliance
• High cost of storage
THE OLD WAY: BRINGING DATA TO COMPUTE
ERP, CRM,
RDBMS, Machines Files, Images, Video, Logs, ClickstreamsExternal Data Sources
Data
Archives
EDWs
Marts SearchServersDocument Stores Storage

Data Center Network
TIME-TO-RESULTS
Data Copy Analysis In-Place Analysis
Existing Primary Storage
Hadoop on a Stick
Have you ever
copied 100TB from
Primary Storage to
a Hadoop system?
How long does it
take to copy
100TB from one
place to another
over a 10Gb link?
>24 Hours
Data Center Network
Existing Primary Storage
Hadoop Compute Nodes
Reading
relevant
data to
analysis

THE NEW WAY: BRINGING COMPUTE TO DATA
Multi-workload analytic platform
• Bring applications to data
• Combine different workloads on
common data (i.e. SQL + Search)
• True BI agility
4
EDWs
Marts Storage
Search
Servers
Documents
Archives
ERP, CRM,
RDBMS, Machines
Files, Images, Video, Logs,
Clickstreams
External Data Sources
Active archive
• Full fidelity original data
• Indefinite time, any source
• Lowest cost storage
1
Data management,
transformations
• One source of data for all analytics
• Persisted state of transformed data
• Significantly faster & cheaper
2
Self-service exploratory BI
• Simple search + BI tools
• “Schema on read” agility
• Reduce BI user backlog requests
3
1
1
4 3
2

EMC ISILON SCALE OUT ARCHITECTURE
Web
Apps
Cloud
HadoopArchive
Linux/Unix
Mac/iOS
Windows

Simple is Smart
• Isilon has integrated ‚RAID‘, Volume
Manager and Filesystem layers into OneFS!
• The resulting HW-Architecture based on
Intel processors is therefore MUCH simpler
• Uses internal Storage
• Data protection with FEC accross nodes
ISILON HW-TOPOLOGY
ClientsC
LAN
C
Clients
Clients
Clients
Isilon Node
GB/10GB
Ethernet
Isilon
SAS
Isilon Node
SAS
Isilon Node
SAS
Infiniband
…
Clients

Isilon Single Volume Simplicity
EMPTYEMPTYEMPTYEMPTYEMPTYFULLFULLFULLFULLBALANCEDBALANCEDBALANCEDBALANCEDBALANCED
60 sec
Upgrade
SCALE-OUT ARCHITECTURE
• Directories and files striped across
nodes in a single volume
• Adding capacity dynamically by
adding node(s)
• It’s a 60 seconds task until new
capacity is available
• Re-balancing is happening in
background
• Data protection with FEC across
nodes
• Leverages Intel technology

Data Layout
File Striping
 Data is striped across the nodes
– Not across disks
– FEC not RAID
 Data breakdown:
– FS block size 8 KB
– 16 blocks per stripe unit
– 128 KB stripe width per drive
Clients
File
Write
Data
Stripe
Unit
Data
Stripe
Unit
File
Data
stripe
unit
Data
stripe
unit
Parity
stripe
unit

• Multi-threaded daemon runs on all nodes
– Services both NN and DN protocols
– Translates HDFS RPCs to POSIX system calls
– Stateless, underlying FS handles coherency
ISI_HDFS_D
OneFS Node
isi_hdfs_d
Thread
Request VFS
OneFS
Syscall
Response

HDFS IMPLEMENTED LIKE A NAS PROTOCOL
OneFS runs a daemon that
speaks NameNode and
DataNode natively
OneFS Clustered FileSystem
OneFS Node
NameNode
DataNode
OneFS Node
NameNode
DataNode
OneFS Node
NameNode
DataNode
OneFS Node
NameNode
DataNode
Hadoop
Node
DFSClient
1) Request(“/file”)
2) Response
(block locations) 3) GetBlock(block)

HDFS: STANDARD HADOOP CLUSTER
HDFS
file
file
copy2
file
copy3
node
info
file
node
info
file
copy2
file
copy3
file
node
info
file
copy2
file
copy3
file
node
info
file
copy2
file
copy3
Node
reply
Node
reply
Node
reply
Node
reply
node
reply
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
node
info
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
Data
Compute
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
Compute
Data
Name node
3X
NFS
Name node
Decision Support
Databases
Web Click
data
OLAP
EDW
HTTP
CIFS
FTP
NFS
Landing Zone
Servers
Step 1:
Data is copied into the
Landing Zone
Step 2:
Data is copied into the
Cluster (3 times)
Step 3:
Hadoop Jobs are run

node
info
node
info
node
info
node
info
MAP
Reduce
MAP
Reduce
MAP
Reduce
MAP
Reduce
Decision
Support
Web
Click
data
OLAP
EDW
Step 1:
Jobs are run
Compute Cluster
HDFS: SHARED DATA LAKE
Hadoop

AND WHAT ABOUT PERFORMANCE ?
Compute
Nodes
Isilon
Nodes

TRADITIONAL COMPUTE WITH LOCAL DAS
Rack Ethernet Switch
Compute
Shuffle+HDFS
SATA
1+ Gbps
Core Ethernet Switch
Compute
1 Gbps
…
Shuffle+HDFS
Compute…
Shuffle+HDFS
Compute
Shuffle+HDFS
SATA
1+ Gbps
Compute
1 Gbps
Shuffle+HDFS
Compute…
Shuffle+HDFS
The ratio of compute and disk
space/performance is fixed.
Non-local HDFS I/O (20-90% of
HDFS I/O) will go through
Ethernet.
Local disk usage is shared
between shuffle I/O (60% of all
I/O during terasort) and HDFS
I/O.

DAS I/O PIPELINE
SAS HBA
Each drive averages 30-50 MB/sec (read or
write) with a typical Hadoop workload
CPU/Bus/Memory
Dual 10 Gbps Ethernet NIC
48-port 10 Gbps Switch
8 SAS ports × 6 Gbps6000
600
PCI 2.0 x8 bus4000
PCI 2.0 x8 bus4000
10 Gbps × 2 directions × 2 ports5000
Total
MB/sec
CPU/memory bandwidth12,000
Non-blocking 48-port Switch120,000
Hadoop DAS Server
Where is the bottleneck?

IMPROVING EFFICIENCY WITH SHARED STORAGE
Isilon InfiniBand Switch
Compute
Shuffle
SATA
1+ Gbps
10 Gbps
Core Ethernet Switch
Compute
Shuffle
10 Gbps
… …
IB
Compute
Shuffle
SATA
10 Gbps
Compute
Shuffle
10 Gbps
…
…
IB
…
The number of compute and Isilon nodes
using Intel technology can be adjusted
independently to achieve the optimal ratio
of compute and I/O bandwidth
HDFS I/O ALWAYS comes through a rack-
local Isilon node which collects data blocks
from all other Isilon nodes across the
InfiniBand fabric (see animation)
(used only for MR copy phase) 1+ Gbps (used only for MR copy phase)
Shuffle I/O (75% of all I/O during terasort)
remains on local storage. This can be flash
for optimal performance.
Isilon
HDFS
Isilon
HDFS
Isilon
HDFS
Isilon
HDFS

ISILON I/O PIPELINE
SAS HBA
Each drive averages 30-50 MB/sec (read or
write) with a typical Hadoop workload
CPU/Bus/Memory
Dual 10 Gbps Ethernet NIC
48-port 10 Gbps Switch
8 SAS ports × 6 Gbps6000
400
PCI 2.0 x8 bus4000
PCI 2.0 x8 bus4000
10 Gbps × 2 directions × 2 ports5000
Total
MB/sec
Intel CPU/memory bandwidth12,000
Non-blocking 48-port Switch120,000
Isilon Node Isilon X400 maximum throughput (per node)740
10 Gbps × 2 directions × 2 ports (per node)5000
Hadoop Compute Server
The network is NOT a bottleneck!
Isilon Node…

Traditional Hadoop+DAS
Isilon Enabled Hadoop
Data Handling not
required on Isilon
17m:32s 30m:18s 20m:50
16m:00s
Hadoop+DAS
55MB/s Node Throughput
Compute:
Time To Results:
vHadoop+Isilon
85MB/s Node Throughput
Compute:
Time To Results:
30m 18s
68m 40s
16m 00s
16m 00s
1 TB Terasort Test
HADOOP JOB CYCLE COMPARISON: TIME TO RESULTS
(THROUGHPUT)

PHASE 1: INGEST
IDC PERFORMANCE VALIDATION
0
50
100
150
200
250
300
350
400
450
NFS Write 10 GB NFS Read 10 GB
Isilon
DAS Cluster
Runtime [s]
Isilon: 4x x410 Nodes
+ 7 Compute Nodes
DAS: 7x (Compute+DAS Nodes)
4,2x
faster
36x
faster

PHASE 1I: COMPUTE RUNNING ON INTEL XEON PROCESSORS
IDC PERFORMANCE VALIDATION
0
500
1000
1500
2000
2500
3000
TeraGen TeraSort TeraValidate
Isilon
DAS Cluster
Runtime [s]
Isilon: 4x x410 Nodes
+ 7 Compute Nodes
DAS: 7x (Compute+DAS Nodes)
2,6x
faster
1,5x
faster
1,5x
faster

ESG PERFORMANCE VALIDATION

CAPACITY UTILIZATION,
DATA TIERING,
PROTECTION

NODE TYPES
SCALE-OUT NAS PRODUCT FAMILY POWERED BY INTEL
Purpose-Built for IOPs-
Intensive, Random Access,
File-Based Applications
Flexible Solution for
High Concurrent and
Sequential Throughput
Applications
Ideal for Cost-
Effective, Large-
Capacity Storage
S-Series
X-Series
NL-Series
Capacity
Performance
Ideal for Cost-Effective,
Large-Capacity Storage
HD-Series

POLICY BASED DATA PLACEMENT
Policy
<30 days
>30 days
S210
NL400
30 days-
2years
>2 years
NL400
HD400
 Multiple archive tiers
 Policy-based heuristic tiering
 Eliminate data migration
 Cost-optimized data placement
 Transparent to applications and users
 Optimize storage resources while matching
storage resources with data requirements

TIERING TO EXTERNAL OJECT STORE
SMART TIERING TO CLOUD
Key Features
Benefits
 Stub to Cloud of choice
 Extending workflow of SmartPools to
CloudPools
 Optional ability to send encrypted
data to the cloud
 Compression for efficient transport
 Simple management from a familiar
interface
 Seamless placement and availability
of data per policy
 Enable offsite DP & archive
 Transparent integration with offsite
stores
 One Accessible namespace
SmartPool -> CloudPool
Clients
SMB | NFS | RAN
OneFS
Service
Provider
Public
Cloud
Private
Cloud

Isilon Shared storageHadoop DAS
COMPARE- UTILIZATION AND CHOICE
Shared Storage
Hadoop Storage 3x copies
# copies
Number of
Nodes
+1 +2:1 +2 +3:1 +3 +4
3 2+1 (33%) 4+2 (33%) 3x -- -- --
4 3+1 (25%) 6+2 (25%) 2+2 (50%) 9+3 (25%) 4x --
5 4+1 (20%) 8+2 (20%) 3+2 (40%) 12+3 (20%) 4x 5x
6 5+1 (17%) 10+2 (17%) 4+2 (33%) 15+3 (17%) 3+3 (50%) 5x
7 6+1 (14%) 12+2 (14%) 5+2 (29%) 15+3 (17%) 4+3 (43%) 5x
8 7+1 (13%) 14+2 (12.5%) 6+2 (25%) 15+3 (17%) 5+3 (38%) 4+4 (50%)
9 8+1 (11%) 16+2 (11%) 7+2 (22%) 15+3 (17%) 6+3 (33%) 5+4 (44%)
10 9+1 (10%) 16+2 (11%) 8+2 (20%) 15+3 (17%) 7+3 (30%) 6+4 (40%)
12 11+1 (8%) 16+2 (11%) 10+2 (17%) 15+3 (17%) 9+3 (25%) 8+4 (33%)
14 13+1 (7%) 16+2 (11%) 12+2 (14%) 15+3 (17%) 11+3 (21%) 10+4 (29%)
D
1
D
2
D
3
D
4
P
0
P
1
D
5
D
6
D
7
D
8
P
0
P
1
D
9
D
1
1
D
1
2
D
1
0
P
0
P
1
Node1 Node2 Node3 Node4 Node5 Node6
Protection
Group
Protection
Group
Protection
Group
D
6
D
7
D
8
D
9
D
1
0
D
1
1
D
1
2
D
1
D
2
D
3
D
4
D
5
Data Stripes of 128kB
1 2 43
Triple
replication

INDUSTRY SOLUTION: MIRROR COPY
• Mirrored copy in a backup site
• Benefit: Achieves Local
reconstruction on hardware
failure
• Shortcoming: Storage
overhead -> 2.66x
Primary Secondary

INDUSTRY SOLUTION: DISTRIBUTED ERASURE
CODING
• Distributing fragments across sites
• Benefit: Achieves low Storage Overhead
~ 1.6x
• Shortcoming: Disk/Node failure
requires fragments to be fetched over
the WAN.
Site 1 Site 2
Site 3 Site 4

ECS HYBRID ERASURE CODING
• Data is written into chunks 3 copies
• Once a chunk fills to 128 MB, erasure coding starts
• Once it is completed and data is protected the 3 copies are
deleted.
A
A
AA

VIPR MODEL: BEST OF BOTH WORLDS
• Achieves low Storage Overhead ~
1.8x
• Local hardware failure recovery
requires no WAN traffic.
• Handles local hardware and full data
center failures
– Disk, Node, Rack, Data Center are
failure domains
Site 1 Site 2
Site 3 Site 4

DATA LAKE FOUNDATIONS WITH ECS
L.A.
Social Networks,
UGC
Public records
Location
Data
Internet
Of Things
Emerging
Traditional
Unstructured
Data
Dark Data
Structured
Data
Efficient Global Scale Big Data Analytics
Shared storage
Built-in high availability
Metered chargebacks
Multi-tenant

MULTIPLE TENANTS
 Each “Authentication Zone” is associated with
 A Network Zone
 List of Auth source associations (trusted or untrusted)
 A root folder in OneFS
IP Pool 1
IP Pool 2
IP Pool 3
AD 1
/ifs/deptC
Access
Zone1
Access
Zone2
Access
Zone3
/ifs/deptB
/ifs/deptA
AD 2
or
LDAP

MULTIPLE TENANTS WITH ECS
Authorization Provider
(AD/LDAP)
Namespace Admin
Create Namespace
Object Users
Namespace 1 Namespace 2 Namespace 3
Add Namespace &
System Admins
Object UsersObject Users
Object Buckets Object Buckets Object Buckets
System Admin
Assigns Object user to a
namespace

NFS
HDFS
SMB, NFS,
HTTP, FTP,
HDFS 1.x
...
HDFS 2.x
...
Node
reply
Node
reply
Node
reply
Node
reply
name
node
name
node
name
node
name
node
datanode
NFS
SMB
SMB
NFS MAP Reduce
MAP Reduce
MAP Reduce
MAP Reduce
MAP Reduce
MAP Reduce
SUPPORT FOR MULTIPLE ANALYTICS APPLICATIONS

SUMMARY: COMPARING SIDE BY SIDE
DAS Isilon
Simultaneous Hadoop Distributions r a
File/Object Level Access Control Lists a a
User Controlled Snapshot R a
De-duplication R a
WORM (SEC 17a-4) r a
POSIX Compliance r a
Independent Scaling (Storage/Compute) r a
Hadoop Distribution Portability r a
SQL Support (HAWQ, Impala, TEZ) a a
Encryption a2
SED
Data Tiering a3
a
Hadoop Distribution Support 1 All
Disaster Recovery Full File Copy
Snap
Replicate
DAS Isilon
Data-Set Management Ingested In-place
Protection overhead 200% ~20%
NameNode Redundancy (HA) Active/Passive N-to-N
Ability to edit files/objects r a
NFS v3, v4 r a
SMB 1, 2 r a
HTTP r a
FTP r a
Object (Proprietary) r a
HDFS v1 r a
HDFS v2 a1
a
Simultaneous Multi-Protocol r a
1. Only one version at a time
2. Software-Only (~30% impact to performance)
3. Available only in HDFS 2.5+

P I V O T A L B I G D A T A S U I T E
V M W A R E V C L O U D S U I T E
EMC DATA LAKE FOUNDATION: ISILON + ECS
VCE VBLOCK | XTREMIO | DATA DOMAIN
O P E N
A N A L Y T I C S
T O O L B O X
D A T A A N D A N A L Y T I C S C A T A L O G
A D V A N C E D A N A L Y T I C SA P P L I C A T I O N S
A T S C A L E
D A T A
P R O C E S S I N G
GREENPLUM
DATABASE
HAWQ
SPRING XD PIVOTAL HDSPARK
REDIS
RABBITMQ
GEMFIRE
BDS ON PIVOTAL
CLOUD FOUNDRY
H A D O O P
PLATFORMMANAGER
DATAGOVERNOR
DATA
MANAGER
INGEST
MANAGER
ANALYTICS
MANAGER
EMC BUSINESS DATA LAKE

FOR MORE INFORMATION ....
.... SEE US AT THE BOOTH
THANK YOU !

QUESTIONS ?
Dr. Stefan Radtke
CTO, EMEA
EMC Emerging Technology Division
Phone: +49-176-34434460
E-Mail: Stefan.Radtke@emc.com
Linkedin: http://de.linkedin.com/in/drstefanradtke
Blog: http://stefanradtke.blogspot.com

In-Place analytics with Unified Data Access

In-Place analytics with Unified Data Access

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to In-Place analytics with Unified Data Access

Similar to In-Place analytics with Unified Data Access (20)

More from DataWorks Summit

More from DataWorks Summit (20)

Recently uploaded

Recently uploaded (20)

In-Place analytics with Unified Data Access