More Related Content Similar to The Efficient Use of Cyberinfrastructure to Enable Data Analysis Collaboration (20) More from Cybera Inc. (20) The Efficient Use of Cyberinfrastructure to Enable Data Analysis Collaboration1. The Efficient Use of Cyberinfrastructure
to Enable Data Analysis Collaboration
Summit ’09 October 15, 2009
Dave Fellinger
CTO
© 2009 DataDirect Networks, Inc. All Rights Reserved.
© 2008 DataDirect Networks, Inc. All Rights Reserved.
2. Company Data at a Glance
• Fast Growing Data Infrastructure Provider for Companies who
Demand Extreme Performance for their Large Content Files and
Unstructured Data
• Integrated Portfolio of Extreme Storage Platforms, Intuitive
Storage Management Software and Consulting Services
• Over 10 Years of Stability and Experience
• Over $130M in Annual Revenue
• Growing, Profitable and Hiring
• Over 200 Petabytes Installed Worldwide
• Clients include XBOX LIVE, Slide & Saudi Aramco
• Global Partners include IBM, Sony and Dell
• Expanding Globally, with Established Offices
• Europe, India, Australia, Asia Pacific and Japan
“DDN could find itself a market leading provider for the Internet computing era
in the same way that EMC did in the transactional era and NetApp did in the
distributed era”
© 2009 DataDirect Networks, Inc. All Rights Reserved.
3. Storage Designed for the Most
Extreme Environments in the World
• Sites That Have Zero Tolerance For Data Latency
• Sites That Demand Sub-Millisecond Application Response Times
• Sites that Stream Extreme HD Content on a Massive Scale
• Sites with Our Systems Delivering Over 100 Gigabytes Per Second of
Consistent Throughput
• Sites that Research International Scientific Data
• The Most Sophisticated and Demanding High Performance Computing
Environments
• Powering 8 of the Top 10 and
• Powering 50 of the Top 100
Supercomputer Sites
© 2009 DataDirect Networks, Inc. All Rights Reserved.
4. Data Distribution Evolution
• The Problem
• A large number of sites need simultaneous
access to local data for detailed analysis.
• The Solution
• Utilize a simplified object based file system to
provide simultaneous synchronous instances
of data based on policy.
© 2009 DataDirect Networks, Inc. All Rights Reserved.
5. Data Distribution Today
• The “Library Method”
• Large data warehouses are queried for data in
an iterative manner.
• MODIS as an example.
• All MODIS image data is stored at NASA
Goddard on both disk and tape.
• Inquiries generally spawn additional inquires
requiring time, study, and network bandwidth.
© 2009 DataDirect Networks, Inc. All Rights Reserved.
6. How is it Done Today?
URL explosion in objects
Map Objects to individual file systems Hundreds or thousands of building
blocks like this
Within file system files and folders are
named 1, 2, 3, 4…. for hashing and rapid No common management framework
index lookups Database
Huge investment in custom engineering
Must manage # of files per folder, # of to cobble together a global namespace
folders in each file system
Heavy reliance on expensive CDNs
Multiple IOPS to get a file limits # of file
reads per second and increases latency Internally developed replication schemes
LAN
NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS
NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS NFS/HTTP/CIFS
Local File System Local File System Local File System Local File System Local File System Local File System
Local File System Local File System Local File System Local File System Local File System Local File System
RAID RAID RAID
Disk Disk Disk
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
7. Multi-site Content Delivery
SITE A SITE C
SITE D
Database
SITE B Tracking
File
Locations
Replication Software (x10)
Administrator (x5)
Storage/File System(s)
SITE E
RAID/LUNs/Fibre Channel (x10 – Thousands)
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
8. What’s within each site?
Complexity!
• Lots of things to manage
• No automation or coordination
Hundreds of filers • Must maintain a path to find every file
• Provisioning more capacity increases complexity
Dozens – Hundreds of File Systems!
Multiple File
Servers
Fibre Channel
Switching,
Cables, SFPs
Multiple
RAIDs
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
9. DDN’s Cloud Storage Initiative
! Take a clean look at the issues of massively scalable
file storage & distribution
! Understand best practices from leading edge
customers
! Develop a tailored solution
– Work with key users every step of the way
– Eliminate the need for complex multi-vendor integration
– Minimize customers having to write custom code
– Focus on file reads per second rather than IOPs
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only9
10. Scalable File Delivery
with WOS Clouds
• Single Management Point
• Single Namespace for Billions of Files
• Fully Automated, Load Balancing & Self Healing
• Automated Best-Path File Retrieval
• Multi-site Policy-Based Replication
• Add Capacity Online in Seconds
• Easy!
Network
Connection
WOS Nodes
Database
Administrator Storing WOS
Object IDs
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
11. WOS Puts & Gets - 1
Users
Users
Customer
Application
Servers
WOS-B WOS-LIB WOS-LIB
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
12. WOS Puts & Gets - 2
User uploads a
User uploads a
file to application
file to application
Users
Users server.
server.
Customer
Application
Servers
WOS-LIB WOS-LIB WOS-LIB
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
13. WOS Puts & Gets - 3
Users
Users
Customer Application makes
Application makes
Application a call to the WOS-
a call to the WOS-
Servers LIB to store (PUT)
LIB to store (PUT)
WOS-LIB WOS-LIB WOS-LIB
a new object
a new object
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
14. WOS Puts & Gets - 4
Users
Users
Customer WOS-LIB stores the object
WOS-LIB stores the object
Application on a node. Subsequent
on a node. Subsequent
Servers objects are automatically
objects are automatically
WOS-LIB WOS-LIB WOS-LIB load balanced across the
load balanced across the
cluster.
cluster.
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
15. WOS Puts & Gets - 5
Users
Users
Customer
Application
Servers
WOS-LIB WOS-LIB WOS-LIB
If the storage
If the storage
Gigabit policy specifies
policy specifies
Ethernet replication, the
replication, the
Network object is replicated
object is replicated
to another node.
to another node.
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
16. WOS Puts & Gets - 6
Users
Users
WOS-LIB returns a unique
WOS-LIB returns a unique
Customer OID = 5718a36143521602 Object ID which the
Object ID which the
Application
application stores in lieu of
application stores in lieu of
Servers
WOS-LIB WOS-LIB WOS-LIB a file path.
a file path.
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
17. WOS Puts & Gets - 7
Users
Users
A User needs
A User needs
the stored file.
the stored file.
Customer
Application
Servers
WOS-LIB WOS-LIB WOS-LIB
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
18. WOS Puts & Gets - 8
Users
Users
Application makes a
Application makes a
call to WOS-LIB to
call to WOS-LIB to
read (GET) the
read (GET) the
object. The unique
object. The unique
Object ID is passed Customer
Object ID is passed
to WOS-LIB.
to WOS-LIB. Application
Servers
WOS-LIB WOS-LIB WOS-LIB
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
19. WOS Puts & Gets - 9
Users
Users
WOS-LIB automatically
WOS-LIB automatically
determines what
determines what
node(s) have the
node(s) have the
requested object,
requested object,
retrieves the object,
retrieves the object,
and returns it to the Customer
and returns it to the
application.
application. Application
Servers
WOS-LIB WOS-LIB WOS-LIB
OID = 5718a36143521602
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
20. WOS Puts & Gets - 10
Users
Users
WOS-LIB automatically
WOS-LIB automatically
determines what
determines what
node(s) have the
node(s) have the
requested object,
requested object,
retrieves the object,
retrieves the object,
and returns it to the Customer
and returns it to the
application.
application. Application
Servers
WOS-LIB WOS-LIB WOS-LIB
OID = 5718a36143521602
Gigabit
Ethernet
Network
OID = 5718a36143521602
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
21. WOS Puts & Gets - 11
Application returns
Application returns
file to user.
file to user.
Users
Users
Customer
Application
Servers
WOS-LIB WOS-LIB WOS-LIB
Gigabit
Ethernet
Network
WOS Cluster
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
22. WOS Zones and Replication
Global Content Distribution - 1
Zones are logical groups of WOS
nodes in which you scope
replication, or data protection
Zone “West” policies.
In this example, we have two Zone “East”
zones which also map to
geographic data centers.
These data centers both serve the
public internet.
Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
23. WOS Zones and Replication
Global Content Distribution - 2
Objects stored utilizing the “Wild”
Zone “West” policy will result in objects being
stored on two distinct nodes within
the “West” zone. Zone “East”
The server performing the store
action can be anywhere on the
network, including the “East” zone
Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
24. WOS Zones and Replication
Global Content Distribution - 3
Objects stored utilizing the
“Safe” policy will result in object
Zone “West” replicas being stored on one
node in the “West” zone and
one node in the “East” zone.
Zone “East”
Again, the server performing the
store action via the WOS-LIB
can be anywhere on the
network.
Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
25. WOS Zones and Replication
Global Content Distribution - 4
The “West” and “East” zones
utilize replication on high
Zone “West” performance WOS nodes for
the most demanding service
environments. Zone “East”
What if you wanted an extra
object copy for Disaster
Recovery?
Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
26. WOS Zones and Replication
Global Content Distribution - 5
Zone “DR”
AN
dW
ee
Sp
H igh
Hi
Data Protection via Replication gh
S
Zone “West”
pe
ed
W
AN
Zone “East”
The “DR” zone utilizes high
capacity WOS nodes.
Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
CYA: West=1, East=1, DR=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
27. WOS Zones and Replication
Global Content Distribution - 6
Zone “DR”
AN
dW
ee
Sp
H igh
Hi
Data Protection via Replication gh
S
Zone “West”
pe
ed
W
AN
Objects stored utilizing the “CYA” Zone “East”
storage policy will result in object
replicas on both coasts as well as
a copy in the “DR” zone.
Data Protection via Replication
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
CYA: West=1, East=1, DR=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
28. WOS Zones and Replication
Global Content Distribution - 7
Zone “DR”
AN
dW
ee
Sp
H igh
Hi
Data Protection via Replication gh
S
Zone “West”
pe
ed
W
AN
Regardless of replication Zone “East”
policy, any object (OID)
can be accessed from
any zone, whether the
object resides in that
Data Protection via Replication
zone or not!
High Spe
ed WAN
Connecti
o n
User Defined Object
User Defined Object Data Protection via Replication
Storage Policies:
Storage Policies:
Wild: West=2, East=0
Wild: West=2, East=0
Wallstreet: East=2, West=0
Wallstreet: East=2, West=0
Safe: West=1, East=1
Safe: West=1, East=1
CYA: West=1, East=1, DR=1
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
29. Data Protection: Drive and Node
Failure Handling - 1
• Replication requires at least two copies of each
object to be stored for a given OID.
• With replication, for maximum performance,
individual objects are stored within 1 disk unit.
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
30. Data Protection: Drive and Node
Failure Handling - 2
• Upon disk drive failure, all objects stored on the
failed drive are noted to be out of policy
compliance and recovery begins.
• Affected objects are copied in parallel to bring the
cluster back into full policy compliance.
!
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
31. Data Protection: Drive and Node
Failure Handling - 3
• Upon disk drive failure, all objects stored on the
failed drive are noted to be out of policy
compliance and recovery begins.
• Affected objects are copied in parallel to bring the
cluster back into full policy compliance.
!
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
32. Data Protection: Drive and Node
Failure Handling - 4
• Policy restoration occurs on a per object basis,
NOT per node, hence only objects that resided on
the failed node will be replicated.
• When the failed node is replaced or returns online,
it simply becomes additional cluster capacity.
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
33. Data Protection: Drive and Node
Failure Handling - 5
• Upon node failure, recovery of all objects stored
on the failed node begins.
• Affected objects are copied in parallel and
distributed to surviving nodes to bring the cluster
back into full policy compliance.
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
34. Data Protection: Drive and Node
Failure Handling - 6
• Upon node failure, recovery of all objects stored
on the failed node begins.
• Affected objects are copied in parallel and
distributed to surviving nodes to bring the cluster
back into full policy compliance.
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
35. Data Protection: Drive and Node
Failure Handling - 7
• Policy restoration occurs on a per object basis,
NOT per disk drive hence only used object space
will be replicated.
• When the failed disk drive is replaced, the
replacement simply becomes additional capacity.
© 2009 DataDirect Networks, Inc. All Rights Reserved. DataDirect Networks Confidential: Internal Use Only
36. Data Distribution Evolution
• The “Push Method”
• Data is automatically replicated based on
policy to scientists with specific requirements.
• ATLAS as an example;
• The LHC at CERN produces more than 1TB
per day.
• Relevant data is replicated to over 300 sites in
the US and Canada for analysis.
• Replication is based on specific interest and is
fully automated.
© 2009 DataDirect Networks, Inc. All Rights Reserved.
37. Conclusion
• To enable multi-point scientific study data must have
attributes of location (site) as well as traditional
ACLs.
• File systems distributing data must have redundant
automation to enable replication over geographies.
• Data distribution methods must be simplified to
reduce latency, maximize network efficiency allowing
improved processing efficiency.
© 2009 DataDirect Networks, Inc. All Rights Reserved.
38. Thank You
Dave Fellinger
dfellinger@ddn.com
© 2009 DataDirect Networks, Inc. All Rights Reserved.
© 2009 DataDirect Networks, Inc. All Rights Reserved.