Your SlideShare is downloading. ×
0
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
pNFS, parallel storage for grid, virtualization and database ...
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

pNFS, parallel storage for grid, virtualization and database ...

1,065

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,065
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
70
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. pNFS, parallel storage for grid, virtualization and database computing Joshua Konkle, Chair NFS SIG
  • 2. SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK . 2 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 3. Abstract pNFS, parallel storage for grid, virtualization and database computing This session will appeal to Virtual Data Center Managers, Database Server administrators, and those that are seeking a fundamental understanding pNFS. This session will cover the four key reasons to start working with NFSv4 today. Explain the storage layouts for parallel NFS; NFSv4.1 Files, Blocks and T10 OSD Objects. We’ll conclude the session with use cases for database access, enterprise and desktop virtualization, including deduplication options. 3 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 4. Tutorial Agenda Introduction to NFS and NFS Special Interest Group NFS v4 – Security, High Availability, Internationalization and Performance (SHIP) pNFS – Layout Overview Files based access Block based access Object based access pNFS – OpenSource Client Status pNFS Use Cases – Virtualization, Database, etc 4 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 5. SNIA’s NFS Special Interest Group NFS SIG drives adoption and understanding of pNFS across vendors to constituents Marketing, industry adoption, Open Source updates NetApp, EMC, Panasas and Sun founders NetApp and Panasas act as co-chairs Deliver Panels/Sessions on NFSv4.1 when possible E.g. SNW Europe in October ‘09, Super Computing ‘10 Learn more about us at: www.snia.org/forums/esf 5 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 6. Background Information Network File System Protocol to make data stored on file servers available to any computer on a network NFS clients are included in all common Operating Systems, e.g. Linux, Solaris, AIX, Windows etc….. Application and OSI layers (remote procedure calls) NFS Server; Inspiration to NAS and appliances Commodity Operating Systems have NFS servers NAS Appliance – Control, Consistency and Cadence Vendors offer commodity hardware, w/ management software 6 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 7. NFSv4 SHIP is sailing Functional Business Benefit ACLs for authorization Compliance, improved access, Security Kerberos for authentication storage efficiency High Client and server lease High Availability, Operations availability management with fail over simplicity, cost containment International Unicode support for utf8 Global file system for multi- characters characters national organizations Multiple read, write, delete Better network utilization for all Performance operations per RPC call NFS clients Delegate locks, read and Leverage NFS client hardware for write procedures to clients better I/O 7 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 8. NFSv4 - HA and Performance High Availability via Leased Lock Client renews lease on server file lock @ n Seconds Client fails, lock is not renewed, server releases lock Server fails, on reboot all files locked for n Seconds Gives clients an n Second grace period to reclaim locks Performance via Delegations File Delegations allow client workloads for single writer and multiple reader Clients can perform all reads/writes in local client cache Delegations are leased and must be renewed Delegations reduce lease lock renewal traffic 8 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 9. The Evolution of Storage 2000 2010? Future Market Adoption Cycles Direct- Networked Attached Storage ? Storage 9 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 10. Evolving Requirements Economic Trends Cheap and fast computing clusters Cheap and fast network (GigE to 10GigE) Performance Exposes single threaded bottlenecks in applications Evolution of computing models Reduced time to market, response time Powerful compute systems Analysis begets more data, at exponential rates Competitive edge (IOPS) 10 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 11. NFS – What’s the problem? In-band data access model Results in Limitations Easy to build, Limited in scale Islands of storage Well-defined failure modes Server and Appliance HW Limited load balancing options Networking and I/O 11 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 12. Performance, Management and Reliability Random I/O and Metadata intensive workloads Memory and CPU are hot spots Load balancing limited to pair of NFS heads Limited to dual-head configuration Compute farms are growing larger in size NFS head can handle a 1000+ NFS clients NFS head hardware comparable to client CPU, I/O, Memory NFS head requires more spindles to distribute the I/O Reliability and availability are challenging Data striping limited to single head and disks Non-disruptive upgrades affect dual-head configurations Access and load balancing are typically limited to a pair of NFS server heads 12 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 13. What is the Solution? 2000 2010 2020 Future NFSv4.1 Parallel NFS Market Adoption Cycles Direct- Networked Scale-Out Attached Storage ? Storage Storage 13 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 14. NFSv4.1 – Parallel Data Storage NFSv4.1 – Three Storage Types Results in Improvements Files – NFSv4.1 Global Name Space Blocks – SCSI Head and Storage scaling Objects – OSD T10 Non disruptive upgrades while maintaining performance NFS Hosts 14 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 15. NFSv4.1 - Parallel NFS 101 NFSv4.1 Client (s) pNFS protocol Standardized: NFSv4.1 Storage-access protocol Files (NFSv4.1) Block (iSCSI, FCP) Object (OSD2) pNFS Control protocol protocol Storage-access Not covered by spec; no protocol generally agreed upon characteristic Metadata Server Control protocol Data Servers 15 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 16. pNFS Operations LAYOUTGET Obtains the data server map from the meta-data server LAYOUTCOMMIT Servers commit the layout and update the meta-data maps LAYOUTRETURN Returns the layout; Or the new layout, if the data is modified GETDEVICEINFO Client gets updated information on a data server in the storage cluster GETDEVICELIST Clients requests the list of all data servers participating in the storage cluster CB_LAYOUT Server recalls the data layout from a client; if conflicts are detected 16 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 17. NFSv4.1 – OpenSource Status Two OpenSource Implementations OpenSolaris and Linux OpenSolaris Client and Server Support only file-based layout Support for multi-device striping already present (NFSv4.1 + pNFS) “Simple Policy Engine” for policy-driven layouts also in the gate Linux Client and Server Support files (NFSv4.1) Support in progress blocks (SCSI), objects (OSD T10) Client consists of generic pNFS client and “plug ins” for “layout drivers” Predicted timeline for Linux: Basic NFSv4.1 features 1H2009 – On Target; delivered Sessions/Trunking NFSv4.1 pNFS and layout drivers by 2H2009 - Behind Linux distributions shipping supported pNFS in 2010 - Behind 17 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 18. pNFS – NFSv4.1 files access Client mounts and opens a NFSv4.1 Client (s) file on the server Servers grants the open and a file stripe map (layout) to the client File Handle Mount, The client can read/write in Open & R/W Request parallel directly to the Get layout Issued in parallel NFSv4.1 data servers Metadata Server Control protocol pNFS, parallel storage for grid, virtualization and database computing Data Servers 18 © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 19. pNFS Blocks Access Model Client mounts and opens a NFSv4.1 Client (s) file on the server Servers grants the open and a block map (layout) to the client Based on the layout Storage Access obtained (read or write); pNFS Protocol SCSI the client can read/write in protocol parallel directly to the SCSI target’s Metadata Server Control protocol pNFS, parallel storage for grid, virtualization and database computing Data Servers 19 © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 20. pNFS Objects Access Model Client mounts and opens NFSv4.1 Client (s) Object Servers grants the open and an object stripe map and object capabilities (layout) to the client Storage Access Based on the layout pNFS Protocol iSCSI obtained (read or write); protocol OSD the client can read/write in parallel directly to the OSD targets Metadata Server Control protocol pNFS, parallel storage for grid, virtualization and database computing Data Servers 20 © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 21. Traditional HPC Use Cases Seismic Data Processing / Geosciences' Applications pNFS Ethernet Solution Broadcast & Video Production Application Server Racks High Performance Streaming Video pNFS FCoE NFS Finite Element Analysis for Modeling & Simulation iSCSI /NFS FC Network iSCSI / FC HPC for Simulation & pNFS Modeling pNFS Storage Data Intensive Searching for Server Heads Computational Infrastructures 21 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 22. pNFS for Virtualization and Databases Original pNFS use case pNFS Ethernet Solution for HyperVisor 100’s of hosts to storage 32 or more HyperVisors in a cluster. 16+ Cores in future ... Single NFS Datastore FCoE Multiple-heads across NFS multiple disks iSCSI /NFS FC Network Trunking iSCSI / FC Directory/File Delegations Storage Heads pNFS Caveat Server Limit on VMs per LUNs 22 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 23. NFSv4.1 – Virtualized Data Center VM DB Cluster Datastore Mount Server:/ pNFS Name Space Server ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 23 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 24. Single NFSv4.1 namespace Striped Volume Striped Volume pNFS Server Name Space ... / HyperVisor Cluster Nodes VM DB HV1 HV2 Srv1 Srv2 Srv3 24 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 25. Single NFSv4.1 datastore pNFS Server Name Space ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 25 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 26. VM Cluster Datastore pNFS Cluster Datastore Server Mount Server:/ Name Space ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 26 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 27. VMs accessing volume w/layout VM pNFS Cluster Datastore Server Mount Server:/ Name Space ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 27 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 28. NFSv4.1 Trunking/Sessions connection Open sessions w/connection Trunking 1. A single connection limits data throughput based on protocol 2. Trunking expands throughput and can reduce latency by opening multiple sessions to the same file handle/server resource Host application consumes 10GigE bandwidth 28 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 29. VM Access using single mount VM pNFS Cluster Datastore Server Mount Server:/ Name Space ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 29 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 30. VM access using pNFS + Trunking VM pNFS Srv2 Server HV1 Name Space Srv1 / VM DB HV1 HV2 Srv1 Srv2 Srv3 30 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 31. NFSv4.1 Directory/File Delegations VM Set NFS Swap File to SSD/Flash I/O Card, single write read/write delegations allow applications to write through changes but keep most data “delegated” on Flash Swap. pNFS Server Srv2 HV1 Name Space Srv1 Flash I/O Card / Delegations available in NFSv4 VM DB Reduce renewals for locks Improve R/W performance Remove getattr storms HV1 HV2 Srv1 Srv2 Srv3 31 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 32. NFSv4.1 – Database enhancements Use Ethernet and pNFS infrastructure for VM Multiple-heads across multiple disks DB Trunking & Delegations Cluster Datastore Mount Server:/ pNFS Name Space Server ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 32 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 33. DB access using pNFS + Trunking Multiple-heads across DB multiple disks Trunking enables highest IOPS and lowest latency Name Space pNFS Server / Srv2 VM DB HV1 Srv1 HV1 HV2 Srv1 Srv2 Srv3 33 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 34. NFSv4.1 – Layout Callbacks DB (Replica) DB Non-disruptive data moves using storage control protocols pNFS Name Space Server ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 34 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 35. NFSv4.1 – Layout Callbacks DB (Replica) DB pNFS Server Name Space ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 35 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 36. NFSv4.1 – Virtualized Data Center VM DB Cluster Datastore Mount Server:/ pNFS Name Space Server ... / HyperVisor Cluster Nodes HV1 HV2 VM DB Srv1 Srv2 Srv3 HV1 HV2 Srv1 Srv2 Srv3 36 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 37. Summary/Call to Action pNFS is the first open standard for parallel I/O across the network Ask vendors to include NFSv4.1 support for client/servers pNFS has wide industry support commercial implementations and open source Start using NFSv4.0 today Eases transition to pNFS 37 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 38. Q&A / Feedback Please send any questions or comments on this presentation to SNIA: tracknetworking@snia.org Many thanks to the following individuals for their contributions to this tutorial. - SNIA Education Committee Mike Eisler, J. Bruce Fields Brian “Beepy” Pawloski Joe White Howard Goldstein Brent Welch David Black Ken Gibson Omer Asad Sachin Chheda Jason Blosil Piyush Shivam Mark Carlson Sorin Faibash Rob Peglar Andy Adamson Dave Hitz Pranoop Ersani Ricardo Labiaga Dave Noveck Learn more about us at: www.snia.org/forums/esf 38 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 39. NFSv4.1 – Status and Overview 2004 – CMU, NetApp and Panasas draft pNFS problem and requirement statements 2005 – CITI, EMC, NetApp and Panasas draft pNFS extensions to NFS 2005 – NetApp and Sun demonstrate pNFS at Connectathon 2005 – pNFS added to NFSv4.1 draft 2006 - 2008 – specification baked Bake/Connect a thons; 29 iterations of NFSv4.1/pNFS spec 2008 – NFSv4.1/pNFS reaches IETF Approval (December) 39 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.
  • 40. pNFS Standards Status NFSv4.1/pNFS were standardized at IETF NFSv4 working group (WG) All done except for RFCs: WG last call (DONE) Area Director review (DONE) IETF last call (DONE) IESG approval for publication (DONE) IANA review (TBD) RFC publication (Expected 2009) Will consist of several documents: NFSv4.1/pNFS/file layout NFSv4.1 protocol description for IDL (rpcgen) compiler blocks layout objects layout netid specification for transport protocol independence (IPv4, IPv6, RDMA) 40 pNFS, parallel storage for grid, virtualization and database computing © 2009 Storage Networking Industry Association. All Rights Reserved.

×