Your SlideShare is downloading. ×
0
Storage Management GOES HERE    PRESENTATION TITLE for the Cloud   Mario Vosschmidt, SNIA Europe Director, LSI
SNIA Europe Legal Notice The material contained in this presentation is copyrighted by SNIA Europe. Member companies and i...
Agenda Cloud computing overview   Cloud Motivation   Cloud Computing overview   Consolidation and Cloud computing Technolo...
Overview  Cloud computing overview    Business model       Deliver Resources – computing, data etc.       Delivery on dema...
Cloud computing trendsSource: Hosting Transformation Summit, Dan Golding Keynote, Sept. 2009
The IDC View                         Storage grows from 9%                         of the total Cloud                     ...
Usage of Cloud Storage today  Elastic demand for web based media (video, eBooks, audio)  Backup to the cloud  Internet “Dr...
Cloud Data Storage ModelStorageManagement                Cloud Data                Cloud Data                 Storage     ...
SNIA CDMI Cloud Data Management Initiative Launched at Fall SNW 2009    Press release listing charter members    Cloud Pav...
What is Cloud Storage?  The use of the term cloud in describing these new models arose from  architecture drawings that ty...
A look at some existing  Cloud APIs       What are some of the offerings and their Data Storage       Interfaces?         ...
Storage IndustryResource Domain ModelAll of these interfaces support some or all of this model. The key to retainingthe si...
Cloud Storage Container• Cloud Storage may be used  similar to a volume/filesystem• DSI Protocols include: WebDAV, NFS, CI...
The Complete Picture
Cloud Data ManagementInterfaceApplicable to three types of Cloud Storage:     Cloud Storage for Cloud Computing           ...
REST Representation State Transfer    Started with Dissertation by Roy Fielding outlining the principles Addressability   ...
Why a RESTful approach for aCloud Storage Standard  Simplicity Rules!  Common Infrastructures in many Languages on many  P...
Cloud Peering                18
Work beyond SNIA    Standardizing Cloud computing interfaces        SNIA : CDMI        OCCI: Open Grid Forum (OGF)        ...
Resource Management  OCCI URI – IaaS Resource alignement       Operations on resources (start, stop, delete, update)
OCCI APIOCCI API Implementation  CRUD – create, retrieve, update and delete  Mapped to http post, get, put, delete  Metada...
OCCI – CDMI Integration
What is missing?  Authentication                          Security  Authorization                           Collaboration ...
THE PROVIDER’S VIEW   of Storage Management
The Provider‘s view  IT Operations    Service Levels (SLA)    Equipment / Software cost       Acquisition       License   ...
Enterprise Resource Management  ERM attributes     Analyse        Quantify and Qualify     Report        Extract and predi...
SMI-S Use Enterprise Management                              Storage Management      Frameworks                           ...
The Storage – ERM Interface  SMI-S      Systems Management Interface      for Storage      WBEM based      XML-CIM cimXML ...
SMI-S Reference Model
CIM Schema – Base Class Structure                                                                       CIM_ManagedElement...
CIM Schema     – ManagedSystem Element
CIM-XML Protocol  Representation of CIM in XML is defined as a DTD  (Document Type Definition)  CIM operations also encode...
CIM-XML Protocol    Operations  Operations can be instigated in either directions     Primarily from Client to Server     ...
CIM-XML ProtocolOperations  Intrinsic Metadata Operations     GetClass     DeleteClass     ModifyClass     EnumerateClasse...
CIM-XML ProtocolOperations  Intrinsic Instance Operations          AssociatorNames     GetInstance                        ...
Storage Management Initiative –Specification (SMI-S) Current version is 1.5    Draft of 1.6 standard is publicly available...
What do I need to know?    Storage Deployment Best Practice
Storage Deployment Planning Know all the requirements beforehand –     Details of all disk spaces needed     Hardware deta...
Block Storage DeploymentPlanning                       List of Apps, Type of Apps                    Disk Space Needs Toda...
Block Storage DeploymentPlanning Existing Cold Space            Replication/Mirroring                  New Storage        ...
Block Storage DeploymentPlanning    Inside the Array…      Segregate like-minded applications on separate disk arrays/pool...
Block Storage DeploymentPlanning    Inside the Array…      Standardize on 2 or 3 sizes of LUNs        – Smaller LUNs for b...
Block Storage DeploymentPlanning   At the Host level…      Set the optimum HBA driver settings        – SAN Topology      ...
Planning for StorageVirtualization    If Storage Virtualization is involved…       Understand the new storage terminology ...
Planning for SSDs   If you plan to have SSDs in the environment      Make sure (evaluate) that the specs meet your workloa...
References  CDMI    http://snia.org/cloud     http://www.snia.org/tech_activities/standards/curr_standards/cdmi/  SMI-S   ...
Summary Cloud Storage is available today Standards are in place for future-proof implementation SNIA is contributing to th...
Upcoming SlideShare
Loading in...5
×

SNIA Tutorials:"Practicalities of Cloud Deployment, Storage Management for cloud by Mario Vosschimdt (SNIA Europe)

984

Published on

Seminar by by Mario Vosschimdt during Infosecurity.be 2011

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
984
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
44
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "SNIA Tutorials:"Practicalities of Cloud Deployment, Storage Management for cloud by Mario Vosschimdt (SNIA Europe)"

  1. 1. Storage Management GOES HERE PRESENTATION TITLE for the Cloud Mario Vosschmidt, SNIA Europe Director, LSI
  2. 2. SNIA Europe Legal Notice The material contained in this presentation is copyrighted by SNIA Europe. Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations. Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the authors personal opinion and current understanding of the issues involved. The author, the presenter, and the SNIA Europe do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK.
  3. 3. Agenda Cloud computing overview Cloud Motivation Cloud Computing overview Consolidation and Cloud computing Technology solutions The management challenge Data management Storage management Summary
  4. 4. Overview Cloud computing overview Business model Deliver Resources – computing, data etc. Delivery on demand Elasticity Pay as you go Infrastructure as a Service (IaaS) Technology drivers Storage virtualization Server virtualization Ubiquitous Networking Resource consolidation and sharing
  5. 5. Cloud computing trendsSource: Hosting Transformation Summit, Dan Golding Keynote, Sept. 2009
  6. 6. The IDC View Storage grows from 9% of the total Cloud Services spend to 14% by 2013 Storage is projected to grow from $1.6 billion to $6.2 billion Storage has highest growth in shareSource: IDC Sept. 2009
  7. 7. Usage of Cloud Storage today Elastic demand for web based media (video, eBooks, audio) Backup to the cloud Internet “Drive” secondary storage Sync of files to the cloud and multiple devices Archive to the cloud Including Compliance, Retention and eDiscovery Storage for Cloud Computing Cloud Application storage
  8. 8. Cloud Data Storage ModelStorageManagement Cloud Data Cloud Data Storage Storage Data Client ManagementAccessMethodsPROVIDER CONSUMER
  9. 9. SNIA CDMI Cloud Data Management Initiative Launched at Fall SNW 2009 Press release listing charter members Cloud Pavilion on show floor Supporting the development and adoption of CDMI, Cloud Storage Marketing, Outreach, Education on Cloud Storage Requirements gathering Premier Organization promoting Cloud Storage and associated Standards
  10. 10. What is Cloud Storage? The use of the term cloud in describing these new models arose from architecture drawings that typically used a cloud as the dominant networking icon. The cloud conceptually represented any to any connectivity in a network, but also an abstraction of concerns such the actual connectivity and the services running in the network that accomplish that connectivity with little manual intervention. 10
  11. 11. A look at some existing Cloud APIs What are some of the offerings and their Data Storage Interfaces? *)*) CRUD – create, retrieve, update, delete 11
  12. 12. Storage IndustryResource Domain ModelAll of these interfaces support some or all of this model. The key to retainingthe simplicity of the cloud, however, is in the use of metadata to drive theunderlying services so that users need not manage the services themselves. 12
  13. 13. Cloud Storage Container• Cloud Storage may be used similar to a volume/filesystem• DSI Protocols include: WebDAV, NFS, CIFS, iSCSI, OSD, others• Existing Management interfaces: SMI-S, Proprietary, Web UI• Billing based on allocated space, Data Requirement (DR) parameters• Resource guarantee (desired and required), consumption• Configuration of DR is an object oriented hierarchy from containers on down to individual data elements 13
  14. 14. The Complete Picture
  15. 15. Cloud Data ManagementInterfaceApplicable to three types of Cloud Storage: Cloud Storage for Cloud Computing Whitepaper at snia.org/cloud – the management interface for the lifecycle of storage in a compute cloud Public Storage Cloud Whitepaper at snia.org/cloud – both a Data Path for the Cloud and a Management Path for the Cloud Data Private Cloud Storage As well as hybrid clouds An API for Storage Vendors selling into Cloud based solutionsSemantics Simple Containers and Data Objects with tagged Metadata Data System Metadata expresses the data requirementsProtocol RESTful *) HTTP as “core” interface style JSON (JavaScript Object Notation)– format of the representations are extensible*) Representational State Transfer, Roy Fielding 2000 15
  16. 16. REST Representation State Transfer Started with Dissertation by Roy Fielding outlining the principles Addressability Every object (resource) is addressable through a unique identifier Uniform, Constrained Interface Use only HTTP verbs and model other semantics in the data model Allows for Familiarity (low learning curve), Interoperability and Scalability Representation Oriented Complexity is in the representations Communicate Statelessly No persistent client-server connections 16
  17. 17. Why a RESTful approach for aCloud Storage Standard Simplicity Rules! Common Infrastructures in many Languages on many Platforms Low learning curve leads to developer adoption Developer adoption creates eco-system around API Eco-system eases adoption by vendors and customers 17
  18. 18. Cloud Peering 18
  19. 19. Work beyond SNIA Standardizing Cloud computing interfaces SNIA : CDMI OCCI: Open Grid Forum (OGF) Open Cloud computing interface working group Open standard APIs for cloud computing Vendor neutral Non-Proprietary OCCI Reference Resource oriented Architecture (ROA) Resources identified by URI *) HTML and other representations Atom/Pub (Atom Publishing Protocol) , JSON, Plain Text*) URI Universal Resource Identifier
  20. 20. Resource Management OCCI URI – IaaS Resource alignement Operations on resources (start, stop, delete, update)
  21. 21. OCCI APIOCCI API Implementation CRUD – create, retrieve, update and delete Mapped to http post, get, put, delete Metadata Exposed via HTTP headers Associations between resources covered Native expression in ATOM OCCI working group coordination with IETF OCCI capabilities Definition, creation, deplyoment, operation and retirement of services Infrastructure lifecycle management
  22. 22. OCCI – CDMI Integration
  23. 23. What is missing? Authentication Security Authorization Collaboration Data Tiering Metadata Data Publishing QoS Services Sharing IT Operations Provision Backup/Recover File Transfer Manage Archive/Compliance Files Services Charge ERM/SRM Folders Provision Containers Move Locking Billing
  24. 24. THE PROVIDER’S VIEW of Storage Management
  25. 25. The Provider‘s view IT Operations Service Levels (SLA) Equipment / Software cost Acquisition License Operational cost Administration, Power/cool, Building, Maintenance Deployment and Implementation ERM (Enterprise Resource Management) Operational stability Optimisation of Administration Vendor Independence Usage assessment and trend/prediction – Act instead of re-act
  26. 26. Enterprise Resource Management ERM attributes Analyse Quantify and Qualify Report Extract and predict, Visualise Monitor Threshold and Alert SNIA SMI lab End-to-End Migrate Availability Management, Capacity Management Costing and Charging Finance Management Performance
  27. 27. SMI-S Use Enterprise Management Storage Management Frameworks Applications CIM/WBEM (XML over HTTP) SMI-S Instrumentation SNIA Array HBA API Switch SNIA-SMI Provider Provider Provider Provider FC Switches FC HBAs Disk Arrays
  28. 28. The Storage – ERM Interface SMI-S Systems Management Interface for Storage WBEM based XML-CIM cimXML SMI-S is an Interface Specification Implemented as provider Client applications connect to the providers Several higher level clients/servers Python PyCIM, PyWBEM Pegasus, YAWN
  29. 29. SMI-S Reference Model
  30. 30. CIM Schema – Base Class Structure CIM_ManagedElementCIM_Capabilities CIM_Configuration CIM_Location CIM_MethodParameters CIM_Setting CIM_StatisticalInformation CIM_SupportAccess CIM_Collection CIM_FRU CIM_ManagedSystemElement CIM_Product CIM_SettingData CIM_StatisticalData CIM_PhysicalElement CIM_LogicalElement CIM_Job CIM_SoftwareIdentity CIM_EnabledLogicalElement
  31. 31. CIM Schema – ManagedSystem Element
  32. 32. CIM-XML Protocol Representation of CIM in XML is defined as a DTD (Document Type Definition) CIM operations also encoded using XML CIM command transport accomplished using HTTP operations POST M-POST
  33. 33. CIM-XML Protocol Operations Operations can be instigated in either directions Primarily from Client to Server Export operations are instigated by Server Used for Event Notification Operations can be partitioned into two categories Metadata operations Instance operations Intrinsic Operations Extrinsic Operations (Method Invocation) Operations come in two varieties Simple requests Multi requests
  34. 34. CIM-XML ProtocolOperations Intrinsic Metadata Operations GetClass DeleteClass ModifyClass EnumerateClasses EnumerateClassNames GetQualifier SetQualifier DeleteQualifier EnumerateQualifiers
  35. 35. CIM-XML ProtocolOperations Intrinsic Instance Operations AssociatorNames GetInstance References – all associations that refer to a given instance CreateInstance ReferenceNames DeleteInstance ModifyInstance Additional Operations EnumerateInstances MethodCall EnumerateInstanceNames ExecuteQuery Associators – all instances ExportIndication associated to a given instance GetProperty ExportAsyncResponse SetProperty
  36. 36. Storage Management Initiative –Specification (SMI-S) Current version is 1.5 Draft of 1.6 standard is publicly available http://www.snia.org/publicreview/ Standard is published as nine books: Overview Part 1 – Common Architecture Part 2 – Common Profiles Part 3 – Block Devices Part 4 – File Systems Part 5 – Fabric Part 6 – Host Elements Part 7 – Information Lifecycle Part 8 – Media Libraries SMI-S – CIM Management frameworks are commercially available today
  37. 37. What do I need to know? Storage Deployment Best Practice
  38. 38. Storage Deployment Planning Know all the requirements beforehand – Details of all disk spaces needed Hardware details of all storage, SAN and host resources available GROWTH in databases for next 1 to 2 yrs List of all applications, filesystems, databases, corresponding hosts Availability requirements, SLAs, DR (replication, clustering) Performance requirements for each fs, DB, and application etc. Processes needing storage team involvement such as application upgrades, refreshes, patching, etc needing snapshots, or backup/restores Don’t forget impact of Disk/Tape backup, de-duplication Tiered storage, SSD, Virtualization at Host and Storage level, Caching, Compression – usage of these technologies and their expected benefits/impacts should be well understood 38
  39. 39. Block Storage DeploymentPlanning List of Apps, Type of Apps Disk Space Needs Today + Next 1-2 yrs + The Surprise Factor! Space Reserve for App/DB Space Reserve or Loss for Processes (Upgrades, DB Snapshots, Short-stroking, Max Refresh cycles, etc) Disk Utilization Limit Vendor Disk Layout Internal or Vendor IOPs Needs Recommendations Array (HSP), Disk (format), File system, RAID Overheads 39
  40. 40. Block Storage DeploymentPlanning Existing Cold Space Replication/Mirroring New Storage Net Cold Space to Purchase Internal and Vendor Vendor Recommendations Performance Best Practices Specific to Storage Array (Stripe size, cache, etc) Configure Storage Array(s) Note Virtualization, SSDs, Automated Tiered Storage can change the above process 40
  41. 41. Block Storage DeploymentPlanning Inside the Array… Segregate like-minded applications on separate disk arrays/pools Set array and LUN level cache settings. Check with Vendor first Review array maintenance related background activities settings RAID Level – Review DB, Storage vendor best practices, and consult your DBAs – General Rule of Thumb - Choose RAID-10 over RAID-5 for write- heavy usage – Consider usage of RAID-5 Vs RAID-10 for table spaces, transaction logs, archive logs, Indexes, Temp space, Sort space, etc – For # of spindles, understand the IOPs requirements for reads/writes as well as the size of the IO Keep in mind the IOPs needs for backups + transactions Pick a suitable segment size based on each app/DB needs Can you stripe on top of a group of LUNs? 41
  42. 42. Block Storage DeploymentPlanning Inside the Array… Standardize on 2 or 3 sizes of LUNs – Smaller LUNs for binaries, OS, Swap, etc – Larger luns for Database files Maintain a balance of ownership of luns among the controllers Allocate LUNs from different RAID-sets (i.e., spread the IO) If you’re not short-stroking, then keep utilization below 80% For Redirect-on-write operation, make SURE the disks you allocate for the deltas are configured same as the original LUNs For Copy-on-write operation, make sure the disks used for Deltas are separate spindles, and not shared with original disks Standardize on Host/Host Group naming style (match it with zones) Check out SNIA Tutorial: Storage Performance 101 42
  43. 43. Block Storage DeploymentPlanning At the Host level… Set the optimum HBA driver settings – SAN Topology – Queue Depth – FiberChannel Speed Test path failover – break the path in every possible combination – Controller failure, just pull it out! – Manual LUN failover to alternate controller – Block port on the switch/director – Unplug fiber cable Know if LUN names are visible by the OS, it helps Make sure LUNs are ingested correctly – i.e., a LOG lun is not configured as a database lun Know the limits – Max # of luns, paths, LUN Zero requirement 43
  44. 44. Planning for StorageVirtualization If Storage Virtualization is involved… Understand the new storage terminology against existing – LUN/Volume/Slice/Partition/Stripe/Pool/Reserved_pool/disk_group, etc, etc… Compatibility check will be more complex, but do it For new install, pick one OS platform and test all features/functions Test how you’ll virtualize existing data/storage Test volume expansion/shrinking Document physical to logical (NPIV) mappings of the WWNs – do this end-to-end Test path failover – break the path in every possible combination Design several storage ‘pools’ with different characteristics – RAID-5, RAID-10, etc – Based on number of underlying disks Check out SNIA Tutorial: – Based on disk type/capacity Five Best Practices In – Based on disk groups having hot spares Virtualization – Based on array specs/type 44
  45. 45. Planning for SSDs If you plan to have SSDs in the environment Make sure (evaluate) that the specs meet your workload/workflow – Refer to the SNIA SSS Performance Test Suite Specification Know the costs for YOUR platform of choice, compared to disks – Refer to the SNIA Enterprise TCO Calculator Know your data! What to place on the SSDs What policies, processes and tools will you employ to – – Put data on SSDs (active and/or performance-critical) – Move data out of SSDs (to Tier1 or SATA) – What suits you? Move data at LUN level, file level or block level? Size the controller/CPU for # of SSD and Disk drives in the array Make sure you factor in the “write amplification” in your tests Know the “Erase Block Size” and align the partitions accordingly Make sure you can measure/quantify the expected performance gains 45
  46. 46. References CDMI http://snia.org/cloud http://www.snia.org/tech_activities/standards/curr_standards/cdmi/ SMI-S SMI Specification http://www.snia.org/tech_activities/standards/curr_standards/smi/ Conformance Testing Program http://www.snia.org/ctp/ SMI-S Developers Group http://groups.google.com/group/smi-s-developers-group SMI-Lab Program http://www.snia.org/forums/smi/tech_programs/lab_program/ SMI-S Central http://www.snia.org/members/smis/ OCCI http://occi-wg.org/
  47. 47. Summary Cloud Storage is available today Standards are in place for future-proof implementation SNIA is contributing to the standardisation Future enhancements are charted Please visit the SNIA Europe website for more information: www.snia-europe.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×