50a volumes

761 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
761
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

50a volumes

  1. 1. Working with Volumes 7/6/2012© 2012 MapR Technologies Volumes 1
  2. 2. Working with Volumes Agenda • Volumes • Exercise: Volumes • Snapshots • Exercise: Snapshots & Schedules • Mirrors • Exercise: Mirrors & Schedules • Best Practices© 2012 MapR Technologies Volumes 2
  3. 3. Working with Volumes Objectives At the end of this module you will be able to: • Explain how volumes relate to the MapR storage architecture • Describe a typical volume layout • Create, modify, move and set permissions on a volume • Explain how MapR snapshots work and how they are different from other types of snapshots • Create a snapshot and a snapshot schedule • Describe mirror volumes • Create a mirror volume and a mirror sync schedule© 2012 MapR Technologies Volumes 3
  4. 4. Volumes© 2012 MapR Technologies Volumes 4
  5. 5. What is a Volume?  Logical unit of storage with policy for: – Replication factor – Ownership – Data protection – Data placement  Contains directories and files  Made of containers  Unique to the MapR distribution Volumes help you manage data© 2012 MapR Technologies Volumes 5
  6. 6. Volumes Let You Manage Data  Backup and load balancing – All or just a portion  Point-in-time recovery  Data placement  Ownership/Permissions on Volumes© 2012 MapR Technologies Volumes 6
  7. 7. Storage Architecture  Nodes  Disks  Storage Pools  Containers – Distributed across cluster – 16(default)-32 GB© 2012 MapR Technologies Volumes 7
  8. 8. Replication 1 1 2 3 2 1 4 1 2 2 4 3 3 1 2 3 4 4 4 5 6 7 8 9© 2012 MapR Technologies Volumes 8
  9. 9. Typical Volume Layout / /binaries /hbase /projects /users /var/mapr /build /test /mjones /jsmith local... Recommendation: set up a volume per user© 2012 MapR Technologies Volumes 9
  10. 10. Applying Policy  Volume permissions  Quota  Topology  Replication Factor  Snapshots  Schedules  Mirrors© 2012 MapR Technologies Volumes 10
  11. 11. Permissions  Delegate all or a subset – dump – Dump the volume – restore – Mirror or restore the volume – m – Modify properties, create/delete snapshots – d – Delete the volume – fc – Full control (admin and ACL change) Volume permissions—not file and directory permissions© 2012 MapR Technologies Volumes 11
  12. 12. Quota  Volume Quotas: – Hard quota – Advisory quota Limited also by per-user quotas© 2012 MapR Technologies Volumes 12
  13. 13. Topology / /rack1  Physical topology describes cluster layout /rack1 /rack2 /rack3 – Generally corresponds to racks (or power drops)  Volume topology specifies what physical topology the volume uses – Expressed as a path – Limits placement of data Topology is a label© 2012 MapR Technologies Volumes 13
  14. 14. Replication Factor  Desired – Re-replicate after configurable timeout  Minimum – Re-replicate immediately Determined by data protection needs© 2012 MapR Technologies Volumes 14
  15. 15. Exercise: Volumes© 2012 MapR Technologies Volumes 15
  16. 16. Creating a Volume maprcli volume create -name name -mount 1 -path path Name and mount path can be different© 2012 MapR Technologies Volumes 16
  17. 17. Setting Permissions maprcli acl edit -name <name> -type volume -user <user> Use acl edit not acl set© 2012 MapR Technologies Volumes 17
  18. 18. Modifying a Volume maprcli volume modify -name name -quota quota© 2012 MapR Technologies Volumes 18
  19. 19. Moving a Volume maprcli volume move -name name -topology path© 2012 MapR Technologies Volumes 19
  20. 20. Snapshots© 2012 MapR Technologies Volumes 20
  21. 21. What is a Snapshot?  Point-in-time image of a volume  Guards against error replication  No initial space penalty  In streaming data, no space penalty at all!  Nearly instantaneous Supports Recovery Point Objective (RPO)© 2012 MapR Technologies Volumes 21
  22. 22. Snapshot Architecture Snapshot Current View of redirected New writes Snapshot Previous area the Data Original to write Continuing toVolume Writing© 2012 MapR Technologies Volumes 22
  23. 23. Where Do Snapshots Go?  The .snapshot directory – Top level of every volume – Does not appear in ls (to prevent recursion) – Same path via NFS or hadoop shell© 2012 MapR Technologies Volumes 23
  24. 24. Schedule  Collection of rules  Makes things happen  Applied to a volume: – Normal: snapshot – Mirror: sync  Specifies how long to keep snapshots© 2012 MapR Technologies Volumes 24
  25. 25. Exercise: Snapshots & Schedules© 2012 MapR Technologies Volumes 25
  26. 26. Creating a Snapshot maprcli volume snapshot create -volume name -snapshotname name© 2012 MapR Technologies Volumes 26
  27. 27. Creating a Schedule maprcli schedule create -schedule {"name":"Schedule-1", "rules": [{"frequency":"weekly", "date":"sun", "time":"7”, "retain":"2w"}]}© 2012 MapR Technologies Volumes 27
  28. 28. Applying a Schedule maprcli schedule list maprcli volume modify -name name -schedule ID© 2012 MapR Technologies Volumes 28
  29. 29. Mirrors© 2012 MapR Technologies Volumes 29
  30. 30. What is a Mirror?  A full, read-only copy of a volume  Use cases: – Remote backup – Research/production – Load balancing – Cascading mirrors to distribute data Can be mounted or not…© 2012 MapR Technologies Volumes 30
  31. 31. Navigating Mirrors mapr.cluster.root // /’/’ Volume “a” at /a X/a/a a’ /a’ Mirror (mounted or not) /b /b b’ /b’ /c /c c’ /c’ X/d /d d’ /d’ d’’ /d’’© 2012 MapR Technologies Volumes 31
  32. 32. Writing to a Mirror Anyway  The .rw directory .rw – Top level of the cluster – Does not appear in ls (to prevent recursion) – Same path via NFS or hadoop shell© 2012 MapR Technologies Volumes 32
  33. 33. Exercise: Mirrors & Schedules© 2012 MapR Technologies Volumes 33
  34. 34. Creating a Mirror maprcli volume create -name name -source name -type 1 -schedule ID© 2012 MapR Technologies Volumes 34
  35. 35. Syncing a Mirror Two ways to sync: maprcli volume mirror start -name mirror maprcli volume mirror push -name source …and of course you can start the mirror in the MapR Control System too.© 2012 MapR Technologies Volumes 35
  36. 36. Best Practices© 2012 MapR Technologies Volumes 36
  37. 37. Best Practices  Set up a /data topology and put all nodes and volumes in it  Set up a /decommisioning topology with nothing in it – Moving nodes to /decommissioning migrates data off when needed  If desired, set up a /cldb topology to isolate CLDB data  Set up a volume per user or project© 2012 MapR Technologies Volumes 37
  38. 38. Questions© 2012 MapR Technologies Volumes 38

×