50a volumes

Working with Volumes
7/6/2012

© 2012 MapR Technologies Volumes 1

Agenda
• Volumes
• Exercise: Volumes
• Snapshots
• Exercise: Snapshots & Schedules
• Mirrors
• Exercise: Mirrors & Schedules
• Best Practices


Objectives
At the end of this module you will be able to:
• Explain how volumes relate to the MapR storage architecture
• Describe a typical volume layout
• Create, modify, move and set permissions on a volume
• Explain how MapR snapshots work and how they are different from other
types of snapshots
• Create a snapshot and a snapshot schedule
• Describe mirror volumes
• Create a mirror volume and a mirror sync schedule


Volumes


What is a Volume?

 Logical unit of storage with
policy for:
– Replication factor
– Ownership
– Data protection
– Data placement
 Contains directories and files
 Made of containers
 Unique to the MapR distribution

Volumes help you manage data

Volumes Let You Manage Data

 Backup and load balancing
– All or just a portion
 Point-in-time recovery
 Data placement
 Ownership/Permissions on
Volumes


Storage Architecture

 Nodes
 Disks
 Storage Pools
 Containers
– Distributed across cluster
– 16(default)-32 GB


Replication

1

1 2

3 2

1 4

1 2

2 4

3
3
1 2 3
4 4
4 5 6
7 8 9


Typical Volume Layout

/

/binaries /hbase /projects /users /var/mapr

/build /test /mjones /jsmith local...

Recommendation: set up a volume per user

Applying Policy

 Volume permissions
 Quota
 Topology
 Replication Factor
 Snapshots
 Schedules
 Mirrors


Permissions

 Delegate all or a subset

– dump – Dump the volume
– restore – Mirror or restore the volume
– m – Modify properties, create/delete snapshots
– d – Delete the volume
– fc – Full control (admin and ACL change)

Volume permissions—not file and directory permissions


Quota

 Volume Quotas:
– Hard quota
– Advisory quota

Limited also by per-user quotas


Topology

/
/rack1
 Physical topology describes cluster
layout
/rack1 /rack2 /rack3
– Generally corresponds to racks
(or power drops)
 Volume topology specifies what
physical topology the volume uses
– Expressed as a path
– Limits placement of data

Topology is a label


Replication Factor

 Desired
– Re-replicate after configurable
timeout
 Minimum
– Re-replicate immediately

Determined by data protection needs


Exercise:
Volumes


Creating a Volume
maprcli volume create
-name name
-mount 1
-path path

Name and mount path can be different

Setting Permissions
maprcli acl edit
-name <name>
-type volume
-user <user>

Use acl edit not acl set


Modifying a Volume
maprcli volume modify
-name name
-quota quota


Moving a Volume
maprcli volume move
-name name
-topology path


Snapshots


What is a Snapshot?

 Point-in-time image of a volume
 Guards against error replication
 No initial space penalty
 In streaming data, no space
penalty at all!
 Nearly instantaneous

Supports Recovery Point Objective (RPO)


Snapshot Architecture

Snapshot
Current View of redirected
New writes Snapshot
Previous area the Data
Original to write
Continuing toVolume
Writing


Where Do Snapshots Go?

 The .snapshot directory
– Top level of every volume
– Does not appear in ls
(to prevent recursion)
– Same path via NFS or hadoop shell


Schedule

 Collection of rules
 Makes things happen
 Applied to a volume:
– Normal: snapshot
– Mirror: sync
 Specifies how long to keep
snapshots


Exercise:
Snapshots & Schedules


Creating a Snapshot

maprcli volume snapshot create
-volume name
-snapshotname name


Creating a Schedule
maprcli schedule create
-schedule
'{"name":"Schedule-1",
"rules":
[{"frequency":"weekly",
"date":"sun",
"time":"7”,
"retain":"2w"}]}'


Applying a Schedule
maprcli schedule list
maprcli volume modify
-name name
-schedule ID


Mirrors


What is a Mirror?

 A full, read-only copy of a volume
 Use cases:
– Remote backup
– Research/production
– Load balancing
– Cascading mirrors to distribute data

Can be mounted or not…

Navigating Mirrors

mapr.cluster.root // /’/’

Volume “a” at /a X/a/a a’
/a’ Mirror (mounted or not)

/b
/b b’
/b’

/c
/c c’
/c’

X/d /d d’
/d’ d’’
/d’’


Writing to a Mirror Anyway

 The .rw directory
.rw – Top level of the cluster
– Does not appear in ls
(to prevent recursion)
– Same path via NFS or hadoop shell


Exercise:
Mirrors & Schedules


Creating a Mirror
maprcli volume create
-name name
-source name
-type 1
-schedule ID


Syncing a Mirror
Two ways to sync: maprcli volume mirror start
-name mirror

maprcli volume mirror push
-name source

…and of course you can
start the mirror in the
MapR Control System
too.


Best Practices


Best Practices

 Set up a /data topology and put all nodes and volumes in it
 Set up a /decommisioning topology with nothing in it
– Moving nodes to /decommissioning migrates data off when needed
 If desired, set up a /cldb topology to isolate CLDB data
 Set up a volume per user or project


Questions


50a volumes

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 50a volumes

Similar to 50a volumes (20)

More from mapr-academy

More from mapr-academy (7)

Recently uploaded

Recently uploaded (20)

50a volumes