Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
491
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MapR NFS 7/6/2012© 2012 MapR Technologies Storage Service Architecture 1
  • 2. MapR NFS Agenda • Before NFS • Exercise: Using hadoop fs • MapR NFS Overview • NFS Mounting Models • Exercise: Mount the Cluster to Itself and Copy a File© 2012 MapR Technologies Storage Service Architecture 2
  • 3. MapR NFS Objectives At the end of this module you will be able to: • Describe the tools that are used with HDFS and their limitations • List the features and benefits of MapR Direct Access NFS • Identify several use cases for NFS on Hadoop • Describe the different models for mounting NFS volumes with MapR© 2012 MapR Technologies Storage Service Architecture 3
  • 4. Before NFS© 2012 MapR Technologies Storage Service Architecture 4
  • 5. Before NFS  Getting work done without MapR NFS – Example: mining web logs for meaningful data • First: get the logs into the cluster – Do you stop writing to the logs? – How long does it take? – What do you do in the meantime? hadoop fs -put, copyfromlocal distcp Flume© 2012 MapR Technologies Storage Service Architecture 5
  • 6. Before NFS  Getting work done without MapR NFS – Example: mining web logs for meaningful data • Next: getting answers from the data – What tools do you use to access data in your cluster? – Who is using these tools? hadoop fs copytolocal© 2012 MapR Technologies Storage Service Architecture 6
  • 7. Exercise: hadoop fs© 2012 MapR Technologies Storage Service Architecture 7
  • 8. MapR NFS Overview© 2012 MapR Technologies Storage Service Architecture 8
  • 9. Introducing Direct Access NFS  Drag and drop  Open files and apps  Watch live data come in  Fully read/write  Multiple concurrent reads/writes  Easy to get data in/out  Same namespace paths as Hadoop  Mount with -o tcp,ver=3,nolock© 2012 MapR Technologies Storage Service Architecture 9
  • 10. Getting Data into the Cluster with NFS  Simply mount your cluster filesystem locally – /mapr/my.cluster.com/myvolume/…  Redirect (or copy) logs to cluster filesystem log data© 2012 MapR Technologies Storage Service Architecture 10
  • 11. Using NFS with Clients  Mount your cluster filesystem locally – /mapr/my.cluster.com/myvolume/…  Access all of the data in your cluster with standard applications – Double-click a file (spreadsheet, report, etc.) • edit, save, etc. – Run a perl script – grep, sed, awk, shell scripts, etc.  Use Hadoop apps (Hive, Pig, Hbase, etc.)© 2012 MapR Technologies Storage Service Architecture 11
  • 12. Using NFS with Clients  Data is constantly changing  How do I view the data at a point in time? – Easy: take a snapshot – cd to the snapshot volume – View the data of interest – Note: overhead for taking a snapshot: none to minimal • The only write overhead is the data that changed – Recall snapshot architecture© 2012 MapR Technologies Storage Service Architecture 12
  • 13. Using NFS with Legacy Applications  What if I have legacy applications that work with my data? – Log analysis, reporting tools, etc.  Mount the cluster where your tools run – Take a snapshot if necessary – Then use your app just as before© 2012 MapR Technologies Storage Service Architecture 13
  • 14. NFS Mounting Models© 2012 MapR Technologies Storage Service Architecture 14
  • 15. NFS Mounting Models  Export to the world – NFS gateway runs on selected gateway hosts  Local server – NFS server runs on local host – Enables local compression and check summing  Export to self – NFS gateway runs on all data nodes, mounted from localhost© 2012 MapR Technologies Storage Service Architecture 15
  • 16. Export to the World NFS NFS Server NFS Server NFS Server NFS Server Client© 2012 MapR Technologies Storage Service Architecture 16
  • 17. Local Server Application NFS Server Client Cluster Nodes© 2012 MapR Technologies Storage Service Architecture 17
  • 18. Universal Export to Self Cluster Nodes Application NFS Cluster Server Node© 2012 MapR Technologies Storage Service Architecture 18
  • 19. Nodes are Identical Application Application NFS NFS Cluster Server Node Cluster Server Node Application NFS Cluster Server Node© 2012 MapR Technologies Storage Service Architecture 19
  • 20. Exercise: Mount the Cluster to Itself and Copy a File© 2012 MapR Technologies Storage Service Architecture 20
  • 21. Questions© 2012 MapR Technologies Storage Service Architecture 21