Your SlideShare is downloading. ×
0
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
52 nfs
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

52 nfs

535

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
535
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. MapR NFS 7/6/2012© 2012 MapR Technologies Storage Service Architecture 1
  • 2. MapR NFS Agenda • Before NFS • Exercise: Using hadoop fs • MapR NFS Overview • NFS Mounting Models • Exercise: Mount the Cluster to Itself and Copy a File© 2012 MapR Technologies Storage Service Architecture 2
  • 3. MapR NFS Objectives At the end of this module you will be able to: • Describe the tools that are used with HDFS and their limitations • List the features and benefits of MapR Direct Access NFS • Identify several use cases for NFS on Hadoop • Describe the different models for mounting NFS volumes with MapR© 2012 MapR Technologies Storage Service Architecture 3
  • 4. Before NFS© 2012 MapR Technologies Storage Service Architecture 4
  • 5. Before NFS  Getting work done without MapR NFS – Example: mining web logs for meaningful data • First: get the logs into the cluster – Do you stop writing to the logs? – How long does it take? – What do you do in the meantime? hadoop fs -put, copyfromlocal distcp Flume© 2012 MapR Technologies Storage Service Architecture 5
  • 6. Before NFS  Getting work done without MapR NFS – Example: mining web logs for meaningful data • Next: getting answers from the data – What tools do you use to access data in your cluster? – Who is using these tools? hadoop fs copytolocal© 2012 MapR Technologies Storage Service Architecture 6
  • 7. Exercise: hadoop fs© 2012 MapR Technologies Storage Service Architecture 7
  • 8. MapR NFS Overview© 2012 MapR Technologies Storage Service Architecture 8
  • 9. Introducing Direct Access NFS  Drag and drop  Open files and apps  Watch live data come in  Fully read/write  Multiple concurrent reads/writes  Easy to get data in/out  Same namespace paths as Hadoop  Mount with -o tcp,ver=3,nolock© 2012 MapR Technologies Storage Service Architecture 9
  • 10. Getting Data into the Cluster with NFS  Simply mount your cluster filesystem locally – /mapr/my.cluster.com/myvolume/…  Redirect (or copy) logs to cluster filesystem log data© 2012 MapR Technologies Storage Service Architecture 10
  • 11. Using NFS with Clients  Mount your cluster filesystem locally – /mapr/my.cluster.com/myvolume/…  Access all of the data in your cluster with standard applications – Double-click a file (spreadsheet, report, etc.) • edit, save, etc. – Run a perl script – grep, sed, awk, shell scripts, etc.  Use Hadoop apps (Hive, Pig, Hbase, etc.)© 2012 MapR Technologies Storage Service Architecture 11
  • 12. Using NFS with Clients  Data is constantly changing  How do I view the data at a point in time? – Easy: take a snapshot – cd to the snapshot volume – View the data of interest – Note: overhead for taking a snapshot: none to minimal • The only write overhead is the data that changed – Recall snapshot architecture© 2012 MapR Technologies Storage Service Architecture 12
  • 13. Using NFS with Legacy Applications  What if I have legacy applications that work with my data? – Log analysis, reporting tools, etc.  Mount the cluster where your tools run – Take a snapshot if necessary – Then use your app just as before© 2012 MapR Technologies Storage Service Architecture 13
  • 14. NFS Mounting Models© 2012 MapR Technologies Storage Service Architecture 14
  • 15. NFS Mounting Models  Export to the world – NFS gateway runs on selected gateway hosts  Local server – NFS server runs on local host – Enables local compression and check summing  Export to self – NFS gateway runs on all data nodes, mounted from localhost© 2012 MapR Technologies Storage Service Architecture 15
  • 16. Export to the World NFS NFS Server NFS Server NFS Server NFS Server Client© 2012 MapR Technologies Storage Service Architecture 16
  • 17. Local Server Application NFS Server Client Cluster Nodes© 2012 MapR Technologies Storage Service Architecture 17
  • 18. Universal Export to Self Cluster Nodes Application NFS Cluster Server Node© 2012 MapR Technologies Storage Service Architecture 18
  • 19. Nodes are Identical Application Application NFS NFS Cluster Server Node Cluster Server Node Application NFS Cluster Server Node© 2012 MapR Technologies Storage Service Architecture 19
  • 20. Exercise: Mount the Cluster to Itself and Copy a File© 2012 MapR Technologies Storage Service Architecture 20
  • 21. Questions© 2012 MapR Technologies Storage Service Architecture 21

×