MapR NFS  7/6/2012© 2012 MapR Technologies   Storage Service Architecture 1
MapR NFS   Agenda   • Before NFS   • Exercise: Using hadoop fs   • MapR NFS Overview   • NFS Mounting Models   • Exercise:...
MapR NFS   Objectives   At the end of this module you will be able to:   • Describe the tools that are used with HDFS and ...
Before NFS© 2012 MapR Technologies   Storage Service Architecture 4
Before NFS     Getting work done without MapR NFS      –   Example: mining web logs for meaningful data          •   Firs...
Before NFS     Getting work done without MapR NFS      –   Example: mining web logs for meaningful data          •   Next...
Exercise:                              hadoop fs© 2012 MapR Technologies   Storage Service Architecture 7
MapR NFS Overview© 2012 MapR Technologies   Storage Service Architecture 8
Introducing Direct Access NFS                                           Drag and drop                                    ...
Getting Data into the Cluster with NFS     Simply mount your cluster filesystem locally      –   /mapr/my.cluster.com/myv...
Using NFS with Clients     Mount your cluster filesystem locally      –   /mapr/my.cluster.com/myvolume/…     Access all...
Using NFS with Clients     Data is constantly changing     How do I view the data at a point in time?      –   Easy: tak...
Using NFS with Legacy Applications     What if I have legacy applications that work with my data?      –   Log analysis, ...
NFS Mounting Models© 2012 MapR Technologies   Storage Service Architecture 14
NFS Mounting Models     Export to the world      –   NFS gateway runs on selected gateway hosts     Local server      – ...
Export to the World                                        NFS                                          NFS               ...
Local Server                             Application                                     NFS                              ...
Universal Export to Self                                                               Cluster Nodes                      ...
Nodes are Identical                  Application                                                                     Appli...
Exercise:                       Mount the Cluster to                       Itself and Copy a File© 2012 MapR Technologies ...
Questions© 2012 MapR Technologies   Storage Service Architecture 21
Upcoming SlideShare
Loading in...5
×

52 nfs

547

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
547
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "52 nfs"

  1. 1. MapR NFS 7/6/2012© 2012 MapR Technologies Storage Service Architecture 1
  2. 2. MapR NFS Agenda • Before NFS • Exercise: Using hadoop fs • MapR NFS Overview • NFS Mounting Models • Exercise: Mount the Cluster to Itself and Copy a File© 2012 MapR Technologies Storage Service Architecture 2
  3. 3. MapR NFS Objectives At the end of this module you will be able to: • Describe the tools that are used with HDFS and their limitations • List the features and benefits of MapR Direct Access NFS • Identify several use cases for NFS on Hadoop • Describe the different models for mounting NFS volumes with MapR© 2012 MapR Technologies Storage Service Architecture 3
  4. 4. Before NFS© 2012 MapR Technologies Storage Service Architecture 4
  5. 5. Before NFS  Getting work done without MapR NFS – Example: mining web logs for meaningful data • First: get the logs into the cluster – Do you stop writing to the logs? – How long does it take? – What do you do in the meantime? hadoop fs -put, copyfromlocal distcp Flume© 2012 MapR Technologies Storage Service Architecture 5
  6. 6. Before NFS  Getting work done without MapR NFS – Example: mining web logs for meaningful data • Next: getting answers from the data – What tools do you use to access data in your cluster? – Who is using these tools? hadoop fs copytolocal© 2012 MapR Technologies Storage Service Architecture 6
  7. 7. Exercise: hadoop fs© 2012 MapR Technologies Storage Service Architecture 7
  8. 8. MapR NFS Overview© 2012 MapR Technologies Storage Service Architecture 8
  9. 9. Introducing Direct Access NFS  Drag and drop  Open files and apps  Watch live data come in  Fully read/write  Multiple concurrent reads/writes  Easy to get data in/out  Same namespace paths as Hadoop  Mount with -o tcp,ver=3,nolock© 2012 MapR Technologies Storage Service Architecture 9
  10. 10. Getting Data into the Cluster with NFS  Simply mount your cluster filesystem locally – /mapr/my.cluster.com/myvolume/…  Redirect (or copy) logs to cluster filesystem log data© 2012 MapR Technologies Storage Service Architecture 10
  11. 11. Using NFS with Clients  Mount your cluster filesystem locally – /mapr/my.cluster.com/myvolume/…  Access all of the data in your cluster with standard applications – Double-click a file (spreadsheet, report, etc.) • edit, save, etc. – Run a perl script – grep, sed, awk, shell scripts, etc.  Use Hadoop apps (Hive, Pig, Hbase, etc.)© 2012 MapR Technologies Storage Service Architecture 11
  12. 12. Using NFS with Clients  Data is constantly changing  How do I view the data at a point in time? – Easy: take a snapshot – cd to the snapshot volume – View the data of interest – Note: overhead for taking a snapshot: none to minimal • The only write overhead is the data that changed – Recall snapshot architecture© 2012 MapR Technologies Storage Service Architecture 12
  13. 13. Using NFS with Legacy Applications  What if I have legacy applications that work with my data? – Log analysis, reporting tools, etc.  Mount the cluster where your tools run – Take a snapshot if necessary – Then use your app just as before© 2012 MapR Technologies Storage Service Architecture 13
  14. 14. NFS Mounting Models© 2012 MapR Technologies Storage Service Architecture 14
  15. 15. NFS Mounting Models  Export to the world – NFS gateway runs on selected gateway hosts  Local server – NFS server runs on local host – Enables local compression and check summing  Export to self – NFS gateway runs on all data nodes, mounted from localhost© 2012 MapR Technologies Storage Service Architecture 15
  16. 16. Export to the World NFS NFS Server NFS Server NFS Server NFS Server Client© 2012 MapR Technologies Storage Service Architecture 16
  17. 17. Local Server Application NFS Server Client Cluster Nodes© 2012 MapR Technologies Storage Service Architecture 17
  18. 18. Universal Export to Self Cluster Nodes Application NFS Cluster Server Node© 2012 MapR Technologies Storage Service Architecture 18
  19. 19. Nodes are Identical Application Application NFS NFS Cluster Server Node Cluster Server Node Application NFS Cluster Server Node© 2012 MapR Technologies Storage Service Architecture 19
  20. 20. Exercise: Mount the Cluster to Itself and Copy a File© 2012 MapR Technologies Storage Service Architecture 20
  21. 21. Questions© 2012 MapR Technologies Storage Service Architecture 21

×