Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Leases and-caching final

341 views

Published on

Leases and-caching final

Published in: Technology
  • Be the first to comment

Leases and-caching final

  1. 1. Soumya Koduri Poornima G Lease Locks and Caching Gluster design Summit May. 2015
  2. 2. 05/17/16 2GlusterFS Meetup Agenda ● Quick intro to Lease locks ● Why implement in Gluster ● Design ● Network Partitions ● Challenges / limitations ● Future enhancements ● Client side caching with leases
  3. 3. 05/17/16 3GlusterFS Meetup ● Locking mechanism, which guarentees the client, certain semantics with respect to the sharing of the file/directory with other clients. ● Granted on regular files (file/directory). ● Protocol for cache consistency and aggressive caching. ● Enhances performance. Quick intro to Lease locks
  4. 4. 05/17/16 4GlusterFS Meetup Quick intro to Lease locks Client 1 Server  Request for  lease on file1 file1 file1  Granted, cache file1 Client 2   Request for  access to file1   Lease Break file1 Flush all the  cached data   Flush file1, Unlock lease 6.Granted access to file1 1 2 6 4 3 5
  5. 5. 05/17/16 5GlusterFS Meetup Why support it in Gluster ● NFS-Ganesha/Samba applications ● Multi-protocol access on Gluster ● A new client side caching xlator based on leases – File data – Directory entries (+ readdirplus) – Metadata
  6. 6. 05/17/16 6GlusterFS Meetup Lease locks and its manifestations ● Linux lease locks: - Read lease - Write lease - Process based ● NFS V4 delegations: - Read delegations - Write delegations - Directory delegations - Client based
  7. 7. 05/17/16 7GlusterFS Meetup Lease locks and its manifestations ● SMB - Oplocks (opurtunistic locking) Level 1 (exclusive lock) Level 2 (Read lock) Batch (handle caching + exclusive lock) Fd based Leases introduced in SMB 2.1 Read Read/Handle Read/Write Read/Write/handles Client based
  8. 8. 05/17/16 8GlusterFS Meetup On Gluster ● Currently supported - Read Lease - Write Lease - fd based ● Plan is to support a super set of all standards: - Handle Lease, Read handle, Read Write Handle Lease - Lease Key support i.e. client based. - Lease upgrade/downgrade. - Directory Lease
  9. 9. 05/17/16 9GlusterFS Meetup Design Gluster Server Locks xlator Lease xlator libgfapi Protocol clnt Protocol server glfs_adv_lock() Lk Request Lk Response Recall Lease Uses Upcall Infra - Stores the lease info - Check conflict,grant/ Recall lease - Timer based recall
  10. 10. 05/17/16 10GlusterFS Meetup Design ● New API introduced in libgfapi to request lease locks. int glfs_*advanced*_lock (); ● Lease xlator on server side above Posix locks: - Maintains the lease state of a file - Checks for conflicts, Grants, Recall, lease locks ● Recall lease is implemented using Gluster file system notifications. ● Timer based recall
  11. 11. 05/17/16 11GlusterFS Meetup Network Parition ● Gluster Server disconnects: ➢ Replica brick will serve the lease state. ➢ Non-replica - the lease state is lost forever. ● Gluster Client disconnects: ➢ All the locks held by this client will be unlocked. ● Gluster Server/Client Reconnects: ➢ The client will replay all the lease locks it has held.
  12. 12. 05/17/16 12GlusterFS Meetup Challenges / Limitations ● Rebalance/Tiering ● Self-Heal ● Client lock replay not enabled. ● Inconsistency when used along with performance xlators ● Delegation throttling ● Tracking non fd based fops and the actual client associated
  13. 13. 05/17/16 13GlusterFS Meetup Future Enhancements ● Super set of all leases ● Upgrade and downgrade leases ● Heuristics based lease grant – Conflict frequency – Limit on leases granted – Quota limits ● Migrate/heal lease state ● Locks recovery during network disconnects
  14. 14. 05/17/16 14GlusterFS Meetup Client side caching
  15. 15. 05/17/16 15GlusterFS Meetup Common challenges of caching ● Challenges with caching ➢ Mostly in memory hence less aggressive ➢ Not distributed cache coherent ➢ Performance hit with cache-validation requests ➢ Not tolerant with file-system notification failures
  16. 16. 05/17/16 16GlusterFS Meetup Benifits of using leases ● Cache Coherency ● Aggressive persistent data caching ● Reduces application-level response time ● Suitable for distributed systems of larger scale with their larger ratio of processor speed to network delay ● Performance improvement in the workloads with little write-sharing
  17. 17. 05/17/16 17GlusterFS Meetup Benifits of using leases ● Directory leases will help in caching readdir and readdirplus coherently ● Well suited for home directory use cases ● Decreases file creation latency
  18. 18. 05/17/16 18GlusterFS Meetup Client cache based on leases Gluster Server Locks xlator Lease xlator Libgfapi/FUSE Protocol clnt Protocol server Open file1 Open + Lease req Lease Granted rsp Lease Cache Open Open+ lease Cache file1 if lease granted Gluster Client
  19. 19. 05/17/16 19GlusterFS Meetup Thank You!

×