SlideShare a Scribd company logo
Leases: An Efficient Fault Tolerant
Mechanism for Distributed File
Presented by Lingxiang Tai
Part I - Problem With Distributed
System Caching
Background
Caching is mechanism to improve the speed and
performance of the system by replicating copies of
frequently accessed data.
● Complexity of ensuring consistency
○ with reams of clients, one can modify data
without consent of the others, cause the stale
data reading for the others.
● Complication of communication and host failtures
(Non-Byzantine failure)
○ Message lost due to these failures
Part II- Lease
Lease
● General Meaning
○ A lease is a contract that give its holder specified right
over property for a limited period of time.
● Caching Meaning
○ A lease is promise by the server that it will not conduct
any writes to the datum covered by your lease before
asking you for consent or waiting for the lease’s expiry
a granted period of time
Read And Write under Lease
● Read
○ First access will get the copy and a term of lease(eg: 10 seconds)
○ Latter read (within 10s) use the cached copy
○ Latter read (beyond 10s) require server to refresh data and Lease
● Write (here only consider write through)
○ client send write to server directly.
○ server defer the request until all the files’ leaseholder granted
approvals or all these leases expired
client
(cache)
server
Read Write
Short Term Lease
● Main Prons
○ Deferring write request, server will wait for a failed client to grant approval,
short term lease expiring fast.
○ When a server crashed, recovery work should wait all those leases granted
before to expire, short term lease make make it quick.
○ Short term lease can minimize the overhead callback in False Sharing cases.
(False Sharing means a client holds a relative long lease, while it’s not currently
accessing it)
Long Term Lease
● Main Prons
○ better for the files accessed repeatedly and have relatively little
write-sharing.
○ Especially, for the read-only file, you can take the lease as a infinite value.
Part III- Lease Analysis
How to chose a lease term
● It is a trade off between minimizing the lease extension overhead versus
minimizing false sharing
● If we can build a function with lease term (variable X) and other target
Metric (variable Y), we can distinguish it much easier.
○ y = Y(x)
Performance parameters
● N number of clients
● R rate of reads for each client
● W rate of writes for each client
● mprop
propagation delay for a message
● mproc
time to process a message (send or receive)
● ε allowance for uncertainty in clocks
● ts
lease term at server
● tc
lease term at client
Server load
● define the Y1
as the observable server load
○ Y1
=
Average latency
● define the Y2
as the observable average latency
○ Y2
=
Average latency(under a delay of network)
● base on average latency,we added 100ms
to the network latency
Conclusion
● Lease have the benefit of increasing the ratio of clients to servers (by reducing the
consistency overhead), thereby reducing the cost (or improving the performance ) of
large-scale systems.
Part IV - Others
Options for lease management
● Separate the files apart according to file’s characteristics, with the different leases to
manage them (improve the both performance for R&W)
● Group a batch of files managed with a same lease (can reduce the number of lease, server
lease overload and so on)
● Let the server multicast the lease extension periodically to all clients, rather client’s
fetching. (reduce the traffic to contact with every client & reduce the server’s track work of
every leaseholder)
Fault Tolerance
● Client Failure
○ No impact to READ,short delay for other client WRITE
● Server Failure
○ Client with old lease can still be accessed to READ but lease extension is shortly
delayed;
○ WRITE operation will be shortly delayed untill server is recovered from crash.
● Physical Clock drift
○ see next sheet...
Physical Clock Drift
● Server clock advances
○ server may allow a WRITE before the term of
a lease held by a previous client has expired
at that client.
● Client clock slows
○ client may still use a lease which the server regards as having expired.
● Server clock slows
○ no inconsistency, but generate extra traffic(sever always gives clients a “expired
lease”)
● Client clock advances
○ no inconsistency, but generate extra traffic (sever always gives clients a “expired
lease”)
Server
Client
Reading Source
Thank You !

More Related Content

What's hot

Os unit 3
Os unit 3Os unit 3
Os unit 3
SandhyaTatekalva
 
Software Cost Estimation Techniques
Software Cost Estimation TechniquesSoftware Cost Estimation Techniques
Software Cost Estimation Techniques
MohamedIbrahim1408
 
Computer network physical layer
Computer network  physical layerComputer network  physical layer
Computer network physical layer
Sweta Kumari Barnwal
 
User interface design
User interface designUser interface design
User interface design
Naveen Sagayaselvaraj
 
Human computerinterface
Human computerinterfaceHuman computerinterface
Human computerinterface
Kumar Aryan
 
HCI : Activity 1
HCI : Activity 1 HCI : Activity 1
HCI : Activity 1
autamata4
 
XP vs Lean vs FDD
XP vs Lean vs FDDXP vs Lean vs FDD
XP vs Lean vs FDD
Suman Guha
 
CCS335 – CLOUD COMPUTING.pptx
CCS335 – CLOUD COMPUTING.pptxCCS335 – CLOUD COMPUTING.pptx
CCS335 – CLOUD COMPUTING.pptx
NiviV4
 
OS file systems
OS file systemsOS file systems
OS file systems
anand hd
 
Ubiquitous Computing
Ubiquitous ComputingUbiquitous Computing
Ubiquitous Computing
u065932
 
Inter Process Communication
Inter Process CommunicationInter Process Communication
Inter Process Communication
Adeel Rasheed
 
Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...
Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...
Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...
Maciek Jozwiak
 
Screen based controls in HCI
Screen based controls in HCIScreen based controls in HCI
Screen based controls in HCI
Venkateswara Rao V
 
SLA Agreement, types and Life Cycle
SLA Agreement, types and Life Cycle SLA Agreement, types and Life Cycle
SLA Agreement, types and Life Cycle
Dr Neelesh Jain
 
Unix memory management
Unix memory managementUnix memory management
Unix memory management
Tech_MX
 
What is Human Computer Interraction
What is Human Computer InterractionWhat is Human Computer Interraction
What is Human Computer Interraction
praeeth palliyaguru
 
Application Performance, Test and Monitoring
Application Performance, Test and MonitoringApplication Performance, Test and Monitoring
Application Performance, Test and Monitoring
Dony Riyanto
 
file sharing semantics by Umar Danjuma Maiwada
file sharing semantics by Umar Danjuma Maiwada file sharing semantics by Umar Danjuma Maiwada
file sharing semantics by Umar Danjuma Maiwada
umardanjumamaiwada
 
RAID Review
RAID ReviewRAID Review
Multimedia operating system
Multimedia operating systemMultimedia operating system
Multimedia operating system
Home
 

What's hot (20)

Os unit 3
Os unit 3Os unit 3
Os unit 3
 
Software Cost Estimation Techniques
Software Cost Estimation TechniquesSoftware Cost Estimation Techniques
Software Cost Estimation Techniques
 
Computer network physical layer
Computer network  physical layerComputer network  physical layer
Computer network physical layer
 
User interface design
User interface designUser interface design
User interface design
 
Human computerinterface
Human computerinterfaceHuman computerinterface
Human computerinterface
 
HCI : Activity 1
HCI : Activity 1 HCI : Activity 1
HCI : Activity 1
 
XP vs Lean vs FDD
XP vs Lean vs FDDXP vs Lean vs FDD
XP vs Lean vs FDD
 
CCS335 – CLOUD COMPUTING.pptx
CCS335 – CLOUD COMPUTING.pptxCCS335 – CLOUD COMPUTING.pptx
CCS335 – CLOUD COMPUTING.pptx
 
OS file systems
OS file systemsOS file systems
OS file systems
 
Ubiquitous Computing
Ubiquitous ComputingUbiquitous Computing
Ubiquitous Computing
 
Inter Process Communication
Inter Process CommunicationInter Process Communication
Inter Process Communication
 
Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...
Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...
Google Spanner - Synchronously-Replicated, Globally-Distributed, Multi-Versio...
 
Screen based controls in HCI
Screen based controls in HCIScreen based controls in HCI
Screen based controls in HCI
 
SLA Agreement, types and Life Cycle
SLA Agreement, types and Life Cycle SLA Agreement, types and Life Cycle
SLA Agreement, types and Life Cycle
 
Unix memory management
Unix memory managementUnix memory management
Unix memory management
 
What is Human Computer Interraction
What is Human Computer InterractionWhat is Human Computer Interraction
What is Human Computer Interraction
 
Application Performance, Test and Monitoring
Application Performance, Test and MonitoringApplication Performance, Test and Monitoring
Application Performance, Test and Monitoring
 
file sharing semantics by Umar Danjuma Maiwada
file sharing semantics by Umar Danjuma Maiwada file sharing semantics by Umar Danjuma Maiwada
file sharing semantics by Umar Danjuma Maiwada
 
RAID Review
RAID ReviewRAID Review
RAID Review
 
Multimedia operating system
Multimedia operating systemMultimedia operating system
Multimedia operating system
 

Similar to [Paper Reading] Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency

Leases and-caching final
Leases and-caching finalLeases and-caching final
Leases and-caching final
Gluster.org
 
A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...
syeda yasmeen
 
ENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEM
ENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEMENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEM
ENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEM
Shakas Technologies
 
Performance and Cost Analysis of Modern Public Cloud Services
Performance and Cost Analysis of Modern Public Cloud ServicesPerformance and Cost Analysis of Modern Public Cloud Services
Performance and Cost Analysis of Modern Public Cloud Services
Md.Saiedur Rahaman
 
IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...
IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...
IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...
1crore projects
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com... A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
nexgentechnology
 
A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...
A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...
A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...
Nexgen Technology
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com... A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
nexgentechnology
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com... A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
nexgentechnology
 
A profit maximization scheme with guaranteed
A profit maximization scheme with guaranteedA profit maximization scheme with guaranteed
A profit maximization scheme with guaranteed
nexgentech15
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...
1crore projects
 
A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...
Pvrtechnologies Nellore
 
Scaling event aggregation at twitter
Scaling event aggregation at twitterScaling event aggregation at twitter
Scaling event aggregation at twitter
lohitvijayarenu
 
High-Speed Reactive Microservices
High-Speed Reactive MicroservicesHigh-Speed Reactive Microservices
High-Speed Reactive Microservices
Rick Hightower
 
J41035862
J41035862J41035862
J41035862
IJERA Editor
 
nfs.ppt
nfs.pptnfs.ppt
nfs.ppt
nfs.pptnfs.ppt
nfs.ppt
Shashank161
 
Cloud quiz question answer
Cloud quiz question answerCloud quiz question answer
Cloud quiz question answer
Lahore Garrison University
 
A Profit Maximization Scheme with Guaranteed1 (1)
A Profit Maximization Scheme with Guaranteed1 (1)A Profit Maximization Scheme with Guaranteed1 (1)
A Profit Maximization Scheme with Guaranteed1 (1)
nelampati ramanjaneyulu
 
Cloud hosting Provider in Pune India
Cloud hosting Provider in Pune IndiaCloud hosting Provider in Pune India
Cloud hosting Provider in Pune India
Swati Singh
 

Similar to [Paper Reading] Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency (20)

Leases and-caching final
Leases and-caching finalLeases and-caching final
Leases and-caching final
 
A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...
 
ENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEM
ENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEMENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEM
ENERGY-AWARE LOAD BALANCING AND APPLICATION SCALING FOR THE CLOUD ECOSYSTEM
 
Performance and Cost Analysis of Modern Public Cloud Services
Performance and Cost Analysis of Modern Public Cloud ServicesPerformance and Cost Analysis of Modern Public Cloud Services
Performance and Cost Analysis of Modern Public Cloud Services
 
IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...
IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...
IEEE 2015-2016 A Profit Maximization Scheme with Guaranteed Quality of Servic...
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com... A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 
A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...
A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...
A PROFIT MAXIMIZATION SCHEME WITH GUARANTEED QUALITY OF SERVICE IN CLOUD COMP...
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com... A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com... A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Com...
 
A profit maximization scheme with guaranteed
A profit maximization scheme with guaranteedA profit maximization scheme with guaranteed
A profit maximization scheme with guaranteed
 
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...
A Profit Maximization Scheme with Guaranteed Quality of Service in Cloud Comp...
 
A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...A profit maximization scheme with guaranteed quality of service in cloud comp...
A profit maximization scheme with guaranteed quality of service in cloud comp...
 
Scaling event aggregation at twitter
Scaling event aggregation at twitterScaling event aggregation at twitter
Scaling event aggregation at twitter
 
High-Speed Reactive Microservices
High-Speed Reactive MicroservicesHigh-Speed Reactive Microservices
High-Speed Reactive Microservices
 
J41035862
J41035862J41035862
J41035862
 
nfs.ppt
nfs.pptnfs.ppt
nfs.ppt
 
nfs.ppt
nfs.pptnfs.ppt
nfs.ppt
 
Cloud quiz question answer
Cloud quiz question answerCloud quiz question answer
Cloud quiz question answer
 
A Profit Maximization Scheme with Guaranteed1 (1)
A Profit Maximization Scheme with Guaranteed1 (1)A Profit Maximization Scheme with Guaranteed1 (1)
A Profit Maximization Scheme with Guaranteed1 (1)
 
Cloud hosting Provider in Pune India
Cloud hosting Provider in Pune IndiaCloud hosting Provider in Pune India
Cloud hosting Provider in Pune India
 

More from PingCAP

[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...
[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...
[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...
PingCAP
 
[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data
[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data
[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data
PingCAP
 
[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...
[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...
[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...
PingCAP
 
[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree
[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree
[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree
PingCAP
 
[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms
[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms
[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms
PingCAP
 
[Paper Reading] QAGen: Generating query-aware test databases
[Paper Reading] QAGen: Generating query-aware test databases[Paper Reading] QAGen: Generating query-aware test databases
[Paper Reading] QAGen: Generating query-aware test databases
PingCAP
 
[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...
[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...
[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...
PingCAP
 
[Paperreading] Paxos made easy (by sen han)
[Paperreading]  Paxos made easy (by sen han)[Paperreading]  Paxos made easy (by sen han)
[Paperreading] Paxos made easy (by sen han)
PingCAP
 
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
PingCAP
 
[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...
[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...
[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...
PingCAP
 
The Dark Side Of Go -- Go runtime related problems in TiDB in production
The Dark Side Of Go -- Go runtime related problems in TiDB  in productionThe Dark Side Of Go -- Go runtime related problems in TiDB  in production
The Dark Side Of Go -- Go runtime related problems in TiDB in production
PingCAP
 
TiDB DevCon 2020 Opening Keynote
TiDB DevCon 2020 Opening Keynote TiDB DevCon 2020 Opening Keynote
TiDB DevCon 2020 Opening Keynote
PingCAP
 
Finding Logic Bugs in Database Management Systems
Finding Logic Bugs in Database Management SystemsFinding Logic Bugs in Database Management Systems
Finding Logic Bugs in Database Management Systems
PingCAP
 
Chaos Practice in PingCAP
Chaos Practice in PingCAPChaos Practice in PingCAP
Chaos Practice in PingCAP
PingCAP
 
TiDB at PayPay
TiDB at PayPayTiDB at PayPay
TiDB at PayPay
PingCAP
 
Paper Reading: FPTree
Paper Reading: FPTreePaper Reading: FPTree
Paper Reading: FPTree
PingCAP
 
Paper Reading: Smooth Scan
Paper Reading: Smooth ScanPaper Reading: Smooth Scan
Paper Reading: Smooth Scan
PingCAP
 
Paper Reading: Flexible Paxos
Paper Reading: Flexible PaxosPaper Reading: Flexible Paxos
Paper Reading: Flexible Paxos
PingCAP
 
Paper reading: Cost-based Query Transformation in Oracle
Paper reading: Cost-based Query Transformation in OraclePaper reading: Cost-based Query Transformation in Oracle
Paper reading: Cost-based Query Transformation in Oracle
PingCAP
 
Paper reading: HashKV and beyond
Paper reading: HashKV and beyondPaper reading: HashKV and beyond
Paper reading: HashKV and beyond
PingCAP
 

More from PingCAP (20)

[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...
[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...
[Paper Reading] Efficient Query Processing with Optimistically Compressed Has...
 
[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data
[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data
[Paper Reading]Orca: A Modular Query Optimizer Architecture for Big Data
 
[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...
[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...
[Paper Reading]KVSSD: Close integration of LSM trees and flash translation la...
 
[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree
[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree
[Paper Reading]Chucky: A Succinct Cuckoo Filter for LSM-Tree
 
[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms
[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms
[Paper Reading]The Bw-Tree: A B-tree for New Hardware Platforms
 
[Paper Reading] QAGen: Generating query-aware test databases
[Paper Reading] QAGen: Generating query-aware test databases[Paper Reading] QAGen: Generating query-aware test databases
[Paper Reading] QAGen: Generating query-aware test databases
 
[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...
[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...
[Paper reading] Interleaving with Coroutines: A Practical Approach for Robust...
 
[Paperreading] Paxos made easy (by sen han)
[Paperreading]  Paxos made easy (by sen han)[Paperreading]  Paxos made easy (by sen han)
[Paperreading] Paxos made easy (by sen han)
 
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
[Paper Reading] Generalized Sub-Query Fusion for Eliminating Redundant I/O fr...
 
[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...
[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...
[Paper Reading] Steering Query Optimizers: A Practical Take on Big Data Workl...
 
The Dark Side Of Go -- Go runtime related problems in TiDB in production
The Dark Side Of Go -- Go runtime related problems in TiDB  in productionThe Dark Side Of Go -- Go runtime related problems in TiDB  in production
The Dark Side Of Go -- Go runtime related problems in TiDB in production
 
TiDB DevCon 2020 Opening Keynote
TiDB DevCon 2020 Opening Keynote TiDB DevCon 2020 Opening Keynote
TiDB DevCon 2020 Opening Keynote
 
Finding Logic Bugs in Database Management Systems
Finding Logic Bugs in Database Management SystemsFinding Logic Bugs in Database Management Systems
Finding Logic Bugs in Database Management Systems
 
Chaos Practice in PingCAP
Chaos Practice in PingCAPChaos Practice in PingCAP
Chaos Practice in PingCAP
 
TiDB at PayPay
TiDB at PayPayTiDB at PayPay
TiDB at PayPay
 
Paper Reading: FPTree
Paper Reading: FPTreePaper Reading: FPTree
Paper Reading: FPTree
 
Paper Reading: Smooth Scan
Paper Reading: Smooth ScanPaper Reading: Smooth Scan
Paper Reading: Smooth Scan
 
Paper Reading: Flexible Paxos
Paper Reading: Flexible PaxosPaper Reading: Flexible Paxos
Paper Reading: Flexible Paxos
 
Paper reading: Cost-based Query Transformation in Oracle
Paper reading: Cost-based Query Transformation in OraclePaper reading: Cost-based Query Transformation in Oracle
Paper reading: Cost-based Query Transformation in Oracle
 
Paper reading: HashKV and beyond
Paper reading: HashKV and beyondPaper reading: HashKV and beyond
Paper reading: HashKV and beyond
 

Recently uploaded

Series of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.pptSeries of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.ppt
PauloRodrigues104553
 
bank management system in java and mysql report1.pdf
bank management system in java and mysql report1.pdfbank management system in java and mysql report1.pdf
bank management system in java and mysql report1.pdf
Divyam548318
 
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
ihlasbinance2003
 
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdfBPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
MIGUELANGEL966976
 
New techniques for characterising damage in rock slopes.pdf
New techniques for characterising damage in rock slopes.pdfNew techniques for characterising damage in rock slopes.pdf
New techniques for characterising damage in rock slopes.pdf
wisnuprabawa3
 
digital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdfdigital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdf
drwaing
 
PPT on GRP pipes manufacturing and testing
PPT on GRP pipes manufacturing and testingPPT on GRP pipes manufacturing and testing
PPT on GRP pipes manufacturing and testing
anoopmanoharan2
 
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
obonagu
 
Swimming pool mechanical components design.pptx
Swimming pool  mechanical components design.pptxSwimming pool  mechanical components design.pptx
Swimming pool mechanical components design.pptx
yokeleetan1
 
DfMAy 2024 - key insights and contributions
DfMAy 2024 - key insights and contributionsDfMAy 2024 - key insights and contributions
DfMAy 2024 - key insights and contributions
gestioneergodomus
 
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODELDEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
gerogepatton
 
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
thanhdowork
 
Self-Control of Emotions by Slidesgo.pptx
Self-Control of Emotions by Slidesgo.pptxSelf-Control of Emotions by Slidesgo.pptx
Self-Control of Emotions by Slidesgo.pptx
iemerc2024
 
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTSHeap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Soumen Santra
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
obonagu
 
PROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.ppt
PROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.pptPROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.ppt
PROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.ppt
bhadouriyakaku
 
Modelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdfModelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdf
camseq
 
Literature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptxLiterature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptx
Dr Ramhari Poudyal
 
sieving analysis and results interpretation
sieving analysis and results interpretationsieving analysis and results interpretation
sieving analysis and results interpretation
ssuser36d3051
 
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressions
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressionsKuberTENes Birthday Bash Guadalajara - K8sGPT first impressions
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressions
Victor Morales
 

Recently uploaded (20)

Series of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.pptSeries of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.ppt
 
bank management system in java and mysql report1.pdf
bank management system in java and mysql report1.pdfbank management system in java and mysql report1.pdf
bank management system in java and mysql report1.pdf
 
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
 
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdfBPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
BPV-GUI-01-Guide-for-ASME-Review-Teams-(General)-10-10-2023.pdf
 
New techniques for characterising damage in rock slopes.pdf
New techniques for characterising damage in rock slopes.pdfNew techniques for characterising damage in rock slopes.pdf
New techniques for characterising damage in rock slopes.pdf
 
digital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdfdigital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdf
 
PPT on GRP pipes manufacturing and testing
PPT on GRP pipes manufacturing and testingPPT on GRP pipes manufacturing and testing
PPT on GRP pipes manufacturing and testing
 
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
在线办理(ANU毕业证书)澳洲国立大学毕业证录取通知书一模一样
 
Swimming pool mechanical components design.pptx
Swimming pool  mechanical components design.pptxSwimming pool  mechanical components design.pptx
Swimming pool mechanical components design.pptx
 
DfMAy 2024 - key insights and contributions
DfMAy 2024 - key insights and contributionsDfMAy 2024 - key insights and contributions
DfMAy 2024 - key insights and contributions
 
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODELDEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
DEEP LEARNING FOR SMART GRID INTRUSION DETECTION: A HYBRID CNN-LSTM-BASED MODEL
 
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
 
Self-Control of Emotions by Slidesgo.pptx
Self-Control of Emotions by Slidesgo.pptxSelf-Control of Emotions by Slidesgo.pptx
Self-Control of Emotions by Slidesgo.pptx
 
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTSHeap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
Heap Sort (SS).ppt FOR ENGINEERING GRADUATES, BCA, MCA, MTECH, BSC STUDENTS
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
 
PROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.ppt
PROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.pptPROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.ppt
PROJECT FORMAT FOR EVS AMITY UNIVERSITY GWALIOR.ppt
 
Modelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdfModelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdf
 
Literature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptxLiterature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptx
 
sieving analysis and results interpretation
sieving analysis and results interpretationsieving analysis and results interpretation
sieving analysis and results interpretation
 
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressions
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressionsKuberTENes Birthday Bash Guadalajara - K8sGPT first impressions
KuberTENes Birthday Bash Guadalajara - K8sGPT first impressions
 

[Paper Reading] Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency

  • 1. Leases: An Efficient Fault Tolerant Mechanism for Distributed File Presented by Lingxiang Tai
  • 2. Part I - Problem With Distributed System Caching
  • 3. Background Caching is mechanism to improve the speed and performance of the system by replicating copies of frequently accessed data. ● Complexity of ensuring consistency ○ with reams of clients, one can modify data without consent of the others, cause the stale data reading for the others. ● Complication of communication and host failtures (Non-Byzantine failure) ○ Message lost due to these failures
  • 5. Lease ● General Meaning ○ A lease is a contract that give its holder specified right over property for a limited period of time. ● Caching Meaning ○ A lease is promise by the server that it will not conduct any writes to the datum covered by your lease before asking you for consent or waiting for the lease’s expiry a granted period of time
  • 6. Read And Write under Lease ● Read ○ First access will get the copy and a term of lease(eg: 10 seconds) ○ Latter read (within 10s) use the cached copy ○ Latter read (beyond 10s) require server to refresh data and Lease ● Write (here only consider write through) ○ client send write to server directly. ○ server defer the request until all the files’ leaseholder granted approvals or all these leases expired client (cache) server Read Write
  • 7. Short Term Lease ● Main Prons ○ Deferring write request, server will wait for a failed client to grant approval, short term lease expiring fast. ○ When a server crashed, recovery work should wait all those leases granted before to expire, short term lease make make it quick. ○ Short term lease can minimize the overhead callback in False Sharing cases. (False Sharing means a client holds a relative long lease, while it’s not currently accessing it)
  • 8. Long Term Lease ● Main Prons ○ better for the files accessed repeatedly and have relatively little write-sharing. ○ Especially, for the read-only file, you can take the lease as a infinite value.
  • 9. Part III- Lease Analysis
  • 10. How to chose a lease term ● It is a trade off between minimizing the lease extension overhead versus minimizing false sharing ● If we can build a function with lease term (variable X) and other target Metric (variable Y), we can distinguish it much easier. ○ y = Y(x)
  • 11. Performance parameters ● N number of clients ● R rate of reads for each client ● W rate of writes for each client ● mprop propagation delay for a message ● mproc time to process a message (send or receive) ● ε allowance for uncertainty in clocks ● ts lease term at server ● tc lease term at client
  • 12. Server load ● define the Y1 as the observable server load ○ Y1 =
  • 13. Average latency ● define the Y2 as the observable average latency ○ Y2 =
  • 14. Average latency(under a delay of network) ● base on average latency,we added 100ms to the network latency
  • 15. Conclusion ● Lease have the benefit of increasing the ratio of clients to servers (by reducing the consistency overhead), thereby reducing the cost (or improving the performance ) of large-scale systems.
  • 16. Part IV - Others
  • 17. Options for lease management ● Separate the files apart according to file’s characteristics, with the different leases to manage them (improve the both performance for R&W) ● Group a batch of files managed with a same lease (can reduce the number of lease, server lease overload and so on) ● Let the server multicast the lease extension periodically to all clients, rather client’s fetching. (reduce the traffic to contact with every client & reduce the server’s track work of every leaseholder)
  • 18. Fault Tolerance ● Client Failure ○ No impact to READ,short delay for other client WRITE ● Server Failure ○ Client with old lease can still be accessed to READ but lease extension is shortly delayed; ○ WRITE operation will be shortly delayed untill server is recovered from crash. ● Physical Clock drift ○ see next sheet...
  • 19. Physical Clock Drift ● Server clock advances ○ server may allow a WRITE before the term of a lease held by a previous client has expired at that client. ● Client clock slows ○ client may still use a lease which the server regards as having expired. ● Server clock slows ○ no inconsistency, but generate extra traffic(sever always gives clients a “expired lease”) ● Client clock advances ○ no inconsistency, but generate extra traffic (sever always gives clients a “expired lease”) Server Client