Uploaded byMiguel Araújo
Evaluating Data Freshness in Large Scale Replicated Databases
The document discusses the evaluation of data freshness in large-scale replicated databases, particularly focusing on MySQL replication mechanisms. It outlines different replication protocols, the impact of replication topology on scalability, and presents an experimental approach to measuring propagation delay and data freshness. Results indicate that delay in data freshness increases with workload, number of replicas, and hops, highlighting limitations in MySQL's scalability concerning data freshness.
![[INSIGHT OUT 2011] A23 database io performance measuring planning(alex)](https://cdn.slidesharecdn.com/ss_thumbnails/insightout2011a23databaseioperformancemeasuringplanningalex-111114181900-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Evaluating Data Freshness in Large Scale Replicated Databases
- 1. Evaluating Data Freshnessin Large Scale Replicated Databases Miguel Araújo and José Pereira Partially funded by project ReD - Resilient Database Clusters (PDTC / EIA-EIA / 109044 / 2008)
- 2.
- 3.
- 4. Introduction • Why replicate? • Replicating data improves fault-tolerance • Allows the construction of large scale systems • Improves performance
- 5. Introduction • Why replicate? • Replicating data improves fault-tolerance • Allows the construction of large scale systems • Improves performance • There are different replication protocols: • Lazy schemes use separate transactions for execution and propagation • Eager schemes distribute updates to replicas in the context of the original updating transaction
- 6. Introduction • Why replicate? • Replicating data improves fault-tolerance • Allows the construction of large scale systems • Improves performance • There are different replication protocols: • Lazy schemes use separate transactions for execution and propagation • Eager schemes distribute updates to replicas in the context of the original updating transaction • Most database management systems implement lazy master-slave replication The application has to deal with stale data in order to avoid data inconsistency among replicas
- 7. Background Replication Mechanism •Replication mechanism of MySQL: 1. The master records changes to its data in its binary log (these records are called binary log events) 2. The slave copies the master’s binary log events to its own log (relay log) 3. The slave replays the events in the relay log, applying the changes to its own data
- 8. Background Replication Topologies •MySQL in particular allows almost any configuration of master and slave, as long as each server has at most one master
- 9. Background Replication Topologies •MySQL in particular allows almost any configuration of master and slave, as long as each server has at most one master Master and multiple slaves
- 10. Background Replication Topologies •MySQL in particular allows almost any configuration of master and slave, as long as each server has at most one master Master and multiple slaves Chain
- 11.
- 12.
- 13. Background Replication Topologies The ring topology is the only one that allows update- everywhere replication (if the application does not found any conflicts) Ring
- 14. Background Data Freshness •But... These techniques do not provide data freshness guarantees • So, asynchronous replication leads to periods of time that copies of the same data diverge • MySQL replication is commonly known as being very fast • There have not been systematic efforts to characterize data freshness in MySQL
- 15.
- 16. Goal Assessing the impact of replication topology in MySQL, towards maximizing scalability and data freshness
- 17. Agenda Approach Experiments - Workload - Setting - Results Conclusions
- 18. Measuring Propagation Delay Approach • Our approach is based on using a centralized probe • It periodically query each of the replicas, thus discovering what has been the last update applied • By comparing such positions, it should be possible to discover the propagation delay There are however several challenges that have to be tackled to obtain correct results:
- 19.
- 20. Measuring Propagation Delay Approach • Measuring updates
- 21. Measuring Propagation Delay Approach • Measuring updates • Non-simultaneous probing Problem: cannot probe master and slave simultaneously, so readings are not comparable !"#$%& '(")% !*+,$*&
- 22. Measuring Propagation Delay Approach • Measuring updates Solution: interpolate with a line the time-log position values of the master and then compare each replica • Non-simultaneous probing value point with it. At the end, calculate the average Problem: cannot probe master and slave simultaneously, so readings are not comparable ! " ."/)&"#0,0"1 ! !"#$%& " ! #$%&'( " ! " ! '(")% " !)*)%$#,(- ")*)#'$+( !*+,$*& 20%(
- 23.
- 24. Measuring Propagation Delay Approach • Eliminating quiet periods Problem: sampling twice without updates erroneously biases the estimate !"#$%& '(")% !*+,$*&
- 25. Measuring Propagation Delay Approach • Eliminating quiet periods Solution: select periods where line segments from both replicas have a positive slope Problem: sampling twice without updates erroneously biases the estimate !"#$%& '(")% !*+,$*&
- 26. Measuring Propagation Delay Approach • Eliminating quiet periods
- 27. Measuring Propagation Delay Approach • Eliminating quiet periods • Dealing with variability
- 28. Measuring Propagation Delay Approach • Eliminating quiet periods • Dealing with variability Solution: consider a sufficient amount of samples and assume that each probe happens after half of the observed round-trip
- 29. Experiments Workload • Wehave chosen the workload model defined by TPC-C benchmark • TPC-C is composed mostly by update intensive transactions • The master server is almost entirely dedicated to update transactions even in a small scale experimental setting, mimicking what would happen in a very large scale MySQL setup • Each client is attached to a database server and produces a stream of transaction requests
- 30. Experiments Setting • Tworeplication schemes were installed and configured. A five machine topology of master and multiple slaves, and a five machine topology in daisy chain • All machines are connected through a LAN, and are named PD01 to PD06. Being PD01 the master instance, PD04 the remote machine in which interrogation client executed, and the others the slave instances Machines: HP’s - lab SO OS: Linux Ubuntu Server 2.6.31-14 ✓ 20 clients CPU: 2 Intel(R) Core(TM)2 CPU 6400 - 2.13GHz ✓ 80 clients ✓ 40 clients MEM: 1 GB ✓ 100 clients DBMS: MySQL 5.1.39 ✓ 60 clients Scale Factor(warehouses): 2 Duration: 20 minutes
- 31. Results • Results forthe different number of TPC-C clients on the master and multiple slaves topology PD02 PD03 PD05 PD06 11000 9900 8800 7700 Delay Average (µs) 6600 5500 4400 3300 2200 1100 0 20 40 60 80 100 Clients
- 32. Results • Results forthe different number of TPC-C clients on the chain topology PD02 PD03 PD05 PD06 32000 28800 25600 22400 Delay Average (µs) 19200 16000 12800 9600 6400 3200 0 20 40 60 80 100 Clients
- 33.
- 34. Conclusions We wereable to measure freshness accurately and with a realistic workload
- 35. Conclusions We wereable to measure freshness accurately and with a realistic workload Results show how much:
- 36. Conclusions We wereable to measure freshness accurately and with a realistic workload Results show how much: • Delay grows with workload • Delay grows with number of replicas attached • Delay grows with number of hops
- 37. Conclusions We wereable to measure freshness accurately and with a realistic workload Results show how much: • Delay grows with workload • Delay grows with number of replicas attached • Delay grows with number of hops The conclusion is that the scalability of MySQL using its replication mechanism is limited when data freshness is a concern
- 38. Questions? Thank you