SlideShare a Scribd company logo
1 of 20
OLTP
Benchmark
B Y: S A I K AT G A R A I
outline
Introduction And Architecture
Workload
Performance vs Cost Comparison
Evolving Workload Mixtures
Conclusion
Introduction
What is OLTP?
OLTP is an operational system that supports transaction-oriented applications in a 3-tier
architecture. It administers the day to day transaction of an organization. OLTP is basically focused
on query processing, maintaining data integrity in multi-access environments as well as
effectiveness that is measured by the total number of transactions per second. The full form of
OLTP is Online Transaction Processing.
What is benchmarking?
A Benchmark is the act of running a computer program, a set of programs, or other operations, in
order to assess the relative performance of an object, normally by running a number of standard
tests and trials against it.
Benchmark Model
OLTPArchitecture :
Requirements of OLTP :
Scalability: the ability to drive the system under test at high transactional rates (i.e., test for max
throughput);
Fine-Grained Rate Control: the ability to control the rate of requests with great precision;
Mixed and Evolving Workloads: the ability to support mixed workloads, and to change the rate,
composition, and access distribution of the workloads dynamically over time;
Synthetic and Real Data & Workloads: support for both synthetic and real data sets and workloads
(and thus both programmatically-generated and trace-based executions);
Ease of Deployment/Portability: the ability to deploy experimental settings and run experiments
easily on a variety of DBMSs and DBaaSs using an integrated framework;
Extensibility: the ability to extend the set of workloads and systems supported with a minimal
engineering effort (e.g., centralizing SQL dialect localizations);
Lightweight, Fine-Grained Statistics Gathering: the ability to collect detailed statistics of both client-
side activity and server-side resource utilization with minimum impact on the overall performance;
Workloads
AuctionMark
Epinions
JPAB
ResourceStresser
SEATS
TATP
TPC-C
Twitter
Wikipedia
YCSB
Profile info for the benchmark workloads
Performance v/s Cost Comparison
The OLTP-Bench is used to measure the performance-vs-cost ratio of a single
DBaaS provider
Using the YCSB and Wikipedia benchmarks, we used OLTP-Bench to deploy
databases on five different instance sizes on Amazon’s RDS platform
Performance v/s Cost Comparison
Performance v/s Cost Comparison
We then ran each benchmark separately at its maximum speed for a total of 30
minutes.
 During that time, OLTP-Bench calculates the average maximum sustained
throughput and the 95th percentile latency from the middle 20 minutes of the
experiment’s run.
Performance v/s Cost Comparison
Performance v/s Cost Comparison
As for the price/performance ratio, the results for YCSB, the L instance yields
the best cost/performance ratio, with good overall throughput and low latency.
As for the price/performance ratio, the results for Wikipedia suggest that the
XL-HM instance is the best choice for this workload. Although the 2XL-HM and
4XL-HM instances provide better performance, the additional cost incurred is
more .
Evolving Workload Mixtures
OLTP-Bench’s ability to smoothly evolve the transaction mixture during an
experiment.
Choosing YCSB as our target workload since it is composed of a series of
simple transactions.
Using MySQL we run the system.
Representation using charts
Comparing DBaaS Providers
Network latency is an important factor for OLTP workloads
In this example all of the workers on virtual machines hosted
in the same datacenter region
The test is done test to ensure fairness between the
DBaaS providers
Cost V/s Performance
Conclusion
Benchmarking has always been central for high-performance and efficient data
management system design, deployment and maintenance.
The advent of cloud-based data management solutions with their highly shared
resources, non-user controlled system tuning, and novel pricing schemes further
exacerbate the need for careful benchmarking.
Thank You

More Related Content

Similar to OLTP benchmark by Saikat Garai, Presidency college, Bangalore.

CS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docx
CS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docxCS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docx
CS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docx
faithxdunce63732
 
% Razones para alta disponibilidad
% Razones para alta disponibilidad% Razones para alta disponibilidad
% Razones para alta disponibilidad
CPVEN
 
VMware Report Draft v2.1
VMware Report Draft v2.1VMware Report Draft v2.1
VMware Report Draft v2.1
John White
 
Iaetsd appliances of harmonizing model in cloud
Iaetsd appliances of harmonizing model in cloudIaetsd appliances of harmonizing model in cloud
Iaetsd appliances of harmonizing model in cloud
Iaetsd Iaetsd
 
B2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draftB2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draft
Steve Feldman
 
A multi path routing algorithm for ip
A multi path routing algorithm for ipA multi path routing algorithm for ip
A multi path routing algorithm for ip
Alvianus Dengen
 

Similar to OLTP benchmark by Saikat Garai, Presidency college, Bangalore. (20)

2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
 
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
 
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
 
Building a new CTL model checker using Web Services
Building a new CTL model checker using Web ServicesBuilding a new CTL model checker using Web Services
Building a new CTL model checker using Web Services
 
CS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docx
CS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docxCS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docx
CS 301 Computer ArchitectureStudent # 1 EID 09Kingdom of .docx
 
% Razones para alta disponibilidad
% Razones para alta disponibilidad% Razones para alta disponibilidad
% Razones para alta disponibilidad
 
VMware Report Draft v2.1
VMware Report Draft v2.1VMware Report Draft v2.1
VMware Report Draft v2.1
 
Three Microsoft Azure SQL Managed Instances offered better SQL Server perform...
Three Microsoft Azure SQL Managed Instances offered better SQL Server perform...Three Microsoft Azure SQL Managed Instances offered better SQL Server perform...
Three Microsoft Azure SQL Managed Instances offered better SQL Server perform...
 
Iaetsd appliances of harmonizing model in cloud
Iaetsd appliances of harmonizing model in cloudIaetsd appliances of harmonizing model in cloud
Iaetsd appliances of harmonizing model in cloud
 
Data warehousing testing strategies cognos
Data warehousing testing strategies cognosData warehousing testing strategies cognos
Data warehousing testing strategies cognos
 
SPROJReport (1)
SPROJReport (1)SPROJReport (1)
SPROJReport (1)
 
Distributed Performance testing by funkload
Distributed Performance testing by funkloadDistributed Performance testing by funkload
Distributed Performance testing by funkload
 
Oracle databasecapacityanalysisusingstatisticalmethods
Oracle databasecapacityanalysisusingstatisticalmethodsOracle databasecapacityanalysisusingstatisticalmethods
Oracle databasecapacityanalysisusingstatisticalmethods
 
Traffic Simulator
Traffic SimulatorTraffic Simulator
Traffic Simulator
 
Workload-Aware Data Management in Shared-Nothing Distributed OLTP Databases
Workload-Aware Data Management in Shared-Nothing Distributed OLTP DatabasesWorkload-Aware Data Management in Shared-Nothing Distributed OLTP Databases
Workload-Aware Data Management in Shared-Nothing Distributed OLTP Databases
 
B2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draftB2 2005 introduction_load_testing_blackboard_primer_draft
B2 2005 introduction_load_testing_blackboard_primer_draft
 
Anton Serputko Workshop “Тестування продуктивності”
Anton Serputko Workshop “Тестування продуктивності”Anton Serputko Workshop “Тестування продуктивності”
Anton Serputko Workshop “Тестування продуктивності”
 
A multi path routing algorithm for ip
A multi path routing algorithm for ipA multi path routing algorithm for ip
A multi path routing algorithm for ip
 
Performance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei RadovPerformance testing in scope of migration to cloud by Serghei Radov
Performance testing in scope of migration to cloud by Serghei Radov
 
Mahout low-overhead datacenter traffic management using end-host-based ...
Mahout   low-overhead  datacenter  traffic management  using end-host-based  ...Mahout   low-overhead  datacenter  traffic management  using end-host-based  ...
Mahout low-overhead datacenter traffic management using end-host-based ...
 

Recently uploaded

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 

OLTP benchmark by Saikat Garai, Presidency college, Bangalore.

  • 1. OLTP Benchmark B Y: S A I K AT G A R A I
  • 2. outline Introduction And Architecture Workload Performance vs Cost Comparison Evolving Workload Mixtures Conclusion
  • 3. Introduction What is OLTP? OLTP is an operational system that supports transaction-oriented applications in a 3-tier architecture. It administers the day to day transaction of an organization. OLTP is basically focused on query processing, maintaining data integrity in multi-access environments as well as effectiveness that is measured by the total number of transactions per second. The full form of OLTP is Online Transaction Processing. What is benchmarking? A Benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it.
  • 7. Scalability: the ability to drive the system under test at high transactional rates (i.e., test for max throughput); Fine-Grained Rate Control: the ability to control the rate of requests with great precision; Mixed and Evolving Workloads: the ability to support mixed workloads, and to change the rate, composition, and access distribution of the workloads dynamically over time; Synthetic and Real Data & Workloads: support for both synthetic and real data sets and workloads (and thus both programmatically-generated and trace-based executions); Ease of Deployment/Portability: the ability to deploy experimental settings and run experiments easily on a variety of DBMSs and DBaaSs using an integrated framework; Extensibility: the ability to extend the set of workloads and systems supported with a minimal engineering effort (e.g., centralizing SQL dialect localizations); Lightweight, Fine-Grained Statistics Gathering: the ability to collect detailed statistics of both client- side activity and server-side resource utilization with minimum impact on the overall performance;
  • 9. Profile info for the benchmark workloads
  • 10. Performance v/s Cost Comparison The OLTP-Bench is used to measure the performance-vs-cost ratio of a single DBaaS provider Using the YCSB and Wikipedia benchmarks, we used OLTP-Bench to deploy databases on five different instance sizes on Amazon’s RDS platform
  • 11. Performance v/s Cost Comparison
  • 12. Performance v/s Cost Comparison We then ran each benchmark separately at its maximum speed for a total of 30 minutes.  During that time, OLTP-Bench calculates the average maximum sustained throughput and the 95th percentile latency from the middle 20 minutes of the experiment’s run.
  • 13. Performance v/s Cost Comparison
  • 14. Performance v/s Cost Comparison As for the price/performance ratio, the results for YCSB, the L instance yields the best cost/performance ratio, with good overall throughput and low latency. As for the price/performance ratio, the results for Wikipedia suggest that the XL-HM instance is the best choice for this workload. Although the 2XL-HM and 4XL-HM instances provide better performance, the additional cost incurred is more .
  • 15. Evolving Workload Mixtures OLTP-Bench’s ability to smoothly evolve the transaction mixture during an experiment. Choosing YCSB as our target workload since it is composed of a series of simple transactions. Using MySQL we run the system.
  • 17. Comparing DBaaS Providers Network latency is an important factor for OLTP workloads In this example all of the workers on virtual machines hosted in the same datacenter region The test is done test to ensure fairness between the DBaaS providers
  • 19. Conclusion Benchmarking has always been central for high-performance and efficient data management system design, deployment and maintenance. The advent of cloud-based data management solutions with their highly shared resources, non-user controlled system tuning, and novel pricing schemes further exacerbate the need for careful benchmarking.