OLTP-Bench

eXascale Infolab
eXascale InfolabeXascale Infolab
OLTP-Bench Framework: An Extensible Testbed for 
3 September 2014 
VLDB14, Hangzhou, CHINA 
Benchmarking Relational Databases 
Djellel Eddine Difallah, Andy Pavlo, Carlo Curino, Philippe Cudré-Mauroux
How many researchers wasted 
time writing benchmarking 
infrastructure to run their 
experiments? 
Measuring Reproducibility in Computer Systems Research – Collberg et al. (20% success)
Experimental 
Research Process 
We continuously re-invent the 
wheel. 
Development 
Create the prototype 
Search for Benchmarks (ad-hoc workloads) 
Search for Datasets (synthetic data) 
Building a test toolkit 
Measure 
Monitor different resources 
Analysis 
Plot and make sense of the results
OLTP-Bench is an open-source 
“batteries-included” DBMS 
benchmarking testbed tailored 
for OLTP/Web workloads
OLTP-Bench 
16 Workloads 
Tested on 8 DBMSs 
Rich metrics 
Fine-grained Rate control 
Workload Mixture 
Extensible
OLTP-Bench Architecture 
● Tight and dynamic control on parallel load generation 
● Statistics gathering 
● SQL dialect handling 
Parallel Connections vs Performance
Current Workloads and Benchmarks 
Class Benchmark Application Domain 
Transactional 
AuctionMark 
CH-benCHmark 
SEATS 
TATP 
TPC-C 
Voter 
SmallBank 
On-line Auctions 
Mixture of OLTP and OLAP 
On-line Airline Ticketing 
Caller Location App 
Order Processing 
Talent Show Voting 
Banking System 
Web-Oriented 
Epinions 
LinkBench 
Twitter 
Wikipedia 
Social Networking 
Social Networking 
Social Networking 
On-line Encyclopedia 
Feature Testing 
ResourceStresser 
YCSB 
JPAB 
SIBench 
Isolated Resource Stresser 
Scalable Key-value Store 
Object-Relational Mapping 
Transactional Isolation 
JDBC Compliant DBMSs 
MS SQL Server 
Oracle 
MySQL 
Postgres 
DB2 
NuoDB 
SQLite 
Apache Derby 
MonetDB
Features 
showcase
Fine-Grained Rate 
Control 
MySQL running Wikipedia 
workload at increasing 
throughput. Demonstrating tight 
control of transactional 
throughput imposed on the 
system (and saturation)
Simulating Evolving 
Skew 
MySQL running Twitter with an 
evolving skew (alternating zipfian 
and uniform). Different skew 
imposes stress on different 
resources.
Multi-tenancy 
Comparing 3 relational databases under multi-tenancy resource contention. (No DBMS 
is perfect)
Changing Mixture over time 
MySQL running YCSB with a varying transaction mixture 
(highlighting various resource bottlenecks) 
Parallel Connections vs Performance
Future Work 
“We explicitly avoid to define benchmark rules, as they are (often) restrictive, 
arbitrary, biased, and not-future-proof” 
- Djellel, Andy, Carlo, Philippe 
The project received an NSF Grant!! 
Possible directions: 
● Better tooling (e.g. real-time visualization) 
● Synchronized distributed clients 
● Support for NoSQL 
● A public repository for experiments and results
Q&A 
https://github.com/oltpbenchmark/ 
External contributors and acknowledgements: 
Evan Jones, Barzan Mozfari, Dimitri Vorona, Ben Reilly, Yu Su, Adam Seering, Simon Krenger, Tommy Reilly, Mark Callaghan, Keving Grittner, David L. 
Day, Nik Lanham, Awajeet Arya, Flavio Sousa, Stein Petter Tronstad, Lance FAng, Rodrigo Felix de Almeida, Woonhak kang, Florian Funke, Ahmad 
Hassan, Ivo Jimenez,Mickaël Hoerdt, Dimokritos Stamatakis, Benjamin Reilly, Mark Callaghan, Zheng Da, Xi Chen, Jean de Lavarene, Ning Zhang, Erik 
Paulson, Alec Jindal, ADD YOUR NAME HERE
1 of 14

Recommended

Destination Sequenced Distance Vector Routing (DSDV) by
Destination Sequenced Distance Vector Routing (DSDV)Destination Sequenced Distance Vector Routing (DSDV)
Destination Sequenced Distance Vector Routing (DSDV)Arun Chokkalingam
2.3K views19 slides
Mac protocols for ad hoc wireless networks by
Mac protocols for ad hoc wireless networks Mac protocols for ad hoc wireless networks
Mac protocols for ad hoc wireless networks Divya Tiwari
18.1K views69 slides
Clustering: Large Databases in data mining by
Clustering: Large Databases in data miningClustering: Large Databases in data mining
Clustering: Large Databases in data miningZHAO Sam
8.1K views28 slides
WSN-Routing Protocols Energy Efficient Routing by
WSN-Routing Protocols Energy Efficient RoutingWSN-Routing Protocols Energy Efficient Routing
WSN-Routing Protocols Energy Efficient RoutingArun Chokkalingam
1.2K views25 slides
Birch by
BirchBirch
BirchBinod Malla
2.3K views12 slides

More Related Content

What's hot

Sensor Protocols for Information via Negotiation (SPIN) by
Sensor Protocols for Information via Negotiation (SPIN)Sensor Protocols for Information via Negotiation (SPIN)
Sensor Protocols for Information via Negotiation (SPIN)rajivagarwal23dei
8.5K views16 slides
Atm by
AtmAtm
AtmMohd Arif
2.3K views31 slides
Multicast routing protocols in adhoc networks by
Multicast routing protocols in adhoc networksMulticast routing protocols in adhoc networks
Multicast routing protocols in adhoc networksPradeep Kumar TS
5.3K views17 slides
Packet radio protocol by
Packet radio protocolPacket radio protocol
Packet radio protocolPriya Kaushal
10.8K views57 slides
Demand assigned and packet reservation multiple access by
Demand assigned and packet reservation multiple accessDemand assigned and packet reservation multiple access
Demand assigned and packet reservation multiple accessGowriLatha1
903 views10 slides
Wot by
WotWot
WotPRAVEENSRC2113003011
3.8K views34 slides

What's hot(20)

Sensor Protocols for Information via Negotiation (SPIN) by rajivagarwal23dei
Sensor Protocols for Information via Negotiation (SPIN)Sensor Protocols for Information via Negotiation (SPIN)
Sensor Protocols for Information via Negotiation (SPIN)
rajivagarwal23dei8.5K views
Multicast routing protocols in adhoc networks by Pradeep Kumar TS
Multicast routing protocols in adhoc networksMulticast routing protocols in adhoc networks
Multicast routing protocols in adhoc networks
Pradeep Kumar TS5.3K views
Packet radio protocol by Priya Kaushal
Packet radio protocolPacket radio protocol
Packet radio protocol
Priya Kaushal10.8K views
Demand assigned and packet reservation multiple access by GowriLatha1
Demand assigned and packet reservation multiple accessDemand assigned and packet reservation multiple access
Demand assigned and packet reservation multiple access
GowriLatha1903 views
Unit 1 architecture of distributed systems by karan2190
Unit 1 architecture of distributed systemsUnit 1 architecture of distributed systems
Unit 1 architecture of distributed systems
karan2190144.8K views
2. Distributed Systems Hardware & Software concepts by Prajakta Rane
2. Distributed Systems Hardware & Software concepts2. Distributed Systems Hardware & Software concepts
2. Distributed Systems Hardware & Software concepts
Prajakta Rane4.8K views
TCP over wireless slides by Mahesh Rajawat
TCP over wireless slidesTCP over wireless slides
TCP over wireless slides
Mahesh Rajawat12.9K views
IOT System Management with NETCONF-YANG.pptx by ArchanaPandiyan
IOT System Management with NETCONF-YANG.pptxIOT System Management with NETCONF-YANG.pptx
IOT System Management with NETCONF-YANG.pptx
ArchanaPandiyan2.7K views
mobile ad-hoc network (MANET) and its applications by Aman Gupta
mobile ad-hoc network (MANET) and its applicationsmobile ad-hoc network (MANET) and its applications
mobile ad-hoc network (MANET) and its applications
Aman Gupta6K views
Ip packet delivery by rajisri2
Ip packet deliveryIp packet delivery
Ip packet delivery
rajisri21.2K views
Classification of routing protocols by Menaga Selvaraj
Classification of routing protocolsClassification of routing protocols
Classification of routing protocols
Menaga Selvaraj2.6K views
Cloud Application architecture styles by Nilay Shrivastava
Cloud Application architecture styles Cloud Application architecture styles
Cloud Application architecture styles
Nilay Shrivastava2.8K views
Mac protocols of adhoc network by shashi712
Mac protocols of adhoc networkMac protocols of adhoc network
Mac protocols of adhoc network
shashi71223.4K views
WIRELESS TRANSMISSION by junnubabu
WIRELESS TRANSMISSIONWIRELESS TRANSMISSION
WIRELESS TRANSMISSION
junnubabu21.8K views

Viewers also liked

Simultaneous OLTP and OLAP in ERP by
Simultaneous OLTP and OLAP in ERPSimultaneous OLTP and OLAP in ERP
Simultaneous OLTP and OLAP in ERPChad Gates
1.9K views23 slides
OLAP v/s OLTP by
OLAP v/s OLTPOLAP v/s OLTP
OLAP v/s OLTPahsan irfan
8.9K views14 slides
Olap, oltp and data mining by
Olap, oltp and data miningOlap, oltp and data mining
Olap, oltp and data miningzafrii
7.7K views17 slides
OLTP vs OLAP by
OLTP vs OLAPOLTP vs OLAP
OLTP vs OLAPBI_Solutions
6.2K views8 slides
OLAP by
OLAPOLAP
OLAPAshir Ali
100K views22 slides
Oltp vs olap by
Oltp vs olapOltp vs olap
Oltp vs olapMr. Fmhyudin
61.6K views19 slides

Viewers also liked(8)

Simultaneous OLTP and OLAP in ERP by Chad Gates
Simultaneous OLTP and OLAP in ERPSimultaneous OLTP and OLAP in ERP
Simultaneous OLTP and OLAP in ERP
Chad Gates1.9K views
OLAP v/s OLTP by ahsan irfan
OLAP v/s OLTPOLAP v/s OLTP
OLAP v/s OLTP
ahsan irfan8.9K views
Olap, oltp and data mining by zafrii
Olap, oltp and data miningOlap, oltp and data mining
Olap, oltp and data mining
zafrii7.7K views
OLAP & DATA WAREHOUSE by Zalpa Rathod
OLAP & DATA WAREHOUSEOLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSE
Zalpa Rathod73.2K views

Similar to OLTP-Bench

The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015 by
The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015
The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015Ioan Toma
1.2K views36 slides
How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I... by
How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I...How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I...
How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I...DevOps.com
98 views20 slides
Workflow Provenance: From Modelling to Reporting by
Workflow Provenance: From Modelling to ReportingWorkflow Provenance: From Modelling to Reporting
Workflow Provenance: From Modelling to ReportingRayhan Ferdous
135 views30 slides
Ops Jumpstart: MongoDB Administration 101 by
Ops Jumpstart: MongoDB Administration 101Ops Jumpstart: MongoDB Administration 101
Ops Jumpstart: MongoDB Administration 101MongoDB
967 views29 slides
Advances in Scientific Workflow Environments by
Advances in Scientific Workflow EnvironmentsAdvances in Scientific Workflow Environments
Advances in Scientific Workflow EnvironmentsCarole Goble
1.1K views41 slides
No sq lv1_0 by
No sq lv1_0No sq lv1_0
No sq lv1_0Tuan Luong
285 views35 slides

Similar to OLTP-Bench(20)

The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015 by Ioan Toma
The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015
The LDBC Social Network Benchmark Interactive Workload - SIGMOD 2015
Ioan Toma1.2K views
How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I... by DevOps.com
How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I...How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I...
How the Automation of a Benchmark Famework Keeps Pace with the Dev Cycle at I...
DevOps.com98 views
Workflow Provenance: From Modelling to Reporting by Rayhan Ferdous
Workflow Provenance: From Modelling to ReportingWorkflow Provenance: From Modelling to Reporting
Workflow Provenance: From Modelling to Reporting
Rayhan Ferdous135 views
Ops Jumpstart: MongoDB Administration 101 by MongoDB
Ops Jumpstart: MongoDB Administration 101Ops Jumpstart: MongoDB Administration 101
Ops Jumpstart: MongoDB Administration 101
MongoDB967 views
Advances in Scientific Workflow Environments by Carole Goble
Advances in Scientific Workflow EnvironmentsAdvances in Scientific Workflow Environments
Advances in Scientific Workflow Environments
Carole Goble1.1K views
NoSQL Introduction, Theory, Implementations by Firat Atagun
NoSQL Introduction, Theory, ImplementationsNoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, Implementations
Firat Atagun1.8K views
AWS re:Invent 2016: Large-Scale, Cloud-Based Analysis of Cancer Genomes: Less... by Amazon Web Services
AWS re:Invent 2016: Large-Scale, Cloud-Based Analysis of Cancer Genomes: Less...AWS re:Invent 2016: Large-Scale, Cloud-Based Analysis of Cancer Genomes: Less...
AWS re:Invent 2016: Large-Scale, Cloud-Based Analysis of Cancer Genomes: Less...
Amazon Web Services2.1K views
An introduction to Workload Modelling for Cloud Applications by Ravi Yogesh
An introduction to Workload Modelling for Cloud ApplicationsAn introduction to Workload Modelling for Cloud Applications
An introduction to Workload Modelling for Cloud Applications
Ravi Yogesh1.6K views
WBDB 2014 Benchmarking Virtualized Hadoop Clusters by t_ivanov
WBDB 2014 Benchmarking Virtualized Hadoop ClustersWBDB 2014 Benchmarking Virtualized Hadoop Clusters
WBDB 2014 Benchmarking Virtualized Hadoop Clusters
t_ivanov640 views
performance_tuning.pdf by Alexadiaz52
performance_tuning.pdfperformance_tuning.pdf
performance_tuning.pdf
Alexadiaz522 views
performance_tuning.pdf by Alexadiaz52
performance_tuning.pdfperformance_tuning.pdf
performance_tuning.pdf
Alexadiaz5231 views
An Overview of VIEW by Shiyong Lu
An Overview of VIEWAn Overview of VIEW
An Overview of VIEW
Shiyong Lu614 views
The Impact of Columnar File Formats on SQL-on-Hadoop Engine Performance: A St... by t_ivanov
The Impact of Columnar File Formats on SQL-on-Hadoop Engine Performance: A St...The Impact of Columnar File Formats on SQL-on-Hadoop Engine Performance: A St...
The Impact of Columnar File Formats on SQL-on-Hadoop Engine Performance: A St...
t_ivanov385 views
MongoDB Days UK: Building an Enterprise Data Fabric at Royal Bank of Scotland... by MongoDB
MongoDB Days UK: Building an Enterprise Data Fabric at Royal Bank of Scotland...MongoDB Days UK: Building an Enterprise Data Fabric at Royal Bank of Scotland...
MongoDB Days UK: Building an Enterprise Data Fabric at Royal Bank of Scotland...
MongoDB7.8K views
Cjoin by blogboy
CjoinCjoin
Cjoin
blogboy1.2K views
Azure DocumentDb Training - Resource Model by Marco Parenzan
Azure DocumentDb Training - Resource ModelAzure DocumentDb Training - Resource Model
Azure DocumentDb Training - Resource Model
Marco Parenzan268 views
Kylin and Druid Presentation by argonauts007
Kylin and Druid PresentationKylin and Druid Presentation
Kylin and Druid Presentation
argonauts0076.6K views

More from eXascale Infolab

Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction by
Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link PredictionBeyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction
Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link PredictioneXascale Infolab
287 views30 slides
It Takes Two: Instrumenting the Interaction between In-Memory Databases and S... by
It Takes Two: Instrumenting the Interaction between In-Memory Databases and S...It Takes Two: Instrumenting the Interaction between In-Memory Databases and S...
It Takes Two: Instrumenting the Interaction between In-Memory Databases and S...eXascale Infolab
167 views16 slides
Representation Learning on Complex Graphs by
Representation Learning on Complex GraphsRepresentation Learning on Complex Graphs
Representation Learning on Complex GraphseXascale Infolab
539 views33 slides
A force directed approach for offline gps trajectory map by
A force directed approach for offline gps trajectory mapA force directed approach for offline gps trajectory map
A force directed approach for offline gps trajectory mapeXascale Infolab
459 views12 slides
Cikm 2018 by
Cikm 2018Cikm 2018
Cikm 2018eXascale Infolab
871 views18 slides
HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit... by
HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit...HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit...
HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit...eXascale Infolab
787 views20 slides

More from eXascale Infolab(20)

Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction by eXascale Infolab
Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link PredictionBeyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction
Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction
eXascale Infolab287 views
It Takes Two: Instrumenting the Interaction between In-Memory Databases and S... by eXascale Infolab
It Takes Two: Instrumenting the Interaction between In-Memory Databases and S...It Takes Two: Instrumenting the Interaction between In-Memory Databases and S...
It Takes Two: Instrumenting the Interaction between In-Memory Databases and S...
eXascale Infolab167 views
Representation Learning on Complex Graphs by eXascale Infolab
Representation Learning on Complex GraphsRepresentation Learning on Complex Graphs
Representation Learning on Complex Graphs
eXascale Infolab539 views
A force directed approach for offline gps trajectory map by eXascale Infolab
A force directed approach for offline gps trajectory mapA force directed approach for offline gps trajectory map
A force directed approach for offline gps trajectory map
eXascale Infolab459 views
HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit... by eXascale Infolab
HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit...HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit...
HistoSketch: Fast Similarity-Preserving Sketching of Streaming Histograms wit...
eXascale Infolab787 views
SwissLink: High-Precision, Context-Free Entity Linking Exploiting Unambiguous... by eXascale Infolab
SwissLink: High-Precision, Context-Free Entity Linking Exploiting Unambiguous...SwissLink: High-Precision, Context-Free Entity Linking Exploiting Unambiguous...
SwissLink: High-Precision, Context-Free Entity Linking Exploiting Unambiguous...
eXascale Infolab1.2K views
Dependency-Driven Analytics: A Compass for Uncharted Data Oceans by eXascale Infolab
Dependency-Driven Analytics: A Compass for Uncharted Data OceansDependency-Driven Analytics: A Compass for Uncharted Data Oceans
Dependency-Driven Analytics: A Compass for Uncharted Data Oceans
eXascale Infolab687 views
SANAPHOR: Ontology-based Coreference Resolution by eXascale Infolab
SANAPHOR: Ontology-based Coreference ResolutionSANAPHOR: Ontology-based Coreference Resolution
SANAPHOR: Ontology-based Coreference Resolution
eXascale Infolab1.1K views
Efficient, Scalable, and Provenance-Aware Management of Linked Data by eXascale Infolab
Efficient, Scalable, and Provenance-Aware Management of Linked DataEfficient, Scalable, and Provenance-Aware Management of Linked Data
Efficient, Scalable, and Provenance-Aware Management of Linked Data
eXascale Infolab713 views
LDOW2015 - Uduvudu: a Graph-Aware and Adaptive UI Engine for Linked Data by eXascale Infolab
LDOW2015 - Uduvudu: a Graph-Aware and Adaptive UI Engine for Linked DataLDOW2015 - Uduvudu: a Graph-Aware and Adaptive UI Engine for Linked Data
LDOW2015 - Uduvudu: a Graph-Aware and Adaptive UI Engine for Linked Data
eXascale Infolab4K views
Executing Provenance-Enabled Queries over Web Data by eXascale Infolab
Executing Provenance-Enabled Queries over Web DataExecuting Provenance-Enabled Queries over Web Data
Executing Provenance-Enabled Queries over Web Data
eXascale Infolab1.5K views
The Dynamics of Micro-Task Crowdsourcing by eXascale Infolab
The Dynamics of Micro-Task CrowdsourcingThe Dynamics of Micro-Task Crowdsourcing
The Dynamics of Micro-Task Crowdsourcing
eXascale Infolab1.6K views
Fixing the Domain and Range of Properties in Linked Data by Context Disambigu... by eXascale Infolab
Fixing the Domain and Range of Properties in Linked Data by Context Disambigu...Fixing the Domain and Range of Properties in Linked Data by Context Disambigu...
Fixing the Domain and Range of Properties in Linked Data by Context Disambigu...
eXascale Infolab3.1K views
CIKM14: Fixing grammatical errors by preposition ranking by eXascale Infolab
CIKM14: Fixing grammatical errors by preposition rankingCIKM14: Fixing grammatical errors by preposition ranking
CIKM14: Fixing grammatical errors by preposition ranking
eXascale Infolab1.7K views
Internet Infrastructures for Big Data (Verisign's Distinguished Speaker Series) by eXascale Infolab
Internet Infrastructures for Big Data (Verisign's Distinguished Speaker Series)Internet Infrastructures for Big Data (Verisign's Distinguished Speaker Series)
Internet Infrastructures for Big Data (Verisign's Distinguished Speaker Series)
eXascale Infolab663 views

Recently uploaded

Advanced API Mocking Techniques by
Advanced API Mocking TechniquesAdvanced API Mocking Techniques
Advanced API Mocking TechniquesDimpy Adhikary
19 views11 slides
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t... by
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...Deltares
9 views26 slides
MariaDB stored procedures and why they should be improved by
MariaDB stored procedures and why they should be improvedMariaDB stored procedures and why they should be improved
MariaDB stored procedures and why they should be improvedFederico Razzoli
8 views32 slides
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ... by
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...Donato Onofri
795 views34 slides
ict act 1.pptx by
ict act 1.pptxict act 1.pptx
ict act 1.pptxsanjaniarun08
13 views17 slides
AI and Ml presentation .pptx by
AI and Ml presentation .pptxAI and Ml presentation .pptx
AI and Ml presentation .pptxFayazAli87
11 views15 slides

Recently uploaded(20)

Advanced API Mocking Techniques by Dimpy Adhikary
Advanced API Mocking TechniquesAdvanced API Mocking Techniques
Advanced API Mocking Techniques
Dimpy Adhikary19 views
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t... by Deltares
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...
Deltares9 views
MariaDB stored procedures and why they should be improved by Federico Razzoli
MariaDB stored procedures and why they should be improvedMariaDB stored procedures and why they should be improved
MariaDB stored procedures and why they should be improved
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ... by Donato Onofri
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...
Donato Onofri795 views
AI and Ml presentation .pptx by FayazAli87
AI and Ml presentation .pptxAI and Ml presentation .pptx
AI and Ml presentation .pptx
FayazAli8711 views
Dev-Cloud Conference 2023 - Continuous Deployment Showdown: Traditionelles CI... by Marc Müller
Dev-Cloud Conference 2023 - Continuous Deployment Showdown: Traditionelles CI...Dev-Cloud Conference 2023 - Continuous Deployment Showdown: Traditionelles CI...
Dev-Cloud Conference 2023 - Continuous Deployment Showdown: Traditionelles CI...
Marc Müller37 views
Fleet Management Software in India by Fleetable
Fleet Management Software in India Fleet Management Software in India
Fleet Management Software in India
Fleetable11 views
Airline Booking Software by SharmiMehta
Airline Booking SoftwareAirline Booking Software
Airline Booking Software
SharmiMehta5 views
DSD-INT 2023 The Danube Hazardous Substances Model - Kovacs by Deltares
DSD-INT 2023 The Danube Hazardous Substances Model - KovacsDSD-INT 2023 The Danube Hazardous Substances Model - Kovacs
DSD-INT 2023 The Danube Hazardous Substances Model - Kovacs
Deltares8 views
SUGCON ANZ Presentation V2.1 Final.pptx by Jack Spektor
SUGCON ANZ Presentation V2.1 Final.pptxSUGCON ANZ Presentation V2.1 Final.pptx
SUGCON ANZ Presentation V2.1 Final.pptx
Jack Spektor22 views
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J... by Deltares
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
Deltares9 views
Software evolution understanding: Automatic extraction of software identifier... by Ra'Fat Al-Msie'deen
Software evolution understanding: Automatic extraction of software identifier...Software evolution understanding: Automatic extraction of software identifier...
Software evolution understanding: Automatic extraction of software identifier...
DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut... by Deltares
DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut...DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut...
DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut...
Deltares7 views
.NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra... by Marc Müller
.NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra....NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra...
.NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra...
Marc Müller38 views

OLTP-Bench

  • 1. OLTP-Bench Framework: An Extensible Testbed for 3 September 2014 VLDB14, Hangzhou, CHINA Benchmarking Relational Databases Djellel Eddine Difallah, Andy Pavlo, Carlo Curino, Philippe Cudré-Mauroux
  • 2. How many researchers wasted time writing benchmarking infrastructure to run their experiments? Measuring Reproducibility in Computer Systems Research – Collberg et al. (20% success)
  • 3. Experimental Research Process We continuously re-invent the wheel. Development Create the prototype Search for Benchmarks (ad-hoc workloads) Search for Datasets (synthetic data) Building a test toolkit Measure Monitor different resources Analysis Plot and make sense of the results
  • 4. OLTP-Bench is an open-source “batteries-included” DBMS benchmarking testbed tailored for OLTP/Web workloads
  • 5. OLTP-Bench 16 Workloads Tested on 8 DBMSs Rich metrics Fine-grained Rate control Workload Mixture Extensible
  • 6. OLTP-Bench Architecture ● Tight and dynamic control on parallel load generation ● Statistics gathering ● SQL dialect handling Parallel Connections vs Performance
  • 7. Current Workloads and Benchmarks Class Benchmark Application Domain Transactional AuctionMark CH-benCHmark SEATS TATP TPC-C Voter SmallBank On-line Auctions Mixture of OLTP and OLAP On-line Airline Ticketing Caller Location App Order Processing Talent Show Voting Banking System Web-Oriented Epinions LinkBench Twitter Wikipedia Social Networking Social Networking Social Networking On-line Encyclopedia Feature Testing ResourceStresser YCSB JPAB SIBench Isolated Resource Stresser Scalable Key-value Store Object-Relational Mapping Transactional Isolation JDBC Compliant DBMSs MS SQL Server Oracle MySQL Postgres DB2 NuoDB SQLite Apache Derby MonetDB
  • 9. Fine-Grained Rate Control MySQL running Wikipedia workload at increasing throughput. Demonstrating tight control of transactional throughput imposed on the system (and saturation)
  • 10. Simulating Evolving Skew MySQL running Twitter with an evolving skew (alternating zipfian and uniform). Different skew imposes stress on different resources.
  • 11. Multi-tenancy Comparing 3 relational databases under multi-tenancy resource contention. (No DBMS is perfect)
  • 12. Changing Mixture over time MySQL running YCSB with a varying transaction mixture (highlighting various resource bottlenecks) Parallel Connections vs Performance
  • 13. Future Work “We explicitly avoid to define benchmark rules, as they are (often) restrictive, arbitrary, biased, and not-future-proof” - Djellel, Andy, Carlo, Philippe The project received an NSF Grant!! Possible directions: ● Better tooling (e.g. real-time visualization) ● Synchronized distributed clients ● Support for NoSQL ● A public repository for experiments and results
  • 14. Q&A https://github.com/oltpbenchmark/ External contributors and acknowledgements: Evan Jones, Barzan Mozfari, Dimitri Vorona, Ben Reilly, Yu Su, Adam Seering, Simon Krenger, Tommy Reilly, Mark Callaghan, Keving Grittner, David L. Day, Nik Lanham, Awajeet Arya, Flavio Sousa, Stein Petter Tronstad, Lance FAng, Rodrigo Felix de Almeida, Woonhak kang, Florian Funke, Ahmad Hassan, Ivo Jimenez,Mickaël Hoerdt, Dimokritos Stamatakis, Benjamin Reilly, Mark Callaghan, Zheng Da, Xi Chen, Jean de Lavarene, Ning Zhang, Erik Paulson, Alec Jindal, ADD YOUR NAME HERE