The document discusses asynchronous replication of databases in large scale systems. It covers consistency criteria for replicated systems, asynchronous replication models including primary copy and update-everywhere, challenges of scaling replication across large systems, and limitations of current full replication techniques with respect to coordination overhead and data freshness at large scales. Examples of MySQL replication are provided and conclusions are presented.
An implementation of visualization and Transportation of warehouse on Electronic Batch Recording (EBR) application regarding Manufacturing Execution Systems (MES)
Replication is useful in improving the availability of data by coping data at multiple sites.
Either a relation or a fragment can be replicated at one or more sites.
Fully redundant databases are those in which every site contains a copy of the entire database.
Depending on the availability and redundancy factor there are three types of replications:
Full replication.
No replication.
Partial replication.
This presentation was provided by Marcy Vana of the Washington University in St. Louis during the NISO virtual conference, The Computer Campus: Integrating Information Systems and Services, held on August 15, 2018.
As opposed to databases for which established benchmarks have been driving the advancement of the field since a long time, workflow engines still lack a well-accepted benchmark that allows to give a fair comparison of their performance. In this talk we discuss the reasons and propose how to address the main challenges related to benchmarking these complex middleware systems at the core of business process automation and service composition solutions. In particular, we look at how to generate a representative workload and how to define suitable performance metrics. You will learn how to use our framework to measure the performance and resource consumption of your BPMN engine and compare different configurations to tune its performance in your concrete real-life project. The talk will also present preliminary experimental results obtained while benchmarking popular open source engines.
An implementation of visualization and Transportation of warehouse on Electronic Batch Recording (EBR) application regarding Manufacturing Execution Systems (MES)
Replication is useful in improving the availability of data by coping data at multiple sites.
Either a relation or a fragment can be replicated at one or more sites.
Fully redundant databases are those in which every site contains a copy of the entire database.
Depending on the availability and redundancy factor there are three types of replications:
Full replication.
No replication.
Partial replication.
This presentation was provided by Marcy Vana of the Washington University in St. Louis during the NISO virtual conference, The Computer Campus: Integrating Information Systems and Services, held on August 15, 2018.
As opposed to databases for which established benchmarks have been driving the advancement of the field since a long time, workflow engines still lack a well-accepted benchmark that allows to give a fair comparison of their performance. In this talk we discuss the reasons and propose how to address the main challenges related to benchmarking these complex middleware systems at the core of business process automation and service composition solutions. In particular, we look at how to generate a representative workload and how to define suitable performance metrics. You will learn how to use our framework to measure the performance and resource consumption of your BPMN engine and compare different configurations to tune its performance in your concrete real-life project. The talk will also present preliminary experimental results obtained while benchmarking popular open source engines.
PROJECT-BASED MICROCONTROLLER SYSTEM LABORATORY USING BK300 DEVELOPMENT BOARD...ijesajournal
Microcontroller system is one of the vital subjects offered by students during the sequence of study in
universities and other colleges of science, engineering and technology in the world. In this paper, we solve
the problem of student comprehension and skill development in embedded system design using
microcontroller chip PIC16F887 by demonstration of hands-on laboratory experiments. Also,
developments of software code, circuit diagram simulation were carried out. This is to help students
connect their theoretical knowledge with the practical experience. Each of the experiments was carried out
using BK300 development board, PICKit3 programmer, Proteus 8.0 software. Our years of experience in
the teaching of microcontroller course and the active involvement of students as manifested in complete indepth hands-on laboratory projects on real life problem solving. Laboratory session with the development
board and software demonstrated in this article is unambiguous. Future embedded system laboratory
session could be designed around ATMel lines of Microcontrollers.
Microcontroller system is one of the vital subjects offered by students during the sequence of study in universities and other colleges of science, engineering and technology in the world. In this paper, we solve the problem of student comprehension and skill development in embedded system design using microcontroller chip PIC16F887 by demonstration of hands-on laboratory experiments. Also, developments of software code, circuit diagram simulation were carried out. This is to help students connect their theoretical knowledge with the practical experience. Each of the experiments was carried out using BK300 development board, PICKit3 programmer, Proteus 8.0 software. Our years of experience in the teaching of microcontroller course and the active involvement of students as manifested in complete in-depth hands-on laboratory projects on real life problem solving. Laboratory session with the development board and software demonstrated in this article is unambiguous. Future embedded system laboratory session could be designed around ATMel lines of Microcontrollers.
Industrial Transfer Learning: a research area of the chair for technologies and management of digital transformation from the university of Wuppertal, Germany.
For more information, see here: https://www.tmdt.uni-wuppertal.de/de
Build systems orchestrate how human-readable source code is translated into executable programs. In a software project, source code changes can induce changes in the build system (aka. build co-changes). It is difficult for developers to identify when build co-changes are necessary due to the complexity of build systems. Prediction of build co-changes works well if there is a sufficient amount of training data to build a model. However, in practice, for new projects, there exists a limited number of changes. Using training data from other projects to predict the build co-changes in a new project can help improve the performance of the build co-change prediction. We refer to this problem as cross-project build co-change prediction.
In this paper, we propose CroBuild, a novel cross-project build co-change prediction approach that iteratively learns new classifiers. CroBuild constructs an ensemble of classifiers by iteratively building classifiers and assigning them weights according to its prediction error rate. Given that only a small proportion of code changes are build co-changing, we also propose an imbalance-aware approach that learns a threshold boundary between those code changes that are build co-changing and those that are not in order to construct classifiers in each iteration. To examine the benefits of CroBuild, we perform experiments on 4 large datasets including Mozilla, Eclipse-core, Lucene, and Jazz, comprising a total of 50,884 changes. On average, across the 4 datasets, CroBuild achieves a F1-score of up to 0.408. We also compare CroBuild with other approaches such as a basic model, AdaBoost proposed by Freund et al., and TrAdaBoost proposed by Dai et al.. On average, across the 4 datasets, the CroBuild approach yields an improvement in F1-scores of 41.54%, 36.63%, and 36.97% over the basic model, AdaBoost, and TrAdaBoost, respectively.
TMS - Schedule of Presentations and ReportsMichel Alves
Ten Minute Speech - Schedule of Presentations and Reports. Subjects: dissertation themes, results using capacity-constrained distribution, image-based reconstruction with color consistency, seamless montage and stats from slideshare!
Continual/Lifelong Learning with Deep Architectures, Vincenzo LomonacoData Science Milan
Humans have the extraordinary ability to learn continually from experience. Not only can we apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of AI is building an artificial continually learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex skills and knowledge.
"Continual Learning" (CL) is indeed a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the topic, we’ll implement different Continual Learning strategies and assess them on common vision benchmarks. We’ll conclude the workshop with a look at possible real world applications of CL.
Vincenzo Lomonaco is a Deep Learning PhD student at the University of Bologna and founder of ContinualAI.org. He is also the PhD students representative at the Department of Computer Science of Engineering (DISI) and teaching assistant of the courses “Machine Learning” and “Computer Architectures” in the same department. Previously, he was a Machine Learning software engineer at IDL in-line Devices and a Master Student at the University of Bologna where he graduated cum laude in 2015 with the dissertation “Deep Learning for Computer Vision: a Comparison Between CNNs and HTMs on Object Recognition Tasks".
Part 2.3 of the slides I wrote for the course "Advanced Computer Architectures", which I taught in the framework of the Advanced Masters Programme in Artificial Intelligence of the Catholic University of Leuven, Leuven (B)
COSMOS:
DevOps for Complex Cyber-physical Systems
Sebastiano Panichella
Zurich University of Applied Sciences (ZHAW)
Workshop on Adaptive CPSoS (WASOS) 2023
Deploying and Monitoring Heterogeneous Machine Learning Applications with Cli...Databricks
Machine learning is being deployed in a growing number of applications which demand real-time, accurate, and robust predictions under heavy serving loads. However, most machine learning frameworks and systems only address model training and not deployment.
Clipper is an open-source, general-purpose model-serving system that addresses these challenges. Interposing between applications that consume predictions and the machine-learning models that produce predictions, Clipper simplifies the model deployment process by adopting a modular serving architecture and isolating models in their own containers, allowing them to be evaluated using the same runtime environment as that used during training. Clipper’s modular architecture provides simple mechanisms for scaling out models to meet increased throughput demands and performing fine-grained physical resource allocation for each model. Further, by abstracting models behind a uniform serving interface, Clipper allows developers to compose many machine-learning models within a single application to support increasingly common techniques such as ensemble methods, multi-armed bandit algorithms, and prediction cascades.
In this talk I will provide an overview of the Clipper serving system and discuss how to get started using Clipper to serve Apache Spark and TensorFlow models on Kubernetes. I will then discuss some recent work on statistical performance monitoring for machine learning models.
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
MySQL Webinar, presented on the 25th of April, 2024.
Summary:
MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out.
With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration.
Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications.
In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.
MySQL Router - Explore The Secrets (MySQL Belgian Days 2024)Miguel Araújo
Talk given at MySQL Belgian Days 2024.
The goal of this talk is to describe Router's architecture, highlight its role/features in MySQL Architectures, and go through other features that are less known and/or new, namely:
- Group Replication notifications
- Connection sharing/reuse
- TLS Session caching
- REST API
- MRS
- RW Splitting
PROJECT-BASED MICROCONTROLLER SYSTEM LABORATORY USING BK300 DEVELOPMENT BOARD...ijesajournal
Microcontroller system is one of the vital subjects offered by students during the sequence of study in
universities and other colleges of science, engineering and technology in the world. In this paper, we solve
the problem of student comprehension and skill development in embedded system design using
microcontroller chip PIC16F887 by demonstration of hands-on laboratory experiments. Also,
developments of software code, circuit diagram simulation were carried out. This is to help students
connect their theoretical knowledge with the practical experience. Each of the experiments was carried out
using BK300 development board, PICKit3 programmer, Proteus 8.0 software. Our years of experience in
the teaching of microcontroller course and the active involvement of students as manifested in complete indepth hands-on laboratory projects on real life problem solving. Laboratory session with the development
board and software demonstrated in this article is unambiguous. Future embedded system laboratory
session could be designed around ATMel lines of Microcontrollers.
Microcontroller system is one of the vital subjects offered by students during the sequence of study in universities and other colleges of science, engineering and technology in the world. In this paper, we solve the problem of student comprehension and skill development in embedded system design using microcontroller chip PIC16F887 by demonstration of hands-on laboratory experiments. Also, developments of software code, circuit diagram simulation were carried out. This is to help students connect their theoretical knowledge with the practical experience. Each of the experiments was carried out using BK300 development board, PICKit3 programmer, Proteus 8.0 software. Our years of experience in the teaching of microcontroller course and the active involvement of students as manifested in complete in-depth hands-on laboratory projects on real life problem solving. Laboratory session with the development board and software demonstrated in this article is unambiguous. Future embedded system laboratory session could be designed around ATMel lines of Microcontrollers.
Industrial Transfer Learning: a research area of the chair for technologies and management of digital transformation from the university of Wuppertal, Germany.
For more information, see here: https://www.tmdt.uni-wuppertal.de/de
Build systems orchestrate how human-readable source code is translated into executable programs. In a software project, source code changes can induce changes in the build system (aka. build co-changes). It is difficult for developers to identify when build co-changes are necessary due to the complexity of build systems. Prediction of build co-changes works well if there is a sufficient amount of training data to build a model. However, in practice, for new projects, there exists a limited number of changes. Using training data from other projects to predict the build co-changes in a new project can help improve the performance of the build co-change prediction. We refer to this problem as cross-project build co-change prediction.
In this paper, we propose CroBuild, a novel cross-project build co-change prediction approach that iteratively learns new classifiers. CroBuild constructs an ensemble of classifiers by iteratively building classifiers and assigning them weights according to its prediction error rate. Given that only a small proportion of code changes are build co-changing, we also propose an imbalance-aware approach that learns a threshold boundary between those code changes that are build co-changing and those that are not in order to construct classifiers in each iteration. To examine the benefits of CroBuild, we perform experiments on 4 large datasets including Mozilla, Eclipse-core, Lucene, and Jazz, comprising a total of 50,884 changes. On average, across the 4 datasets, CroBuild achieves a F1-score of up to 0.408. We also compare CroBuild with other approaches such as a basic model, AdaBoost proposed by Freund et al., and TrAdaBoost proposed by Dai et al.. On average, across the 4 datasets, the CroBuild approach yields an improvement in F1-scores of 41.54%, 36.63%, and 36.97% over the basic model, AdaBoost, and TrAdaBoost, respectively.
TMS - Schedule of Presentations and ReportsMichel Alves
Ten Minute Speech - Schedule of Presentations and Reports. Subjects: dissertation themes, results using capacity-constrained distribution, image-based reconstruction with color consistency, seamless montage and stats from slideshare!
Continual/Lifelong Learning with Deep Architectures, Vincenzo LomonacoData Science Milan
Humans have the extraordinary ability to learn continually from experience. Not only can we apply previously learned knowledge and skills to new situations, we can also use these as the foundation for later learning. One of the grand goals of AI is building an artificial continually learning agent that constructs a sophisticated understanding of the world from its own experience through the autonomous incremental development of ever more complex skills and knowledge.
"Continual Learning" (CL) is indeed a fast emerging topic in AI concerning the ability to efficiently improve the performance of a deep model over time, dealing with a long (and possibly unlimited) sequence of data/tasks. In this workshop, after a brief introduction of the topic, we’ll implement different Continual Learning strategies and assess them on common vision benchmarks. We’ll conclude the workshop with a look at possible real world applications of CL.
Vincenzo Lomonaco is a Deep Learning PhD student at the University of Bologna and founder of ContinualAI.org. He is also the PhD students representative at the Department of Computer Science of Engineering (DISI) and teaching assistant of the courses “Machine Learning” and “Computer Architectures” in the same department. Previously, he was a Machine Learning software engineer at IDL in-line Devices and a Master Student at the University of Bologna where he graduated cum laude in 2015 with the dissertation “Deep Learning for Computer Vision: a Comparison Between CNNs and HTMs on Object Recognition Tasks".
Part 2.3 of the slides I wrote for the course "Advanced Computer Architectures", which I taught in the framework of the Advanced Masters Programme in Artificial Intelligence of the Catholic University of Leuven, Leuven (B)
COSMOS:
DevOps for Complex Cyber-physical Systems
Sebastiano Panichella
Zurich University of Applied Sciences (ZHAW)
Workshop on Adaptive CPSoS (WASOS) 2023
Deploying and Monitoring Heterogeneous Machine Learning Applications with Cli...Databricks
Machine learning is being deployed in a growing number of applications which demand real-time, accurate, and robust predictions under heavy serving loads. However, most machine learning frameworks and systems only address model training and not deployment.
Clipper is an open-source, general-purpose model-serving system that addresses these challenges. Interposing between applications that consume predictions and the machine-learning models that produce predictions, Clipper simplifies the model deployment process by adopting a modular serving architecture and isolating models in their own containers, allowing them to be evaluated using the same runtime environment as that used during training. Clipper’s modular architecture provides simple mechanisms for scaling out models to meet increased throughput demands and performing fine-grained physical resource allocation for each model. Further, by abstracting models behind a uniform serving interface, Clipper allows developers to compose many machine-learning models within a single application to support increasingly common techniques such as ensemble methods, multi-armed bandit algorithms, and prediction cascades.
In this talk I will provide an overview of the Clipper serving system and discuss how to get started using Clipper to serve Apache Spark and TensorFlow models on Kubernetes. I will then discuss some recent work on statistical performance monitoring for machine learning models.
Similar to Asynchronous Replication of Databases (20)
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
MySQL Webinar, presented on the 25th of April, 2024.
Summary:
MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out.
With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration.
Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications.
In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.
MySQL Router - Explore The Secrets (MySQL Belgian Days 2024)Miguel Araújo
Talk given at MySQL Belgian Days 2024.
The goal of this talk is to describe Router's architecture, highlight its role/features in MySQL Architectures, and go through other features that are less known and/or new, namely:
- Group Replication notifications
- Connection sharing/reuse
- TLS Session caching
- REST API
- MRS
- RW Splitting
MySQL Shell/AdminAPI - MySQL Architectures Made Easy For All!Miguel Araújo
Talk given at MySQL Belgian Days 2024.
Covers all the MySQL Architectures supported to ensure business continuity with a focus on business requirements, tecnicalities, and features: InnoDB Cluster, InnoDB ClusterSet, InnoDB Cluster Read Replicas and InnoDB ReplicaSet.
A special focus is given on the AdminAPI of MySQL Shell, with its main features, recommendations, and the latest additions and features.
Deep Dive into MySQL InnoDB Cluster Read Scale-out Capabilities.pdfMiguel Araújo
MySQL's first Innovation Release is out, 8.1.0, and with it, we're introducing MySQL InnoDB Cluster Read Replicas.
The main purpose of secondaries on MySQL InnoDB Cluster is to be ready to take over when a primary member has failed (High Availability). This is done using MySQL Group Replication. Another commonly used purpose for the secondaries is to use them to offload read workloads away from the primary. With MySQL InnoDB Cluster Read Replicas, it's now possible to add asynchronous replicas to the database topology, to be used to offload read traffic away from primary or secondaries, to have dedicated read replicas, special purpose read replicas (e.g. for reporting), or to scale beyond what the secondaries can handle by adding multiple read replicas.
This talk covers the read replicas functionality, showcase its usage in different database architectures, and include a demonstration on its setup and management.
MySQL Database Architectures - High Availability and Disaster Recovery SolutionMiguel Araújo
MySQL InnoDB ClusterSet brings multi-datacenter capabilities to our solutions and makes it very easy to set up a disaster recovery architecture. Think multiple MySQL InnoDB Clusters into one single database architecture, fully managed from MySQL Shell and with full MySQL Router integration to make it easy to access the entire architecture.
This presentation covers the various solutions of MySQL for High Availability, Replication, and Disaster Recovery, with a special focus on InnoDB ClusterSet:
- The various features of InnoDB Clusterset
- How to setup MySQL InnoDB ClusterSet
- Ways to migrate from an existing MySQL InnoDB Cluster into MySQL InnoDB ClusterSet
- How to deal with various failures
- The various features of router integration make the connection to the database architecture easy.
Disaster Recovery with MySQL InnoDB ClusterSet - What is it and how do I use it?Miguel Araújo
MySQL InnoDB ClusterSet brings multi-datacenter capabilities to our solutions and make it very easy to setup a disaster recovery architecture. Think multiple MySQL InnoDB Clusters into one single database architecture, fully managed from MySQL Shell and with full MySQL Router integration to make it easy to access the entire architecture.
This presentation covers:
- The various features of InnoDB Clusterset
- How to setup MySQL InnoDB ClusterSet
- Ways to migrate from an existing MySQL InnoDB Cluster into MySQL InnoDB ClusterSet
- How to deal with various failures
- The various features of router integration which makes connection to the database architecture easy.
Session presented at Oracle Developer Live - MySQL, 2020. Recording available at https://developer.oracle.com/developer-live/mysql/
Abstract:
MySQL Shell is the new, advanced command-line client and editor for MySQL. It sends SQL statements to MySQL server, supports both the classic MySQL protocol and the newer X protocol, and provides scripting capabilities for JavaScript and Python. But there's more to MySQL Shell than meets the eye. It delivers a natural and powerful interface for all DevOps tasks related to MySQL by providing APIs for development and administration. This session covers MySQL Shell's core features, along with demonstrations of how to use the various APIs and how to extend MySQL Shell. We’ll address the regular interaction with databases, the built-in tools that make DBAs and developers’ lives easier, the easy and flawless set up of HA architectures, and the plugins and extensions framework.
MySQL InnoDB Cluster / ReplicaSet - Making Provisioning & Troubleshooting as ...Miguel Araújo
Session presented at pre-FOSDEM MySQL Days 2020.
MySQL InnoDB Cluster and ReplicaSet provide failover/high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use.
Recent enhancements and features added to MySQL Shell make the management of InnoDB Clusters / ReplicaSets even more powerful and effortless! Full instance provisioning and cluster troubleshooting became easy as 1, 2, 3.
MySQL InnoDB Cluster / ReplicaSet - TutorialMiguel Araújo
Tutorial given at pre-FOSDEM MySQL Days 2020 on MySQL InnoDB Cluster and ReplicaSet, a fully integrated product built on MySQL technology, by MySQL.
MySQL InnoDB Cluster and ReplicaSet provide failover/high availability and scaling features baked in; providing an integrated end-to-end solution that is easy to use.
MySQL InnoDB Cluster: Management and Troubleshooting with MySQL ShellMiguel Araújo
MySQL InnoDB Cluster and MySQL Shell session presented at Oracle CodeOne2019.
Abstract:
MySQL InnoDB Cluster provides a built-in high-availability solution for MySQL. Combining MySQL Group Replication with MySQL Router and MySQL Shell into an integrated full-stack solution, InnoDB Cluster provides easy setup and management of MySQL instances into a fault-tolerant database service. MySQL Shell is the “control panel” of InnoDB Cluster, enabling the easy and straightforward configuration and management of InnoDB clusters by providing a scriptable and interactive API: the AdminAPI. Recent enhancements and features added to MySQL Shell make the management of InnoDB clusters even more powerful and smoother. Attend this session to get an overview of the latest developments and improved InnoDB Cluster administration tasks.
Notes:
The slideshow includes a video that cannot be seen in slideshare/PDF. If you're interested in it please check the following blog post: https://mysqlhighavailability.com/mysql-innodb-cluster-automatic-node-provisioning/
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...Miguel Araújo
MySQL Shell is an interactive multi-language console interface that supports development and administration for the MySQL Server. It can be used to perform data queries or updates, and administration operations through scriptable DevOps APIs.
It is by nature a unified interface for all DevOps tasks related to MySQL, supporting SQL, JavaScript and Python, with an extensive set of features for both developers and DBAs.
This session demonstrates how can a DevOps-engineer day be better with the MySQL Shell. From the development tasks to the operations, culminating in the management of MySQL InnoDB Cluster (The HA solution for MySQL) with the most recent and advanced features.
Session presented at pre-FOSDEM MySQL Day 2019 (https://lefred.be/content/pre-fosdem-mysql-day-2019/)
MySQL Shell is an interactive JavaScript, Python, and SQL command-line client supporting development and administration for the MySQL Server and InnoDB clusters. MySQL InnoDB cluster provides a complete high-availability (HA) solution for MySQL, tightly integrating MySQL Server, MySQL Group Replication, MySQL Router, and MySQL Shell to provide an easy-to-use full-stack solution for HA. MySQL Shell provides a natural interface for all 'development and operations tasks related to MySQL, by supporting scripting with development and administration APIs. Learn more in this session.
MySQL 8 High Availability with InnoDB ClustersMiguel Araújo
MySQL’s InnoDB cluster provides a high-level, easy-to-use solution for MySQL high availability. Combining MySQL Group Replication with MySQL Router and the MySQL Shell into an integrated solution, InnoDB clusters offer easy setup and management of MySQL instances into a fault-tolerant database service. In this session learn how to set up a basic InnoDB cluster, integrate it with applications, and recognize and react to common failure scenarios that would otherwise lead to a database outage.
- Workshop presentation
FOSDEM'18: MySQL InnoDB Cluster - MySQL HA Made Easy!Miguel Araújo
MySQL InnoDB Cluster provides a built-in High Availability solution for MySQL. It tightly integrates MySQL Server, Group Replication, MySQL Router and MySQL Shell providing an easy-to-use full stack solution for HA.
MySQL Shell main goal is to provide a natural interface for all 'DevOps' tasks related to MySQL, by supporting scripting with development and administration APIs. To allow an easy and straightforward configuration and administration of InnoDB Clusters, the Shell provides a scriptable API - the AdminAPI. This API hides the complexity associated with configuring, provisioning and managing everything without sacrificing power, flexibility or security.
Join this session to understand the key points of MySQL InnoDB Cluster and to learn how to use the Shell and the AdminAPI to configure and manage InnoDB Clusters.
Session presented at FOSDEM'18.
Live demos not available in the slide deck, but recorded session available at: https://fosdem.org/2018/schedule/event/mysql_innodb_cluster/
MySQL Shell - The DevOps Tool for MySQLMiguel Araújo
Automation wasn’t always easy within MySQL-related operations, but now MySQL Shell can be used to make this integration better. For developers and DBAs, this session purpose was to show the power of the new MySQL Shell for development, operations, automation, orchestration, setup, maintenance, and management of InnoDB clusters.
Live demos not available in the slide deck. If you're interested in knowing more about those, feel free to contact me.
(From Oracle Open World 2017)
MySQL Proxy. A powerful, flexible MySQL toolbox.Miguel Araújo
MySQL-Proxy is a software application that, as the name suggests, sits between your client and MySQL server(s) allowing you to monitor, analyse or transform that communication. It communicates over the network using the MySQL network protocol and as so, in its the most basic configuration, Proxy simply interposes itself between the server and clients, passing queries from the client to the server and its responses on the other way around. This opens the possibility of changing the communication packets when needed, allowing thus being used for multiple purposes being the most remarkable query analysis, query filtering and modification, load balancing, failover, query injection and pooling.
On this session I'll present and give you a global overview of MySQL-Proxy and the concepts behind it. Use-cases, technical overview and architecture will follow. And of course, everyone will want to see it working so that'll be included as well in parallel with detailed explanation on how you can use it to fulfill your needs.
Evaluating Data Freshness in Large Scale Replicated Databases
Asynchronous Replication of Databases
1. Survey Paper - UCE15
Asynchronous Replication of
Databases in Large Scale
State Of The Art
Master's Degree in Computer Engineering
University of Minho
2. Survey Paper - UCE15
Agenda
Introduction
Consistency Criteria
Asynchronous Replication
Replication in Large Scale and Data Freshness
MySQL
Conclusions
Master's Degree in Computer Engineering
University of Minho
3. Survey Paper - UCE15
Introduction
Master's Degree in Computer Engineering
University of Minho
4. Survey Paper - UCE15
Introduction
Why replicate data?
Master's Degree in Computer Engineering
University of Minho
5. Survey Paper - UCE15
Introduction
Why replicate data?
Fault-Tolerance
Master's Degree in Computer Engineering
University of Minho
6. Survey Paper - UCE15
Introduction
Why replicate data?
Fault-Tolerance
Performance
Master's Degree in Computer Engineering
University of Minho
7. Survey Paper - UCE15
Introduction
Why replicate data?
Fault-Tolerance
Performance
Master's Degree in Computer Engineering
University of Minho
8. Survey Paper - UCE15
Introduction
Why replicate data?
Fault-Tolerance
Performance
However...
Master's Degree in Computer Engineering
University of Minho
9. Survey Paper - UCE15
Introduction
Why replicate data?
Fault-Tolerance
Performance
However...
Introduces a constant trade-off between consistency and
performance
Master's Degree in Computer Engineering
University of Minho
10. Survey Paper - UCE15
Introduction
Why replicate data?
Fault-Tolerance
Performance
However...
Introduces a constant trade-off between consistency and
performance
Need to use adequate replication mechanisms / protocols
Master's Degree in Computer Engineering
University of Minho
11. Survey Paper - UCE15
Introduction
Master's Degree in Computer Engineering
University of Minho
12. Survey Paper - UCE15
Introduction
Classification of replication protocols:
Master's Degree in Computer Engineering
University of Minho
13. Survey Paper - UCE15
Introduction
Classification of replication protocols:
When can updates be performed:
Master's Degree in Computer Engineering
University of Minho
14. Survey Paper - UCE15
Introduction
Classification of replication protocols:
When can updates be performed:
Who can perform updates:
Master's Degree in Computer Engineering
University of Minho
15. Survey Paper - UCE15
Introduction
Classification of replication protocols:
When can updates be performed:
- Lazy (also know as asynchronous)
- Eager (also know as synchronous)
Who can perform updates:
Master's Degree in Computer Engineering
University of Minho
16. Survey Paper - UCE15
Introduction
Classification of replication protocols:
When can updates be performed:
- Lazy (also know as asynchronous)
- Eager (also know as synchronous)
Who can perform updates:
- Primary copy
- Update-everywhere
Master's Degree in Computer Engineering
University of Minho
17. Survey Paper - UCE15
Consistency Criteria
Master's Degree in Computer Engineering
University of Minho
18. Survey Paper - UCE15
Consistency Criteria
Key issue, since replication is a solution to achieve fault-tolerance
Master's Degree in Computer Engineering
University of Minho
19. Survey Paper - UCE15
Consistency Criteria
Key issue, since replication is a solution to achieve fault-tolerance
Correct behaviour in a replicated system must ensure linearizability
(also know as one copy equivalence):
Master's Degree in Computer Engineering
University of Minho
20. Survey Paper - UCE15
Consistency Criteria
Key issue, since replication is a solution to achieve fault-tolerance
Correct behaviour in a replicated system must ensure linearizability
(also know as one copy equivalence):
‣ Gives the illusion that a replicated database system is single
Master's Degree in Computer Engineering
University of Minho
21. Survey Paper - UCE15
Consistency Criteria
Key issue, since replication is a solution to achieve fault-tolerance
Correct behaviour in a replicated system must ensure linearizability
(also know as one copy equivalence):
‣ Gives the illusion that a replicated database system is single
‣ Maintain order
Master's Degree in Computer Engineering
University of Minho
22. Survey Paper - UCE15
Consistency Criteria
Key issue, since replication is a solution to achieve fault-tolerance
Correct behaviour in a replicated system must ensure linearizability
(also know as one copy equivalence):
‣ Gives the illusion that a replicated database system is single
‣ Maintain order
‣ Atomicity
Master's Degree in Computer Engineering
University of Minho
23. Survey Paper - UCE15
Consistency Criteria
Other Criteria:
Increasing popularity of Snapshot Isolation (SI)
Strong Serializability
Strong Session 1SR
Master's Degree in Computer Engineering
University of Minho
24. Survey Paper - UCE15
Asynchronous Replication
Master's Degree in Computer Engineering
University of Minho
25. Survey Paper - UCE15
Asynchronous Replication
Lazy schemes update replicas using separate transactions
Master's Degree in Computer Engineering
University of Minho
26. Survey Paper - UCE15
Asynchronous Replication
Lazy schemes update replicas using separate transactions
Due to the complexity and performance of eager replication, there is
a wide spectrum of lazy schemes
Master's Degree in Computer Engineering
University of Minho
27. Survey Paper - UCE15
Asynchronous Replication
Lazy schemes update replicas using separate transactions
Due to the complexity and performance of eager replication, there is
a wide spectrum of lazy schemes
Lazy Replication Models:
Master's Degree in Computer Engineering
University of Minho
28. Survey Paper - UCE15
Asynchronous Replication
Lazy schemes update replicas using separate transactions
Due to the complexity and performance of eager replication, there is
a wide spectrum of lazy schemes
Lazy Replication Models:
Primary copy
Master's Degree in Computer Engineering
University of Minho
29. Survey Paper - UCE15
Asynchronous Replication
Lazy schemes update replicas using separate transactions
Due to the complexity and performance of eager replication, there is
a wide spectrum of lazy schemes
Lazy Replication Models:
Primary copy
Update-everywhere
Master's Degree in Computer Engineering
University of Minho
30. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Master's Degree in Computer Engineering
University of Minho
31. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Current techniques have attained some degree of scalability,
however there are two main limitations:
Master's Degree in Computer Engineering
University of Minho
32. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Current techniques have attained some degree of scalability,
however there are two main limitations:
Most solutions adopt a full replication model
Master's Degree in Computer Engineering
University of Minho
33. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Current techniques have attained some degree of scalability,
however there are two main limitations:
Most solutions adopt a full replication model
➡ Coordination overhead
Master's Degree in Computer Engineering
University of Minho
34. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Current techniques have attained some degree of scalability,
however there are two main limitations:
Most solutions adopt a full replication model
➡ Coordination overhead
Most solutions rely on 1-copy-serializability
Master's Degree in Computer Engineering
University of Minho
35. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Current techniques have attained some degree of scalability,
however there are two main limitations:
Most solutions adopt a full replication model
➡ Coordination overhead
Most solutions rely on 1-copy-serializability
➡ Limits concurrency
Master's Degree in Computer Engineering
University of Minho
36. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Current techniques have attained some degree of scalability,
however there are two main limitations:
Most solutions adopt a full replication model
➡ Coordination overhead
Most solutions rely on 1-copy-serializability
➡ Limits concurrency
➡ Limits scalability of the system
Master's Degree in Computer Engineering
University of Minho
37. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Master's Degree in Computer Engineering
University of Minho
38. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Although serializability is guaranteed on lazy replication system with
concurrency control techniques and consistency criterion, these
techniques do not provide data freshness guarantees:
Master's Degree in Computer Engineering
University of Minho
39. Survey Paper - UCE15
Replication in Large Scale and Data
Freshness
Transactions may see stale data
They may be serialized in an order different from the one in
which they were submitted
Master's Degree in Computer Engineering
University of Minho
40. Survey Paper - UCE15
MySQL
Master's Degree in Computer Engineering
University of Minho
41. Survey Paper - UCE15
MySQL
MySQL implements asynchronous master-slave replication
Master's Degree in Computer Engineering
University of Minho
42. Survey Paper - UCE15
MySQL
MySQL implements asynchronous master-slave replication
Uses the primary-copy replication method, supporting two formats
of replication:
Master's Degree in Computer Engineering
University of Minho
43. Survey Paper - UCE15
MySQL
MySQL implements asynchronous master-slave replication
Uses the primary-copy replication method, supporting two formats
of replication:
Statement-based
Master's Degree in Computer Engineering
University of Minho
44. Survey Paper - UCE15
MySQL
MySQL implements asynchronous master-slave replication
Uses the primary-copy replication method, supporting two formats
of replication:
Statement-based
Row-based
Master's Degree in Computer Engineering
University of Minho
45. Survey Paper - UCE15
MySQL
Replication topologies:
Master's Degree in Computer Engineering
University of Minho
46. Survey Paper - UCE15
MySQL
Replication topologies:
Master and Multiple Slaves
Master's Degree in Computer Engineering
University of Minho
47. Survey Paper - UCE15
MySQL
Replication topologies:
Master and Multiple Slaves Ring
Master's Degree in Computer Engineering
University of Minho
48. Survey Paper - UCE15
MySQL
Replication topologies:
Master and Multiple Slaves Ring Tree
Master's Degree in Computer Engineering
University of Minho
49. Survey Paper - UCE15
Conclusions
Master's Degree in Computer Engineering
University of Minho
50. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Master's Degree in Computer Engineering
University of Minho
51. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Lazy protocols have better performance, but inconsistencies among copies may occur
Master's Degree in Computer Engineering
University of Minho
52. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Lazy protocols have better performance, but inconsistencies among copies may occur
Primary-copy approach introduces a single point of failure, but simplifies replica control
Master's Degree in Computer Engineering
University of Minho
53. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Lazy protocols have better performance, but inconsistencies among copies may occur
Primary-copy approach introduces a single point of failure, but simplifies replica control
Update-everywhere method speeds up data access but makes replica coordination more
complex and expensive
Master's Degree in Computer Engineering
University of Minho
54. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Lazy protocols have better performance, but inconsistencies among copies may occur
Primary-copy approach introduces a single point of failure, but simplifies replica control
Update-everywhere method speeds up data access but makes replica coordination more
complex and expensive
1-copy serializability limits concurrency and thus scalability of the system
Master's Degree in Computer Engineering
University of Minho
55. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Lazy protocols have better performance, but inconsistencies among copies may occur
Primary-copy approach introduces a single point of failure, but simplifies replica control
Update-everywhere method speeds up data access but makes replica coordination more
complex and expensive
1-copy serializability limits concurrency and thus scalability of the system
Concurrency control techniques and consistency criterion in lazy replication systems do not
provide data freshness
Master's Degree in Computer Engineering
University of Minho
56. Survey Paper - UCE15
Conclusions
Eager protocols are not appropriate for large scale systems
Lazy protocols have better performance, but inconsistencies among copies may occur
Primary-copy approach introduces a single point of failure, but simplifies replica control
Update-everywhere method speeds up data access but makes replica coordination more
complex and expensive
1-copy serializability limits concurrency and thus scalability of the system
Concurrency control techniques and consistency criterion in lazy replication systems do not
provide data freshness
Some consistency techniques to improve data freshness emerged, but have a trade-off
between consistency and performance
Master's Degree in Computer Engineering
University of Minho