Part II of our free self-training slides on MySQL Cluster.
In this part we cover 'Detailed Concepts':
* Data Distribution & Partitioning
* Two Phase Commit Protocol
* Transaction Resources
Severalnines Self-Training: MySQL® Cluster - Part VIISeveralnines
Part VII of our free self-training slides on MySQL Cluster.
In this installment, we cover ’Management and Administration'
* Backup and Restore
* Geographical Redundancy
* Online and Offline Operations
* Ndbinfo tables
* Reporting
* Single User Mode
* Scaling MySQL Cluster
Severalnines Training: MySQL Cluster - Part XSeveralnines
Part X of our self-training slides on MySQL Cluster, focused on troubleshooting MySQL Cluster
Topics:
- common problems encountered by users
- error logs and trace files
- recovery and escalation procedures
Severalnines Self-Training: MySQL® Cluster - Part VISeveralnines
Part VI of our free self-training slides on MySQL Cluster.
In this part we cover ’Configuration and Installation'
* Data Node configuration
* SQL Node configuration
* Important parameters
* Installation
* Upgrading
Conference tutorial: MySQL Cluster as NoSQLSeveralnines
Slides from the 'MySQL Cluster as NoSQL' tutorial at Percona Live MySQL Conference 2012 in London.
Tutorial covers:
*MySQL Cluster administration
* NoSQL options for MySQL Cluster and when to use what
* Memcached (installation and configuration)
* Cluster/J
* NDBAPI
* Benchmarking of different access methods on a live cluster
Severalnines Self-Training: MySQL® Cluster - Part VIISeveralnines
Part VII of our free self-training slides on MySQL Cluster.
In this installment, we cover ’Management and Administration'
* Backup and Restore
* Geographical Redundancy
* Online and Offline Operations
* Ndbinfo tables
* Reporting
* Single User Mode
* Scaling MySQL Cluster
Severalnines Training: MySQL Cluster - Part XSeveralnines
Part X of our self-training slides on MySQL Cluster, focused on troubleshooting MySQL Cluster
Topics:
- common problems encountered by users
- error logs and trace files
- recovery and escalation procedures
Severalnines Self-Training: MySQL® Cluster - Part VISeveralnines
Part VI of our free self-training slides on MySQL Cluster.
In this part we cover ’Configuration and Installation'
* Data Node configuration
* SQL Node configuration
* Important parameters
* Installation
* Upgrading
Conference tutorial: MySQL Cluster as NoSQLSeveralnines
Slides from the 'MySQL Cluster as NoSQL' tutorial at Percona Live MySQL Conference 2012 in London.
Tutorial covers:
*MySQL Cluster administration
* NoSQL options for MySQL Cluster and when to use what
* Memcached (installation and configuration)
* Cluster/J
* NDBAPI
* Benchmarking of different access methods on a live cluster
Slides presented at Great Indian Developer Summit 2016 at the session MySQL: What's new on April 29 2016.
Contains information about the new MySQL Document Store released in April 2016.
This presentation was written by Wagner Bianchi for the presentation on the Oracle Consulting Team/Professional Services meeting that took place in San Francisco/CA.
MySQL Performance Tuning: The Perfect Scalability (OOW2019)Mirko Ortensi
“When data grows, MySQL multiplies.” When data grows, a standard configuration does not fit anymore. This hands-on lab walks through the ways MySQL has to adapt to an ever-increasing amount of data in context of tuning, under the heading “multiplication”:
—Scaling up through partitioning and how partitioning can help manage/archive data
—How to relocate binlog, undo, redo, tablespaces, and more on different mount points to take advantage of multiple storages
—All improvements to parallel slave replication
—A quick OS approach, to verify swapping and affinity tuning take the most out of the machine.
Spectrum Scale - Diversified analytic solution based on various storage servi...Wei Gong
This slides describe diversified analytic solutions based on Spectrum Scale with various deployment mode, such as storage rich-server, share storage, IBM DeepFlash 150 and Elastic Storage Server. It deep dives several advanced data management features and solutions for BD&A workload derived from Spectrum Scale.
Database as a Service on the Oracle Database Appliance PlatformMaris Elsins
Speaker: Marc Fielding, Co-speaker: Maris Elsins.
Oracle Database Appliance provides a robust, highly-available, cost-effective, and surprisingly scalable platform for database as a service environment. By leveraging Oracle Enterprise Manager's self-service features, databases can be provisioned on a self-service basis to a cluster of Oracle Database Appliance machines. Discover how multiple ODA devices can be managed together to provide both high availability and incremental, cost-effective scalability. Hear real-world lessons learned from successful database consolidation implementations.
In this Introduction to GlusterFS webinar, introduction and review of the GlusterFS architecture and key functionalities. Learn how GlusterFS is deployed in the datacenter, in the cloud, or between the two. We’ll also cover a brief update on GlusterFS v3.3 which is currently in beta.
Webinar Sept 22: Gluster Partners with Redapt to Deliver Scale-Out NAS StorageGlusterFS
Gluster has partnered with Redapt, Inc., an innovative data center architecture and infrastructure solutions provider, to integrate GlusterFS with hardware providing customers with highly-scalable NAS storage technology for on-premise, virtual and cloud environments. Gluster's storage technology enables Redapt to offer a comprehensive, cost-effective storage solution delivering the scalability, performance and reliability that companies need to effectively run their data centers.
This webinar will provide an overview of the partnership, benefits of the joint solution, and include use cases of how customers today are deploying the joint solution. .
Best Practices for Becoming an Exceptional Postgres DBA EDB
Drawing from our teams who support hundreds of Postgres instances and production database systems for customers worldwide, this presentation provides real-real best practices from the nation's top DBAs. Learn top-notch monitoring and maintenance practices, get resource planning advice that can help prevent, resolve, or eliminate common issues, learning top database tuning tricks for increasing system performance and ultimately, gain greater insight into how to improve your effectiveness as a DBA.
Slides presented at Great Indian Developer Summit 2016 at the session MySQL: What's new on April 29 2016.
Contains information about the new MySQL Document Store released in April 2016.
This presentation was written by Wagner Bianchi for the presentation on the Oracle Consulting Team/Professional Services meeting that took place in San Francisco/CA.
MySQL Performance Tuning: The Perfect Scalability (OOW2019)Mirko Ortensi
“When data grows, MySQL multiplies.” When data grows, a standard configuration does not fit anymore. This hands-on lab walks through the ways MySQL has to adapt to an ever-increasing amount of data in context of tuning, under the heading “multiplication”:
—Scaling up through partitioning and how partitioning can help manage/archive data
—How to relocate binlog, undo, redo, tablespaces, and more on different mount points to take advantage of multiple storages
—All improvements to parallel slave replication
—A quick OS approach, to verify swapping and affinity tuning take the most out of the machine.
Spectrum Scale - Diversified analytic solution based on various storage servi...Wei Gong
This slides describe diversified analytic solutions based on Spectrum Scale with various deployment mode, such as storage rich-server, share storage, IBM DeepFlash 150 and Elastic Storage Server. It deep dives several advanced data management features and solutions for BD&A workload derived from Spectrum Scale.
Database as a Service on the Oracle Database Appliance PlatformMaris Elsins
Speaker: Marc Fielding, Co-speaker: Maris Elsins.
Oracle Database Appliance provides a robust, highly-available, cost-effective, and surprisingly scalable platform for database as a service environment. By leveraging Oracle Enterprise Manager's self-service features, databases can be provisioned on a self-service basis to a cluster of Oracle Database Appliance machines. Discover how multiple ODA devices can be managed together to provide both high availability and incremental, cost-effective scalability. Hear real-world lessons learned from successful database consolidation implementations.
In this Introduction to GlusterFS webinar, introduction and review of the GlusterFS architecture and key functionalities. Learn how GlusterFS is deployed in the datacenter, in the cloud, or between the two. We’ll also cover a brief update on GlusterFS v3.3 which is currently in beta.
Webinar Sept 22: Gluster Partners with Redapt to Deliver Scale-Out NAS StorageGlusterFS
Gluster has partnered with Redapt, Inc., an innovative data center architecture and infrastructure solutions provider, to integrate GlusterFS with hardware providing customers with highly-scalable NAS storage technology for on-premise, virtual and cloud environments. Gluster's storage technology enables Redapt to offer a comprehensive, cost-effective storage solution delivering the scalability, performance and reliability that companies need to effectively run their data centers.
This webinar will provide an overview of the partnership, benefits of the joint solution, and include use cases of how customers today are deploying the joint solution. .
Best Practices for Becoming an Exceptional Postgres DBA EDB
Drawing from our teams who support hundreds of Postgres instances and production database systems for customers worldwide, this presentation provides real-real best practices from the nation's top DBAs. Learn top-notch monitoring and maintenance practices, get resource planning advice that can help prevent, resolve, or eliminate common issues, learning top database tuning tricks for increasing system performance and ultimately, gain greater insight into how to improve your effectiveness as a DBA.
The objective of this presentation is to give Oracle DBAs the necessary background information to understand what is doable with MySQL and how to integrate MySQL instances into the Oracle world.
Lessons Learned: Novell Open Enterprise Server Upgrades Made EasyNovell
You've read the documentation, played in the lab, and now you're ready to jump in and upgrade your NetWare environment to Novell Open Enterprise Server 2 on Linux. Attend this session to glean a final few best practices and to learn how to make the most of the migration tools included in the product. You'll also learn about the various pitfalls encountered during real-world upgrades, as well as the solutions used to resolve them.
The new Novell File Management Suite is drawing accolades from customers, analysts and industry watchers alike. This session will help you dive in and see exactly what the product can do for your organization. We'll focus on the product's capabilities and its many use cases. We'll also explore the way it can help you better understand your organization's storage usage and give you the tools to begin automating the management of storage resources.
The new Novell File Management Suite is drawing accolades from customers, analysts and industry watchers alike. This session will help you dive in and see exactly what the product can do for your organization. We'll focus on the product's capabilities and its many use cases. We'll also explore the way it can help you better understand your organization's storage usage and give you the tools to begin automating the management of storage resources.
Novell Storage Manager: Your Secret Weapon for Simplified File and User Manag...Novell
See how the popular Novell Storage Manager can help you manage file storage and user administration like never before. Leveraging user identities and roles, you can customize policies based on your business rules, thereby automating redundant tasks and reducing the heavy manual effort typically required for file management. Attend this session to hear from the experts on architecture, deployment patterns and how to get the most bang for your buck!
DIY DBaaS: A guide to building your own full-featured DBaaSSeveralnines
More so than ever, businesses need to ensure that their databases are resilient, secure, and always available to support their operations. Database-as-a-Service (DBaaS) solutions have become a popular way for organizations to manage their databases efficiently, leveraging cloud infrastructure and advanced set-and-forget automation.
However, consuming DBaaS from providers comes with many compromises. In this guide, we’ll show you how you can build your own flexible DBaaS, your way. We’ll demonstrate how it is possible to get the full spectrum of DBaaS capabilities along with workload access and portability, and avoid surrendering control to a third-party.
From architectural and design considerations to operational requirements, we’ll take you through the process step-by-step, providing all the necessary information and guidance to help you build a DBaaS solution that is tailor-made to your unique use case. So get ready to dive in and learn how to build your own custom DBaaS solution from scratch!
We created this guide to help developers understand:
- Traditional vs. Sovereign DBaaS implementation models
- The DBaaS environment, elements and design principles
- Using a Day 2 operations framework to develop your blueprint
- The 8 key operations that form the foundation of a complete DBaaS
- Bringing the Day 2 ops framework to life with a provisional architecture
- How you can abstract the orchestration layer with Severalnines solutions
Cloud's future runs through Sovereign DBaaSSeveralnines
Sovereign DBaaS is a new way to do DBaaS that allows you to reliably scale your open-source database ops without being limited to a specific environment or ceding control of your infrastructure to third-party service providers.
With Sovereign DBaaS, users can leverage the benefits of modern deployment strategies, e.g. public cloud, hybrid, etc., with additional security, compliance, and risk mitigation. So what exactly is Sovereign DBaaS and why should you choose one?
Presented by Sanjeev Mohan, Principal Analyst at SanjMo and former Gartner Research VP, and Vinay Joosery, CEO of Severalnines, this webinar dives into the future of the cloud and database management and introduces a new solution, Sovereign DBaaS.
The state of the cloud and its current challenges
What is Sovereign DBaaS?
Agenda:
- Key features of Sovereign DBaaS
- Why you should choose a Sovereign DBaaS
- How you can implement Sovereign DBaaS with Severalnines
- Q&A
Tips to drive maria db cluster performance for nextcloudSeveralnines
Nextcloud requires a database to store administrative data for the platform. A poorly performing database can have a serious impact on performance and availability of Nextcloud. MariaDB Cluster is the recommended database backend for production installations that require high availability and performance.
This talk is a deep dive into how to design and optimize MariaDB Galera Cluster for Nextcloud. We will cover 5 tips on how to significantly improve performance and stability.
Agenda:
Overview of Nextcloud architecture
Database architecture design
Database proxy
MariaDB and InnoDB performance tuning
Nextcloud performance tuning
Q&A
Working with the Moodle Database: The BasicsSeveralnines
Managing the database behind Moodle is key to improving performance and achieving uptime for your users. In this training video we will talk about the Moodle database including topics like configuration, monitoring, and schema management as well as show you how ClusterControl can help with the management of your eLearning LMS systems.
SysAdmin Working from Home? Tips to Automate MySQL, MariaDB, Postgres & MongoDBSeveralnines
Are you an SysAdmin who is now responsible for your companies database operations? Then this is the webinar for you. Learn from a Senior DBA the basics you need to know to keep things up-and-running and how automation can help.
(slides) Polyglot persistence: utilizing open source databases as a Swiss poc...Severalnines
Over the past few years, VidaXL has become a European market leader in the online retail of slow moving consumer goods. When a company achieved over 50% year over year growth for the past 9 years, there is hardly enough time to overhaul existing systems. This means existing systems will be stretched to the maximum of their capabilities, and often additional performance will be gained by utilizing a large variety of datastores.
Webinar slides: How to Migrate from Oracle DB to MariaDBSeveralnines
Watch this webinar replay as we walk you through all you need to know to plan and execute a successful migration from Oracle database to MariaDB.
Over the years MariaDB has gained Enterprise support and maturity to run critical and complex data transaction systems. With the recent version, MariaDB has added some great new features such as SQL_Mode=Oracle compatibility, making the transition process easier than ever before.
Whether you’re planning to migrate from Oracle database to MariaDB manually or with the help of a commercial tool to automate the entire migration process, you need to know all the possible bottlenecks and methods involved in the process and the results validation.
Migrating from Oracle database to MariaDB can come with a number of benefits: lower cost of ownership, access to and use of an open source database engine, tight integration with the web, wide circle of MariaDB database professionals and more.
Find out how it could benefit your organisation!
AGENDA
- A brief introduction to the platform
- Oracle vs MariaDB
- Platform support
- Installation process
- Database access
- Backup process
- Controlling query execution
- Security
- Replication options
- Migration
- Planning and development strategy
- Assessment or preliminary check
- Data type mapping
- Migration tools
- Migration process
- Testing
- Post-migration
- Monitoring and Alerting
- Performance Management
- Backup Management
- High availability
- Upgrades
- Scaling
- Staff training
SPEAKER
Bartlomiej Oles, Senior Support Engineer at Severalnines, is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Webinar slides: How to Automate & Manage PostgreSQL with ClusterControlSeveralnines
Running PostgreSQL in production comes with the responsibility for a business critical environment; this includes high availability, disaster recovery, and performance. Ops staff worry whether databases are up and running, if backups are taken and tested for integrity, whether there are performance problems that might affect end user experience, if failover will work properly in case of server failure without breaking applications, and the list goes on.
ClusterControl can be used to operationalize your PostgreSQL footprint across your enterprise. It offers a standard way of deploying high-availability replication setups with auto-failover, integrated with load balancers offering a single endpoint to applications. It provides constant health and performance monitoring through rich dashboards, as well as backup management and point-in-time recovery
See how much time and effort can be saved, as well as risks mitigated, with the help of a unified management platform over the more traditional, manual methods.
We’ve seen a 152% increase in ClusterControl installations by PostgreSQL users last year, so make sure you don’t miss out on the trend!
AGENDA
- Managing PostgreSQL “the old way”:
- Common challenges
- Important tasks to perform
- Tools that are available to help
- PostgreSQL automation and management with ClusterControl:
- Deployment
- Backup and recovery
- HA setups
- Failover
- Monitoring
- Live Demo
SPEAKER
Sebastian Insausti, Support Engineer at Severalnines, has loved technology since his childhood, when he did his first computer course (Windows 3.11). And from that moment he was decided on what his profession would be. He has since built up experience with MySQL, PostgreSQL, HAProxy, WAF (ModSecurity), Linux (RedHat, CentOS, OL, Ubuntu server), Monitoring (Nagios), Networking and Virtualization (VMWare, Proxmox, Hyper-V, RHEV).
Prior to joining Severalnines, Sebastian worked as a consultant to state companies in security, database replication and high availability scenarios. He’s also a speaker and has given a few talks locally on InnoDB Cluster and MySQL Enterprise together with an Oracle team. Previous to that, he worked for a Mexican company as chief of sysadmin department as well as for a local ISP (Internet Service Provider), where he managed customers' servers and connectivity.
Webinar slides: How to Manage Replication Failover Processes for MySQL, Maria...Severalnines
Failover is the process of moving to a healthy standby component, during a failure or maintenance event, in order to preserve uptime. The quicker it can be done, the faster you can be back online. However, failover can be tricky for transactional database systems as we strive to preserve data integrity - especially in asynchronous or semi-synchronous topologies. There are risks associated, from diverging datasets to loss of data. Failing over due to incorrect reasoning, e.g., failed heartbeats in the case of network partitioning, can also cause significant harm.
This webinar replay gives a detailed overview of what failover processes may look like in MySQL, MariaDB and PostgreSQL replication setups. We’ve covered the dangers related to the failover process, and discuss the tradeoffs between failover speed and data integrity. We’ve found out about how to shield applications from database failures with the help of proxies. And we've finally had a look at how ClusterControl manages the failover process, and how it can be configured for both assisted and automated failover.
So if you’re looking at minimizing downtime and meet your SLAs through an automated or semi-automated approach, then this webinar replay is for you!
AGENDA
- An introduction to failover - what, when, how
- in MySQL / MariaDB
- in PostgreSQL
- To automate or not to automate
- Understanding the failover process
- Orchestrating failover across the whole HA stack
- Difficult problems
- Network partitioning
- Missed heartbeats
- Split brain
- From assisted to fully automated failover with ClusterControl
- Demo
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
What if …
- Traditional, labour-intensive backup and archive practices for your MySQL, MariaDB, MongoDB and PostgreSQL databases were a thing of the past?
- You could have one backup management solution for all your business data?
- You could ensure integrity of all your backups?
- You could leverage the competitive pricing and almost limitless capacity of cloud-based backup while meeting cost, manageability, and compliance requirements from the business.
Welcome to our webinar on Backup Management with ClusterControl.
ClusterControl’s centralized backup management for open source databases provides you with hot backups of large datasets, point in time recovery in a couple of clicks, at-rest and in-transit data encryption, data integrity via automatic restore verification, cloud backups (AWS, Google and Azure) for Disaster Recovery, retention policies to ensure compliance, and automated alerts and reporting.
Whether you are looking at rebuilding your existing backup infrastructure, or updating it, this webinar is for you!
AGENDA
- Backup and recovery management of local or remote databases
- Logical or physical backups
- Full or Incremental backups
- Position or time-based Point in Time Recovery (for MySQL and PostgreSQL)
- Upload to the cloud (Amazon S3, Google Cloud Storage, Azure Storage)
- Encryption of backup data
- Compression of backup data
- One centralized backup system for your open source databases (Demo)
- Schedule, manage and operate backups
- Define backup policies, retention, history
- Validation - Automatic restore verification
- Backup reporting
SPEAKER
Bartlomiej Oles, Senior Support Engineer at Severalnines, is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Disaster Recovery Planning for MySQL & MariaDBSeveralnines
Bart Oles - Severalnines AB
Organizations need an appropriate disaster recovery plan to mitigate the impact of downtime. But how much should a business invest? Designing a highly available system comes at a cost, and not all businesses and indeed not all applications need five 9's availability.
We will explain fundamental disaster recovery concepts and walk you through the relevant options from the MySQL & MariaDB ecosystem to meet different tiers of disaster recovery requirements, and demonstrate how to automate an appropriate disaster recovery plan.
Krzysztof Ksiazek - Severalnines AB
So, you are a developer or sysadmin and showed some abilities in dealing with databases issues. And now, you have been elected to the role of DBA. And as you start managing the databases, you wonder…
* How do I tune them to make best use of the hardware?
* How do I optimize the Operating System?
* How do I best configure MySQL or MariaDB for a specific database workload?
If you're asking yourself the following questions when it comes to optimally running your MySQL or MariaDB databases, then this talk is for you!
We will discuss some of the settings that are most often tweaked and which can bring you significant improvement in the performance of your MySQL or MariaDB database. We will also cover some of the variables which are frequently modified even though they should not.
Performance tuning is not easy, especially if you're not an experienced DBA, but you can go a surprisingly long way with a few basic guidelines.
Performance Tuning Cheat Sheet for MongoDBSeveralnines
Bart Oles - Severalnines AB
Database performance affects organizational performance, and we tend to look for quick fixes when under stress. But how can we better understand our database workload and factors that may cause harm to it? What are the limitations in MongoDB that could potentially impact cluster performance?
In this talk, we will show you how to identify the factors that limit database performance. We will start with the free MongoDB Cloud monitoring tools. Then we will move on to log files and queries. To be able to achieve optimal use of hardware resources, we will take a look into kernel optimization and other crucial OS settings. Finally, we will look into how to examine performance of MongoDB replication.
Advanced MySql Data-at-Rest Encryption in Percona ServerSeveralnines
Iwo Panowicz - Percona & Bart Oles - Severalnines AB
The purpose of the talk is to present data-at-rest encryption implementation in Percona Server for MySQL.
Differences between Oracle's MySQL and MariaDB implementation.
- How it is implemented?
- What is encrypted:
- Tablespaces?
- General tablespace?
- Double write buffer/parallel double write buffer?
- Temporary tablespaces? (KEY BLOCKS)
- Binlogs?
- Slow/general/error logs?
- MyISAM? MyRocks? X?
- Performance overhead.
- Backups?
- Transportable tablespaces. Transfer key.
- Plugins
- Keyrings in general
- Key rotation?
- General-Purpose Keyring Key-Management Functions
- Keyring_file
- Is useful? How to make it profitable?
- Keyring Vault
- How does it work?
- How to make a transition from keyring_file
Polyglot Persistence Utilizing Open Source Databases as a Swiss Pocket KnifeSeveralnines
Art Van Scheppingen - vidaXL & Bart Oles - Severalnines AB
Over the past few years, VidaXL has become a European market leader in the online retail of slow moving consumer goods. When a company achieved over 50% year over year growth for the past 9 years, there is hardly enough time to overhaul existing systems. This means existing systems will be stretched to the maximum of their capabilities, and often additional performance will be gained by utilizing a large variety of datastores.
Polyglot persistence reigns in rapidly growing environments and the traditional one-size-fits-all strategy of monoglots is over.
VidaXL has a broad landscape of datastores, ranging from traditional SQL data stores, like MySQL or PostgreSQL alongside more recent load balancing technologies such as ProxySQL, to document stores like MongoDB and search engines such as SOLR and Elasticsearch.
Webinar slides: Free Monitoring (on Steroids) for MySQL, MariaDB, PostgreSQL ...Severalnines
Traditional server monitoring tools are not built for modern distributed database architectures. Let’s face it, most production databases today run in some kind of high availability setup - from simpler master-slave replication to multi-master clusters fronted by redundant load balancers. Operations teams deal with dozens, often hundreds of services that make up the database environment.
This is why we built ClusterControl - to address modern, highly distributed database setups based on replication or clustering. We wanted something that could provide a systems view of all the components of a distributed cluster, including load balancers.
Watch this replay of a webinar on free database monitoring using ClusterControl Community Edition. We show you how to monitor all your MySQL, MariaDB, PostgreSQL and MongoDB systems from a single point of control - whether they are deployed as Galera Clusters, sharded clusters or replication setups across on-prem and cloud data centers. We also see how to use Advisors in order to improve performance.
AGENDA
- Requirements for monitoring distributed database systems
- Cloud-based vs On-prem monitoring solutions
- Agent-based vs Agentless monitoring
- Deepdive into ClusterControl Community Edition
- Architecture
- Metrics Collection
- Trending
- Dashboards
- Queries
- Performance Advisors
- Other features available to Community users
SPEAKER
Bartlomiej Oles is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Webinar slides: An Introduction to Performance Monitoring for PostgreSQLSeveralnines
To operate PostgreSQL efficiently, you need to have insight into database performance and make sure it is at optimal levels.
With that in mind, we dive into monitoring PostgreSQL for performance in this webinar replay.
PostgreSQL offers many metrics through various status overviews and commands, but which ones really matter to you? How do you trend and alert on them? What is the meaning behind the metrics? And what are some of the most common causes for performance problems in production?
We discuss this and more in ordinary, plain DBA language. We also have a look at some of the tools available for PostgreSQL monitoring and trending; and we’ll show you how to leverage ClusterControl’s PostgreSQL metrics, dashboards, custom alerting and other features to track and optimize the performance of your system.
AGENDA
- PostgreSQL architecture overview
- Performance problems in production
- Common causes
- Key PostgreSQL metrics and their meaning
- Tuning for performance
- Performance monitoring tools
- Impact of monitoring on performance
- How to use ClusterControl to identify performance issues
- Demo
SPEAKER
Sebastian Insausti, Support Engineer at Severalnines, has loved technology since his childhood, when he did his first computer course (Windows 3.11). And from that moment he was decided on what his profession would be. He has since built up experience with MySQL, PostgreSQL, HAProxy, WAF (ModSecurity), Linux (RedHat, CentOS, OL, Ubuntu server), Monitoring (Nagios), Networking and Virtualization (VMWare, Proxmox, Hyper-V, RHEV).
Prior to joining Severalnines, Sebastian worked as a consultant to state companies in security, database replication and high availability scenarios. He’s also a speaker and has given a few talks locally on InnoDB Cluster and MySQL Enterprise together with an Oracle team. Previous to that, he worked for a Mexican company as chief of sysadmin department as well as for a local ISP (Internet Service Provider), where he managed customers' servers and connectivity.
This webinar builds upon a related blog post by Sebastian: https://severalnines.com/blog/performance-cheat-sheet-postgresql.
Webinar slides: Our Guide to MySQL & MariaDB Performance TuningSeveralnines
If you’re asking yourself the following questions when it comes to optimally running your MySQL or MariaDB databases:
- How do I tune them to make best use of the hardware?
- How do I optimize the Operating System?
- How do I best configure MySQL or MariaDB for a specific database workload?
Then this replay is for you!
We discuss some of the settings that are most often tweaked and which can bring you significant improvement in the performance of your MySQL or MariaDB database. We also cover some of the variables which are frequently modified even though they should not.
Performance tuning is not easy, especially if you’re not an experienced DBA, but you can go a surprisingly long way with a few basic guidelines.
This webinar builds upon blog posts by Krzysztof from the ‘Become a MySQL DBA’ series.
AGENDA
- What to tune and why?
- Tuning process
- Operating system tuning
- Memory
- I/O performance
- MySQL configuration tuning
- Memory
- I/O performance
- Useful tools
- Do’s and do not’s of MySQL tuning
- Changes in MySQL 8.0
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
Webinar slides: Migrating to Galera Cluster for MySQL and MariaDBSeveralnines
Galera Cluster is a mainstream option for high availability MySQL and MariaDB. And though it has established itself as a credible replacement for traditional MySQL master-slave architectures, it is not a drop-in replacement.
While Galera Cluster has some characteristics that make it unsuitable for certain use cases, most applications can still be adapted to run on it.
The benefits are clear: multi-master InnoDB setup with built-in failover and read scalability.
But how do you migrate? Does the schema or application change? What are the limitations? Can a migration be done online, without service interruption? What are the potential risks?
In this webinar, Severalnines Support Engineer Bart Oles walks you through what you need to know in order to migrate from standalone or a master-slave MySQL/MariaDB setup to Galera Cluster.
AGENDA
- Application use cases for Galera
- Schema design
- Events and Triggers
- Query design
- Migrating the schema
- Load balancer and VIP
- Loading initial data into the cluster
- Limitations:
- Cluster technology
- Application vendor support
- Performing Online Migration to Galera
- Operational management checklist
- Belts and suspenders: Plan B
- Demo
SPEAKER
Bartlomiej Oles is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
Webinar slides: How to Measure Database Availability?Severalnines
Database availability is notoriously hard to measure and report on, although it is an important KPI in any SLA between you and your customer. We often define availability in terms of 9’s (e.g. 99.9% or 99.999%), although there is often a lack of understanding of what these numbers might mean, or how we can measure them.
Is the database available if an instance is up and running, but it is unable to serve any requests? Or if response times are excessively long, so that users consider the service unusable? Is the impact of one longer outage the same as multiple shorter outages? How do partial outages affect database availability, where some users are unable to use the service while others are completely unaffected?
Not agreeing on precise definitions with your customer might lead to dissatisfaction. The database team might be reporting that they have met their availability goals, while the customer is dissatisfied with the service. In this webinar, we will discuss the different factors that affect database availability. We will then see how you can measure your database availability in a realistic way.
AGENDA
- Defining availability targets
- Critical business functions
- Customer needs
- Duration and frequency of downtime
- Planned vs unplanned downtime
- SLA
- Measuring the database availability
- Failover/Switchover time
- Recovery time
- Upgrade time
- Queries latency
- Restoration time from backup
- Service outage time
- Instrumentation and tools to measure database availability:
- Free & open-source tools
- CC's Operational Report
- Paid tools
SPEAKER
Bartlomiej Oles is a MySQL and Oracle DBA, with over 15 years experience in managing highly available production systems at IBM, Nordea Bank, Acxiom, Lufthansa, and other Fortune 500 companies. In the past five years, his focus has been on building and applying automation tools to manage multi-datacenter database environments.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
How world-class product teams are winning in the AI era by CEO and Founder, P...
Severalnines Self-Training: MySQL® Cluster - Part II
1. MySQL Cluster Training
presented by severalnines.com
Address:
Contact: SeveralninesAB
Jean-Jérôme Schmidt c/o SICS, Box 1263
Email: services@severalnines.com Isafjordsgatan22
SE-164-29 Kista
Copyright 2011 Severalnines AB Control your database infrastructure 1
2. Introduction
• At Severalnines, we believe in sharing information
and knowledge; we all come from an open source
background
• We know a lot of things about MySQL Cluster and
think that MySQL Cluster is a great technology
• These free MySQL Cluster Training slides are a
contribution of ours to the knowledge and information
sharing that‟s common practice in the open source
community
• If you have any questions on these slides or would
like to book an actual training class, please contact
us at: services@severalnines.com
Copyright 2011 Severalnines AB Control your database infrastructure 2
3. Training Slides - Concept
• Over the coming weeks we will be chronologically
releasing slides for the different sections of our
MySQL Cluster Training program on our website.
• The full agenda of the training with all of its modules
is outlined in the next slides so that you can see what
topics will be covered over the coming weeks.
• Particularly specialised topics such as Cluster/J or
NDB API are not fully covered in the slides. We
recommend our instructor-led training classes for
such topics.
• Please contact us for more details:
services@severalnines.com
Copyright 2011 Severalnines AB Control your database infrastructure 3
4. Full Training Agenda (1/4)
• MySQL Cluster Introduction
– MySQL eco system
– Scale up, scale out, and sharding
– MySQL Cluster Architecture
– Use cases
– Features
– Node types and Roles
• Detailed Concepts
– Data Distribution
– Verifying data distribution
– Access Methods
– Partitioning
– Node failures and failure detection
– Network Partitioning
– Transactions and Locking
– Consistency Model
– Redo logging and Checkpointing
Copyright 2011 Severalnines AB Control your database infrastructure 4
5. Agenda (2/4)
• Installing MySQL Cluster
– Setting up MySQL Cluster
– Starting/stopping nodes
– Recovery and restarts
– Upgrading configuration
– Upgrading Cluster
• Performance Tuning (instructor-led only; contact us at services@severalnines.com)
– Differences compared to Innodb/MyISAM
– Designing efficient and fast applications
– Identifying bottlenecks
– Tweaking configuration (OS and MySQL Cluster)
– Query Tuning
– Schema Design
– Index Tuning
Copyright 2011 Severalnines AB Control your database infrastructure 5
6. Agenda (3/4)
• Management and Administration
– Backup and Restore
– Geographical Replication
– Online and offline operations
– Ndbinfo tables
– Reporting
– Single user mode
– Scaling Cluster
• Disk Data
– Use cases
– Limitations
– Best practice configuration
• Designing a Cluster
– Capacity Planning and Dimensioning
– Hardware recommendations
– Best practice Configuration
– Storage calculations
Copyright 2011 Severalnines AB Control your database infrastructure 6
7. Agenda (4/4)
• Resolving Issues
– Common problems
– Error logs and Tracefiles
– Recovery and Escalation procedures
• Connectivity Overview
– NDBAPI
– Cluster/J
– LDAP
• Severalnines Tools
– Monitoring and Management
– Benchmarking
– Sandboxes
– Configuration and capacity planning
• Conclusion
Copyright 2011 Severalnines AB Control your database infrastructure 7
8. Agenda: Lab Exercises
(only applicable to instructor-led training classes)
• Lab Exercises
– Installing and Loading data into MySQL Cluster
– Starting/stopping nodes, recovery
– Query tuning
– Backup and Restore
– Configuration Upgrade
• Would you like to try something particular?
– This is possible too, speak with your instructor
Copyright 2011 Severalnines AB Control your database infrastructure 8
9. Prerequisites
• Readers / Participants have understanding of SQL and basic database concepts.
• Laptops/PCs for hands-on exercises
• Linux: 1GB RAM
• Windows: 2GB RAM
• Approx. 20GB disk space and Virtualbox installed.
• Virtualbox can be downloaded for free at http://www.virtualbox.org/wiki/Downloads
• MySQL Cluster version 7.1 or later
Copyright 2011 Severalnines AB Control your database infrastructure 9
10. Part 2
Section 2: Detailed Concepts
Copyright 2011 Severalnines AB Control your database infrastructure 10
11. Topics covered in Part 2
• Data Distribution & Partitioning
• Two Phase Commit Protocol
• Transaction Resources
Copyright 2011 Severalnines AB Control your database infrastructure 11
12. Data Distribution and Partitioning
Copyright 2011 Severalnines AB Control your database infrastructure 12
13. MySQL Cluster Architecture
Application Application Application
R/I/U/D R/I/U/D R/I/U/D
Java, C++, LDAP
Application
R/I/U/D
Native API
DATA DATA
NODE NODE
Management
Management
Server
DATA DATA
Server
NODE NODE
Copyright 2011 Severalnines AB Control your database infrastructure 13
14. Data Distribution
• MySQL Cluster automatically partitions tables and
distributes the data on the partitions.
– It is possible to use user-defined partitioning.
• A table is divided into as many partitions as you have
data nodes and data is “striped” on the partitions.
• Hashing of the Primary Key determines the Partition
that should hold the data - automatic distribution.
• It is possible to hash on part of the primary key –
great for sharding and locality of data!
Copyright 2011 Severalnines AB Control your database infrastructure 14
15. Partitioning and Data Distribution
• With two data nodes, data node 1 and data node 2,
each table is divided into two Partitions.
• Subidis the Primary Key.
• Hashing on the Primary Key (subid) determines the
Partition. It is also possible to hash on part of the PK.
– Odd PKs Green
– Even PKs Red
STORAGE LAYER
subid data
DATA DATA
NODE 1 NODE 2 1 A Partition 0
3 B
2 C Partition 1
P0 P1 4 D
Copyright 2011 Severalnines AB Control your database infrastructure 15
16. Replicas
• To provide for redundancy and fast failover partitions
are synchronouslyreplicated.
• Two replicas (two copies of data) is most common
– it is possible to have one (previous slide), two, three and four
replicas.
– NoOfReplicas=2
STORAGE LAYER
DATA DATA
NODE 1 NODE 2
P0 S0
Copyright 2011 Severalnines AB Control your database infrastructure 16
17. Replicas
• To provide for redundancy and fast failover partitions
are synchronouslyreplicated from a PRIMARY to a
SECONDARY partition.
– When a change ( ) is made on P0 it is synchronously
replicated to S0.
– The change is persisted at transaction COMMIT.
– Either both P0 or S0 is update or NOTHING AT ALL
STORAGE LAYER
DATA DATA
NODE 1 NODE 2
P0 S0
Copyright 2011 Severalnines AB Control your database infrastructure 17
18. Data Distribution - disk logging
• After the COMMIT, the data is in main memory.
– But changes are REDO LOGGED, but the REDO LOG is
synced to disk every xth ms (1000 ms is recommended).
• Controlled by TimeBetweenGlobalCheckpoints
• Similar to innodb-flush-log-at-trx_commit=2
– The data is also checkpointed to disk.
• More on this later.
• The disk logging makes it possible to recovery a
completely failed cluster.
Copyright 2011 Severalnines AB Control your database infrastructure 18
19. Replicas
• To provide for redundancy and fast failover partitions
are synchronouslyreplicated.
• Two replicas (two copies of data) is most common
– it is possible to have one (previous slide), two, three and four
replicas.
– NoOfReplicas=2
STORAGE LAYER
subid data
DATA DATA
NODE 1 NODE 2 1 A Partition 0
3 B
2 C Partition 1
P0 P1
4 D
S1 S0
Px == PRIMARY Partition x
Sx == SECONDARY Partition x
Copyright 2011 Severalnines AB Control your database infrastructure 19
20. Node groups
• Data Nodes sharing the same data data belongs to a
node group.
• A node group contains as many nodes as you have
replicas (NoOfReplicas=2 is used in the picture
below).
STORAGE LAYER
subid data
DATA DATA
NODE 1 NODE 2 1 A Partition 0
3 B
2 C Partition 1
P0 P1
4 D
S1 S0
Node group 0 Px == PRIMARY Partition x
Sx == SECONDARY Parttionx
Copyright 2011 Severalnines AB Control your database infrastructure 20
21. Two Replicas – Four Data Nodes
• Four data nodes – four partitions – two replicas
• Four data nodes and two replicas two node groups
– No of node groups = #data nodes / #replicas
STORAGE LAYER subid data
1 A
Partition 0
2 B
DATA DATA DATA DATA
NODE 1 NODE 2 NODE 3 NODE 4 3 C
Partition 1
4 D
P0 P1 P2 P3 5 E
Partition 2
6 F
S1 S0 S3 S2
7 G
Partition 3
Node group 0 Node group 1 8 H
Px == PRIMARY Partition x
Sx == SECONDARY Parttionx
Copyright 2011 Severalnines AB Control your database infrastructure 21
22. Three Replicas – Three Data Nodes
• Three data nodes – Three partitions
– More replicas gives slower writes.
• Three data nodes and three replicas one node
group. This is not common.
– Two replicas is common practice/best practice
STORAGE LAYER
DATA DATA DATA subid data
NODE 1 NODE 2 NODE 3
1 A Partition 0
2 B
P0 P1 P2 3 C Partition 1
S1 S0 S0 4 D
S2 S2 S1 5 E Partition
6 F 2
Node group 0
Px == PRIMARY Partition x
Sx == SECONDARY
Parttionx
Copyright 2011 Severalnines AB Control your database infrastructure 22
23. Recommendations
• Two Replicas recommended - gives best tradeoff
between Performance and Availability
• Three Replicas (or Four) will give slower writes (, is
less tested (not so many deployments), and less
deployments are made with this.
• Cost of write (approx)
– 1 replica (no redundancy): cost X
– 2 replicas: cost 2X
– 3 replicas: cost 3X
Copyright 2011 Severalnines AB Control your database infrastructure 23
24. Verifying Data Distribution
• A helper program called ndb_desc shows how many
records are in each partition:
ndb_desc –c<mgmt_host:1186> –d<database>
-p<tablename>
Copyright 2011 Severalnines AB Control your database infrastructure 24
25. Two Phase Commit Protocol
Copyright 2011 Severalnines AB Control your database infrastructure 25
26. View of the Data Node
• Each node has as active Transaction Coordinator
(TC).
– Every Transaction starts at the TC
– One transaction consists of one or more Operations.
• Operation is SELECT, INSERT, UPDATE, DELETE
• Each node has 1-4 Local Query Handlers (LQH)
– Executes the operations
TC
• ACCess stores a hash table
LQH
– Primary Key and Unique
• TUPle stores data records ACC TUP
– ACC hash index links to record in TUP
Copyright 2011 Severalnines AB Control your database infrastructure 26
27. Two Phase Commit Protocol
• 2PC facilitates the synchronous replication
– To safeguard that both Primary and Secondary Partitions are
modified in a Consistent Way
– Locking (Row level)
• 2PC consists of two phases
– Prepare Phase
• Grab locks
• Make a update to a shadow copy
– Commit Phase
• Make the shadow copy the real copy
• Release locks
• 2PC protocol is only involved in
– Update, Delete, Insert
• Reads are not 2PC (would be very slow and
pointless)
Copyright 2011 Severalnines AB Control your database infrastructure 27
28. Two Phase Commit Protocol
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
– Two data nodes - PK hashes to Primary P0.
– The same applies to UPDATES and DELETEs
TC TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 28
29. Phase 1 - Prepare Phase
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
• The protocol starts at the TC that has the Primary
Partition for the PK
TC TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 29
30. Phase 1 - Prepare Phase
• TC starts by calculating a hash (md5sum) on the
Entire Primary key.
– md5sum(PK)
– This gives 128 bit hash value, divided up in 2 x 64-bit
INDEX_HASH PARTITION_HASH
– Actually further calculations are made
• PARTITION= PARTITION_HASH % NO_OF_PARTITIONS
• INDEX_POS=INDEX_HASH % NO_OF_BUCKETS
– LH*3 Algorithm is what is really being used.
• Now MySQL Cluster will know what nodes and
partitions are involved.
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
– INDEX_POS=23
– PARTITION=P0
Copyright 2011 Severalnines AB Control your database infrastructure 30
31. Phase 1 - Prepare Phase
• INSERT INTO t1(id, data) VALUES (1, „hello‟)
• Prepare on the local LQH
TC TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 31
32. Phase 1 - Prepare Phase
• Insert into the hash table in ACC and write record in
TUP (allocate space for the record).
– The Index Entry is locked now
TC TC
PREPARE
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello
POS=23
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 32
33. Phase 1 - Prepare Phase
• Send PREPARE message to the other LQH for the
secondary Partition (S0), and do exactly the same
thing.
TC TC
PREPARE
PREPARE
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 33
34. Phase 1 - Prepare Phase
• Send back PREPARE OK to TC
TC PREPARE OK TC
PREPARE
PREPARE
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 34
35. Phase 2 - Commit Phase
• Send COMMIT message to REMOTE LQH first
– Release locks, simplifies error handling, ops waiting on lock
can resume faster.
TC COMMIT TC
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 35
36. Phase 2 - Commit Phase
• Send COMMIT to LQH of P0
– Release locks
TC COMMIT TC
COMMIT
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 36
37. Phase 2 - Commit Phase
• Send COMMIT OK to TC
TC COMMIT TC
COMMIT OK
COMMIT
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 37
38. Phase 2 - Commit Phase
• Send COMMIT ACK to application
COMMIT ACK
TC COMMIT TC
COMMIT OK
COMMIT
LQH LQH
ACC TUP ACC TUP
P0 P0 S0 S0
1 hello 1 hello
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 38
39. Failure Handling
• If the Transaction Coordinator fails another TC will
take over.
• The new TC will ask the nodes “Send me a list of
outstanding transactions from the failed TC”
• The new TC will
– ABORT transactions in the Prepare Phase
– Force COMMIT of transactions IFF another node has seen a
COMMIT message.
• The Node that has failed will get the changes when it recovers.
Copyright 2011 Severalnines AB Control your database infrastructure 39
40. Transaction Timers
• Deadlock detection – waiting for lock
– If a transaction is waiting too long time for a lock it is aborted
• TransactionDeadLockDetectionTimeout=3000 [ms]
• Inactive Transactions – holding a lock
– If a transaction is holding a lock too long it is aborted
• TransactionInactiveTimeout=60000 [ms]
– Never set TransactioInactiveTimeout=0
Copyright 2011 Severalnines AB Control your database infrastructure 40
41. Locking
• Committed Read Isolation level is used
• This means that a transaction
– Will read last committed data unless it has itself modified this
data.
• Locks are acquired in the Prepare Phase and
released in the Commit Phase.
Copyright 2011 Severalnines AB Control your database infrastructure 41
43. Transaction and Operation Records
• Each transaction requires on Transaction Record
(TR)
– When it was started
– Controlled by MaxNoOfConcurrentTransactions
• Also each Operation requires one Operation Record
(OR) in TC and one Local Operation Record (LOR)
in LQH
– What transaction it belongs to
– Operation Type
– Controlled by MaxNoOfConcurrentOperations
– and MaxNoOfLocalOperations
Copyright 2011 Severalnines AB Control your database infrastructure 43
44. Transaction and Operation Records
• Assume
– MaxNoOfConcurrentTransactions=3
– MaxNoOfConcurrentOperations=3
– MaxNoOfLocalOperations=4
• By default there are 10% more Local Ops than Conc Ops.
• Query
– INSERT INTO t1 (id,data) VALUES(1,‟x‟);
• One transaction
• One operation
Copyright 2011 Severalnines AB Control your database infrastructure 44
45. Transaction Starts (prepare phase)
• INSERT INTO t1 (id,data) VALUES(1,‟x‟);
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 45
46. Transaction Starts (prepare phase)
• Allocate a Transaction Record at TC
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 46
47. Transaction Starts (prepare phase)
• Allocate a Operation Record at TC
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 47
48. Transaction Starts (prepare phase)
• Allocate a Local Operation Record at LQH
TR
TC TC
OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 48
49. Transaction Starts (prepare phase)
• With many parallel transactions
TR TR
TC TC
OR OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 49
50. Transaction Starts (prepare phase)
• Wrongly dimensioned for failover
• Not enough TRs / ORs
TR TR
TC TC
OR OR
PREPARE
PREPARE
LOR LQH LQH LOR
ACC TUP ACC TUP
P0 P0 S0 S0
IndexMemory DataMemory IndexMemory DataMemory
DATA NODE 1 DATA NODE 2
Copyright 2011 Severalnines AB Control your database infrastructure 50
51. Transaction and Operation Records
• Define
– MaxNoOfConcurrentTransactions= 2 x<max_threads>
– MaxNoOfConcurrentOperations=
MaxConcurrentTransactionsx<ops in trans>
– MaxNoOfLocalOperations=1.1 x
MaxNoOfConcurrentOperations
• 1.1x is by default
• Seldom needed to change this.
Copyright 2011 Severalnines AB Control your database infrastructure 51
52. Coming next in Part 3:
Detailed Concepts
Access Methods
Redo Logging and Check-pointing
Recovery
Node Failures and Error Detection
Network Partitioning
Copyright 2011 Severalnines AB Control your database infrastructure 52
53. We hope these training slides are
useful to you!
Please visit our website to view the
next section of this training.
For any questions, comments, feedback or to
book a training class, please contact us at:
services@severalnines.com
Thank you!
Copyright 2011 Severalnines AB Control your database infrastructure 53
Dear Jury, Just as the electric grid revolutionized access to electricity 100 years ago, we at Severalnines believe that Cloud Computing will revolutionize IT where organizations will be able to plug into extremely powerful computing resources over the network. We have already seen the beginnings of this new wave, where the current infrastructure stack is being challenged and disrupted by a whole set of new technologies. For instance, in the database market, over 40 startups have received funding over the past 18 months. Severalnines is not building yet another database product, we believe there are already a lot of good technologies available. To manage a database costs 4 times the purchase price, and yet, very few companies are addressing this problem. Severalnines focuses on solutions to address this underserved segment. The founders of the company have a solid background in databases, having been at MySQL since 2003. The company develops a management platform which is database and cloud agnostic.We are database independent since we do not know who, if anybody, will be the next MySQL of the cloud.We are cloud independent, since we do not want to depend on any cloud vendor (e.g. Amazon or Rackspace) to avoid vendor lock-in. After the Amazon EC2 downtime during the Easter break, hundreds of affected companies have realized the importance of this. There is also a commercial aspect for avoiding vendor lock-in. Severalnines enhances productivity of organizations by attacking the biggest cost associated with database systems. We are a Swedish startup, hosted by SICS in Kista. There is also a small but very efficient development capacity in Singapore. Just as MySQL became a major brand and placed Sweden on the global software infrastructure map, we believe Severalnines can become a serious global player in the emerging Cloud space. We are very thankful that an organization like Eurocloud exists, and would like to thank the jury for considering our application. Kind regards,Vinay Joosery Severalnines AB
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Dear Jury, Just as the electric grid revolutionized access to electricity 100 years ago, we at Severalnines believe that Cloud Computing will revolutionize IT where organizations will be able to plug into extremely powerful computing resources over the network. We have already seen the beginnings of this new wave, where the current infrastructure stack is being challenged and disrupted by a whole set of new technologies. For instance, in the database market, over 40 startups have received funding over the past 18 months. Severalnines is not building yet another database product, we believe there are already a lot of good technologies available. To manage a database costs 4 times the purchase price, and yet, very few companies are addressing this problem. Severalnines focuses on solutions to address this underserved segment. The founders of the company have a solid background in databases, having been at MySQL since 2003. The company develops a management platform which is database and cloud agnostic.We are database independent since we do not know who, if anybody, will be the next MySQL of the cloud.We are cloud independent, since we do not want to depend on any cloud vendor (e.g. Amazon or Rackspace) to avoid vendor lock-in. After the Amazon EC2 downtime during the Easter break, hundreds of affected companies have realized the importance of this. There is also a commercial aspect for avoiding vendor lock-in. Severalnines enhances productivity of organizations by attacking the biggest cost associated with database systems. We are a Swedish startup, hosted by SICS in Kista. There is also a small but very efficient development capacity in Singapore. Just as MySQL became a major brand and placed Sweden on the global software infrastructure map, we believe Severalnines can become a serious global player in the emerging Cloud space. We are very thankful that an organization like Eurocloud exists, and would like to thank the jury for considering our application. Kind regards,Vinay Joosery Severalnines AB
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Dear Jury, Just as the electric grid revolutionized access to electricity 100 years ago, we at Severalnines believe that Cloud Computing will revolutionize IT where organizations will be able to plug into extremely powerful computing resources over the network. We have already seen the beginnings of this new wave, where the current infrastructure stack is being challenged and disrupted by a whole set of new technologies. For instance, in the database market, over 40 startups have received funding over the past 18 months. Severalnines is not building yet another database product, we believe there are already a lot of good technologies available. To manage a database costs 4 times the purchase price, and yet, very few companies are addressing this problem. Severalnines focuses on solutions to address this underserved segment. The founders of the company have a solid background in databases, having been at MySQL since 2003. The company develops a management platform which is database and cloud agnostic.We are database independent since we do not know who, if anybody, will be the next MySQL of the cloud.We are cloud independent, since we do not want to depend on any cloud vendor (e.g. Amazon or Rackspace) to avoid vendor lock-in. After the Amazon EC2 downtime during the Easter break, hundreds of affected companies have realized the importance of this. There is also a commercial aspect for avoiding vendor lock-in. Severalnines enhances productivity of organizations by attacking the biggest cost associated with database systems. We are a Swedish startup, hosted by SICS in Kista. There is also a small but very efficient development capacity in Singapore. Just as MySQL became a major brand and placed Sweden on the global software infrastructure map, we believe Severalnines can become a serious global player in the emerging Cloud space. We are very thankful that an organization like Eurocloud exists, and would like to thank the jury for considering our application. Kind regards,Vinay Joosery Severalnines AB
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com
Severalnines has been offering its products free of charge since 2007, while the founders were employed at MySQL. These products are the de-facto standard tools to assist MySQL customers and users in deploying their MySQL clusters. More information about Severalnines at www.severalnines.com